Welcome to the PLI Blog! Our blog post will cover topics such as new AI research, cross-disciplinary applications, societal implications, and more!

About PLI BLog

Getting Started with Large Language Models with Princeton Language and Intelligence
Jan. 10, 2025

Large language models have transformed how we conduct research. In this session, we will talk about how one can:
(1) fine-tune a language model on their own data;
(2) evaluate models on benchmarks;
(3) use large-language-model-powered embedding tools for clustering, classification, and other tasks;
(4) access frontier large language…

Princeton Language and Intelligence at NeurIPS 2024
Dec. 10, 2024

NeurIPS, the top AI and ML conference, is being held in Vancouver from December 10 to December 15. Princeton Language and Intelligence is proud to feature the work of our students, post-docs, and faculty that will be showcased at the conference. 

Presented at NeurIPS 2024The Road Less…
SimPO: A New Way to Teach AI Models to Follow Human Preferences
Dec. 2, 2024

Yu Meng1*, Mengzhou Xia2*, Danqi Chen2

*Equal Contribution
1Computer Science Department,…

Enabling GPTs on Expert-Level Question-Answering in Environmental Science and Engineering – Early Insights
July 24, 2024

Jun-Jie Zhu, Meiqi Yang, Jinyue Jiang, Yiming Bai, Zhiyong Jason Ren

Department of Civil and Environmental Engineering and Andlinger Center for Energy and the Environment, Princeton University

 

Research Gap

Our discussion (Zhu et al.,…

The Mechanistic Basis of Abrupt Learning In An In-Context Classification Task
June 27, 2024

Gautam Reddy

Introduction 

A striking feature of large language models is their ability for in-context learning. In-context learning (ICL) is the ability to predict the response to a query based on illustrative examples presented…

Self-Supervised Reinforcement Learning with Contrastive Representations
June 20, 2024

Chongyi Zheng, Benjamin Eysenbach

Unsupervised learning is really powerful. It lies at the heart of large language models (just predict the next token), generative image models (predict what…

Spectral Transformers
June 10, 2024

Naman Agarwal, Daniel Suo, Xinyi Chen, Elad Hazan

One of the biggest challenges for the…

Mamba-2: Algorithms and Systems
June 3, 2024

The theoretical framework of structured state space duality (SSD) (see part 1 and 2 of this blogpost series) connects SSMs and (linear) attention through structured matrices. As mentioned in Part I, this connection allows us to derive new algorithms for selective SSMs that are faster than the parallel associative scan in Mamba-1 by…

Princeton Language and Intelligence at ICLR 2024
May 8, 2024

The International Conference on Learning Representations (ICLR) is the premier gathering of professionals dedicated to the advancement of the branch of artificial intelligence called representation learning, but generally referred to as deep learning. Taking place between May 7th…

Using LESS Data to Tune Models
May 7, 2024

Mengzhou Xia*
Sadhika Malladi*

This post is based on the following work:
LESS: Selecting Influential Data for Targeted Instruction Tuning
Mengzhou Xia*1, Sadhika Malladi*1, Suchin Gururangan2, Sanjeev Arora1, Danqi Chen1
*denotes equal…