The International Conference on Learning Representations (ICLR) is the premier gathering of professionals dedicated to the advancement of the branch of artificial intelligence called representation learning, but generally referred to as deep learning. Taking place between May 7th to 11th in Vienna, ICLR is globally renowned for presenting and publishing cutting-edge research on all aspects of deep learning used in the fields of artificial intelligence, statistics and data science, as well as important application areas such as machine vision, computational biology, speech recognition, text understanding, gaming, and robotics.

Princeton Language and Intelligence is proud to feature the work of our students, post-docs, and faculty that will be showcased at the conference. 

Presented at ICLR 2024

BooookScore: A systematic exploration of book-length summarization in the era of LLMs 

_{Main Conference (Oral)}

Authors: Yapei Chang, Kyle Lo, Tanya Goyal, Mohit Iyyer

Links: Paper | Code

Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!

_{Main Conference (Oral)}

Authors: Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen, Ruoxi Jia, Prateek Mittal, Peter Henderson

Links: Paper | Website | Code 

SWE-bench: Can Language Models Resolve Real-world Github Issues?

_{Main Conference (Oral)}

Authors: Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, Karthik Narasimhan

Links: Paper | Code 

Catastrophic Jailbreak of Open-source LLMs via Exploiting Generation

_{Main Conference (Spotlight)}

Authors: Yangsibo Huang, Samyak Gupta, Mengzhou Xia, Kai Li, Danqi Chen

Links: Paper | Website

On the Provable Advantage of Unsupervised Pretraining

_{Main Conference (Spotlight)}

Authors: Jiawei Ge, Shange Tang, Jianqing Fan, Chi Jin

Links: Paper

Provable Offline Preference-Based Reinforcement Learning

_{Main Conference (Spotlight)}

Authors: Wenhao Zhan, Masatoshi Uehara, Nathan Kallus, Jason Lee, Wen Sun

Links: Paper

Provable Reward-Agnostic Preference-Based Reinforcement Learning

_{Main Conference (Spotlight)}

Authors: Wenhao Zhan, Masatoshi Uehara, Wen Sun, Jason Lee

Links: Paper

Stabilizing Contrastive RL: Techniques for Robotic Goal Reaching from Offline Data

_{Main Conference (Spotlight)}

Authors: Chongyi Zheng, Benjamin Eysenbach, Homer Walke, Patrick Yin, Kuan Fang, Ruslan Salakhutdinov, Sergey Levine

Links: Paper

A Quadratic Synchronization Rule for Distributed Deep Learning

_{Main Conference}

Authors: Xinran Gu, Kaifeng Lyu, Sanjeev Arora, Jingzhao Zhang, Longbo Huang

Links: Paper

Abstractors and relational cross-attention: An inductive bias for explicit relational reasoning in Transformers

_{Main Conference}

Authors: Awni Altabaa, Taylor Webb, Jonathan Cohen, John Lafferty

Links: Paper

Adaptive Regret for Bandits Made Possible: Two Queries Suffice

_{Main Conference}

Authors: Zhou Lu, Qiuyi Zhang, Xinyi Chen, Fred Zhang, David Woodruff, Elad Hazan

Links: Paper

BaDExpert: Extracting Backdoor Functionality for Accurate Backdoor Input Detection

_{Main Conference}

Authors: Tinghao Xie, Xiangyu Qi, Ping He, Yiming Li, Jiachen (Tianhao) Wang, Prateek Mittal

Links: Paper

Bridging State and History Representations: Understanding Self-Predictive RL

_{Main Conference}

Authors: Tianwei Ni, Benjamin Eysenbach, Erfan Seyedsalehi, Michel Ma, Clement Gehring, Aditya Mahajan, Pierre-Luc Bacon

Links: Paper

Closing the Gap between TD Learning and Supervised Learning - A Generalisation Point of View

_{Main Conference}

Authors: Raj Ghugare, Matthieu Geist, Glen Berseth, Benjamin Eysenbach

Links: Paper

COLLIE: Systematic Construction of Constrained Text Generation Tasks

_{Main Conference}

Authors: Shunyu Yao, Howard Chen, Austin Hanjie, Runzhe Yang, Karthik Narasimhan

Links: Paper

Consistency Models as a Rich and Efficient Policy Class for Reinforcement Learning

_{Main Conference}

Authors: Zihan Ding, Chi Jin

Links: Paper

Contrastive Difference Predictive Coding

_{Main Conference}

Authors: Chongyi Zheng, Ruslan Salakhutdinov, Benjamin Eysenbach

Links: Paper

Depthwise Hyperparameter Transfer in Residual Networks: Dynamics and Scaling Limit

_{Main Conference}

Authors: Blake Bordelon, Lorenzo Noci, Mufan Li, Boris Hanin, Cengiz Pehlevan

Links: Paper

Detecting Pretraining Data from Large Language Models

_{Main Conference}

Authors: Weijia Shi, Anirudh Ajith, Mengzhou Xia, Yangsibo Huang, Daogao Liu, Terra Blevins, Danqi Chen, Luke Zettlemoyer

Links: Paper

Dichotomy of Early and Late Phase Implicit Biases Can Provably Induce Grokking

_{Main Conference}

Dichotomy of Early and Late Phase Implicit Biases Can Provably Induce Grokking

Authors: Kaifeng Lyu, Jikai Jin, Zhiyuan Li, Simon Du, Jason Lee, Wei Hu

Links: Paper

Evaluating Large Language Models at Evaluating Instruction Following

_{Main Conference}

Authors: Zhiyuan Zeng, Jiatong Yu, Tianyu Gao, Yu Meng, Tanya Goyal, Danqi Chen

Links: Paper

Game-Theoretic Robust Reinforcement Learning Handles Temporally-Coupled Perturbations

_{Main Conference}

Authors: Yongyuan Liang, Yanchao Sun, Ruijie Zheng, Xiangyu Liu, Benjamin Eysenbach, Tuomas Sandholm, Furong Huang, Stephen McAleer

Links: Paper

Horizon-Free Regret for Linear Markov Decision Processes

_{Main Conference}

Authors: Zhang Zihan, Jason Lee, Yuxin Chen, Simon Du

Links: Paper

ImageNet-OOD: Deciphering Modern Out-of-Distribution Detection Algorithms

_{Main Conference}

Authors: William Yang, Byron Zhang, Olga Russakovsky

Links: Paper

Implicit Maximum a Posteriori Filtering via Adaptive Optimization

_{Main Conference}

Authors: Gianluca M. Bencomo, Jake C. Snell, Thomas L. Griffiths

Links: Paper

Learning Hierarchical Polynomials with Three-Layer Neural Networks

_{Main Conference}

Authors: Zihao Wang, Eshaan Nichani, Jason Lee

Links: Paper

Learning with Language-Guided State Abstractions

_{Main Conference}

Authors: Andi Peng, Ilia Sucholutsky, Belinda Z. Li, Theodore R. Sumers, Thomas L. Griffiths, Jacob Andreas, Julie A. Shah

Links: Paper

Llemma: An Open Language Model for Mathematics

_{Main Conference}

Authors: Zhangir Azerbayev, Hailey Schoelkopf, Keiran Paster, Marco Dos Santos, Stephen McAleer, Qiaochu Jiang, Jia Deng, Stella R Biderman, Sean Welleck

Links: Paper

Maximum Likelihood Estimation is All You Need for Well-Specified Covariate Shift

_{Main Conference}

Authors: Jiawei Ge, Shange Tang, Jianqing Fan, Cong Ma, Chi Jin

Links: Paper

PARL: A Unified Framework for Policy Alignment in Reinforcement Learning

_{Main Conference}

Authors: Souradip Chakraborty, Amrit Bedi, Alec Koppel, Huazheng Wang, Dinesh Manocha, Mengdi Wang, Furong Huang

Links: Paper

Principled Architecture-aware Scaling of Hyperparameters

_{Main Conference}

Authors: Wuyang Chen, Junru Wu, Zhangyang Wang, Boris Hanin

Links: Paper

Privacy-Preserving In-Context Learning for Large Language Models

_{Main Conference}

Authors: Tong Wu, Ashwinee Panda, Jiachen (Tianhao) Wang, Prateek Mittal

Links: Paper

Provably Efficient CVaR RL in Low-rank MDPs

_{Main Conference}

Authors: Yulai Zhao, Wenhao Zhan, Xiaoyan Hu, Ho-fung Leung, Farzan Farnia, Wen Sun, Jason Lee

Links: Paper

Sample-Efficient Learning of POMDPs with Multiple Observations In Hindsight

_{Main Conference}

Authors: Jiacheng Guo, Minshuo Chen, Huan Wang, Caiming Xiong, Mengdi Wang, Yu Bai

Links: Paper

Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning

_{Main Conference}

Authors: Mengzhou Xia, Tianyu Gao, Zhiyuan Zeng, Danqi Chen

Links: Paper

Skill-Mix: a Flexible and Expandable Family of Evaluations for AI Models

_{Main Conference}

Authors: Dingli Yu, Simran Kaur, Arushi Gupta, Jonah Brown-Cohen, Anirudh Goyal, Sanjeev Arora

Links: Paper

Teach LLMs to Phish: Stealing Private Information from Language Models

_{Main Conference}

Authors: Ashwinee Panda, Christopher A. Choquette-Choo, Zhengming Zhang, Yaoqing Yang, Prateek Mittal

Links: Paper

Teaching Arithmetic to Small Transformers

_{Main Conference}

Authors: Nayoung Lee, Kartik Sreenivasan, Jason Lee, Kangwook Lee, Dimitris Papailiopoulos

Links: Paper

5th Workshop on African Natural Language Processing

_{AfricaNLP 2024 Workshop}

Authors: Happy Buzaaba, Bonaventure F.P. Dossou, David Adelani, Hady Elsahar, Constantine Lignos, Atnafu Lambebo Tonja, Salomey Osei, Anuloluwapo Aremu, Clemencia Siro, Shamsuddeen Muhammad, Tajuddeen Gwadabe, Perez Ogayo, Israel Abebe Azime, Kayode K. Olaleye

Links: Website

Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications

_{SeT LLM Workshop (Best Paper Award)}

Authors: Boyi Wei, Kaixuan Huang, Yangsibo Huang, Tinghao Xie, Xiangyu Qi, Mengzhou Xia, Prateek Mittal, Mengdi Wang, Peter Henderson

Links: Paper  | Website | Code 

Vision-Language Dataset Distillation

_{Data Problems for Foundation Models (DPFM) Workshop}

Authors: Xindi Wu Byron Zhang Zhiwei Deng Olga Russakovsky

Links: Paper | Website | Code

What's in Your "Safe" Data?: Identifying Benign Data that Breaks Safety

_{Data Problems for Foundation Models (DPFM) Workshop (Best Paper Award)}

Authors: Luxi He, Mengzhou Xia, Peter Henderson

Links: Paper