DATA8005: Advanced Natural Language Processing

Course Information

Instructor

Tao Yu
Office Hour: Wednesday 4 - 5pm @IDS P307H

Lecture

Friday 1:30 - 4:20pm @Learning Commons CPD 2.25

Course Description

Natural language processing (NLP) is the study of human language from a computational perspective. This course is an introductory graduate-level course on natural language processing aimed at students who are interested in doing cutting-edge research in the field. In this class, we will cover recent developments on core techniques and modern advances in NLP, especially in the era of large language models. We will also survey some recent NLP research topics including language grounding, agents, multimodality, interactivity, and interoperability for NLP. Students will gain the necessary skills and experience to understand, design, implement, and test large language models through a final project. We will also introduce cutting-edge research topics and learn how to conduct NLP research through paper readings and discussions. We will potentially also host invited speakers for talks.

Prerequisites

We require students to have prior knowledge undergraduate linear algebra, probability and statistics, machine learning, or deep learning. Familiarity with Python programming is required. Introduction to natural language processing is recommended.

Course Materials

There is no required textbook for this course (Natural Language Processing by Jacob Eisenstein is recommended if you would like to read more about NLP). Readings from papers, blogs, tutorials, and book chapters will be posted on the course website.

Grading

30%: Class presentation
- You (a group of two) will present on the ~2-3 papers on an NLP topic assigned for a particular day (~60 minutes). We will do scheduling and signups at the beginning of the semester. The goal is to educate the others in the class about the topic, so do think about how to best cover the material (survey and review more papers/blogs on the topic), do a good job with slides, and be prepared for lots of questions.
- Papers will be chosen for each topic (ToD). You are still welcome to suggest relevant papers that you like to present (and coordinate with the instructor). It is your job to decide what to cover in the lecture and how to divide the work with your partner.
- You are also required to meet with the instructor before the lecture. Please send your draft slides via email before the meeting and we will go over your slides during the meeting.
25%: Course participation
- Each class will cover 2 topics. You are expected to read these papers before class, add in comments (answer, ask, or add >2 high-quality questions/comments, suggest related papers). Please come to the class prepared with several points that will substantially contribute to the group discussion. Your participation grade will be determined based on attendance and more importantly, substantial contributions to paper discussions both on Slack and in class.
45%: Final project
- The final project is an opportunity for open-ended exploration of concepts in the course. This project should constitute novel work beyond directly implementing concepts from lecture and should result in a report that roughly reads like an NLP/ML conference submission in terms of presentation and scope. You can work on the final project in a group of 2-3. You are allowed to integrate the project with ongoing research or projects from other classes (assuming the other instructor also approves).
- Deliverables
  - 10% Proposal: Partway through the semester, you will submit a brief proposal (around 1 page) explaining your idea and plan including what problem or task you want to address, what dataset(s) you want to work on, what metrics you need to employ, what baselines you would like to compare with. Project group registration: week 5; proposal due: week 7.
  - 35% Writeup: Your final project report should be 4-8 pages—use your discretion about the length. Groups of two should have reports closer to 8 pages. The scope should be similar to that of an ACL paper (ICLR style files). You should present a novel idea, discuss related work, describe your implementation or what you did, give results, and provide discussion or error analysis. Due: 11:59 pm on Dec 15
  - Presentation: You will give lightning talks (around 3 minutes) about your project (ToD).
- More guidelines will be announced soon.

Course Schedule

Date	Topic	Material	Event	Due
Week 1 Sep 6	Canceled due to bad weather
Week 2 Sep 13	Introduction (Tao Yu) [slides]	Readings Human Language Understanding & Reasoning M. Collins, Notes 1
Week 3 Sep 20	Introduction to LLMs (Tao Yu) [slides]	Readings J & M 7.3-7.5 Byte-pair encoding paper Attention Is All You Need Language Models are Few-Shot Learners (GPT-3) Others Talk on the GPT Tokenizer by Andrej Karpathy The Annotated Transformer Scaling Laws for Neural Language Models Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Talk on building GPT: from scratch, in code, spelled out by Andrej Karpathy	Registration Out
Week 4 Sep 27	Introduction to LLMs (Tao Yu) [slides] The Llama 3 Herd of Models [slides]	Readings The Llama 3 Herd of Models Gemma 2 : Improving Open Language Models at a Practical Size
Week 5 Oct 4	LM post-training 2: SFT, instruction tuning (Sihui Ji, Tianzhe Chu) [slides] LM data and evaluation (Jianrui Wu, Tianle Li) [slides]	Readings The Llama 3 Herd of Models (post-training part only) Self-Instruct : Aligning Language Model with Self-Generated Instructions Multitask Prompted Training Enables Zero-Shot Task Generalization The FineWeb Datasets : Decanting the Web for the Finest Text Data at Scale DoReMi : Optimizing Data Mixtures Speeds Up Language Model Pretraining Chatbot Arena : An Open Platform for Evaluating LLMs by Human Preference		Project registration Due
Week 6 Oct 11	No class
Week 7 Oct 18	No class
Week 8 Oct 25	LM safety, bias, and privacy (Yifeng Lin, Pinglu Gong, Fengyi Xu) [slides] LM post-training 2: alignment, RLHF/DPO (Runhui Huang, Yiyang Wang) [slides]	Readings Jailbreaking Black Box Large Language Models in Twenty Queries Extracting Training Data from Large Language Models Universal and Transferable Adversarial Attacks on Aligned Language Models Training language models to follow instructions with human feedback Proximal Policy Optimization Algorithms Direct Preference Optimization 2: Your Language Model is Secretly a Reward Model		Project proposal Due
Week 9 Nov 1	Efficient LM adaptation (Sidi Yang, Yatai Ji, Jing Xiong) [slides] Efficient LM training (Qi Guicheng, Shen Che, Zijian Ye) [slides]	Readings LoRA : Low-Rank Adaptation of Large Language Models QLoRA : Efficient Finetuning of Quantized LLMs LoRA Learns Less and Forgets Less FlashAttention : Fast and Memory-Efficient Exact Attention with IO-Awareness Mamba : Linear-Time Sequence Modeling with Selective State Spaces Overview on Mixture of Experts
Week 10 Nov 8	Multimodal LMs 1 (Chenming Zhu, Pei Zhou, Yi Zhang) [slides] Multimodal LMs 2 (Mengzhao Chen, Tianshuo Yang, Chengqi Duan) [slides]	Readings Learning Transferable Visual Models From Natural Language Supervision Flamingo : a Visual Language Model for Few-Shot Learning Visual Instruction Tuning Qwen2-VL : Enhancing Vision-Language Model Perception of the World at Any Resolution Chameleon : Mixed-Modal Early-Fusion Foundation Models GLIDE : Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models
Week 11 Nov 15	LLM/VLMs + Robotics 1 (Feng Chen, Ruizhe Liu) [slides] LLM/VLMs + Robotics 2 (Yi Chen, Lu Qiu) [slides]	Readings GenSim : Generating Robotic Simulation Tasks via Large Language Models RoboGen : Towards Unleashing Infinite Data for Automated Robot Learning via Generative Simulation RL-VLM-F : Reinforcement Learning from Vision Language Foundation Model Feedback OpenVLA : An Open-Source Vision-Language-Action Model LAPA : Latent Action Pretraining from Videos Do As I Can, Not As I Say : Grounding Language in Robotic Affordances Genie : Generative Interactive Environments RT-2 : Vision-Language-Action Models Transfer Web Knowledge to Robotic Control Vision Language Models are In-Context Value Learners Others Toward General-Purpose Robots via Foundation Models : A Survey and Meta-Analysis Project GR00T : A Blueprint for Generalist Robotics - Jim Fan, NVIDIA
Week 12 Nov 22	LLM/VLMs as Agents (Xinyuan Wang, Bowen Wang) [slides] Agents in the digital and physical world (Tao Yu)	Readings ReAct : Synergizing Reasoning and Acting in Language Models OSWorld : Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments Attacking Vision-Language Computer Agents via Pop-ups Video PreTraining (VPT) : Learning to Act by Watching Unlabeled Online Videos EMNLP 2024 Tutorial on Language Agents : Foundations, Prospects, and Risks AI Agents That Matter
Week 13 Nov 29	Embodied AI (Guest lecture : Yanchao Yang)
Week 15 Dec 6	No class
Week 16 Dec 13	No class