DATA8005: Advanced Natural Language Processing

Course Information

Instructor

• Tao Yu
• Office Hour: By appointment

Lecture

• Wednesday 2:00 - 4:45pm @Graduate House P603

Course Description

Natural language processing (NLP) is the study of human language from a computational perspective. This course is an introductory graduate-level course on natural language processing aimed at students who are interested in doing cutting-edge research in the field. In this class, we will cover recent developments on core techniques and modern advances in NLP, especially in the era of large language models. We will also survey some recent NLP research topics including language grounding, agents, multimodality, interactivity, and interoperability for NLP. Students will gain the necessary skills and experience to understand, design, implement, and test large language models through a final project. We will also introduce cutting-edge research topics and learn how to conduct NLP research through paper readings and discussions. We will potentially also host invited speakers for talks.

Prerequisites

We require students to have prior knowledge undergraduate linear algebra, probability and statistics, machine learning, or deep learning. Familiarity with Python programming is required. Introduction to natural language processing is recommended.

Grading

• 40%: Class presentation
  • You (a group of two) will present on the ~2-3 papers on an NLP topic assigned for a particular day (~60 minutes). We will do scheduling and signups at the beginning of the semester. The goal is to educate the others in the class about the topic, so do think about how to best cover the material (survey and review more papers/blogs on the topic), do a good job with slides, and be prepared for lots of questions.
  • Papers will be chosen for each topic (ToD). You are still welcome to suggest relevant papers that you like to present (and coordinate with the instructor). It is your job to decide what to cover in the lecture and how to divide the work with your partner.
  • You are also required to meet with the instructor before the lecture. Please send your draft slides via email before the meeting and we will go over your slides during the meeting.
• 25%: Course participation
  • Each class will cover 2 topics. You are expected to read these papers before class, add in comments (answer, ask, or add >2 high-quality questions/comments, suggest related papers). Please come to the class prepared with several points that will substantially contribute to the group discussion. Your participation grade will be determined based on attendance and more importantly, substantial contributions to paper discussions both on Slack and in class.
• 35%: Final project
  • The final project is an opportunity for open-ended exploration of concepts in the course. This project should constitute novel work beyond directly implementing concepts from lecture and should result in a report that roughly reads like an NLP/ML conference submission in terms of presentation and scope. You can work on the final project in a group of 2-3. You are allowed to integrate the project with ongoing research or projects from other classes (assuming the other instructor also approves).
  • Deliverables
    • 10% Proposal: Partway through the semester, you will submit a brief proposal (around 1 page) explaining your idea and plan including what problem or task you want to address, what dataset(s) you want to work on, what metrics you need to employ, what baselines you would like to compare with. Project group registration: week 5; proposal due: week 7.
    • 25% Writeup: Your final project report should be 4-8 pages—use your discretion about the length. Groups of two should have reports closer to 8 pages. The scope should be similar to that of an ACL paper (ICLR style files). You should present a novel idea, discuss related work, describe your implementation or what you did, give results, and provide discussion or error analysis. Due: 11:59 pm on Dec 15
  • More guidelines will be announced soon.

Course Schedule