Tao Yu (余涛)

Assistant Professor
Department of Computer Science
Musketeers Foundation Institute of Data Science
The University of Hong Kong

tao.yu.nlp [AT] gmail.com

Bio

Tao Yu is an Assistant Professor of Computer Science at The University of Hong Kong and a director of the XLANG Lab (as part of the HKU NLP Group). He spent one year in the UW NLP Group working with Noah Smith, Luke Zettlemoyer, and Mari Ostendorf. He completed his Ph.D. in Computer Science from Yale University, advised by Dragomir Radev and master's at Columbia University advised by Owen Rambow and Kathleen McKeown.

Tao has received the Google and Amazon faculty research awards (Google Research Scholar Award 2023, Amazon Research Award 2022). His main research interest is in Natural Language Processing. His research aims to build language model agents that transform (“grounding”) language instructions into code or actions executable in real-world environments, including databases, web applications, and the physical world etc,. It lies at the heart of the next generation of natural language interfaces that can interact with and learn from these real-world environments to facilitate human interaction with data analysis, web applications, and robotic instruction through conversation. It involves:

We are actively looking for strong and motivated students to join our group! If you are interested in working with us, please read recent papers, fill in the form with thoughts on extensions. Sorry, I'm afraid I generally can't respond to all individual emails.

Publications

Most recent publications on Google Scholar.
* indicates equal contribution.

OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments

Tianbao Xie, Danyang Zhang, Jixuan Chen, Xiaochuan Li, Siheng Zhao, Ruisheng Cao, Toh Jing Hua, Zhoujun Cheng, Dongchan Shin, Fangyu Lei, Yitao Liu, Yiheng Xu, Shuyan Zhou, Silvio Savarese, Caiming Xiong, Victor Zhong, Tao Yu

Preprint 2024

Paper Poster Code

OpenAgents: An Open Platform for Language Agents in the Wild

Tianbao Xie*, Fan Zhou*, Zhoujun Cheng*, Peng Shi*, Luoxuan Weng*, Yitao Liu*, Toh Jing Hua, Junning Zhao, Qian Liu, Che Liu, Leo Z. Liu, Yiheng Xu, Hongjin Su, Dongchan Shin, Caiming Xiong, Tao Yu

Preprint 2023

Paper Poster Code

Lemur: Harmonizing Natural Language and Code for Language Agents

Yiheng Xu*, Hongjin Su*, Chen Xing*, Boyu Mi, Qian Liu, Weijia Shi, Binyuan Hui, Fan Zhou, Yitao Liu, Tianbao Xie, Zhoujun Cheng, Siheng Zhao, Lingpeng Kong, Bailin Wang, Caiming Xiong, Tao Yu

ICLR 2024, Spotlight

Paper Poster Code

Text2Reward: Automated Dense Reward Function Generation for Reinforcement Learning

Tianbao Xie*, Siheng Zhao*, Chen Henry Wu, Yitao Liu, Qian Luo, Victor Zhong, Yanchao Yang, Tao Yu

ICLR 2024, Spotlight

Paper Poster Code

One Embedder, Any Task: Instruction-Finetuned Text Embeddings

Hongjin Su*, Weijia Shi*, Jungo Kasai, Yizhong Wang, Yushi Hu, Mari Ostendorf, Wen-tau Yih, Noah A Smith, Luke Zettlemoyer, Tao Yu

ACL Findings 2023

Paper Poster Code Data

Coder Reviewer Reranking for Code Generation

Tianyi Zhang, Tao Yu, Tatsunori B Hashimoto, Mike Lewis, Wen-tau Yih, Daniel Fried, Sida I Wang

ICML 2023

Paper Code

DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation

Yuhang Lai*, Chengxi Li*, Yiming Wang*, Tianyi Zhang*, Ruiqi Zhong*, Luke Zettlemoyer, Scott Wen-tau Yih, Daniel Fried, Sida Wang, Tao Yu

ICML 2023

Paper Poster Code Data

Binding Language Models in Symbolic Languages

Zhoujun Cheng*, Tianbao Xie*, Peng Shi, Chengzu Li, Rahul Nadkarni, Yushi Hu, Caiming Xiong, Dragomir Radev, Mari Ostendorf, Luke Zettlemoyer, Noah A Smith, Tao Yu

ICLR 2023, Spotlight

Paper Poster Code

Selective Annotation Makes Language Models Better Few-Shot Learners

Hongjin Su, Jungo Kasai, Chen Henry Wu, Weijia Shi, Tianlu Wang, Jiayi Xin, Rui Zhang, Mari Ostendorf, Luke Zettlemoyer, Noah A. Smith, Tao Yu

ICLR 2023

Paper Poster Code

UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models

Tianbao Xie*, Chen Henry Wu*, Peng Shi, Ruiqi Zhong, Torsten Scholak, Michihiro Yasunaga, Chien-Sheng Wu, Ming Zhong, Pengcheng Yin, Sida Wang, Victor Zhong, Bailin Wang, Chengzu Li, Connor Boyle, Ansong Ni, Ziyu Yao, Dragomir Radev, Caiming Xiong, Lingpeng Kong, Rui Zhang, Noah A. Smith, Luke Zettlemoyer, Tao Yu

EMNLP 2022

Paper Poster Code

In-Context Learning for Few-Shot Dialogue State Tracking

Yushi Hu, Chia-Hsuan Lee, Tianbao Xie, Tao Yu, Noah A. Smith, Mari Ostendorf

EMNLP Findings 2022

Paper Code

ZeroGen: Efficient Zero-shot Learning via Dataset Generation

Jiacheng Ye*, Jiahui Gao*, Qintong Li, Hang Xu, Jiangtao Feng, Zhiyong Wu, Tao Yu, Lingpeng Kong

EMNLP 2022

Paper Code

GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsing

Tao Yu, Chien-Sheng Wu, Xi Victoria Lin, Bailin Wang, Yi Chern Tan, Xinyi Yang, Dragomir Radev, Richard Socher, Caiming Xiong

ICLR 2021

Paper Code

Semantic Evaluation for Text-to-SQL with Distilled Test Suites

Ruiqi Zhong, Tao Yu, Dan Klein

EMNLP 2020

Paper Code

Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task

Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, Zilin Zhang and Dragomir Radev

EMNLP 2018

Paper Poster Code Data

OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments

Tianbao Xie, Danyang Zhang, Jixuan Chen, Xiaochuan Li, Siheng Zhao, Ruisheng Cao, Toh Jing Hua, Zhoujun Cheng, Dongchan Shin, Fangyu Lei, Yitao Liu, Yiheng Xu, Shuyan Zhou, Silvio Savarese, Caiming Xiong, Victor Zhong, Tao Yu

Preprint 2024

Paper Poster Code

OpenAgents: An Open Platform for Language Agents in the Wild

Tianbao Xie*, Fan Zhou*, Zhoujun Cheng*, Peng Shi*, Luoxuan Weng*, Yitao Liu*, Toh Jing Hua, Junning Zhao, Qian Liu, Che Liu, Leo Z. Liu, Yiheng Xu, Hongjin Su, Dongchan Shin, Caiming Xiong, Tao Yu

Preprint 2023

Paper Poster Code

Lemur: Harmonizing Natural Language and Code for Language Agents

Yiheng Xu*, Hongjin Su*, Chen Xing*, Boyu Mi, Qian Liu, Weijia Shi, Binyuan Hui, Fan Zhou, Yitao Liu, Tianbao Xie, Zhoujun Cheng, Siheng Zhao, Lingpeng Kong, Bailin Wang, Caiming Xiong, Tao Yu

ICLR 2024, Spotlight

Paper Poster Code

Text2Reward: Automated Dense Reward Function Generation for Reinforcement Learning

Tianbao Xie*, Siheng Zhao*, Chen Henry Wu, Yitao Liu, Qian Luo, Victor Zhong, Yanchao Yang, Tao Yu

ICLR 2024, Spotlight

Paper Poster Code

One Embedder, Any Task: Instruction-Finetuned Text Embeddings

Hongjin Su*, Weijia Shi*, Jungo Kasai, Yizhong Wang, Yushi Hu, Mari Ostendorf, Wen-tau Yih, Noah A Smith, Luke Zettlemoyer, Tao Yu

ACL Findings 2023

Paper Poster Code Data

Coder Reviewer Reranking for Code Generation

Tianyi Zhang, Tao Yu, Tatsunori B Hashimoto, Mike Lewis, Wen-tau Yih, Daniel Fried, Sida I Wang

ICML 2023

Paper Code

DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation

Yuhang Lai*, Chengxi Li*, Yiming Wang*, Tianyi Zhang*, Ruiqi Zhong*, Luke Zettlemoyer, Scott Wen-tau Yih, Daniel Fried, Sida Wang, Tao Yu

ICML 2023

Paper Poster Code Data

Compositional Exemplars for In-context Learning

Jiacheng Ye, Zhiyong Wu, Jiangtao Feng, Tao Yu, and Lingpeng Kong

ICML 2023

Paper Code

Binding Language Models in Symbolic Languages

Zhoujun Cheng*, Tianbao Xie*, Peng Shi, Chengzu Li, Rahul Nadkarni, Yushi Hu, Caiming Xiong, Dragomir Radev, Mari Ostendorf, Luke Zettlemoyer, Noah A Smith, Tao Yu

ICLR 2023, Spotlight

Paper Poster Code

Selective Annotation Makes Language Models Better Few-Shot Learners

Hongjin Su, Jungo Kasai, Chen Henry Wu, Weijia Shi, Tianlu Wang, Jiayi Xin, Rui Zhang, Mari Ostendorf, Luke Zettlemoyer, Noah A. Smith, Tao Yu

ICLR 2023

Paper Poster Code

UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models

Tianbao Xie*, Chen Henry Wu*, Peng Shi, Ruiqi Zhong, Torsten Scholak, Michihiro Yasunaga, Chien-Sheng Wu, Ming Zhong, Pengcheng Yin, Sida Wang, Victor Zhong, Bailin Wang, Chengzu Li, Connor Boyle, Ansong Ni, Ziyu Yao, Dragomir Radev, Caiming Xiong, Lingpeng Kong, Rui Zhang, Noah A. Smith, Luke Zettlemoyer, Tao Yu

EMNLP 2022

Paper Poster Code

In-Context Learning for Few-Shot Dialogue State Tracking

Yushi Hu, Chia-Hsuan Lee, Tianbao Xie, Tao Yu, Noah A. Smith, Mari Ostendorf

EMNLP Findings 2022

Paper Code

ZeroGen: Efficient Zero-shot Learning via Dataset Generation

Jiacheng Ye*, Jiahui Gao*, Qintong Li, Hang Xu, Jiangtao Feng, Zhiyong Wu, Tao Yu, Lingpeng Kong

EMNLP 2022

Paper Code

ProGen: Progressive Zero-shot Dataset Generation via In-context Feedback

Jiacheng Ye, Jiahui Gao, Zhiyong Wu, Jiangtao Feng, Tao Yu, and Lingpeng Kong

EMNLP Findings 2022

Paper Code

Augmenting Multi-Turn Text-to-SQL Datasets with Self-Play

Qi Liu, Zihuiwen Ye, Tao Yu, Phil Blunsom, Linfeng Song

EMNLP Findings 2022

Paper Code

NL2INTERFACE: Interactive Visualization Interface Generation from Natural Language Queries

Yiru Chen, Ryan Li, Austin Mac, Tianbao Xie, Tao Yu, Eugene Wu

IEEE Visualization Conference NLVIZ Workshop, 2022

Paper

Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models

with the BIG-bench team (442 authors)

TMLR 2023

Paper Code

FOLIO: Natural Language Reasoning with First-Order Logic

with Simeng Han, Rui Zhang, Alexander R Fabbri, Xi Victoria Lin, Caiming Xiong, Dragomir Radev and many authors

Preprint, 2022

Paper Code

DYLE: Dynamic Latent Extraction for Abstractive Long-Input Summarization

Ziming Mao*, Chen Henry Wu*, Ansong Ni, Yusen Zhang, Rui Zhang, Tao Yu, Budhaditya Deb, Chenguang Zhu, Ahmed H Awadallah, Dragomir Radev

ACL 2022

Paper Code

An Exploratory Study on Long Dialogue Summarization: What Works and What's Next

Yusen Zhang*, Ansong Ni*, Tao Yu, Rui Zhang, Chenguang Zhu, Budhaditya Deb, Asli Celikyilmaz, Ahmed Hassan Awadallah, Dragomir Radev

EMNLP Findings 2021. Short Paper

Paper Code

SummerTime: Text Summarization Toolkit for Non-experts

Ansong Ni, Zhangir Azerbayev, Mutethia Mutuma, Troy Feng, Yusen Zhang, Tao Yu, Ahmed Hassan Awadallah, Dragomir Radev

EMNLP 2021. Demo Track

Paper Code

Testing Cross-Database Semantic Parsers Using Canonical Utterances

Heather Lent, Semih Yavuz, Tao Yu, Tong Niu, Yingbo Zhou, Dragomir Radev, Xi Victoria Lin

EMNLP 2021 Workshop: Evaluation & Comparison of NLP Systems. Best Paper Award

Paper Code

Logic-Consistency Text Generation from Semantic Parses

Chang Shu, Yusen Zhang, Xiangyu Dong, Peng Shi, Tao Yu, Rui Zhang

ACL Findings 2021

Paper Code

QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization

Ming Zhong*, Da Yin*, Tao Yu, Ahmad Zaidi, Mutethia Mutuma, Rahul Jha, Ahmed Hassan Awadallah, Asli Celikyilmaz, Yang Liu, Xipeng Qiu and Dragomir Radev

NAACL 2021

Paper Code

DART: Open-Domain Structured Data Record to Text Generation

with Linyong Nan, Dragomir Radev, Rui Zhang, Neha Verma, Xi Victoria Lin, Caiming Xiong, Richard Socher and many authors.

NAACL 2021

Paper Data

SCoRe: Pre-Training for Context Representation in Conversational Semantic Parsing

Tao Yu, Rui Zhang, Alex Polozov, Christopher Meek, Ahmed Hassan Awadallah

ICLR 2021

Paper Code

GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsing

Tao Yu, Chien-Sheng Wu, Xi Victoria Lin, Bailin Wang, Yi Chern Tan, Xinyi Yang, Dragomir Radev, Richard Socher, Caiming Xiong

ICLR 2021

Paper Code

Semantic Evaluation for Text-to-SQL with Distilled Test Suites

Ruiqi Zhong, Tao Yu, Dan Klein

EMNLP 2020

Paper Code

Did You Ask a Good Question? A Cross-Domain Question Intention Classification Benchmark for Text-to-SQL

Yusen Zhang, Xiangyu Dong, Shuaichen Chang, Tao Yu, Peng Shi, Rui Zhang

EMNLP 2020 Workshop on Interactive and Executable Semantic Parsing. Short Paper

Paper Data

CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databases

Tao Yu, Rui Zhang He Yang Er, Suyi Li, Eric Xue, Bo Pang, Xi Victoria Lin, Yi Chern Tan, Tianze Shi, Zihan Li, Youxuan Jiang, Michihiro Yasunaga, Sungrok Shim, Tao Chen, Alexander Fabbri, Zifan Li, Luyao Chen, Yuwen Zhang, Shreya Dixit, Vincent Zhang, Caiming Xiong, Richard Socher, Walter Lasecki, Dragomir Radev

EMNLP 2019

Paper Data

Editing-Based SQL Query Generation for Cross-Domain Context-Dependent Questions

Rui Zhang, Tao Yu, He Yang Er, Sungrok Shim, Eric Xue, Xi Victoria Lin, Tianze Shi, Caiming Xiong, Richard Socher, Dragomir Radev

EMNLP 2019

Paper Code

SParC: Cross-Domain Semantic Parsing in Context

Tao Yu, Rui Zhang, Michihiro Yasunaga, Yi Chern Tan, Xi Victoria Lin, Suyi Li, Heyang Er, Irene Li, Bo Pang, Tao Chen, Emily Ji, Shreya Dixit, David Proctor, Sungrok Shim, Jonathan Kraft, Vincent Zhang, Caiming Xiong, Richard Socher and Dragomir Radev

ACL 2019

Paper Code Data

Twitter Sentiment in New York City Parks as Measure of Well-being

Richard A Plunz, Yijia Zhou, Maria Isabel Carrasco Vintimilla, Kathleen Mckeown, Tao Yu, Laura Uguccioni, Maria Paola Sutto

Landscape and Urban Planning 2019

Paper Code Data

Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task

Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, Zilin Zhang and Dragomir Radev

EMNLP 2018

Paper Poster Code Data

SyntaxSQLNet: Syntax Tree Networks for Complex and Cross-Domain Text-to-SQL Task

Tao Yu, Michihiro Yasunaga, Kai Yang, Rui Zhang, Dongxu Wang, Zifan Li and Dragomir Radev

EMNLP 2018

Paper Code

TypeSQL: Knowledge-based Type-Aware Neural Text-to-SQL Generation

Tao Yu, Zifan Li, Zilin Zhang, Rui Zhang, Dragomir Radev

NAACL 2018. Short Paper

Paper Code

Cross-lingual Sentiment Transfer with Limited Resources

Mohammad Sadegh Rasooli, Noura Farra, Axinia Radeva, Tao Yu, and Kathleen McKeown

Machine Translation 2017

Paper Code

The Columbia-GWU System at the 2016 TAC KBP BeSt Evaluation

Owen Rambow, Tao Yu, Axinia Radeva, Sardar Hamidian, Alexander R. Fabbri, Debanjan Ghosh, Christopher Hidey, Tianrui Peng, Mona Diab, Kathleen McKeown, Smaranda Muresan

NIST TAC KBP Workshop, 2016

Paper Slides

Talks and Presentations

Invited Speaker, Workshop on Open-World Agents,
NeurIPS 2024

Tutorial on Language Agents: Foundations, Prospects, and Risks,
EMNLP 2024

Invited Panelist, Workshop on LLM Agents,
ICLR 2024

Advancing Natural Language Interfaces with Language Models as Agents,
Columbia NLP seminar, April 2023
Cornell DB seminar, May 2023
Microsoft Research Asia, May 2023
Apple KP Tech Talks, June 2023
Keynote at VLDB 2023 1st International Workshop on Databases and Large Language Models, Sept. 2023
AI NEW HORIZONS 2023: A Symposium with Scientific Leaders, Nov. 2023
Morgan Stanley ML Speaker Seminar, Dec. 2023
MILA ML4Code Seminar, Dec. 2023
2nd Table Representation Learning Workshop at NeurIPS 2023, Dec. 2023
Instacart Distinguished Speaker Series, Jan. 2024

Neuro-Symbolic Approaches: Large Language Models + Tool Use,
ACL 2023 Tutorial on Complex Reasoning over Natural Language, July 2023

Building Natural Language Interfaces with Large Language Models,
Amazon AWS, Nov. 2022

Few-shot In-context Learning with Large Language Models,
AllState Tech Talks, Jun. 2022

UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models,
ServiceNow Research (Prev. ElementAI), Feb. 2022

SCoRe: Pre-Training for Context Representation in Conversational Semantic Parsing,
Google Research, Apr. 2021

Students

Tianbao Xie, Ph.D. student, 2022

Hongjin Su, Ph.D. student, 2022

Yiheng Xu, Ph.D. student, 2022, co-advised with Lingpeng Kong

Jiacheng Ye, Ph.D. student, 2022, co-advised with Lingpeng Kong

Zhoujun Cheng, Intern, 2022, SJTU BS/MS

Fan Zhou, Intern, 2023, SJTU BS/MS

Leo Liu, Intern, 2023, UW BS/MS → UT Austin PhD

Chen Henry Wu, Intern, 2022, Tsinghua BS → CMU PhD

Ryan Li, Intern, 2022, UW BS → Stanford MS

Chengzu Li, Intern, 2022, Xi'an Jiaotong BS → Cambridge PhD

Shuyang Jiang, Intern, 2023, SJTU BS → Fudan PhD

Yiming Wang, Intern, 2022, PKU BS → Harvard MS

Yuhang Lai, Intern, 2022, BIT BS → Fudan MS

Chengxi Li, Intern, 2022, HIT BS

Ming Zhong, Summer Intern, 2020, Fudan MS → UIUC PhD

Da Yin, Summer Intern, 2020, PKU BS → UCLA PhD

Naihao Deng, Summer Intern, 2020, UMich BS → UMich PhD

Yusen Zhang, Summer Intern, 2020, Emory MS → PSU PhD

Michihiro Yasunaga, Project Student, 2018-19, Yale BS → Stanford PhD

Service

Organizing Committee
ACL 2023
SUKI: Structured and Unstructured Knowledge Integration Workshop@NAACL 2022
IntEx-SemPar: Interactive and Executable Semantic Parsing Workshop@EMNLP 2020

Program Committee/Reviewer
Nature
TACL
ACL Rolling Review
ACL: 2020, 2021, 2022
EMNLP: 2019, 2020, 2021, 2022
ICLR: 2022,
ICML: 2023
NeurIPS: 2022
NAACL: 2019, 2021
COLING: 2020, 2022
AACL-IJCNLP: 2020

Resume

Full Resume in PDF.

Misc.

I did a cycling tour (~2 weeks) at the top of the world, Tibet (avg elevation: ~4500 meters). I am also a student pilot. I enjoy hiking, travelling, and cooking. I ski and skate, and I am learning tennis.

I am from Ningdu (a less developed but beautiful county), Jiangxi Province in China. I’ve lived in (stay for over 3 months) about 20 cities including Zhongshan, Beijing, Shanghai, Salt Lake City, New York City, San Francisco, New Haven, Columbus, Honolulu, San Diego, Seattle, and Hong Kong etc. I've also visited over 60 cities around the world.

Acknowledgement

This website uses the website design and template by Martin Saveski
Tao Yu (余涛)
tao.yu.nlp [AT] gmail.com