Model, Data and Task Engineering for NLP

Shafiq Joty
Associate Professor in the School of Computer Science and Engineering
Nanyang Technological University (NTU)
Professor Jeffrey Flanigan

Join us on Zoom:

Description: With the advent of deep learning and neural methods, NLP research over the last decade has shifted from feature engineering to model engineering, primarily focusing on inventing new architectures  for NLP problems. Two other related factors that are getting more attention only recently are: how to better use the available data, and which objectives or tasks to optimize; referred to as data engineering and task engineering, respectively. In this talk, I will present our recent work along these three dimensions: model, data and task engineering for NLP. In particular, I will first present novel neural architectures for parsing texts into hierarchical structures at the sentence and discourse level, and efficient parallel encoding of such structures for better language understanding and generation. I will then present effective data augmentation methods for supervised and unsupervised machine translation and other cross-lingual tasks. Finally, I will present a new objective for text generation tasks that aims to mitigate the degeneration issues prevalent in neural generation models and a unified multitask framework for lifelong few-shot language learning based on prompt tuning. With empirical results, I will argue that while model engineering is crucial to the advancement of the field, the other two factors are more important to build robust NLP systems.

Speaker Bio: Shafiq Joty is a tenured Associate Professor in the School of Computer Science and Engineering (SCSE) at NTU, where he founded the NTU-NLP group and currently leads the group. He is also a senior research manager and a founding member of Salesforce Research Asia, where he leads the NLP group. He received his PhD in Computer Science from the University of British Columbia in February 2014. 

Shafiq’s research has primarily focused on developing language analysis tools (e.g., syntactic parsers, language models, NER, discourse parser, coherence models) and downstream NLP applications including machine translation, question answering, text summarization, controllable generation and vision-language tasks. A significant part of his current research focuses on multilingual processing and robustness of NLP models.

Shafiq served as a senior area chair for ACL’22 and EMNLP’21 in Machine Learning (ML) and NLP Applications tracks respectively, and area chair for ACL'19-21, EMNLP'19, NAACL’21 and EACL’21 in ML, QA and Discourse tracks. He is an action editor for ACL-RR and previously served as an associate editor for ACM Transactions on Asian and Low Resource Language Processing. He gave tutorials at ACL’19, ICDM’18 and COLING’18 on discourse processing and conversation modeling. His research contributed to 15 patents and more than 95 papers in top-tier NLP and ML conferences and journals including ACL, EMNLP, NAACL, NeurIPS, ICML, ICLR, CVPR, ECCV, ICCV, CL and JAIR. His major awards include NTU College of Engineering Young Faculty Award (Research), NSERC CGS-D Doctoral award and Microsoft Research Excellent Intern award. More about him can be found here.