Toward better Immersive Environments: Bridging Experience and Expression in NLP and Beyond

Roxana Girju
Professor Linguistics and Computer Science Affiliate
University of Illinois at Urbana-Champaign
Professor Jeffery Flanigan

Join us on Zoom:

Description: Human communication is multimodal and multisensory, integrating multiple sensory and effector modalities such as speech, gestures, body posture and movement, and senses like touch, smell, and taste. Even our knowledge is embodied, verbal and nonverbal, leading to various forms of understanding. In my talk, I will discuss two lines of work to bridge the gap between human lived experience and linguistic expression. I will start with the connection between cognition and perception, as well as between emotions and the senses, toward a semantic organization of the lexical spaces in the experiential domain. I will briefly summarize our investigations of semantic structures in the lexicon as manifested by co-expressions of form–meaning mapping in a typologically diverse set of 499 world languages. Such research could inspire significant advances in Natural Language Processing (NLP), especially for tasks involving languages which lack dedicated lexical resources and can benefit from language transfer from large, shared cross-lingual semantic spaces. 

In the second part of my talk, I will introduce various projects on emotion and empathy at the intersection of NLP and HCI (Human-Computer Interaction). As digital social platforms and mobile technologies are becoming more prevalent and robust, the use of Artificial Intelligence (AI) in facilitating human communication will grow. At the same time, the current wave of digital disruption is forcing us to think of new ways to design interactions in spaces that accommodate our multifaceted lives, meeting us where we live, work, create, and play. This will pave the way for the development of intuitive, adaptive, and effective empathic AI interfaces that better address the needs of socially and culturally diverse communities. Besides clear applications in healthcare and the workplace, such an approach will allow various communities of practice to investigate the areas of intersection between AI, on one side, and human communication, perceptual needs and social and cultural values, on the other.

Speaker Bio: Roxana Girju is professor of Linguistics, Computer Science affiliate, and part-time faculty member at the Beckman Institute of Advanced Science and Technology, University of Illinois at Urbana-Champaign (UIUC). Dr. Girju earned a B.Sc. in Computer Science from "Politehnica" University of Bucharest (1995) and a Ph.D. degree in Computer Science from the University of Texas at Dallas (2002). She is the co-founder and former director of the successful CS+Linguistics joint undergraduate program at UIUC. Located at the intersection of language – technology – society, Dr. Girju’s research interests include designing, building, and testing Natural Language Processing systems that facilitate empathic human–human and human–computer communication. She has participated in many shared tasks in NLP and has been an advocate for the integration of STEMM with Humanities, Social Sciences, and the Arts. Dr. Girju also hosts the ‘Creative Language Technologies’, a podcast on embodied experience as a useful stance for designing and building the next generation of human language technologies.