Learning to Interact With the World: Towards Building Intelligent Robotic Manipulation Systems

Lin Shao
Stanford University
Professor Dejan Milutinovic

Join us on Zoom:  https://ucsc.zoom.us/s/97339709714

Description:  Building general-purpose robots that intelligently sense and act in the physical world is
extremely challenging. For example, a robot operating in a household will need to perform a
broad range of tasks, such as preparing food in the kitchen and cleaning the floor in the living
room. It must be able to handle diverse objects and various unstructured environments.
Moreover, it needs to quickly master new tasks and follow human instructions. To help robots
achieve these abilities, we need to understand the robotic system comprehensively and build it

This talk will present my work towards building intelligent robotic manipulation systems from
three aspects. I will first discuss how to train robots to master primitive skills, such as grasping,
hanging, and placing, to interact with various objects. Next, I will introduce a learning framework
for robots to interact with complex surrounding environments. We propose a model-based
manipulation learning approach to efficiently produce policies for diverse manipulation tasks,
leveraging a learning-augmented differentiable simulation. Finally, I will discuss how to train
robots to understand different concepts through natural language and visual demonstrations to
better interact with humans. When robots perceive the initial scenes and receive instructions
such as “open the door,” they can generate motion trajectories to accomplish these tasks
accordingly. I will conclude this talk by discussing my future research plan.

Speakers Bio:  Lin Shao received his Ph.D. at Stanford University, advised by Professor Jeannette Bohg. His
research interests lie at the intersection of robotics and artificial intelligence. His long-term
research goal is to build general-purpose robotic systems that intelligently perform a diverse
range of tasks in a large variety of environments in the physical world. Specifically, his research
focuses on developing algorithms and systems to provide robots with the abilities of perception
and manipulation. He received his M.S. in computational and applied mathematics from
Stanford University and his B.S. in geochemistry from Nanjing University.