The Toyota Research Institute (TRI) announced an innovative generative AI approach based on Diffusion Policy to teach Robots new dexterity skills quickly and confidently. This advancement significantly improves the utility of robots and is a step toward creating “Large Behavior Models (LBM)” for robots, analogous to the Large Language Models (LLM) that have revolutionized recently conversational AI.
“Our robotics research aims to amplify people rather than replace them,” said Gill Pratt , CEO of TRI and chief scientist at Toyota Motor Corporation. “This new teaching technique is highly effective and produces high-performance behaviors, allowing robots to more effectively amplify people in many ways.”
Previous cutting-edge techniques for teaching robots new behaviors were slow, inconsistent, ineffective, and often limited to highly defined tasks performed in highly constrained environments. Robotics specialists had to spend many hours writing sophisticated code and/or using numerous trial-and-error cycles to program behaviors.
TRI has already taught robots more than 60 difficult, high-dexterity skills using the new approach, including pouring liquids, using tools and manipulating deformable objects. These achievements were achieved without writing a single line of new code; the only change was to provide new data to the robot. Building on this success, TRI has set an ambitious goal of teaching hundreds of new skills by the end of the year and 1,000 by the end of 2024.
Today’s news also highlights that robots can be taught to operate in new scenarios and perform a wide range of behaviors. These skills are not limited to just “pick and place” or simply picking up objects and depositing them elsewhere. TRI robots can now interact with the world in a variety of ways, which will one day allow them to help people in everyday situations and in unpredictable, constantly changing environments.
“The tasks I see these robots perform are simply amazing; even a year ago, I couldn’t have predicted that we were anywhere near this level of diverse dexterity,” said Russ Tedrake , vice president of Robotics Research at TRI. Dr. Tedrake, also Toyota Professor of Electrical and Computer Engineering, Aeronautics and Astronautics, and Mechanical Engineering at MIT, explained: “What’s exciting about this new approach is the pace and reliability with which we can add new skills. Because these skills work directly from camera images and touch detection, using only learned representations, they are able to perform well even on tasks involving deformable objects, fabrics and liquids, which has traditionally been very difficult for robots.”
Technical details:
TRI’s robotic behavior model learns from a teacher’s haptic demonstrations, combined with a linguistic description of the target. It then uses an AI-based Broadcast Policy to learn the demonstrated skill. This process allows new behavior to be deployed autonomously from dozens of demonstrations. This approach not only produces consistent, repeatable and effective results, but it does so extremely quickly.
Key TRI research achievements for this novel development include:
- Dissemination Policy: TRI and our collaborators in Professor Song ‘s group at Columbia University developed a powerful new generative AI approach to behavioral learning. This approach, called Diffusion Policy , allows for quick and easy teaching of behavior from demonstration.
- Custom Robotic Platform: TRI’s robotic platform is specifically made to perform highly dexterous two-arm manipulation tasks, with a focus on enabling haptic feedback and haptic sensing.
- Production line: TRI robots have already learned 60 dexterity skills, with the goal of reaching hundreds by the end of this year and 1,000 by the end of 2024.
- Drake: Part of our (not so) secret recipe is Drake , a model-based design for robotics that provides us with a cutting-edge toolset and simulation platform. Drake’s high level of realism allows us to develop both in simulations and reality at a much greater scale and speed than would otherwise be possible. Our internal set of robots is built using Drake’s systems and optimization frameworks, and we’ve made Drake open source to catalyze the work of the entire robotics community.
- Safety: At TRI, safety is at the heart of our robotics efforts. We have designed our system with robust safety measures, powered by Drake and our custom robotic control package, to ensure our robots respect safety guarantees such as not colliding with themselves or their environment.
SOURCE: PRNewswire