Google’s DeepMind robotics team has unveiled a groundbreaking set of AI systems designed to enhance the development of versatile robots for everyday use. The newly introduced systems, namely AutoRT, SARA-RT, and RT-Trajectory, aim to revolutionize the way robots collect data, improve their speed, and enhance their understanding of the world.
AutoRT leverages large language models (LLMs) to enable robots to comprehend practical human goals. By accumulating extensive training data, AutoRT enables robots to better prepare for real-world scenarios. The system combines LLMs, such as Large Language Models or Visual Language Models, with robot control models, resulting in the deployment of robots in diverse environments to collect training data.
The Self-Adaptive Robust Attention for Robotics Transformers (SARA-RT) system takes Robotics Transformer (RT) models to the next level of efficiency. By incorporating scalable attention mechanisms, SARA-RT-2 models showcased improved accuracy by 10.6% and performed 14% faster than previous RT-2 models. This breakthrough demonstrates significant advancements in computational efficiency without compromising quality.
In addition, the RT-Trajectory model introduces visual outlines to describe robot motions in training videos. By overlaying 2D trajectory sketches on each video, RT-Trajectory provides valuable visual cues for the model to learn robot control policies. In testing, an arm controlled by RT-Trajectory achieved a remarkable task success rate of 63%, compared to 29% with existing RT models. This adaptability makes RT-Trajectory a versatile tool that can generate trajectories by observing human demonstrations or accepting hand-drawn sketches, adapting to different robot platforms.
The comprehensive real-world evaluations carried out over seven months demonstrated AutoRT’s ability to safely coordinate up to 20 robots simultaneously. The dataset collected during this period included 77,000 robotic trials across 6,650 unique tasks in various office buildings. These advancements represent significant progress in robotics and bring us closer to a future in which intelligent robots navigate complex environments seamlessly.
By harnessing the power of AI and large language models, Google DeepMind’s innovations pave the way for the rapid development of advanced robotics, revolutionizing the way we interact with and benefit from these intelligent machines.