A new framework developed by MIT’s Improbable AI Lab is set to revolutionize robot planning, allowing machines to execute complex tasks that involve multiple steps with greater precision. The Compositional Foundation Models for Hierarchical Planning (HiP) framework utilizes the expertise of three different foundation models to develop detailed, feasible plans for robots in various settings, including households, factories, and construction sites.
Unlike previous multimodal models, which rely on paired vision, language, and action data, HiP takes a different approach. It utilizes three distinct foundation models trained on different data modalities. Each model captures a different aspect of the decision-making process and collaborates with the others when making decisions.
One of the significant advantages of HiP is that it eliminates the need for paired vision, language, and action data, which can be challenging to obtain. This makes the planning process more transparent and accessible. Furthermore, by incorporating linguistic, physical, and environmental intelligence into a robot, HiP offers a more cost-effective and efficient solution compared to monolithic foundation models.
Jim Fan, an AI researcher at NVIDIA, commended the HiP framework for its decomposition of the complex task of planning into three constituent models. According to Fan, this approach makes decision-making more tractable and transparent, proving to be a significant advancement in the field.
The possibilities for HiP are vast. The system can aid robots in household chores like putting away items or loading the dishwasher correctly. It can also assist in multi-step construction and manufacturing tasks, such as stacking and arranging different materials in specific sequences.
The innovation brought by the HiP framework paves the way for highly capable and intelligent robots that can seamlessly perform complex tasks across various domains. With further development and implementation, the potential applications of this technology are boundless.
The source of the article is from the blog motopaddock.nl