Robotics Foundation Models: How One AI Model Learns to Control Different Robots

A robotic arm sorts packages in a warehouse, a humanoid folds clothes, and another machine handles delicate equipment in a laboratory. These robots have different bodies and responsibilities, yet many of the skills they need are surprisingly similar. They must recognise objects, understand instructions, plan movements and respond when something unexpected happens. Traditionally, teaching these abilities required considerable work for each machine separately. Robotics foundation models offer another approach: training artificial intelligence on experiences collected from different robots so that useful knowledge can be shared between them. By 2026, research teams at Google DeepMind, NVIDIA and Physical Intelligence have demonstrated important advances in this field. Their work suggests a future in which robots can adapt to new tasks more easily, although making such systems consistently reliable remains a major challenge.

What Are Robotics Foundation Models and How Do They Work?

A robotics foundation model is an artificial intelligence system trained to understand and perform a broad range of physical tasks. Instead of learning only one operation, such as moving identical boxes between fixed positions, the model develops knowledge that may be useful across different activities. This knowledge can include recognising household objects, understanding how items should be handled and identifying actions needed to complete instructions. Much of the training comes from demonstrations involving robots interacting with real or simulated environments. By analysing many examples, the AI learns relationships between what a machine sees, what it needs to accomplish and how it should move.

The difference from traditional robotic programming is particularly noticeable when circumstances change. A conventional industrial robot often performs exceptionally well when every object arrives at a predictable location. However, even small changes may require adjustments to its programmed movements. A foundation model can potentially respond more flexibly because it has learned from varied examples rather than memorising a single sequence. For instance, if a cup appears in an unfamiliar position, an AI-controlled robot may identify its location and calculate a suitable way to grasp it. Whether this succeeds depends on the training data, the robot’s sensors and the quality of its movement controls.

Many advanced systems use a method known as vision-language-action, or VLA. The vision component processes information from cameras, while language understanding helps the robot interpret instructions expressed in ordinary words. The action component then produces commands that can guide physical movement. Someone might ask a robot to place several objects inside a container without explaining every movement required. The model must identify the objects, understand the intended destination and select appropriate actions. Although this sounds similar to how a person approaches everyday tasks, robots still need specialised control systems to translate their decisions into safe and accurate movements.

How One AI Model Learns From Different Robot Designs

Robots do not share the same physical structure. A stationary robotic arm may have six or seven movable joints, while a humanoid has legs, arms and hands that must work together. Mobile robots introduce additional challenges because they need to navigate through their surroundings before manipulating objects. Researchers describe these differences using the term embodiment. A foundation model trained across several embodiments must distinguish between general knowledge about a task and the movements required by a particular machine. Understanding that a bottle needs to remain upright is useful to many robots, but the correct method of lifting it depends on the available gripper, joints and sensors.

To develop transferable skills, researchers collect information from robots performing different activities. Training examples may contain camera recordings, descriptions of tasks, movement data and the results of individual actions. Some demonstrations come from people operating robots remotely, while others are produced through automated experiments or simulated environments. The AI searches for patterns shared across these examples. It may learn that reaching towards an object, positioning a gripper and lifting carefully form a useful sequence, even when the exact movements differ between machines. This process helps the model build general knowledge that can later be adapted to unfamiliar robotic equipment.

An important milestone came with the Open X-Embodiment research project announced in 2023. Google DeepMind and collaborators from 33 research laboratories combined datasets covering 22 different robot types. The work demonstrated how robotic experience gathered by separate research groups could contribute to shared learning systems. Rather than developing every model exclusively from information collected on one machine, researchers could investigate methods that learn from varied hardware. The project provided a foundation for subsequent research into cross-robot learning. However, access to a large shared dataset does not automatically guarantee successful control of every robot, and individual machines may still require additional training.

How Foundation Models Transfer Skills Between Different Robots

The ability to transfer learned behaviour is one of the most promising aspects of robotics foundation models. Imagine a robotic arm that has learned to recognise kitchen objects and place them into designated containers. Another robot, equipped with different joints and a different gripper, might use the same general understanding of the task. It would still need to determine how to approach each object and perform the movements using its own hardware. The shared AI knowledge can reduce the amount of learning required, but it cannot remove the physical differences between machines. A robot with a simple two-finger gripper, for example, cannot automatically reproduce every movement made by a five-fingered robotic hand.

This approach is known as cross-embodiment learning. Its central principle is that different machines can benefit from common training experiences without using identical motor commands. A robot may learn what needs to happen while adapting how it performs the action. Researchers achieve this through training methods that connect visual observations, task descriptions and physical movements. Additional demonstrations from a new robot can improve how effectively the shared model controls that machine. This is especially useful for manufacturers developing several robotic designs, since some learned abilities can potentially be reused instead of taught independently from the beginning.

Google DeepMind provided a notable example with Gemini Robotics 1.5, introduced in September 2025. Its research demonstrated the transfer of behaviours between different robotic systems, including ALOHA 2, a dual-arm Franka configuration and Apptronik’s Apollo humanoid. Some tasks learned with one configuration could be carried out by another without separate specialised training for that particular task and robot combination. These experiments represented meaningful progress towards more general robotic intelligence. Nevertheless, the results were obtained using selected machines and evaluation conditions. They do not mean that a single model can already operate every commercial robot or complete any unfamiliar activity without preparation.

Real Examples From Google DeepMind, NVIDIA and Physical Intelligence

Google DeepMind’s Gemini Robotics family combines visual understanding, natural-language interpretation and physical action generation. Its research has included manipulating objects, arranging items and performing tasks involving several consecutive decisions. The related Gemini Robotics-ER systems focus more heavily on understanding physical environments and reasoning about appropriate actions. Gemini Robotics-ER 1.6, presented in April 2026, introduced improvements involving spatial understanding, information from multiple cameras and evaluating whether tasks had been completed successfully. These capabilities address a practical problem: a robot must recognise not only what action to attempt, but also whether that action produced the intended outcome.

NVIDIA has pursued another approach through its Isaac GR00T family of foundation models, developed primarily for humanoid robotics. The research combines visual information, language instructions and movement generation to support a broader range of physical activities. Models introduced during 2026, including GR00T N1.6 and N1.7, reflected continuing efforts to improve general humanoid capabilities. NVIDIA also provides development tools and simulation environments that allow researchers to practise movements before testing them on physical machines. Simulation is particularly useful when tasks involve expensive equipment or potentially damaging mistakes. However, successful virtual training must still be validated under real physical conditions, where objects, surfaces and sensors may behave differently.

Physical Intelligence has also contributed to research into robots that can perform varied manipulation tasks. Its π0.5 model, presented in April 2025, explored how robots could generalise learned behaviour to unfamiliar environments. This addresses a common weakness of conventional robotic systems: a machine may perform a task successfully in one room but struggle when the arrangement of furniture or objects changes. Training across diverse examples can help reduce dependence on fixed surroundings. Experiments involving household manipulation have illustrated the potential of this approach, although they should not be confused with fully independent domestic robots capable of handling every situation safely and reliably.

How Robotics Foundation Models Could Change Everyday Work

Manufacturing is one of the industries where more adaptable robotic intelligence could provide practical advantages. Industrial robots are already widely used for welding, assembly, painting and moving materials, but many installations depend on carefully controlled working conditions. When products change or production lines are reorganised, engineers may need to adjust the equipment and its software. Foundation models could reduce some of this work by helping robots recognise new objects and learn additional procedures from demonstrations. For smaller manufacturers that frequently change their products, greater flexibility could make robotic automation more practical. However, performance, maintenance requirements and installation costs will still influence whether adoption makes economic sense.

Warehouses and logistics centres offer another potential application. Many existing machines already move goods efficiently, but handling irregular packages or unexpected arrangements can be more difficult than transporting standard containers. A robot with stronger general manipulation skills might recognise objects of different sizes, choose an appropriate grip and adjust when items are moved. In laboratories, similar abilities could support repetitive activities such as organising supplies or transporting equipment. These applications are particularly attractive where work involves variation rather than one perfectly repeated movement. Even so, deploying robots in busy workplaces requires extensive testing because people, equipment and obstacles may behave unpredictably.

Domestic assistance is a longer-term possibility that attracts considerable research attention. Tasks such as clearing tables, collecting household objects and sorting laundry appear straightforward to humans but involve complicated decisions for robots. A towel may change shape when lifted, a glass may be slippery, and furniture can restrict movement. Foundation models can help robots interpret instructions and adapt learned manipulation skills, but safe household operation involves much more than completing isolated demonstrations. A practical domestic assistant would need reliable navigation, careful interaction with people and the ability to recognise when a task exceeds its capabilities. These requirements explain why impressive laboratory results have not yet translated into universally capable household robots.

Current Challenges and the Future of General-Purpose Robotics

Safety remains one of the most serious challenges for AI-controlled machines. A language model that produces an incorrect answer creates a different kind of problem from a robot that makes an incorrect physical movement. A poorly chosen action might damage expensive equipment, break fragile objects or put someone at risk. Foundation models can misunderstand requests or misjudge the position and properties of objects, particularly in unfamiliar conditions. Consequently, physical robots need safeguards beyond the AI model itself. These may include collision detection, restrictions on movement, emergency stopping systems and supervision. Real-world testing must examine how machines respond to mistakes and unexpected events, not only how well they perform tasks under favourable conditions.

Another obstacle is the difficulty of obtaining sufficient high-quality training data. Internet-scale text and image collections are widely available for general AI research, but examples of physical robotic actions are more expensive to produce. Researchers need information about movements, contact with objects and the results of interactions. Simulation can supplement these examples and allow robots to practise without damaging equipment, yet virtual environments cannot reproduce every physical detail perfectly. Small differences in friction, weight, lighting or camera measurements may cause difficulties when a model moves from simulation to a real machine. Developers therefore continue combining shared datasets, simulated experience and additional training on individual robots.

By October 2026, robotics foundation models have demonstrated that AI knowledge can be reused across different robotic designs and applied to a growing variety of physical tasks. The next stage involves turning experimental capabilities into dependable systems that can operate for long periods without constant intervention. Improvements in training methods, spatial reasoning, computing efficiency and safety evaluation will all be important. The likely direction is not a single AI capable of instantly controlling every machine, but families of adaptable robots that share useful knowledge while retaining specialised hardware and control mechanisms. Such systems could gradually reduce the effort required to teach robots new skills and make automation practical in environments that are currently difficult to manage.