The medicine is out of sight. Before a robot can retrieve it, it needs a reason to walk toward the right drawer. The HomeBody research team connects that remembered location with a library of actions, so a high-level model can choose the next skill and use its result to plan again. Its kitchen demonstrations give an abstract robotics problem an ordinary shape: remembering where to look is only the beginning of getting something done.

A high-level model chooses among skills

HomeBody’s project page contrasts two arrangements. In a common three-part approach, a vision-language model interprets a goal, a learned action model produces commands and a controller executes motion. HomeBody instead lets the high-level model call reusable skills directly. Navigate, pick, open and place become an action vocabulary, with lower-level systems doing the physical work behind each request.

Consider opening the drawer. Choosing that action is one decision. Finding the handle, reaching it and coordinating the pull are other problems. HomeBody describes grasp planning, camera calibration, motion planning and whole-body control below the skill interface. Execution feedback can then tell the planner that it needs another action. The model organizes the sequence without deciding every joint movement along the way.

That separation helps explain where a task can fail. A sensible plan may lead to the wrong target, a failed grasp or an object that has moved. Each problem needs a response at the right level. A local motion adjustment can correct one kind of mistake, while returning a failed result to the planner allows a different next step. Fluent reasoning alone does not close a hand around an object.

Remember a place beyond the current camera view

The team’s project description starts with exploration. Camera observations, LiDAR, positions and selected waypoints contribute to a spatial model of the kitchen. Stored views give the planner something to return to when an object leaves the immediate camera frame. The resulting digital twin connects a remembered place with the room the robot has to move through.

In one demonstration, the robot is asked to tidy several coffee bags and discard specified cartons. In another, it retrieves medicine that is initially out of sight, opens a drawer with one hand, hands the item to a person, and uses the other hand for a separate disposal task. These are the researchers' demonstrations, not independent trials by Newsroom. Their value is that they show the system coordinating repeated navigation and manipulation within one environment rather than completing a single isolated grasp.

Imagine turning away from a kitchen drawer. The current view no longer contains it, but its location can still guide your next move. A robot needs that connection in a form its planner can use. HomeBody’s stored observations and spatial representation provide the proposed link. The project’s comparisons between reconstructed and real views offer a way to examine the model of the room, alongside the physical task videos.

Now imagine someone has moved the item after exploration. A remembered location can guide a search, but the next observation still needs to establish what is there. That is a useful variation for an evaluation: keep the instruction the same, change the object’s position and examine how the system detects the mismatch and chooses another action.

Read the code release one stage at a time

The September repository originally said “code is coming soon.” Its current code release schedule now marks SIM, covering navigation and pick-and-place in simulation, released on October 5. REAL2SIM, the exploration and scene-reconstruction pipeline, is marked released on October 16. As checked on October 26, REAL, the real-world deployment package, is still marked Planned against an October 23 target, with its usage section saying Coming soon. A target date is not evidence that the code has shipped. Those stages give researchers more to inspect while keeping a simulation run distinct from reproducing the complete physical system.

Release trackWhat the repository labels availableWhat that does not establish
SIMNavigation and pick-and-place simulation, released October 5A physical robot reproduced the task
REAL2SIMExploration and scene reconstruction, released October 16The full real-world deployment package is released
REALStill marked Planned at the October 26 checkAvailability merely because its October 23 target has passed

The demonstrations use a Unitree G1 humanoid . Its sensors, reach and two-handed movement set the physical choices available to the planner. Understanding a result means keeping that hardware beside the skills and environment. A useful controlled comparison would hold the task fixed while changing one component, so an improvement can be connected to the memory, planning or execution that produced it.

A similar distinction appears in the circuit benchmark article, where passing an earlier test leaves the final physical result to be demonstrated separately.

An embodied agent has to connect a decision with a physical consequence. For this prototype, look for task definitions, failed attempts, recovery behavior and comparisons with alternative architectures as the release develops. The project page still does not supply a linked full paper. Available code is a route to investigation, rather than a measured success rate across homes the team has not shown.

Our analysis of Nvidia’s physical-AI platform places simulation within the wider robotics toolchain. HomeBody gives that layer a particular job: help connect a room with remembered observations and the next skill. Follow one object through the sequence and ask where the target comes from, what movement is requested and how success returns to the planner. That is a useful set of questions to bring to the next stage of the release.

FAQ

What happens after HomeBody remembers where an object is?

The planner chooses a skill, and specialized systems still have to move, grasp or open something and report the result. The kitchen demonstrations show this on selected tasks.