Humanoid robots are increasingly capable of performing complex real-world tasks, but many still struggle with dexterous object manipulation and navigation in dynamic environments.

Carnegie Mellon University (CMU) and the Bosch Center for AI have now developed a new AI-based system designed to improve whole-body coordination and contact-rich manipulation in humanoid robots.

The framework combines body movement, hand dexterity, and predictive understanding of physical interactions, enabling robots to handle tasks such as folding cloth better, inserting objects, scooping items, and carrying fragile materials in real-world settings.

Last month, a team led by researchers from ETH Zurich developed a framework linking large language models with ROS, enabling robots to understand natural language instructions and perform real-world physical tasks autonomously.

Dexterous robot learning

Researchers developed a new AI framework designed to improve whole-body humanoid manipulation in contact-rich environments. The system, called Humanoid Transformer with Touch Dreaming (HTD), combines imitation learning with predictive touch modeling, allowing robots to anticipate how physical contact, force, and tactile feedback will evolve while handling objects.

Unlike conventional systems that rely mainly on vision and motion sensing, HTD incorporates distributed tactile sensing to improve awareness during complex interactions. The model predicts future actions alongside hand-joint forces and tactile representations, helping robots better manage tasks such as object insertion, cloth handling, scooping, and carrying fragile items, reports TechXplore.

Instead of reconstructing raw tactile sensor data, the framework generates compact tactile latent representations through a slowly updated target network. This approach enables the AI system to focus on meaningful contact patterns while filtering out noisy sensor fluctuations, improving manipulation stability and responsiveness.

The platform also separates lower-body balance control from upper-body manipulation tasks. A reinforcement learning-based controller stabilizes torso orientation, velocity, and balance during movement, while inverse kinematics and hand retargeting handle upper-body positioning and dexterous hand motions. The lower-body controller was trained in simulation using a teacher-student method, enabling the robot to learn robust motion planning from realistic sensory observations.

“During training, we replay retargeted arm motions from the AMASS dataset so that the controller learns to remain stable under realistic upper-body disturbances. The final student controller is what we deploy on the real robot,” said Ding Zhao and Yaru Niu, part of the CMU Safe AI Lab, as reported by TechXplore.

Smarter robot control

The HTD system improves humanoid manipulation by reducing interference between balance control and dexterous object handling.

Tests showed the system separates lower-body stability from upper-body and hand manipulation, enabling balance during complex tasks. It manages posture and motion in the lower body, while inverse kinematics and hand retargeting control upper-body movements. It also predicts forces and tactile feedback within the policy, avoiding separate tactile pre-training or extra world models.

“Our experimental results were strong. Across five real-world tasks, namely insert-T, book organization, towel folding, cat litter scooping, and tea serving, HTD achieved a 90.9 percent relative improvement in average success rate over the stronger ACT baseline,” said Zhao, as reported by TechXplore.

The ablation studies showed that simply adding touch as an additional input was insufficient. Predicting tactile signals in a latent space proved more effective than predicting raw tactile signals directly, resulting in a 30 percent relative improvement in success rate compared to raw tactile-based prediction.

The team notes that their new platform and AI-based controller could enhance humanoid robots for real-world deployment in domestic, service, and industrial environments. Future work will scale the learning framework and test it in human–robot collaboration settings by incorporating visual data and human demonstrations.

Researchers say the goal is to improve generalization across different robot designs, including varied dexterous hands and tactile sensor setups. Long-term efforts focus on creating scalable, embodiment-robust systems that adapt to diverse tasks and learn from both human and robot experience.