The technology may also find applications in lightweight augmented reality headsets, allowing them to map indoor environments without draining batteries. The chip combines specialized hardware with a compact mapping algorithm that dramatically reduces the memory and energy needed to create 3D representations of the world around a robot.

Mapping without burden

Building detailed 3D maps typically requires robots to process large volumes of image data and store complex representations of their surroundings. These tasks often demand significant memory and power, making them difficult to deploy on small battery-powered devices.

Instead of relying on conventional voxel-based maps, which represent environments using millions of tiny cubes, the MIT team uses flexible ellipsoid shapes known as Gaussians. These shapes can more efficiently represent curved objects and open spaces while requiring far less memory.

The researchers paired the chip with a mapping algorithm called GMMap, which creates 3D maps from depth images in a single pass. This allows the system to discard image data almost immediately instead of repeatedly storing and processing it.

“At any point in time, we only need to store a few pixels in memory, which significantly reduces the memory footprint our algorithm requires,” said Peter Zhi Xuan Li, one of the study’s co-lead authors.

The system also avoids another common challenge in mapping. As a robot moves, it often observes the same object from multiple angles, creating overlapping representations that increase map size. The MIT team developed a method to merge overlapping Gaussians directly without returning to the original image data.

Tiny chip, big reach

This approach allowed the researchers to keep much of the active data in fast on-chip memory rather than relying on power-hungry external storage. “By having a dedicated memory that just stores the objects you’ve seen in the previous few frames, you can access the data much more efficiently,” said co-lead author Zih-Sing Fu.

In tests involving a variety of previously recorded environments, Gleanmer generated detailed 3D maps in real time while consuming about 6 milliwatts of power. According to the researchers, this is roughly 2.5% of the energy required by the best existing chip designed for map construction.

The chip can also reconstruct obstacles and free space directly from live data streamed from an iPhone camera. By reusing compact Gaussian representations during path planning, the system enables robots to calculate collision-free routes using about 20% of the energy typically required.

“This paper showcases a key example of how you can leverage co-design of the algorithm and hardware to really push energy efficiency,” said Vivienne Sze, professor of electrical engineering and computer science at MIT and senior author of the study.

The researchers believe future versions could become even more efficient by placing computing resources closer to onboard sensors. Beyond robotics, the team is also exploring whether Gaussian-based representations could help computing systems process technical drawings and complex schematics more efficiently.

Recommended Articles