MIT Develops Advanced AI to Improve Wireless Vision Systems
Researchers at the Massachusetts Institute of Technology (MIT) have made a significant breakthrough in the field of robotics and artificial intelligence (AI). Their decade-long research on enabling robots to manipulate hidden objects using surface-penetrating radio signals has taken a leap forward with the help of generative AI models. This new model can create a partial reconstruction of a hidden object based on the reflected radio signals and fill in the missing parts of its shape. This has resulted in more precise shape reconstructions and could further enhance a robot’s ability to interact with non-visible objects.
A New Approach to Mitigate Previous Limitations
The previous methods had a significant bottleneck that restricted the precision of the outcomes. The new generative AI model helps to overcome this limitation, producing more accurate reconstructions. An advanced system that can accurately reconstruct an entire room, including all furniture, using this technology has also been unveiled by the researchers. This system operates on wireless signals sent by a stationary radar that are reflected by people moving in the space.
Benefits of the New System
This innovative technology not only enhances the capabilities of robots but also respects the privacy of individuals in the vicinity. It eliminates the need for mounting a wireless sensor on a mobile robot to scan the environment, a requirement in many existing methods. Applications of this technology range from enabling warehouse robots to inspect packaged items before shipping to reduce wastage, to allowing smart home robots to detect a person’s location in a room, thereby improving the safety and efficiency of human-robot interaction.
Generative AI: A Leap in Wireless Vision
“What we’ve done now is develop generative AI models that help us understand wireless reflections. That opens up many interesting new applications, but technically it’s also a qualitative leap in capabilities, from the ability to close gaps we couldn’t see before to the ability to interpret reflections and reconstruct entire scenes,” says Fadel Adib, an associate professor in the Department of Electrical Engineering and Computer Science and director of the Signal Kinetics group in the MIT Media Lab. The papers on these techniques will be presented at the IEEE Conference on Computer Vision and Pattern Recognition.
Enhancing the Accuracy of 3D Shape Reconstructions
Millimeter wave (mmWave) signals, the same type used in Wi-Fi, can penetrate common obstacles and reflect off hidden objects. However, these waves are usually specularly reflective, meaning they reflect in a single direction after hitting a surface. Large portions of the surface will reflect signals away from the mmWave sensor, making these areas virtually invisible. To overcome this, the researchers used a generative AI model to fill in parts that are missing in a partial reconstruction.
Wave-Former: A Step Forward in Wireless Vision
The complete system, named Wave-Former, passes potential object surfaces based on mmWave reflections to the generative AI model to complete the shape. It then refines the surfaces until a complete reconstruction is achieved. Wave-Former has demonstrated the ability to create faithful reconstructions of approximately 70 everyday items, increasing accuracy by almost 20 percent compared to existing methods.
Seeing “Ghosts”
The team has also developed an advanced system that fully reconstructs entire indoor scenes using mmWave reflections of people moving around a room. These secondary reflections create so-called “ghost signals” which, though often discarded as noise, contain information about the layout of the room. The researchers used a similar training method to teach a generative AI model to interpret these rough scene reconstructions and understand the behavior of multipath mmWave reflections. The result is a system, called RISE, that generates reconstructions twice as precise as existing techniques.
Looking ahead, the researchers aim to improve the granularity and detail of their reconstructions. They also aspire to build large base models for wireless signals, potentially opening up new applications. This research has been supported in part by the National Science Foundation (NSF), the MIT Media Lab, and Amazon.
For more details, read the original article Here.
