Researchers combined artificial intelligence and radar used to track speeders to develop a system that will allow vehicles to spot hazards hidden around corners. Image courtesy of the researchers.



"This will enable cars to see occluded objects that today's lidar and camera sensors cannot record, for example, allowing a self-driving vehicle to see around a dangerous intersection," said Felix Heide, an assistant professor of computer science at Princeton University and one of researchers. "The radar sensors are also relatively low-cost, especially compared to lidar sensors, and scale to mass production."

In a paper presented June 16 at this Conference on Computer Vision and Pattern Recognition (CVPR), the researchers described how the system is able to distinguish objects including cars, bicyclists and pedestrians and gauge their direction and oncoming speed.

"The proposed approach allows for collision warning for pedestrians and cyclists in real-world autonomous driving scenarios—before seeing them with exist direct line-of-sight sensors," the authors write.

In recent years, engineers have developed a variety of sensor systems that allow cars to detect other objects on the road. Many of them rely on lidar or cameras using visible or near-infrared light, and such sensors preventing collisions are now common on modern cars. But optical sensing is difficult to use to spot items out of the car's line of sight. In earlier research, Heide's team has used light to see objects hidden around corners. But those efforts currently are not practical for use in cars both because they require high-powered lasers and are restricted to short ranges.

In conducting that earlier research, Heide and his colleagues wondered whether it would be possible to create a system to detect hazards out of the car's line of sight using imaging radar instead of visible light. The signal loss at smooth surfaces is much lower for , and radar is a proven technology for tracking objects. The challenge is that radar's spatial resolution—used for picturing objects around corners such as cars and bikes—is relatively low. However, the researchers believe that they could create algorithms to interpret the radar data to allow the to function.

"The algorithms that we developed are highly efficient and fit on current generation automotive hardware systems," Heide said. "So, you might see this technology already in the next generation of vehicles."

To allow the system to distinguish objects, Heide's team processed part of the that standard radars consider background noise rather than usable information. The team applied artificial intelligence techniques to refine the processing and read the images. Fangyin Wei, a in computer science and one of the paper's lead authors, said the computer running the system had to learn to recognize cyclists and pedestrians from a very sparse amount of data.

"First we have to detect if something is there. If there is something there, is it important? Is it a cyclist or a pedestrian?" she said. "Then we have to locate it."

Wei said the system currently detects pedestrians and cyclists because the engineers felt those were the most challenging objects because of their small size and varied shape and motion. She said the system could be adjusted to detect vehicles as well.

Heide said the researchers plan to follow the research in a number of directions for applications involving both radar and refinements in signal processing. He said the system has the potential to radically improve automotive safety and it relies on existing radar sensor technology, so readying the system for deployment in the next generation of automobiles should be possible.

"It would certainly go through the very rigorous automotive development cycles," he said. "In terms of integration and bringing it to market, it requires a lot of engineering. But the technology is there, so there is the potential for seeing this very soon in vehicles."

Besides Heide and Wei, the paper's authors include: Jürgen Dickmann, Florian Krause, Werner Ritter, and Nicolas Schiener of Mercedes-Benz AG; Buu Phan and Fahim Mannan of Algolux; Klaus Dietmayer of Ulm University; and Bernard Sick of the University of Kassel.