When Cars Can See: The Massive Opportunity of Road Data

When Cars Can See: The Massive Opportunity of Road Data

Uri Tamir | November 19, 2019

The men and women working city operations from behind the scenes have never played more important – or challenging – roles than they do today. Fiscal constraints across the country are forcing municipal leaders to do more with less, at a time when constituents are rightfully demanding less traffic, safer roads, and more reliable infrastructure. 

As city planners explore the vast array of smart city solutions, many of which require the costly replacement of legacy systems and existing infrastructure, there is an emerging solution in automotive technology that has the potential to solve a wide range of challenges. Advanced Driver Assistance Systems (ADAS), originally developed for preventing car collisions, can provide new insights and enable automation for some of our cities’ most burning issues. Much like the Internet, a technology first invented for military purposes but which expanded its use cases far beyond what its pioneers first envisioned, ADAS’s benefits are only beginning to be realized. 

Imagine if our cities could:

  • Identify deficient infrastructure in real time, not months, so they can make repairs before collateral damage is caused
  • Map and localize hundreds of thousands of infrastructure assets over thousands of miles of roads, and still be able to fit the data into a handheld USB stick
  • Understand the most dangerous roads and intersections before an accident or a fatality occurred

At Mobileye, we’re building this reality. From powering driver assist features for millions of consumer passenger cars and perfecting the technology required for Level 4 and 5 Autonomous Vehicles (AVs), to our Mobileye 8 Connect product, a retrofit device that can provide collision avoidance warnings on almost any vehicle on the road today, we’re building computer vision and artificial intelligence solutions that teach cars to see.

What do we mean by “computer vision?” If you’re imagining that iconic last scene of the Matrix, where Neo is visualizing the world in green code as he looks down at three agents, I wouldn’t blame you (in fact, I would applaud your taste in sci-fi). Put simply, computer vision comes down to sensing and classifying the surrounding environment and understanding its relevant attributes and semantics.

But we don’t have to wait for AVs or live in the Matrix to put this technology to work and take advantage of its many benefits. Many new cars today come equipped with a forward-facing camera as part of their safety systems. And aftermarket collision avoidance warning systems can be retrofitted into the millions of existing vehicles on the road whose owners wish to make them safer. By retrofitting a vehicle with Mobileye 8 Connect, vehicle owners and fleets get life-saving features such as Forward Collision Warning, Lane Departure Warning, Safe Distance Warning, and Pedestrian & Cyclist Collision Warning – features that the Institute for Highway Safety has found to reduce the frequency and cost of collisions. Our fleet customers have benefited immensely from retrofitting their vehicles with Mobileye solutions, from dramatic reduction in collision-related costs to even improving their fuel economy along the way.

But safety is just the beginning for this technology. The real treasure trove is in the insights we can generate from the data that is being harvested.

In early 2017 we announced a technology called Road Experience ManagementTM (REMTM). REM is an end-to-end mapping and localization engine that uses ADAS cameras to not only sense the environment, but to also localize the perceived objects on a map that is constantly updated. By connecting our systems to the cloud, REM can aggregate data from each car and incorporate it into the map, creating a collective “memory” of the road. As Mobileye-equipped cars drive a road segment, they can localize the position of various objects and markings and update the map with their own experience, continuously improving the map’s accuracy.

Beyond creating more sophisticated ADAS features and enhancing driving capabilities of AVs, REM opened the door to a wide array of data possibilities.

With Mobileye-equipped vehicles driving city streets, we can detect and share the status of many street-level characteristics, such as streetlights, traffic signs, road markings, manholes and potholes. If there’s a disruption to traffic, such as a critical traffic sign getting knocked down, or a tree falling in the road during a storm, cities can know about it and address it in (almost) real-time, not months.

Mobileye recently partnered with Ordnance Survey, the U.K. national mapping company, to create a detailed roadside infrastructure dataset. The information we are harvesting has a variety of use cases to utility companies, chief among them helping to prevent the dozen killed, hundreds injured, and millions of dollars lost each year to unfortunate instances where routine drilling results in unintentional puncturing of below-ground structures in the U.K.

We also recently mapped hundreds of thousands of assets over thousands of miles of Japan’s highway system. Using data harvested by Mobileye-equipped vehicles, we were able create the detailed map in under 24 hours and fit it all into less than 400 megabytes of storage. The information can potentially give municipalities in Japan the ability to analyze the effectiveness of various speed signs, for example.

The Mobileye REM technology can turn vehicles into road surveyors. We can detect potholes, road cracks, faded lane marks, and other dangerous issues and provide that information to cities and states. With this knowledge at their fingertips, city officials, rather than picking road maintenance projects by zip code or according to an outdated road survey, can act with surgical precision before issues become problems, saving time and money in their already tight budgets.

We can even detect when and where ADAS collision warning alerts (near-misses) happen, pinpointing dangerous places across a city. This takes the idea of “driver scoring,” a method commonly used by insurers for telematics applications to improve driver behavior, and expands it to an entire metro area. The result is a map that can identify the most vulnerable areas for cars, cyclists, and pedestrians alike.

All of this can be done through the deployment of cost effective ADAS that road users are beginning to expect from cars navigating their city’s streets. While improving the safety of constituents, entire cities can be transformed in the process.

Collision avoidance technology is not new; Mobileye pioneered the camera-based ADAS technology and has been perfecting it for decades. But the full, vast potential of ADAS beyond safety has not yet been seen. Similar to how the Internet transformed the way the world operates despite its limited initial intentions, we’ve only begun to scratch the surface of ADAS. Today’s forward-thinking city leaders should see ADAS not only for the collisions it has been proven to prevent, but for the future potential it promises for the improved livability of their cities.

Topics: Smart Mobility

Uri Tamir, Senior Director, Strategic Initiatives, Mobileye (Intel)

As Mobileye’s North American Senior Director of Strategic Initiatives, Uri Tamir leads the development of new markets for Mobileye’s latest advancements in collision avoidance technology, sensor’s data, and mapping. Uri oversees Mobileye’s push for smarter and safer roads and represents Mobileye in numerous research projects and partnerships with US DOT, State DOTs, and cities in North America.