Presented By O’Reilly and Intel AI
Put AI to work
Sep 4-5, 2018: Training
Sep 5-7, 2018: Tutorials & Conference
San Francisco, CA

Predicting short-term driving intention using recurrent neural network on sequential data

Zhou Xing (Borgward R&D Silicon Valley)
Implementing AI, Models and Methods
Location: Yosemite BC

Prerequisite knowledge

  • A basic understanding of machine learning, deep learning, recurrent neural network methodology, path planning and prediction for autonomous driving vehicles, and 3D simulations

What you'll learn

  • Explore a system that can predict road users' intentions and associated risks using machine learning techniques such as recurrent neural networks
  • Learn how to generate training data, traffic scenes, and test algorithms performances using a 3D simulation environment


While most of the time, human drivers can predict the simple intentions of other drivers and various on-road behaviors a few seconds in advance, thus rationalizing the associated risks, such reasoning capabilities can be challenging and difficult for an autonomous driving system. Predicting driver intention and behavior is of great importance for the systems that implement safe, defensive path planning and decision making for autonomous driving vehicles. In particular, short-term driving intentions are the fundamental building blocks of relatively long-term and more sophisticated goals, such as overtaking a slow vehicle in front, taking an exit, or merging onto a congested highway.

Zhou Xing shares a methodology that can be used to build and train a predictive driver system, which includes components such as traffic data, a traffic scene generator, a simulation and experimentation platform, a supervised learning framework for sequential data using recurrent neural network (RNN) approach, and validation of the modeling using both quantitative and qualitative methods. The simulation environment can parameterize and configure relatively challenging traffic scenes, customize different vehicle physics and controls for various types of vehicles including cars, SUVs, and trucks, test and utilize a high-definition map of the road model in algorithms, and generate sensor data out of light detection and ranging (lidar), optical wavelength cameras for training deep neural networks and is crucial for driving intention, behavior, and collision risk modeling, since collecting a statistically significant amount of such data as well as experimentation processes in the real world can be extremely time and resource consuming. Standardizing such a testing, scoring system can be very useful to validate and experiment various planning and prediction algorithms of autonomous driving application.

Photo of Zhou Xing

Zhou Xing

Borgward R&D Silicon Valley

Zhou Xing is the director of artificial intelligence for autonomous driving at Borgward R&D Silicon Valley, where he is in charge of the R&D on artificial intelligence for self-driving car technologies. Zhou is an exceptional scientist, and AI researcher and an experimentalist who always believe in data and experimental results. Previously, he was an engineering physicist and staff scientist at Stanford, where he worked at the SLAC National Accelerator Laboratory developing and managing an in-house software system for data analysis, machine learning, sensor and detector read-out, and big data, and a staff data scientist at NIO USA, where he focused on developing state-of-the-art deep learning and reinforcement learning algorithms for autonomous driving vehicles. An innovative entrepreneur, Zhou founded his own startup company Athena Robotics and served as the CTO in charge of full stack of self-driving car software technologies, including perception, prediction, simulation, and planning. Athena Robotics was selected by Y Combinator for the summer batch of 2018. Zhou’s outstanding publication record includes more than 15,000 Google Scholar citations in top-notch science and AI journals and conferences. He is also a frequent invited speaker on the applications of deep neural networks. Zhou holds a BS in physics from the University of Science and Technology of China (USTC), which has the most prestigious physics department in China, and a PhD in experimental particle physics at CERN, France and Switzerland, affiliated with Syracuse University, New York, where his thesis was focusing on using neural network to solve leading-edge scientific problems, including CP violation after the Big Bang.