KARNet: Kalman Filter Augmented Recurrent Neural Network for Learning
  World Models in Autonomous Driving Tasks
Autonomous driving has received a great deal of attention in the automotive
industry and is often seen as the future of transportation. The development of
autonomous driving technology has been greatly accelerated by the growth of
end-to-end machine learning techniques that have been successfully used for
perception, planning, and control tasks. An important aspect of autonomous
driving planning is knowing how the environment evolves in the immediate future
and taking appropriate actions. An autonomous driving system should effectively
use the information collected from the various sensors to form an abstract
representation of the world to maintain situational awareness. For this
purpose, deep learning models can be used to learn compact latent
representations from a stream of incoming data. However, most deep learning
models are trained end-to-end and do not incorporate any prior knowledge (e.g.,
from physics) of the vehicle in the architecture. In this direction, many works
have explored physics-infused neural network (PINN) architectures to infuse
physics models during training. Inspired by this observation, we present a
Kalman filter augmented recurrent neural network architecture to learn the
latent representation of the traffic flow using front camera images only. We
demonstrate the efficacy of the proposed model in both imitation and
reinforcement learning settings using both simulated and real-world datasets.
The results show that incorporating an explicit model of the vehicle (states
estimated using Kalman filtering) in the end-to-end learning significantly
increases performance.
Hemanth Manjunatha, Andrey Pak, Dimitar Filev, Panagiotis Tsiotras
KARNet: Kalman Filter Augmented Recurrent Neural Network for Learning World Models in Autonomous Driving Tasks

###### Abstract

Autonomous driving has received a great deal of attention in the automotive industry and is often seen as the future of transportation. The development of autonomous driving technology has been greatly accelerated by the growth of end-to-end machine learning techniques that have been successfully used for perception, planning, and control tasks. An important aspect of autonomous driving planning is knowing how the environment evolves in the immediate future and taking appropriate actions. An autonomous driving system should effectively use the information collected from the various sensors to form an abstract representation of the world to maintain situational awareness. For this purpose, deep learning models can be used to learn compact latent representations from a stream of incoming data. However, most deep learning models are trained end-to-end and do not incorporate in the architecture any prior knowledge (e.g., from physics) of the vehicle. In this direction, many works have explored physics-infused neural network (PINN) architectures to infuse physics models during training. Inspired by this observation, we present a Kalman filter augmented recurrent neural network architecture to learn the latent representation of the traffic flow using front camera images only. We demonstrate the efficacy of the proposed model in both imitation and reinforcement learning settings using both simulated and real-world datasets. The results show that incorporating an explicit model of the vehicle (states estimated using Kalman filtering) in the end-to-end learning significantly increases performance.

 Autonomous vehicles, Autoencoders, Imitation Learning, Reinforcement Learning, Physics Infused Neural Networks.

## I Introduction

Agents that can learn autonomously while interacting with the world are quickly becoming mainstream due to advances in the machine learning domain [1]. Particularly, stochastic generative modeling and reinforcement learning frameworks have proved successful in learning strategies for complex tasks, often out-performing humans by learning the structure and statistical regularities found in data collected in the real world [2, 3]. These successful theoretical frameworks support the idea that acquiring internal models of the environment, i.e. World Models (WM) is a natural way to achieve desired interaction of the agent with its surroundings. Extensive evidence from recent neuroscience/cognitive science research [4, 5] highlights prediction as one of the core brain functions, even stating that "prediction, not behavior, is the proof of intelligence" with the brain continuously combining future expectations with present sensory inputs during decision making. One can argue that prediction provides an additional advantage from an evolutionary perspective, where the ability to predict the next action is critical for successful survival. Thus, constructing explicit models of the environment, or enabling the learning agent to predict its future, has a fundamental appeal for learning-based methods [6]. In this direction, we present a physics-infused neural network architecture for learning World Models for autonomous driving tasks. By World Model we mean predicting the future frames from consecutive historical frames where a frame is the front view image of the traffic flow [7].

Reinforcement Learning (RL) and Imitation Learning (IL) have gained much traction in autonomous driving as a promising avenue to learn an end-to-end policy that directly maps sensor observations to steering and throttle commands. However, approaches based on RL (or IL) require many training samples collected from a multitude of sensors of different modalities, including camera images, LiDAR scans, and Inertial-Measurement-Units (IMUs). These data modalities generate high-dimensional data that are both spatially and temporally correlated. In order to effectively use the information collected from the various sensors and develop a world model (an abstract description of the world), this high-dimensional data needs to be reduced to low-dimensional representations [8]. To this end, Variational Autoencoders (VAEs) and Recurrent Neural Networks (RNNs) have been extensively used to infer low-dimensional _latent variables_ from temporally correlated data [9, 10, 11, 12]. Furthermore, modeling temporal dependencies using RNN in the data allows to construct a good _prediction model_ (a World Model), which has been shown to benefit RL/IL scenarios [13, 14]. However, learning a good prediction model in an end-to-end fashion comes with the cost of substantial training data, training time, not to mention a significant computational burden. Moreover, end-to-end training can lead to hallucinations where an agent learns World Models that do not adhere to physical laws [9, 15]. Hence, there is a need to consider physics-informed or model-based machine learning approaches [16].

Model-based learning methods, on the other hand, incorporate prior knowledge (e.g., from physics) into the neural network architecture [16, 17]. The explicit model acts as a high-level abstraction of the observed data and can provide rich