# Unifying F1TENTH Autonomous Racing:
Survey, Methods and Benchmarks

Benjamin David Evans1, Raphael Trumpp2, Marco Caccamo2,
Hendrik Willem Jordaan1, and Herman Arnold Engelbrecht1 1 Faculty of
Electrical Engineering, Stellenbosch University, Stellenbosch, 7600, South
Africa, {bdevans, wjordaan<EMAIL_ADDRESS>TUM School of Engineering and
Design, Technical University of Munich, Germany, {raphael.trumpp,
<EMAIL_ADDRESS>Caccamo is supported by an Alexander von Humboldt
Professorship endowed by the German Federal Ministry of Education and
Research.Manuscript received

###### Abstract

The F1TENTH autonomous racing platform, consisting of 1:10 scale RC cars, has
evolved into a leading research platform. The many publications and real-world
competitions span many domains, from classical path planning to novel
learning-based algorithms. Consequently, the field is wide and disjointed,
hindering direct comparison of methods and making it difficult to assess the
state-of-the-art. Therefore, we aim to unify the field by surveying current
approaches, describing common methods and providing benchmark results to
facilitate clear comparison and establish a baseline for future work. We
survey current work in F1TENTH racing in the classical and learning
categories, explaining the different solution approaches. We describe particle
filter localisation, trajectory optimisation and tracking, model predictive
contouring control (MPCC), follow-the-gap and end-to-end reinforcement
learning. We provide an open-source evaluation of benchmark methods and
investigate overlooked factors of control frequency and localisation accuracy
for classical methods and reward signal and training map for learning methods.
The evaluation shows that the optimisation and tracking method achieves the
fastest lap times, followed by the MPCC planner. Finally, our work identifies
and outlines the relevant research aspects to help motivate future work in the
F1TENTH domain.

###### Index Terms:

F1TENTH, Autonomous racing, Survey, Deep reinforcement learning, Trajectory
optimisation

##  I Introduction

The growing field of F1TENTH autonomous racing offers an exciting testbed for
cutting-edge robotics and autonomous vehicle research [1]. The domain
encompasses diverse aspects of autonomous systems, including perception,
planning, control and learning [2]. However, the breath of F1TENTH research
has left the field divided and fragmented, with researchers often focusing on
specific subproblems in isolation. A particular divide exists between
classical and machine learning approaches. This fragmentation hinders
comparison between methods, making it difficult to assess algorithmic
advancement. This paper aims to unify F1TENTH research by providing a
comprehensive overview of the field and introducing a consistent perspective
to unite different disciplines.

Autonomous racing presents a uniquely demanding testbed for the advancement of
autonomous robotics methods. The racing problem is to select control actions
(speed and steering) based on raw sensor data (LiDAR) that move the vehicle
around the track as quickly as possible. The nature of racing provides a
difficult challenge due to non-linear tyre dynamics, unstructured sensor data
and real-time computing requirements [3]. A key challenge is the fundamental
conflict between maximising performance and ensuring safety [4]. Higher speeds
increase operational risk while prioritizing safety sacrifices competitive
advantage. These difficulties promote the improvement of high-performance,
safe, and efficient autonomous vehicle algorithms.

The F1TENTH platform is ideal for research because it enables rapid
prototyping of algorithms, provides a competitive environment and simplifies
software-to-hardware deployment [5]. F1TENTH vehicles, shown in Fig. 1, are
equipped with a 2D-LiDAR for sensing, an onboard computer for processing, and
actuator controllers for the motors [1]. The smaller size of F1TENTH cars
makes them cost-effective and safe for developing new high-performance
algorithms where crashes are inevitable. Finally, it provides a fun topic for
students worldwide [6], and competitions encourage novel research.

Figure 1: The F1TENTH platform provides a development platform for autonomous
algorithms with an accurate simulator allowing rapid deployment.

We unify the field of F1TENTH racing by creating a common solution taxonomy
through the following contributions:

  * •

Survey advances in classical and learning-based approaches for F1TENTH racing
in Section II.

  * •

Describe common classical and reinforcement learning baseline methods in
Section III.

  * •

Provide benchmark results (with open-source software111Code available at:
https://github.com/BDEvan5/f1tenth_benchmarks) to accelerate future work in
Section IV.

  * •

Motivate promising research directions for future work on F1TENTH cars in
Section V.

##  II Literature Survey

###  II-A Classical Approaches

Classical approaches to autonomous racing use estimation, optimisation and
control systems to calculate control commands. Typically, estimation is used
to localise the vehicle on the map, optimisation is used for planning, and
tracking algorithms are used to follow the trajectory. Fig. 2 shows how
classical pipelines use localisation with either offline planning and online
control or online planning and control. We study works related to each of
these functions.

Perception: Perception is the task of building a map and localising the
vehicle on the map. While no methods of map building, i.e., simultaneous
localization and mapping (SLAM), have been built explicitly for F1TENTH
racing, the ROS SLAM toolbox (using graph SLAM) is commonly used [7, 8]. Walsh
et al. [9] implement a computationally efficient method for particle filter
localization. Their approach, and open-source repo, is the standard
localization method used in many other approaches [10, 11]. An extension of
their approach has improved the original implementation by using a higher-
order motion model [12]. A different approach using invertible neural networks
to learn a localisation policy boasted improved computational performance
[13]. However, the particle filter remains the most commonly used method due
to its simplicity and robustness.

Offline trajectory planning: Offline trajectory planning uses an optimisation
algorithm and map of the track to generate an optimal set of waypoints. The
baseline planning method for most works uses a minimum curvature or minimum
time trajectory planning formulation from Heilmeier et al. [14], which
formulates the trajectory planning problem as a quadratic programming problem.
Other approaches have turned to genetic programming algorithms to jointly
optimise the trajectory and the controller [10, 15], but these can lead to
simulator exploitation and unrealistic behaviour [16]. Heilmeier et al.’s
method is the most popular because it is efficient to compute, robust to
different tracks, open-source accessible and provides high-quality results
[17, 18].

Online trajectory planning: At each planning step, online trajectory planning
approaches generate a set of control inputs and a corresponding trajectory for
a receding horizon [19]. Model predictive contouring control (MPCC) has been
used to optimise a trajectory that maximises progress along the centre line
[20]. Since online planning algorithms replan at each timestep, they can more
easily integrate obstacle avoidance and head-to-head racing strategies [20,
21]. While online planning algorithms, such as MPCC, can plan accurate, high-
performance paths, they are limited by requiring accurate vehicle models and
system identification [22]. Additionally, due to the complexity of
implementing optimisation routines, there is little comparison between MPC
formulations (such as between different dynamics models) or other methods.

Figure 2: Classical approaches use either offline planning with a separate
control module, or online planning and control.

Control: The controller generates actuator commands for speed and steering
that cause the vehicle to track the reference trajectory. The most common
control algorithm is pure pursuit, a geometric path tracking algorithm based
on the vehicle dynamics [23]. The method has been advanced by using a speed-
dependant lookahead distance [11], and integrating the tyre dynamics into the
algorithm to improve tracking at high speeds [18]. Model predictive control
(MPC) approaches have also been used for control [24, 25], with the advantage
that they can perform online system identification [26], thus updating the
model as more data is collected [27]. Despite these advantages, the pure
pursuit algorithm remains the most common control algorithm due to its
robustness and simplicity [10, 5, 28].

TABLE I: Comparative analysis of autonomous racing methods, evaluation platforms, and comparisons used. | Reference | Method/Novelty | Category | Simulation |  Comparison | Real World
---|---|---|---|---|---|---
Classical |  O’Kelly et at. (2020) [10, 16] | Genetic optimisation | Offline planing | F1TENTH Gym |  Gradient-based trajectory optimisation [14] | ✓
Li et al. (2022) [21] | Multi-vehicle MPC | Online planning | Custom |  ✗ | ✗
Curinga (2018) [19] | MPCC | Online planning | Custom |  Optimisation & tracking | ✗
Bhargav et al. (2021) [20] | MPCC for overtaking | Online planning | F1TENTH Gym |  ✗ | ✗
Jain et al. (2020) [27] | MPC with gaussian process | Control | F1TENTH Gym |  MPC | ✗
Cataffo et al. (2022) [25] | MPC for obstacle avoidance | Control | F1Tenth.dev |  ✗ | ✗
Wang et al. (2021) [26] | Data-driven MPC | Control | Gazebo |  MPC, adaptive pure pursuit | ✓
Nagy et al. (2023) [24] | Adaptive MPC | Control | F1TENTH Gym |  Pure pursuit | ✓
Sukhil and Behl (2021) [11] | Adaptive pure pursuit | Control | F1Tenth.dev |  Pure pursuit | ✓
Becker et al. (2023) [18] | Model- & acceleration-based pure pursuit | Control | ✗ |  Pure pursuit | ✓
Sezer et at. (2012) [29] | Follow-the-gap | Mapless | Custom |  APF, A* | ✓
Otterness (2019) [30] | Disparity extender | Mapless | ✗ |  ✗ | ✓
Learning |  Hamilton et al. (2022) [31] | DRL vs IL | End-to-end | Gazebo |  IL, SAC, DDPG | ✓
Brunnbauer et al. (2022) [32] | Model-based DRL | End-to-end | PyBullet |  Model-free DRL | ✓
Bosello et al. (2022) [33] | Generalisable DRL | End-to-end | Rviz |  DRL, Dreamer, FTG | ✓
Sun et al. (2023)[34] | IL algorithm comparison | End-to-end | F1TENTH Gym |  IL, PPO, DAgger, EIL | ✓
Sun et al. (2023) [35] | New IL algorithm | End-to-end | F1TENTH Gym |  IL, experts | ✗
Evans et al. (2023) [36] | Trajectory-aided learning reward | End-to-end | F1TENTH Gym |  Cross-track & heading | ✗
Evans et al. (2023) [7] | Comparing DRL architectures | Comparison | F1TENTH Gym |  Optimisation & tracking | ✓
Tătulea-Codrean et al. (2020) [37] | IL to replace MPC | Learned planning | Gazebo |  MPC | ✗
Ghignone et al. (2022) [22] | Trajectory-conditioned learning | Learned planning | F1TENTH Gym |  MPC | ✓
Dwivedi et al. (2022) [28] | Plan-assisted DRL | Learned planning | PyBullet |  Dreamer, MPO | ✗
Trumpp et al. (2023) [38] | RPL for high-speed racing | RPL | F1TENTH Gym |  Optimisation & tracking | ✗
Evans et al. (2021) [39, 40] | Static obstacle avoidance | RPL | Custom |  FTG, end-to-end DRL | ✗
Zhang et al. (2022) [41] | APF-based RPL | RPL | PyBullet |  FTG, MPC, DRL, Dreamer | ✗
Trumpp et al. (2024) [42] | Mapless RPL racing | RPL | F1TENTH Gym |  APF, disparity extender | ✗
Ivanov et al. (2020) [43] | Formal neural network verification | Safe learning | Custom |  DRL | ✓
Musau et al. (2022) [44] | Reachability Theory safety | Safe learning | Gazebo |  IL, DRL | ✓
Evans et al. (2023) [45, 46] | Viability Theory safety | Safe learning | F1TENTH Gym |  DRL, pure pursuit | ✓

Mapless Methods: A commonly used mapless approach for F1TENTH racing is the
Follow-the-Gap (FTG) controller. Proposed by Sezer et at. [29], this is a
reactive method based on identifying gaps in the car’s LiDAR signal. After
finding the gaps, the car’s steering angle is directly defined as the angle in
direction or the widest or furthest. An extension to FTG is the disparity
extender [30]. In this approach, before detecting the gaps in the LiDAR
signal, all disparities in the LiDAR signal are identified. The distance
readings of the LiDAR signal adjacent in some distance to the disparity are
set to the minimum distance in each interval. Typically, the interval is
chosen as the vehicle width plus a safety margin, thus avoiding trajectories
that cut obstacles closely.

###  II-B Learning Approaches

Deep learning approaches use neural networks to replace a part or all of the
racing pipeline. We study learning approaches in the categories of
architecture, algorithm and reward signal. The deep learning architecture
determines the role of the agent in the racing pipeline. The algorithm denotes
the process of training the neural network from random initialisation to a
policy that controls the vehicle. The reward signal is the method of
communicating the desired behaviour to the agent.

Learning architectures: The learning architecture describes the input (state
vector) given to the agent and the role/use of the actions generated. Fig. 3
shows the end-to-end, planning, residual and safe learning architectures for
autonomous racing. We consider approaches using each architecture.

End-to-end learning uses a neural network to replace the entire racing
pipeline. In F1TENTH racing, common inputs for end-to-end agents are single or
multiple downsampled 2D LiDAR scans and the vehicle speed [34, 32]. Many
studies have addressed constant speed racing (driving) [31, 46], and
implemented their approaches on physical vehicles at low, constant speeds [33,
32]. End-to-end approaches generalise well to unseen tracks [33] and transfer
to physical vehicles [7]. A major challenge for end-to-end learning agents is
reliable performance at high speed [36].

Planning agents learn a policy that incorporates the vehicle’s location and
upcoming track (usually in the form of centre points) into the state vector
[22, 28]. Including the track points in the state has been shown to outperform
end-to-end agents in training stability, sample efficient and racing
performance [7]. Another motivation is to replace an online trajectory planner
(MPC) with a neural network to reduce the computational burden [37, 47]. The
limitation of planning agents is they require localisation, restricting them
to mapped environments and putting them in direct competition with
optimisation approaches.

Residual policy learning (RPL) learns an additional policy that is added to a
classical base policy [38]. Residual policies for racing have been learned on
top of pure pursuit controllers [38] and artificial potential fields [41],
resulting in faster convergence and improved stability. Residual policy
learning has been extended to obstacle avoidance [39] and multi-agent
overtaking manoeuvres [42]. Learning residuals promise to harness the
flexibility of neural networks to improve upon classical components, but have
yet to be deployed on physical vehicles.

Safe learning is a method for training neural network controllers while
simultaneously ensuring vehicle safety [45]. Safe learning approaches use a
supervisor (action mask) to ensure that the controls executed on the vehicle
are safe. Actions masks have used formal methods [43], reachability theory
[44], time-to-collision (TTC) [33] and Viability Theory [45] to select which
actions are safe. Safe learning enables safe deployment on physical hardware,
resulting in many approaches having physical evaluations [43, 33, 44, 46]. A
challenge in safe learning is building action masks that accurately discern
safe and unsafe actions for high-performance systems.

Figure 3: End-to-end, planning, residual, and safe agent architectures for
autonomous racing.

Training algorithms: While the majority of learning approaches are trained
with reinforcement learning (RL) [33, 7, 22], imitation learning (IL) has also
been used [34, 31, 35, 48]. Reinforcement learning trains agents to control a
vehicle in an environment from the experience of receiving a state, selecting
an action and receiving a reward. In contrast, imitation learning trains the
network to mimic the behaviour of an expert, as recorded in a dataset. A study
comparing the two found that RL is more robust to unseen states [31], a result
backed up by the poor completion rates shown by IL algorithms in [34].

For training agents, the most common DRL algorithms have been TD3 [36, 43],
SAC [22, 31], and PPO [38, 41]. Other algorithms that have been used are the
deep-Q-networks (DQN) [33], and Dreamer, a model-based algorithm [32]. TD3 has
shown to outperform SAC and DDPG for continuous control in autonomous racing
[7]. It remains to be studied how off-policy algorithms (TD3) compare to on-
policy algorithms (PPO) for autonomous racing.

Rewards signals: There has been much variation in the reward signals used to
train DRL agents for autonomous racing. The reward signal plays a crucial role
in the training process, but is often simply listed as a minor implementation
detail. One of the most common is the progress reward signal, which rewards
vehicles according to their progress along the track [32]. However, due to its
lack of ability to communicate speed information, rewards that include the
vehicle speed, and punish lateral deviation from the centre line have been
favoured [40, 33]. A trajectory-aided learning (TAL) reward has been proposed
that uses an optimal trajectory to train the agent [36]. Reward signal design
for autonomous racing is difficult because the agent must be encouraged to
race at high-speeds, while not being so aggressive that the vehicle crashes.

###  II-C Literature Summary

Table I presents a study of approaches to F1TENTH racing by identifying their
category, evaluation benchmarks and if tested on a physical vehicle. Since
gradient-based offline trajectory optimisation approaches work well in F1TENTH
racing, few studies have designed such approaches specifically for the F1TENTH
platform. Approaches in online planning lack comparison with other methods and
evaluation on physical vehicles. This is possibly due to the complexity of the
method and the computational burden, making them poorly suited to real-time
embedded platforms. In control, the challenge of high-speed tracking, taking
the vehicle’s non-linear dynamics into account, has been attempted with data-
driven MPC [24, 26], and the model- and acceleration-based pure pursuit
algorithm [18]. However, it is difficult to compare these approaches due to
the numerous design decisions (model choice, track, horizon length) that
differ between papers, and work is required to compare these methods directly.

There are many approaches in all learning categories for F1TENTH racing, with
end-to-end being the largest. While learned planning has shown to be more
robust [7], end-to-end methods have been tested on physical platforms a lot
more [31, 32]. Few papers compare classical and learning-based methods,
demonstrating the field’s fragmentation. It is proposed that this is due to
differing motivations for different methods; classical methods aim for high
performance [27], while learning methods aim for generality [33]. Several
simulators have been used, such as PyBullet [32], Gazebo [37], and F1Tenth.dev
(which is a Gazebo-based simulator) [49]. A positive trend is the widespread
adoption of the F1TENTH Gym simulator, allowing for results to be compared as
demonstrated by [33].

Classical racing approaches using estimation, optimisation and control have
proven highly effective in many contexts, offering both high performance and
safety guarantees. However, their reliance on accurate dynamics models and
apriori track knowledge restricts their applicability in unmodelled and
unmapped contexts. In contrast, the follow-the-gap method requires neither a
map nor a dynamics model but often compromises performance. End-to-end
reinforcement learning agents excel in unmapped settings, allowing neural
networks to learn generalisable racing policies. Yet, concerns about the
robustness and the complexity of their training pipelines persist. Recent
efforts to combine the adaptability of neural networks with the reliability of
classical methods, such as those seen in planning, residual, and safe learning
agents, show promise but have yet to achieve widespread adoption.

##  III Racing Methods

We describe the most common racing algorithms from each category to provide an
overview of different solution approaches. We describe the particle filter
algorithm used for localisation, a trajectory planning and control strategy,
the model predictive contouring algorithm, the follow-the-gap method and end-
to-end deep reinforcement learning.

###  III-A Particle Filter Algorithm

The particle filter (PF) algorithm uses the control input and the 2D LiDAR
scan to estimate the vehicle’s pose on the map [9]. The algorithm is a
recursive Bayesian state estimator designed for non-parametric distributions
by representing the belief of the vehicle’s pose by a set of particles [50].

Figure 4: Motion and measurement updates for the particle filter.

Fig. 4 shows the particle filter process for a vehicle in the environment
shown on the right. A set of particles (blue) represents the belief of the
vehicle’s location. The motion update uses the control input and a vehicle
model are used to update the particles to form the posterior belief (red).
After each measurement from the LiDAR scan, weights (orange) are calculated
for each particle based on the probability of the measurement being correct.
The current pose (purple arrow) is estimated by multiplying the particle
locations with the weights.

###  III-B Trajectory Optimisation and Tracking

We consider a two-stage planner that uses the trajectory optimisation method
presented by [14] and the pure pursuit path tracking algorithm from [23]. The
two-stage nature means a trajectory can be calculated offline and tracked
during the race.

Trajectory optimisation: The input into the optimisation is a list of centre
line points and track widths from the map. Fig. 5 shows a track segment
indicating the centre-line points used to calculate the heading angle $\psi$
and segment length $l$. The minimum curvature path generation minimises the
change in heading angle over the segment length.

Figure 5: Centre line, minimum curvature line, heading angle $\psi$, segment
length $L$, and optimisation variable $\alpha$ and limits
$[\alpha_{\text{min}},\alpha_{\text{max}}]$ for a track segment.

The optimisation variable is $\alpha$, which is the (positive or negative)
distance from the track centre measured along the normal vector (perpendicular
to the heading angle). The minimum and maximum values for $\alpha$ are defined
by the track widths. The optimisation can be written as,

$\displaystyle\text{minimise}_{[\alpha_{1}...\alpha_{N}]}\quad$
$\displaystyle\quad\sum_{i=1}^{N}\bigg{(}\frac{d\psi}{dl}\bigg{)}^{2}$ (1)
subject to
$\displaystyle\quad\alpha_{i}\in[\alpha_{i,\text{min}},\alpha_{i,\text{max}}]$
(2)

The minimum time speed profile is calculated as the maximum speed that keeps
the vehicle within the handling limits. For each point on the path $i$, the
largest speed for the next point $v_{\text{i+1}}$ is calculated as,

$v_{i+1}=\sqrt{v_{i}^{2}\pm 2l_{i}\cdot\mu
a_{\text{max}}(1-v_{i}\kappa_{i})}.$ (3)

The inputs into the calculation are the maximum acceleration (based on the
motors), current speed $v_{i}$, segment length $l_{i}$, maximum vehicle
acceleration $a_{\text{max}}$, path curvature $\kappa_{i}$ and friction
coefficient $\mu$. The $\pm$ indicates that the process is repeated in both
forward and reverse path directions.

Pure pursuit: The pure pursuit path tracking algorithm tracks a reference path
using a lookahead distance to select a point to steer towards. Given a
lookahead point that is a lookahead distance $l_{\text{d}}$ away from the
vehicle, and at a relative heading angle of $\phi$, the steering angle
$\delta$ is calculated as,

$\delta=\arctan\bigg{(}\frac{L\sin(\phi)}{l_{\text{d}}}\bigg{)},$ (4)

where $L$ is the length of the vehicle’s wheelbase. The lookahead distance
$l_{\text{d}}$ has constant and speed-dependant parts. Additionally, we limit
the speed based on the steering angle to prevent the algorithm from selecting
a large steering angle if it is going too quickly. The speed action is the
minimum of the raceline speed $v_{\text{raceline}}$ and the speed that reaches
a lateral force to weight ratio of 1.5, calculated as,

$v=\min(v_{\text{raceline}},\sqrt{1.5\cdot g\cdot L/\tan(|\delta|)}).$ (5)

###  III-C Model Predictive Contouring Control

Model predictive control is a receding horizon strategy that calculates a set
of control inputs that optimise an objective function subject to constraints
[4]. Model predictive contouring control (MPCC) maximises the progress along
the centre line subject to the path being on the track and dynamically
feasible.

The optimisation variables consist of the state and control inputs at each
discrete timestep. The state is represented by position $x,y$ and heading
$\theta$, and the control actions are speed $v$ and steering $\delta$. The
state of centre line progress $s$ and corresponding control of centre line
speed $\dot{s}$ are added, resulting in a 4D state $[x,y,\theta,s]$ and 3D
control $[\delta,v,\dot{s}]$. The initial state is set to the vehicle’s
position, and subsequent states are constrained to be the result of the
control actions according to the kinematic bicycle model. The friction is
constrained by ensuring that the lateral force is smaller than the friction
coefficient $\mu$ multiplied by the vehicle’s weight (mass $m$ times
gravitational force $g$) as,

$v^{2}/L\cdot\tan(|\delta|)m<\mu\cdot mg.$ (6)

Objective: The MPCC objective maximises progress along the path using four
terms: contouring error, lag error, progress, and control regularisation. Fig.
6 shows how the contouring and lag errors are measured.

Figure 6: Lag and contouring errors represent vehicle progress along the path.

The $x$ and $y$ coordinates and heading angle of the track are represented as
functions of progress along the centre line path $X(s),~{}Y(s),~{}\Phi(s)$.
Since the true progress along the reference path $s^{\text{true}}$ cannot be
found, an approximate progress $s$ is used. Fig. 6 shows how the lag error
quantifies the difference between the true and approximate progress. The
contouring error quantifies the vehicle’s perpendicular distance from the
trajectory. For the state $(x,y,\theta,s)$, the linearised contouring and lag
errors ($\epsilon^{c}$ and $\epsilon^{l}$) can be calculated as,

$\displaystyle\epsilon^{\text{c}}$
$\displaystyle=\sin(\Phi(s))(x-X(s))-\cos(\Phi(s))(y-Y(s))$ (7)
$\displaystyle\epsilon^{\text{l}}$
$\displaystyle=-\cos(\Phi(s))(x-X(s))-\sin(\Phi(s))(y-Y(s))$ (8)

The progress speed along the centre line is rewarded to encourage the vehicle
to maximise progress along the track. Finally, a control regularisation term
penalises steering angles, encouraging smooth behaviour. Each term is
weighted, and the weights are tuned to ensure satisfactory performance.

###  III-D Follow-the-gap Method

The follow-the-gap (FTG) method, extended to the disparity extender algorithm,
is a simple, mapless approach to autonomous racing [29, 30]. The algorithm
identifies the nearest boundary/obstacle in the LiDAR scan and excludes a
bubble of the beams around it. Then, the largest visible gap in the LiDAR scan
is identified, and the steering angle is calculated to drive towards the
middle of the gap. The speed is selected using a higher speed (5 m/s) for
small steering angles and a low speed (3 m/s) for high steering angles.

###  III-E End-to-end Deep Reinforcement Learning

End-to-end learning agents for autonomous racing use a neural network to map a
state vector $s$ directly to a control action $a$ [7]. Mathematically, the
mapping is written as a policy $\pi$ that maps a state to an action as
$a=\pi(s)$. The policy is a deep neural network of multiple fully connected
layers. The state vector comprises the previous and current LiDAR scans and
the vehicle speed. The control actions are the speed and steering angle used
to control the vehicle.

Figure 7: Traininng setup for training a reinforcement learning agent to race
in a simulator.

Fig. 7 shows the training setup for an end-to-end reinforcement learning
agent. During training, the agent experiences states, selects actions and
receives rewards. The reward, which indicates how good or bad an action is, is
calculated based on the vehicle’s pose and/or the agent’s action. The
reinforcement learning objective is to select actions that maximise the total
reward [51]. The training is split into episodes where the vehicle is randomly
spawned on the track and must drive until it either crashes or completes a
lap.

We use the TD3 algorithm [52], a continuous control, off-policy, actor-critic
reinforcement learning algorithm. The actor network is the policy that selects
actions, and the critic network learns a Q-value function for each state-
action pair. The critic is trained towards targets calculated by the Bellman
update equation. The actor is updated using the policy gradient that maximises
Q-values.

Figure 8: Illustrations of the cross-track and heading, track progress and
trajectory-aided learning reward signals.

To train the agent to complete laps without crashing, the agent is rewarded
with 1 for completing a lap, punished with -1 if the vehicle crashes, and
given a shaped reward to encourage fast racing behaviour. Fig. 8 illustrates
the three shaped reward signals of cross-track and heading error (CTH), track
progress and trajectory-aided learning (TAL).

The cross-track and heading error (CTH) reward punishes the cross-track
distance $d_{\text{c}}$ and promotes speed in the direction of the track. The
normalised vehicle speed $v_{t}$, heading error $\psi$ and cross-track error
$d_{c}$ are used to calculate the reward as,

$r_{\text{CTH}}=v_{\text{t}}\cos\psi-~{}d_{\text{c}},$ (9)

The track progress reward promotes advancement along the track centre line by
rewarding the progress made during each timestep. The current progress $s_{t}$
and previous progress $s_{t-1}$ are used to calculate the reward as,

$r_{\text{progress}}=s_{t}-s_{t-1},$ (10)

The trajectory-aided learning (TAL) reward trains the agent to mimic the
actions selected by the optimisation and tracking planner described in §III-B
[36]. The agent’s action $\mathbf{u}_{\text{agent}}$ and classical action
(optimisation and tracking) $\mathbf{u}_{\text{classic}}$ is used to calculate
the TAL reward as,

$r_{\text{TAL}}=1-|\mathbf{u}_{\text{agent}}-\mathbf{u}_{\text{classic}}|$
(11)

##  IV Benchmark Evaluation

We benchmark the results described in Section III by,

  1. 1.

Investigating the impact of localisation error and control frequency on racing
performance

  2. 2.

Comparing the CTH, progress and TAL reward signals, and training maps on agent
performance

  3. 3.

Providing benchmark results for future studies

For each test, we present the key quantitative result (i.e. lap time) and then
qualitative data (i.e. speed profile plots) to explain the reasons for the
result.

|  |  |
---|---|---|---
Figure 9: Maps of the AUT, ESP, GBR, and MCO race tracks (left to right).

Methodology: We use the open-source F1TENTH autonomous vehicle simulator [5].
The vehicle is represented by the single-track bicycle model (described in
Appendix A), which describes the state in terms of position, speed, steering
angle, yaw, yaw rate, and slip angle [53]. The 2D LiDAR scanner is simulated
with a ray-casting algorithm that measures the distance to the track boundary
for each beam. The simulator and all the tests are written in Python and run
on an Ubuntu computer. The simulator dynamics are updated at a rate of 100 Hz
in all tests, and unless otherwise stated, the planning frequency is 25 Hz.

We use a set of four F1 race tracks that have been rescaled ($\approx$ 1:20)
for F1TENTH racing and had the widths adjusted to a constant value. These maps
have been widely used in the literature [7, 33, 32], since they provide a
difficult challenge for F1TENTH vehicles and allow for results to be easily
compared. Fig. 9 shows the shape of the circuits in Austria (AUT), Spain
(ESP), Great Britain (GBR) and Monaco (MCO). Appendix B provides the track
length, percentage of straight sections and number of corners for each track.

For all tests, 10 laps are run with starting points randomly selected along
the centre line. All the results are seeded for reproducibility. The following
metrics are used,

  * •

Lap time: the mean of the lap times for successfully completed laps.

  * •

Completion rate: the number of successfully completed laps divided by the
total number of test laps

  * •

Progress: the progress made by the vehicle before crashing (used for learning
tests)

The speed and slip angle profiles and the path taken are analysed to better
understand the results by exposing the vehicle behaviour.

###  IV-A Localisation Error Evaluation

Using a particle filter for pose estimation introduces error into the racing
pipeline that many planning and control solutions ignore. Therefore, we
investigate the impact of localisation by recording the lap times using the
true pose (from the simulator) and the estimated pose. The estimated pose
comes from the particle filter, which was tuned to have 1000 particles (see
Appendix C). We test laps on the AUT track with friction values ranging from
0.5 to 1.0.

Figure 10: Lap times using estimated and true friction values on AUT map.
Figure 11: Speed profiles on AUT using friction coefficients of 0.5 and 1.0

Fig. 10 shows the lap times for the optimisation and tracking and MPCC
planners using the estimated and true poses with varying friction values. The
optimisation and tracking planner completes laps at all the friction values.
The MPCC planner using the true pose can complete laps at all friction values,
but using the estimated pose, only up to 0.6. The range of the optimisation
and tracking planner lap times varies from 22 s to 17 s, while the MPCC
planner from 19 s to 17 s. Fig. 11 qualitatively explains this difference
using segments of the speed profiles with friction coefficients of 0.5 and
1.0. The difference between the optimisation and tracking planner has a large,
near constant speed difference. Conversely, the MPCC has a smaller, irregular
gap. This result demonstrates that the MPCC planner is more affected by errors
in the pose than the optimisation and tracking planner because it selects
higher speeds, even at lower friction values.

Figure 12: Friction and control frequency plotted against completion rate for
the optimisation and tracking planner using the estimated and the true poses.

We investigate how control frequency impacts the effect of localisation using
the particle filter with 200 particles (see Appendix C) to expose the impact
of the error. Fig. 12 shows the optimisation and tracking planner completion
rate, tested on the AUT map with varying friction coefficients and control
frequencies. Using the true pose results in the planner achieving a 100%
completion rate for many tests, even at lower control frequencies. In
contrast, using the estimated pose results in much lower completion rates,
especially at high friction coefficients.

Conclusion 1: Localisation error plays a significant role in lap times and
completion rates, and therefore, tests should explicitly study how perception
error and control frequency affect a planner’s performance.

###  IV-B Training Configuration Evaluation

The learning evaluation compares the progress, CTH and TAL reward signals and
investigates the impact of the training map on DRL agent performance. Three
agents are trained for each reward signal on each map for 50,000 steps.

Figure 13: Average progress during training on the AUT, GBR and MCO tracks.

Fig. 13 shows the mean lap progress during training, with the shaded regions
indicating the minimum and maximum. The TAL reward produces the fastest
training, and the agents reach around 90% average progress in around 15k
steps. The CTH reward produces slower training, and the agents converge to
around 70% average progress. The progress reward results in the agents
converging to low average progresses, around 50%.

Figure 14: Completion rate for DRL agents tested on each map.

Fig. 14 shows the completion rate for each agent on each test map. The
progress agent achieves low completion rates on all the maps. The CTH agents
achieve higher completion rates, with some agents achieving around 80%. The
TAL agents achieve the highest completion rates, with 10 of the tests
achieving a 100% completion rate. The MCO map produces the best
generalisation, with all the agents achieving 100% on all test maps.

Figure 15: Box and whisker plot of the slip angles for a lap on the MCO track.

Fig. 15 shows a box and whisker plot of the slip angles for a lap on the MCO
track. The progress and CTH rewards both have median slip angles of around 4
degrees and maximums of over 14 degrees. These are much higher than the TAL’s
median of 18 degrees and a maximum of 5 degrees. This suggests that the TAL
reward outperforms the CTH and progress reward signals due to its trajectories
having lower slip angles [36].

Conclusion 2: The TAL reward has the highest completion rate and produces the
lowest slip angles.

###  IV-C Benchmark Results

We present benchmark results by providing lap times and supporting speed
profiles. We record the average lap times of the optimisation and tracking,
MPCC, follow-the-gap and end-to-end planners. The classical planners use a
friction coefficient of 0.9 and the true pose from the simulator. The end-to-
end agent is trained on GBR with the TAL reward.

Planner | Map
---|---
AUT |  ESP |  GBR |  MCO
Opti. & tracking |  16.79 |  35.92 |  31.24 |  28.08
MPCC |  16.87 |  39.13 |  35.40 |  31.53
Follow-the-gap |  19.10 |  45.78 |  39.34 |  34.99
End-to-end |  19.94 |  46.37 |  40.22 |  34.93
TABLE II: Mean lap times [s] for the optimisation and tracking, MPCC, follow-
the-gap and end-to-end methods.

Table II presents the mean lap times for the planners on the four test maps.
The optimisation and tracking planner achieved the fastest lap time on all the
maps. The MPCC is 0.1 s slower on the AUT map and ranges to 4.16 s slower on
GBR. The follow-the-gap and end-to-end methods achieve similar lap times,
significantly slower than the other two approaches.

Figure 16: Comparison of the speed profiles for the AUT map.

Fig. 16 shows the speed profiles for each planner on the AUT map. Both
classical planners select a range of speeds, slowing down at times (around
40%) and seeing up to the maximum speed. The follow-the-gap planner has a
maximum speed of 5 m/s and the end-to-end agent only selects speeds of above 6
m/s for short periods before slowing down again.

Figure 17: Trajectories with colour indicating speed on the MCO track

Fig. 17 further explains the behaviour by showing trajectory segments for each
planner on the MCO map. The optimisation and tracking planner selects a smooth
profile that slows down in the corners and speeds up in the straighter
sections. The MPCC planner selects a more jagged speed profile by slowing down
more in the corners. The follow-the-gap method often drives in a straight line
and always cuts to the inside of the corner while slowing down. The end-to-end
planner sometimes speeds up, but not to the full speed, and selects a varying
speed profile that appears to change for no reason.

Conclusion 3: Offline trajectory optimisation and tracking achieves the
fastest lap times for F1TENTH autonomous racing. It is followed by MPCC, with
the follow-the-gap method and end-to-end agents being significantly slower.

##  V Conclusion

This paper addressed the fragmented field of F1TENTH autonomous racing. The
literature study described how different methods have been used for the
subproblems of classical and learning-based approaches. The study explained
how the methods relate to one another, creating a taxonomy for future work to
fit into. The methods of particle filter localisation, trajectory optimisation
and tracking, model predictive contouring control, follow-the-gap and end-to-
end deep reinforcement learning were described. The evaluation highlighted the
importance of reporting on the overlooked factors of localisation error for
classical approaches and reward signals for learning methods. The benchmark
evaluation concluded that trajectory optimisation and tracking solutions
provide the fastest lap times due to optimal speed profile selection.

###  V-A Future Work

Vision-based racing: Almost all racing methods use LiDAR as the main input
modality. While LiDAR’s are convenient since they provide geometric
information about the surrounding environment, they are expensive and poorly
suited to real-world applications. In contrast, cameras are cheaper and
contain more expressive information. Therefore, using cameras for high-
performance control should be investigated. Caie et al. [54] started work in
this direction with end-to-end neural networks; however, their solution is
limited to slow speeds and neglects the key racing challenges.

Full-stack racing: Many approaches have focused on a specific section of the
planning pipeline, made possible due to solution modularity. However, further
research is required to investigate the effect of one section on another. For
example, approaches focusing on planning, should consider the impact of noisy,
delayed localisation information. A key point is that evaluations should
include full-stack performance in addition to submodule comparison tests.

Robustness of end-to-end learning: End-to-end learning has shown excellent
potential in various games such as Grand Turismo Sport [55]. However, these
have not yet been seen in real-world robotics tasks due to a lack of
robustness one end-to-end solutions, i.e. many agents still occasionally
crash. This limitation prevents the widespread use of end-to-end neural
networks. Work in autonomous racing and other robotics fields should address
these challenges and design more robust algorithms with safety guarantees.

Simulation-to-reality transfer: The literature study showed that many
approaches are only tested in simulation. While simulation is helpful since
you can easily compare methods, it neglects the true noisy dynamics of
physical systems. Therefore, studies should focus on the differences between
simulation and real-world results. This is specifically needed in the machine
learning approaches, where simulation testing is common.

Mapless solutions: A majority of approaches, especially in the categories of
optimisation-based planning and control, assume that a map of the environment
is available. While this is often the case in racing, it is not the case in
many real-world applications. Therefore, research should investigate solutions
that do not require a map, and thus promise flexibility. This will promote
flexible, adaptable, general solutions that can be used in other robotics
applications.

Multi-agent racing: The crown of racing is multi-vehicle, competitive racing,
where software stacks compete against each other. Competitive, multi-vehicle
racing, similar to real-world events like F1, is a difficult challenge due to
the large solution space and unpredictability of opponents. Classical and
machine learning approaches should be investigated for their ability to
overtake and avoid other cars. Additionally, benchmark methods and
standardised evaluations should be constructed to advance progress.

## References

  * [1] M. O’Kelly, V. Sukhil, H. Abbas, J. Harkins, C. Kao, Y. V. Pant, R. Mangharam, D. Agarwal, M. Behl, P. Burgio _et al._ , “F1/10: An open-source autonomous cyber-physical platform,” _arXiv preprint arXiv:1901.08567_ , 2019\.
  * [2] J. Betz, H. Zheng, A. Liniger, U. Rosolia, P. Karle, M. Behl, V. Krovi, and R. Mangharam, “Autonomous vehicles on the edge: A survey on autonomous vehicle racing,” _IEEE Open Journal of Intelligent Transportation Systems_ , 2022.
  * [3] J. Betz, T. Betz, F. Fent, M. Geisslinger, A. Heilmeier, L. Hermansdorfer, T. Herrmann, S. Huch, P. Karle, M. Lienkamp _et al._ , “Tum autonomous motorsport: An autonomous racing software for the indy autonomous challenge,” _Journal of Field Robotics_ , vol. 40, no. 4, pp. 783–809, 2023\.
  * [4] A. Liniger, A. Domahidi, and M. Morari, “Optimization-based autonomous racing of 1:43 scale RC cars,” _Optimal Control Applications and Methods_ , vol. 36, no. 5, pp. 628–647, 9 2015.
  * [5] M. O’Kelly, H. Zheng, D. Karthik, and R. Mangharam, “F1tenth: An open-source evaluation environment for continuous control and reinforcement learning,” _Proceedings of Machine Learning Research_ , vol. 123, 2020.
  * [6] J. Betz, H. Zheng, Z. Zang, F. Sauerbeck, K. Walas, V. Dimitrov, M. Behl, R. Zheng, J. Biswas, V. Krovi _et al._ , “Teaching autonomous systems hands-on: Leveraging modular small-scale hardware in the robotics classroom,” _arXiv preprint arXiv:2209.11181_ , 2022.
  * [7] B. D. Evans, H. W. Jordaan, and H. A. Engelbrecht, “Comparing deep reinforcement learning architectures for autonomous racing,” _Machine Learning with Applications_ , vol. 14, p. 100496, 2023.
  * [8] N. Gupta, K. Wilson, and Z. Guo, “Optimizing real-time performances for timed-loop racing under f1tenth,” _arXiv preprint arXiv:2212.04549_ , 2022\.
  * [9] C. H. Walsh and S. Karaman, “Cddt: Fast approximate 2d ray casting for accelerated localization,” in _2018 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2018, pp. 3677–3684.
  * [10] M. O’Kelly, H. Zheng, A. Jain, J. Auckley, K. Luong, and R. Mangharam, “Tunercar: A superoptimization toolchain for autonomous racing,” in _2020 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2020, pp. 5356–5362.
  * [11] V. Sukhil and M. Behl, “Adaptive lookahead pure-pursuit for autonomous racing,” _arXiv preprint arXiv:2111.08873_ , 2021.
  * [12] T. Y. Lim, E. Ghignone, N. Baumann, and M. Magno, “Robustness evaluation of localization techniques for autonomous racing,” 2024.
  * [13] Z. Zang, H. Zheng, J. Betz, and R. Mangharam, “Local_inn: Implicit map representation and localization with invertible neural networks,” in _2023 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2023, pp. 11 742–11 748.
  * [14] A. Heilmeier, A. Wischnewski, L. Hermansdorfer, J. Betz, M. Lienkamp, and B. Lohmann, “Minimum curvature trajectory planning and control for an autonomous race car,” _Vehicle System Dynamics_ , vol. 58, no. 10, pp. 1497–1527, 10 2020.
  * [15] J. Klapálek, A. Novák, M. Sojka, and Z. Hanzálek, “Car racing line optimization with genetic algorithm using approximate homeomorphism,” in _2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. IEEE, 2021, pp. 601–607.
  * [16] H. Zheng, J. Betz, and R. Mangharam, “Gradient-free multi-domain optimization for autonomous systems,” _arXiv preprint arXiv:2202.13525_ , 2022.
  * [17] J. Klapálek, M. Sojka, and Z. Hanzálek, “Comparison of control approaches for autonomous race car model,” in _Proceedings of the FISITA 2021 World Congress_. FISITA-International Federation of Automotive Engineering Societies, 2021.
  * [18] J. Becker, N. Imholz, L. Schwarzenbach, E. Ghignone, N. Baumann, and M. Magno, “Model-and acceleration-based pursuit controller for high-performance autonomous racing,” in _2023 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2023, pp. 5276–5283.
  * [19] F. Curinga, “Autonomous racing using model predictive control,” Master’s thesis, KTH, School of Electrical Engineering and Computer Science (EECS), 2017\.
  * [20] J. Bhargav, J. Betz, H. Zheng, and R. Mangharam, “Track based offline policy learning for overtaking maneuvers with autonomous racecars,” _arXiv preprint arXiv:2107.09782_ , 2021.
  * [21] N. Li, E. Goubault, L. Pautet, and S. Putot, “A real-time nmpc controller for autonomous vehicle racing,” in _2022 6th International Conference on Automation, Control and Robots (ICACR)_. IEEE, 2022, pp. 148–155.
  * [22] E. Ghignone, N. Baumann, and M. Magno, “Tc-driver: A trajectory conditioned reinforcement learning approach to zero-shot autonomous racing,” _Field Robotics_ , vol. 3, no. 1, pp. 637–651, 2023.
  * [23] R. C. Coulter, “Implementation of the pure pursuit path tracking algorithm,” Carnegie-Mellon UNIV Pittsburgh PA Robotics INST, Tech. Rep., 1992.
  * [24] T. Nagy, A. Amine, T. X. Nghiem, U. Rosolia, Z. Zang, and R. Mangharam, “Ensemble gaussian processes for adaptive autonomous driving on multi-friction surfaces,” _arXiv preprint arXiv:2303.13694_ , 2023.
  * [25] V. Cataffo, G. Silano, L. Iannelli, V. Puig, and L. Glielmo, “A nonlinear model predictive control strategy for autonomous racing of scale vehicles,” in _2022 IEEE International Conference on Systems, Man, and Cybernetics (SMC)_. IEEE, 2022, pp. 100–105.
  * [26] R. Wang, Y. Han, and U. Vaidya, “Deep koopman data-driven control framework for autonomous racing,” in _Proc. Int. Conf. Robot. Autom.(ICRA) Workshop Opportunities Challenges Auton. Racing_ , 2021, pp. 1–6.
  * [27] A. Jain, M. O’Kelly, P. Chaudhari, and M. Morari, “Bayesrace: Learning to race autonomously using prior experience,” in _Conference on Robot Learning_. PMLR, 2021, pp. 1918–1929.
  * [28] T. Dwivedi, T. Betz, F. Sauerbeck, P. Manivannan, and M. Lienkamp, “Continuous control of autonomous vehicles using plan-assisted deep reinforcement learning,” in _2022 22nd International Conference on Control, Automation and Systems (ICCAS)_. IEEE, 2022, pp. 244–250.
  * [29] V. Sezer and M. Gokasan, “A novel obstacle avoidance algorithm:“follow the gap method”,” _Robotics and Autonomous Systems_ , vol. 60, no. 9, pp. 1123–1134, 2012.
  * [30] N. Otterness, “The ”Disparity Extender” Algorithm, and F1/Tenth.” [Online]. Available: https://www.nathanotterness.com/2019/04/the-disparity-extender-algorithm-and.html
  * [31] N. Hamilton, P. Musau, D. M. Lopez, and T. T. Johnson, “Zero-shot policy transfer in autonomous racing: Reinforcement learning vs imitation learning,” in _2022 IEEE International Conference on Assured Autonomy (ICAA)_. IEEE, 2022, pp. 11–20.
  * [32] A. Brunnbauer, L. Berducci, A. Brandstatter, M. Lechner, R. Hasani, D. Rus, and R. Grosu, “Latent Imagination Facilitates Zero-Shot Transfer in Autonomous Racing,” _2022 International Conference on Robotics and Automation (ICRA)_ , pp. 7513–7520, 5 2022.
  * [33] M. Bosello, R. Tse, and G. Pau, “Train in austria, race in montecarlo: Generalized rl for cross-track f1 tenth lidar-based races,” in _2022 IEEE 19th Annual Consumer Communications & Networking Conference (CCNC)_. IEEE, 2022, pp. 290–298.
  * [34] X. Sun, M. Zhou, Z. Zhuang, S. Yang, J. Betz, and R. Mangharam, “A benchmark comparison of imitation learning-based control policies for autonomous racing,” in _2023 IEEE Intelligent Vehicles Symposium (IV)_. IEEE, 2023, pp. 1–5.
  * [35] X. Sun, S. Yang, and R. Mangharam, “Mega-dagger: Imitation learning with multiple imperfect experts,” _arXiv preprint arXiv:2303.00638_ , 2023.
  * [36] B. D. Evans, H. A. Engelbrecht, and H. W. Jordaan, “High-speed autonomous racing using trajectory-aided deep reinforcement learning,” _IEEE Robotics and Automation Letters_ , vol. 8, no. 9, pp. 5353–5359, 2023.
  * [37] A. Tătulea-Codrean, T. Mariani, and S. Engell, “Design and simulation of a machine-learning and model predictive control approach to autonomous race driving for the f1/10 platform,” _IFAC-PapersOnLine_ , vol. 53, no. 2, pp. 6031–6036, 2020.
  * [38] R. Trumpp, D. Hoornaert, and M. Caccamo, “Residual policy learning for vehicle control of autonomous racing cars,” in _2023 IEEE Intelligent Vehicles Symposium (IV)_ , 2023, pp. 1–6.
  * [39] B. Evans, H. A. Engelbrecht, and H. W. Jordaan, “Learning the subsystem of local planning for autonomous racing,” in _2021 20th International Conference on Advanced Robotics (ICAR)_. IEEE, 2021, pp. 601–606.
  * [40] ——, “Reward signal design for autonomous racing,” in _2021 20th International Conference on Advanced Robotics (ICAR)_. IEEE, 2021, pp. 455–460.
  * [41] R. Zhang, J. Hou, G. Chen, Z. Li, J. Chen, and A. Knoll, “Residual policy learning facilitates efficient model-free autonomous racing,” _IEEE Robotics and Automation Letters_ , vol. 7, no. 4, pp. 11 625–11 632, 2022.
  * [42] R. Trumpp, E. Javanmardi, J. Nakazato, M. Tsukada, and M. Caccamo, “RaceMOP: Mapless online path planning for multi-agent autonomous racing using residual policy learning,” _Under submission_ , 2024.
  * [43] R. Ivanov, T. J. Carpenter, J. Weimer, R. Alur, G. J. Pappas, and I. Lee, “Case study: verifying the safety of an autonomous racing car with a neural network controller,” in _Proceedings of the 23rd International Conference on Hybrid Systems: Computation and Control_ , 2020, pp. 1–7.
  * [44] P. Musau, N. Hamilton, D. M. Lopez, P. Robinette, and T. T. Johnson, “On using real-time reachability for the safety assurance of machine learning controllers,” in _2022 IEEE International Conference on Assured Autonomy (ICAA)_. IEEE, 2022, pp. 1–10.
  * [45] B. D. Evans, H. W. Jordaan, and H. A. Engelbrecht, “Safe reinforcement learning for high-speed autonomous racing,” _Cognitive Robotics_ , vol. 3, pp. 107–126, 2023.
  * [46] B. D. Evans, J. Betz, H. Zheng, H. A. Engelbrecht, R. Mangharam, and H. W. Jordaan, “Bypassing the simulation-to-reality gap: Online reinforcement learning using a supervisor,” in _2023 21st International Conference on Advanced Robotics (ICAR)_. IEEE, 2023, pp. 325–331.
  * [47] M. Luiza Costa Vianna, E. Goubault, and S. Putot, “Neural Network Based Model Predictive Control for an Autonomous Vehicle,” _arXiv e-prints_ , p. arXiv:2107.14573, Jul. 2021.
  * [48] J. Zhang and H.-W. Loidl, “F1tenth: An over-taking algorithm using machine learning,” in _2023 28th International Conference on Automation and Computing (ICAC)_. IEEE, 2023, pp. 01–06.
  * [49] V. S. Babu and M. Behl, “F1tenth.dev-An Open-source ROS based F1/10 Autonomous Racing Simulator,” in _IEEE International Conference on Automation Science and Engineering_ , vol. 2020-Augus. IEEE Computer Society, 8 2020, pp. 1614–1620.
  * [50] S. Thrun, W. Burgard, and D. Fox, _Probabilistic Robotics_. MIT Press, 2005.
  * [51] R. S. Sutton and A. G. Barto, _Reinforcement learning: An introduction_. MIT press, 2018.
  * [52] S. Fujimoto, H. Hoof, and D. Meger, “Addressing function approximation error in actor-critic methods,” in _International conference on machine learning_. PMLR, 2018, pp. 1587–1596.
  * [53] M. Althoff, M. Koschi, and S. Manzinger, “Commonroad: Composable benchmarks for motion planning on roads,” in _2017 IEEE Intelligent Vehicles Symposium (IV)_. IEEE, 2017, pp. 719–726.
  * [54] P. Cai, H. Wang, H. Huang, Y. Liu, and M. Liu, “Vision-based autonomous car racing using deep imitative reinforcement learning,” _IEEE Robotics and Automation Letters_ , vol. 6, no. 4, pp. 7262–7269, 2021.
  * [55] P. R. Wurman, S. Barrett, K. Kawamoto, J. MacGlashan, K. Subramanian, T. J. Walsh, R. Capobianco, A. Devlic, F. Eckert, F. Fuchs, L. Gilpin, P. Khandelwal, V. Kompella, H. C. Lin, P. MacAlpine, D. Oller, T. Seno, C. Sherstan, M. D. Thomure, H. Aghabozorgi, L. Barrett, R. Douglas, D. Whitehead, P. Dürr, P. Stone, M. Spranger, and H. Kitano, “Outracing champion Gran Turismo drivers with deep reinforcement learning,” _Nature 2022 602:7896_ , vol. 602, no. 7896, pp. 223–228, 2 2022\.

##  VI Biography Section

Benjamin David Evans studied for a bachelor’s in Mechatronic Engineering at
the University of Stellenbosch, graduating in 2019. He started a masters,
focused on using reinforcement learning for autonomous racing, which was
upgraded to a PhD. After graduating in 2023, he continues as a postdoctoral
researcher at Stellenbosch University. His interests include the intersection
of classical control and machine learning for autonomous systems.

Raphael Trumpp graduated with a M.Sc. degree in mechanical engineering from
the Technical University of Munich (Germany) in 2021, where he is currently
pursuing a Ph.D. in informatics. His research focuses on machine learning,
especially combining deep reinforcement learning with classical control
methods for interactive multi-agent scenarios.

Marco Caccamo earned his Ph.D. in computer engineering from Scuola Superiore
Sant’Anna (Italy) in 2002. Shortly after graduation, he joined University of
Illinois at Urbana-Champaign as assistant professor in Computer Science and
was promoted to full professor in 2014. Since 2018, Prof. Caccamo has been
appointed to the chair of Cyber-Physical Systems in Production Engineering at
Technical University of Munich (Germany). In 2003, he was awarded an NSF
CAREER Award. He is a recipient of the Alexander von Humboldt Professorship
and he is IEEE Fellow.

Hendrick Willem Jordaan received his bachelor’s in Electrical and Electronic
Engineering with Computer Science and continued to receive his Ph.D degree in
satellite control at Stellenbosch University. He currently acts as a senior
lecturer at Stellenbosch University and is involved in several research
projects regarding advanced control systems as applied to different autonomous
vehicles. His interests include robust and adaptive control systems applied to
practical vehicles. He is a senior member of IEEE.

Herman Engelbrecht received his Ph.D. degree in Electronic Engineering from
Stellenbosch University (South Africa) in 2007. He is currently the Chair of
the Department of Electrical and Electronic Engineering. His research
interests include distributed systems (specifically infrastructure to support
massive multi-user virtual environments) and machine learning (specifically
deep reinforcement learning). Prof Engelbrecht is a Senior Member of the IEEE
and a Member of the ACM.

##  Appendix A Single-Track Model

The F1TENTH-gym’s simulation is based on a single-track model derived from the
CommonRoad framework [53]. This model simplifies the vehicle kinematics to a
two-wheel system but allows for under- and oversteering by introducing a slip
angle $\beta$ and a friction coefficient $\mu$. It accounts for load transfer
between axles, assumes no constant velocity, and is suitable for simulating
manoeuvres close to the vehicle’s physical driving limit.

For a global position $[x,y]$ with yaw angle $\theta$, speed $v$ and steering
angle $\delta$, the state is defined as

$\small\vec{x}=\begin{bmatrix}x&y&\delta&v&\theta&\dot{\theta}&\beta\end{bmatrix}^{\top}.$
(12)

The inputs into the system $u_{1}$ and $u_{2}$ are the steering angle velocity
$v_{\delta}$, and longitudinal acceleration $a$. Using the parameters of
vehicle mass $m$, cornering stiffness $C_{s}$ and wheelbase length $l$ for the
front f and rear r, the height of the centre of gravity $h_{\textbf{cg}}$, and
gravitational constant $g$, the state derivatives are given by,

$\displaystyle\dot{x}_{1}$
$\displaystyle=x_{4}\cos(x_{5}+x_{7}),\quad\quad\dot{x}_{2}=x_{4}\sin(x_{5}+x_{7}),$
(13) $\displaystyle\dot{x}_{3}$
$\displaystyle=u_{1},\quad\quad\dot{x}_{4}=u_{2},\quad\quad\dot{x}_{5}=x_{6}$
$\displaystyle\dot{x}_{6}$ $\displaystyle=\frac{\mu
m}{I_{z}(l_{r}+l_{f})}\big{[}l_{f}C_{S,f}(gl_{r}-u_{2}h_{cg})\cdot x_{3}$
$\displaystyle\quad+(l_{r}C_{S,r}(gl_{f}+u_{2}h_{cg})-l_{f}C_{S,f}(gl_{r}-u_{2}h_{cg}))\cdot
x_{7}$
$\displaystyle\quad-(l_{f}^{2}C_{S,f}(gl_{r}-u_{2}h_{cg})+l_{r}^{2}C_{S,r}(gl_{f}+u_{2}h_{cg}))\cdot\frac{x_{6}}{x_{4}}\big{]},$
$\displaystyle\dot{x}_{7}$
$\displaystyle=\frac{\mu}{x_{4}(l_{r}+l_{f})}\big{[}C_{S,f}(gl_{r}-u_{2}h_{cg})\cdot
x_{3}$
$\displaystyle\quad-(C_{S,r}(gl_{f}+u_{2}h_{cg})+C_{S,f}(gl_{r}-u_{2}h_{cg}))\cdot
x_{7}$
$\displaystyle\quad+(C_{S,r}(gl_{f}+u_{2}h_{cg})l_{r}-C_{S,f}(gl_{r}-u_{2}h_{cg})l_{f})\cdot\frac{x_{6}}{x_{4}}\big{]}$
$\displaystyle\quad-x_{6}.$

Due to singularities, a kinematic model is used for $|v|<0.1$.

##  Appendix B Map Characteristics

We quantify the characteristics of these maps by providing the length,
straight percentage, and number of corners in Table III. The straight
percentage is the length of the track with curvature below 0.1 rad/m divided
by the total track length. The number of corners is the number of sections
with curvature above 0.6 rad/m. The table shows that the ESP is the longest
track with a length of 236.93 m, AUT has the highest % of straight sections
and MCO has 16 corners, more than double the other maps.

Map name |  AUT |  ESP |  GBR |  MCO
---|---|---|---|---
Track length [m] |  94.90 |  236.93 |  201.84 |  178.71
Straight length [%] |  64.92 |  58.98 |  59.45 |  60.60
Corner count |  7 |  7 |  7 |  16
TABLE III: Quantifiable track characteristics

##  Appendix C Particle Filter Tuning

We ran a test lap using the pure pursuit algorithm following the centre line
at a low constant speed of 2 m/s, and evaluated the accuracy using increasing
numbers of particles. We present Fig. 18 to indicate the impact of the number
of particles on the localisation accuracy and computation time. The graph
shows that the error drops from 10 cm for 50 particles to around 4 cm for 1000
particles. Using 1000 particles has a computation time of 10 ms, which is
below our planning period of 40 ms. Therefore, we use 1000 particles for our
tests.

Figure 18: Localisation error and computation time w.r.t the number of
particles.