# Vehicle Type Specific Waypoint Generation

Yunpeng Liu1,2 Jonathan Wilder Lavington1,2 Adam Ścibior1,2 Frank Wood1,2,3
1Inverted AI, 2University of British Columbia, 3Mila

###### Abstract

We develop a generic mechanism for generating vehicle-type specific sequences
of waypoints from a probabilistic foundation model of driving behavior. Many
foundation behavior models are trained on data that does not include vehicle
information, which limits their utility in downstream applications such as
planning. Our novel methodology conditionally specializes such a behavior
predictive model to a vehicle-type by utilizing byproducts of the
reinforcement learning algorithms used to produce vehicle specific
controllers. We show how to compose a vehicle specific value function estimate
with a generic probabilistic behavior model to generate vehicle-type specific
waypoint sequences that are more likely to be physically plausible then their
vehicle-agnostic counterparts.

##  I INTRODUCTION

Behavior models [1] are useful for planning [2] and control [3]. One use of
such models is to do route planning in the presence of other stochastic agents
whose types and goals may not be known [4]. Another is to, for instance,
conditionally propose a sequence of waypoints to be followed by a controlled
ego vehicle which can be used to evaluate the safety or performance of another
egocentric controller [5]. In all cases the sequence of generated waypoints is
generally handed to a lower-level policy that actuates the physical controls
of the vehicle to “achieve” or “follow” these waypoints [6]. In this work we
address the discrepancy in performance which can be observed when a “generic”
behavior model, trained to conditionally propose a sequence of waypoints to be
followed for the purpose of model evaluation and testing, is applied to a
vehicle-type specific lower-level controller.

In this setting, our generic behavior model is defined with a probabilistic
conditional model of multi-agent trajectories trained on data that does not
include labelling of vehicle specific parameters [7]. Realistically such
labeled data can’t be collected in the real world as it would require having
direct access to labeled measurements of vehicle parameters for every vehicle
encountered. For this reason, behavior models are usually trained on overhead
bird-view data [8] or ego-centric bounding box data [9]. Here, only basic
information of each vehicle is known, and nothing about the vehicles’ masses,
torque curves, or other parameters is directly observable.

Such behavior models can be framed as multi-agent policies, and are defined as
a mapping from a history of observations to an action that determines the next
state. Typically in this setting, state includes all agent positions and
orientations and a map in some representation. In some cases, such policies
are even explicitly factorized as the product of individual agents interacting
in a shared, deterministic world [8, 10]. When the agents that comprise these
models cannot directly observe aspects of true world state, they are referred
to as partially observing agents. Generic foundation behavior models are
partially observing in that they are often blind to vehicle specific dynamics,
and as a result, must form an internal “belief" over what the state of the
system is, including the types of all vehicles, given the observations they
can make. The internal uncertainty of such models can additionally stem from
the multi-agent setting itself, where the policy must infer driver intent
[11], style, and more [12]. Such partially observing behavior models become
particularly problematic when they are used to provide waypoints for a lower-
level controller of a particular vehicle-type to follow. Consider the
following example: in self-driving car simulations, it is desirable for all
non-ego agents to behave in highly realistic ways to avoid sim-to-real issues
in training and validation [13]. This means both that the vehicles must behave
realistically with respect to physical behavior (accelerations, steering
alignments, and wheel rotations all being physically realistic), while also
being behaviorally realistic (with the patterns and distribution of behaviors
matching human-like behaviors). A generic behavior model can generally produce
realistically distributed behaviors in aggregate, but often times cannot
produce physically realistic driving between those waypoints. To ensure its
distribution over waypoints is physically tractable, such a model must be
coupled to a lower-level controller that has been trained to follow sequences
of waypoints through interaction with a realistic version of vehicle dynamics
and road conditions [6].

The problem we address in this work arises at this intersection: we show how
to condition a behavioral model so as to generate sequences of waypoints that
are more likely to be achievable by specific vehicle types. For instance, if
we know that the vehicle to be controlled is a heavy truck with a weak engine,
we would like to automatically produce behaviorally diverse and realistic
waypoint sequences that are dynamically achievable by this heavy truck using a
lower-level controller trained to drive such a vehicle.

To achieve this, we first generate a dataset of achievable waypoints for
several vehicle types by manually driving each vehicle in simulation. This
manual driving is done such that we explore a wide envelope of dynamics for
each vehicle type. From these examples, we construct a “waypoint following”
reinforcement learning environment in which we train a vehicle specific policy
and value function that learns how to apply steering and acceleration
controls. More specifically, this vehicle learns to take actions so as to
match reference trajectories at all points in both space and time, given both
the current state, and a sub-sequence of goal waypoints ahead. We then show
how to use the learned value function for a particular vehicle type to
condition a behavioral model to produce waypoint sequences that are achievable
trajectories for the associated vehicle type. We integrate our work into the
CARLA [14] open source driving simulator.

Figure 1: Top row Importance sampled, feasible trajectories are marked in blue
for the waypoint following agent of four different vehicle types (Tesla, Jeep,
shipping truck, fire truck) in the CARLA simulator. Trajectories marked in red
represent trajectories with weights that are close to zero, and thus are
unlikely to be feasible. Bottom Row Manually collected expert trajectories
marked in blue to cover the agent’s full action space inside the waypoint
following environment. RL agent rollouts for each scenario are marked in red.
Scenarios are similar across vehicle types.

##  II METHODS

Our methodology couples solutions to three sub-problems: defining an
environment for training low-level vehicle controllers, using RL to train low-
level waypoint following agents within this environment, and using artifacts
from those training processes within a Bayesian procedure for conditioning a
foundation behavioral model to produce vehicle specific waypoints. We start by
defining the waypoint following environment in which we train our vehicle-type
specific, low-level controllers. We then describe the reinforcement learning
(RL) approach we use, in particular defining the reward, the Markov decision
process (MDP), the value function, and the choice of RL algorithm. We then
review the approximate inference perspective on RL and explain our final,
novel step. In this step we combine a vehicle-type specific value function
with a foundation behavior model to form an approximate posterior distribution
over trajectories that are achievable by a particular vehicle-type and its
associated controller.

###  II-A Waypoint Following Environment

In this paper we define a target trajectory as sequence of waypoints
$k_{1:T}$, where each waypoint is a tuple of target coordinates, speed, and
orientation $k_{t}=(x_{t}^{k},y_{t}^{k},v_{t}^{k},\psi_{t}^{k})$ for $t\in
1,\dots,T$. The waypoint following task is to produce a sequence of actions
$a_{t},t=1,\ldots,T$, consisting of vehicle controls. These controls consist
of steering, throttle, and brake, and must produce a sequence of realized
vehicle positions $\\{x^{s}_{t},y^{s}_{t}\\}_{t=0}^{T}$ which resemble the
target trajectory as closely as possible. Our notion of success in following a
target trajectory is that the vehicle stays within a distance $\epsilon$ away
from its reference trajectory at all time-steps t. That is, we say that a
vehicle successfully followed a target trajectory if for all $t$,

$\displaystyle
d_{t}:=\sqrt{(x_{t}^{s}\\!\\!-\\!\\!x_{t}^{k})^{2}+(y_{t}^{s}\\!\\!-\\!\\!y_{t}^{k})^{2}}\leq\epsilon,$
(1)

where the superscript $s$ indicates the positions generated through
interaction with the simulator, and $k$ defines the desired positions in the
reference trajectory. If there exists a sequence of actions that results in a
vehicle successfully following a trajectory from its initial state, we say
that this trajectory is feasible for the particular vehicle from that specific
initial state. A waypoint following environment consists of one or more
waypoint following tasks which we also refer to as scenarios. We construct
waypoint following environments for each vehicle type we consider where target
trajectories are generated by manual driving. These manually driven
trajectories are picked to maximize state-space coverage to ensure proper
generalization of the policy.

Certain target trajectories may not be feasible due to the dynamics and
kinematic characteristics of a particular vehicle type. For example, maximum
torque or minimum turning radius vary between most vehicle-types and these
among other things heavily influence feasibility of trajectories. Which
trajectories are feasible therefore depends on which vehicle is trying to
follow them. We assume a setting where we have access to a generic
probabilistic model of target trajectories learned by observing the motion of
multiple vehicle-types, and our goal is to produce a model of target
trajectories that can be followed by a specific vehicle-type $e$. In order to
do that, we first describe the RL algorithm and MDP used to learn a vehicle-
specific waypoint following policy. We then show that the artifacts obtained
in this process can be used to construct vehicle-specific target trajectory
models without further interaction with the simulator.

###  II-B Reinforcement Learning for Waypoint Following

First we construct a waypoint following policy $\pi(a_{t}|s_{t})$, which
creates controls based on the vehicle’s state and the target trajectory. In
order to use RL to construct a waypoint following policy, we need to frame the
waypoint following problem as an MDP. Here actions are defined to be the
vehicle controls, i.e. steering angle and single combined pedal
acceleration/braking control (where negative values indicate braking and
positive values indicate acceleration). The reward is then defined as the
negated distance to the target trajectory

$\displaystyle\\!\\!r_{t}=\\!\\!\left(\epsilon\\!-\\!\sqrt{(x_{t}^{s}\\!\\!-\\!\\!x_{t}^{k})^{2}+(y_{t}^{s}\\!\\!-\\!\\!y_{t}^{k})^{2}}\right).$
(2)

The full MDP state contains the vehicle’s absolute position, orientation,
velocity, the target trajectory, and any internal vehicle state, in particular
that of the engine, which may influence its future dynamics. To simplify the
state representation, and ensure that the state can be applied to arbitrary
vehicle types, we consider six primary features: the longitudinal and lateral
velocity of the vehicle, its gear, and the relative longitudinal and lateral
position and orientation of the next target waypoint in the reference frame
attached to the vehicle. Additionally, to allow the policy to better
anticipate future position and heading requirements, we additionally provide
it with a window of $H$ target waypoints into the future. We denote this
state-space of size $3(H+1)$ as

$\displaystyle s_{t}$
$\displaystyle=\left[v_{lon,t}^{s},v_{lat,t}^{s},g_{t},\Delta
x^{s}_{t:t+H},\Delta y^{s}_{t:t+H},\Delta\psi^{s}_{t:t+H}\right],$ (3)
$\displaystyle\\!\\!\\!\\!\\!\\!\\!\\!\begin{bmatrix}\Delta x^{s}_{t+j}\\\
\Delta y^{s}_{t+j}\\\
\end{bmatrix}=\begin{bmatrix}(x^{k}_{t+j+1}-x^{s}_{t})\\\
(y^{k}_{t+j+1}-y^{s}_{t})\\\
\end{bmatrix}^{\top}\\!\\!\begin{bmatrix}cos(\psi^{s}_{t})\
-sin(\psi^{s}_{t})\\\ sin(\psi^{s}_{t})\ \ cos(\psi^{s}_{t})\\\
\end{bmatrix},$ (4)

for each time index displacement $j<H$, with
$\Delta\psi^{s}_{t+j}=(\psi^{k}_{t+j+1}-\psi^{s}_{t})$. We also assume a fixed
distribution over initial states, defined by the initial velocity, position,
orientation, and gear of the vehicle. Finally, we assume that the transition
dynamics are defined by a deterministic external driving simulator (e.g. CARLA
[14] or NVIDIA Drive Sim [15]). For convenience we also define
$\tau_{t}:=\\{a_{t},s_{t},r_{t}\\}$, $\tau=\tau_{1:T}$, and the distribution
over fixed time-horizon trajectories

$\displaystyle
q_{\theta}(\tau)=p(s_{0})\prod\nolimits_{t=0}^{T}p(s_{t+1}|s_{t},a_{t})\pi_{\theta}(a_{t}|s_{t}),$
(5)

for a particular parametric policy $\pi_{\theta}$. In this paper we
incorporate an additional termination condition defined by Equation 1, as well
as a maximum time-limit for interaction.

For a fixed horizon MDP, the optimal policy is one which maximizes the
expected reward ahead. When conditioned on state, this expectation defines the
value function or critic

$\displaystyle V^{\pi_{\theta}}(s_{t})=\mathop{\mathbb{E}}_{\tau_{t:T}\sim
q_{\pi_{\theta}}(\tau_{t:T}|s_{t})}\left[{\sum_{t^{\prime}=t}^{T}r_{t^{\prime}}(a_{t}^{\prime},s_{t}^{\prime})}\right].$
(6)

As is typical in RL, we learn a parametric approximation to this value
function using a neural network $V^{\phi}(s_{t})$ with parameter $\phi$, which
takes the same inputs as the policy. Generally the value function is trained
jointly with the policy, and in this work we specifically use Proximal Policy
Optimization [16]. Training separately for each vehicle type $e$, we obtain
the approximately optimal waypoint following policy $\pi_{\theta_{e}}$ and the
associated value function $V^{\phi_{e}}$. Figure 2(b) shows that policies
obtained in this fashion successfully follow target trajectories known to be
feasible.

###  II-C Reinforcement Learning as Inference

We will briefly discuss the RL as inference (RLAI) framework [17, 18] and
define the mathematical tools used in later subsections. In this setting, RL
is posed as an approximate posterior inference problem over the set of
trajectories conditioned on how well the agent should perform. This creates an
inference problem defined by a set of latent random variables (the
trajectories), and a set of observed random variables describing performance
which are conditionally dependent on the state-action pair at time t. These
observed random variables are called “optimality" variables $O_{t}$, and are
assumed to be Bernoulli distributed. When $O_{t}=1$ we say that the associated
state-action was optimal, while if $O_{t}=0$ we say the state-action was sub-
optimal. The parameters of these Bernoulli distributions are

$\displaystyle p(O_{t}=1|a_{t},s_{t})$
$\displaystyle=\frac{1}{Z_{0}}\exp{r(a_{t},s_{t})},$ (7)

where the normalizing constant $Z_{0}$ is dependent on the range of values the
reward can take on. With optimality defined, we can consider the probability
that a given trajectory, the sequence of state action pairs sampled from a
policy interacting with the environment $\tau$, is itself optimal. For finite
horizon problems, this joint distribution over both latent and observed random
variables, $p(\tau,O_{1:T-1})$, is defined

$\displaystyle
p(s_{1})\prod_{t=1}^{T-1}\Big{[}p(s_{t+1}|s_{t},a_{t})\pi_{0}(a_{t}|s_{t})p(O_{t}|\tau_{t})\Big{]},$
(8)

where $\pi_{0}(a_{t}|s_{t})$ denotes the ‘default’ policy or prior over
actions given states. In general, using this joint distribution to derive an
exact posterior is intractable, therefore we rely on variational inference or
Monte-Carlo methods.

###  II-D Evaluating Feasibility

We are interested in the distribution of trajectories reachable by a
particular vehicle type $e$. We define a posterior over feasible trajectories,
conditioned on optimality, initial conditions, and the vehicle type $e$. Here
the optimality variables depend on how well the agent was able to reproduce
the reference policy through the definition of the reward. The initial
conditions $c=(c_{-U},\ldots,c_{0})$ consist of a ‘burn-in’ sequence of states
of length $U+1$ where each element is similar in nature to part of $s_{t}$
combined with a waypoint $k_{t}^{j}$ from some empirical distribution over
burn-in waypoints

$c_{t}=[v_{lon,t}^{s},v_{lat,t}^{s},g_{t},x_{t}^{j},y_{t}^{j},v_{t}^{j},\psi_{t}^{j}].$
(9)

Which means the posterior is defined according to

$p_{e}(k_{1:T}|O_{1:T},c)=\frac{p_{e}(O_{1:T}|k_{1:T},c)p(k_{1:T}|c)}{p_{e}(O_{1:T}|c)}.$
(10)

In order to characterize this distribution, we start with an estimate of the
likelihood term $p_{e}(O_{1:T}|c,k_{1:T})$, which we arrive at via the the
probabilistic value function [19, 18]

$\displaystyle p_{e}(\textsl{O}_{1:T}|k_{1:T},c)=\int
p_{e}(O_{1:T},\tau|k_{1:T},c)d{\tau}$
$\displaystyle=\int\prod_{t=0}^{T-1}\frac{\exp\left(r(s_{t},a_{t})\right)}{Z_{0}}\pi_{\theta_{e}}(a_{t}|s_{t})p(s_{t+1}|s_{t},a_{t})\mathbb{1}_{[d_{t}\leq\epsilon]}d{\tau}$
$\displaystyle=\frac{1}{Z}\mathop{\mathbb{E}}_{\tau|s_{0}\sim
q_{\pi_{\theta_{e}}}}\left[{\exp\left[\sum^{T-1}_{t=0}r_{t}(a_{t},s_{t})\right]}\right],$

where $Z=\prod_{t=0}^{T-1}Z_{0}$. As a reminder, note that the reference
trajectory $k_{1:T}$ is consumed by both $r(s_{t},a_{t})$ and $d_{t}$. Here we
include an indicator function $\mathbb{1}_{[d_{t}\leq\epsilon]}$ to denote
that the posterior only includes support over trajectories which are feasible.
We then take the log of both sides, to give a expectation that is more
familiar

$\displaystyle\log p_{e}(\textsl{O}_{1:T}|k_{1:T},c)=\log
p_{e}(O_{1:T}|s_{0})$
$\displaystyle=\log\left[\mathop{\mathbb{E}}_{\tau|s_{0}\sim
q_{\pi_{\theta_{e}}}}\left[{\exp\left[\sum^{T-1}_{t=0}r_{t}(a_{t},s_{t})\right]}\right]\right]-\log(Z).$
(11)

Using Jensen’s inequality we recover an upper bound to the artifact that
classical RL algorithms train

$\displaystyle\log p_{e}(O_{1:T}|s_{0})$
$\displaystyle\geq\mathop{\mathbb{E}}_{\tau|s_{0}\sim
q_{\pi_{\theta_{e}}}}\left[{\sum^{T-1}_{t=0}r_{t}}\right]-\log(Z)$ (12)
$\displaystyle=V^{\pi_{\theta_{e}}}(s_{0})-\log(Z)$ (13)

Using this value function, we produce a surrogate likelihood function along
with its associated normalizing constant,

$\displaystyle p_{e}(O_{1:T}|c)$
$\displaystyle=\mathop{\mathbb{E}}_{p_{e}(k_{1:T}|c)}\left[\exp(V^{\pi_{\theta_{e}}}(s_{0})\right]-\log(Z).$
(14)

Because both $p(O_{1:T}|c)$ and $p(O_{1:T}|s_{0})$ are now easy to estimate
and $p(s_{0}|c)$ can be sampled from directly, we use importance sampling to
gather examples which are close to the desired posterior distribution

$p_{e}(k_{1:T}|O_{1:T},c)\approx\frac{\exp(V^{\pi_{\theta_{e}}}(s_{0}))p_{e}(s_{0}|c)}{\mathop{\mathbb{E}}_{p_{e}(k_{1:T}|c)}\left[\exp(V^{\pi_{\theta_{e}}}(s_{0})\right]}.$
(15)

Using the right hand side of (15) to score, we apply importance weighting on
top of the vehicle planner proposals to increase the percentage of feasible
trajectories generated for a particular vehicle type, as shown in Algorithm 1.

Algorithm 1 Planner Refinement Via Re-sampling

1:Input: Planner $p(k_{1:T})$, Value function $V_{\pi_{\theta_{e}}}(s_{t})$,
Number of Importance Samples $L$

2:for $\ell=0,1,2...$ do

3:Sample rollout $k_{1:T}^{\ell}$ from $p(k_{1:T}|c)$

4:Evaluate un-normalized $w$ using equation 15, store.

5:end for

6:Select a rollout $k_{1:T}^{i^{\ell}}$ by sampling an index with probability
proportional to the weights  (16)

7:Output: Deploy agent using ITRA trajectory $k_{1:T}^{i^{\ell}}$.

##  III EXPERIMENTS

###  III-A Performance of waypoint follower

To obtain the value function for each vehicle type, we train four waypoint
following agents which include: a fire truck, a small shipping truck, a Tesla
Model 3 and a Jeep Wrangler. Each of these vehicles is provided by CARLA [14],
a high fidelity simulator for autonomous driving. It provides a variety of
vehicle blueprints with different dynamics. To train a vehicle specific
waypoint following agent, we collect vehicle specific waypoints inside CARLA
by driving manually as shown in Figure 1. We collect four different scenarios
which were designed to cover the agent’s full action space for each vehicle
type. The scenarios include left-turn, right-turn, fully-stopping and an
S-shaped trajectory. We record the following attributes for each vehicle type:
position, yaw, velocity and action (steering angle and throttle) at each frame
given by the simulator. Each record consists of 120 to 180 waypoints.

We train each vehicle to match manually gathered trajectories using proximal
policy optimization (PPO) [16]. As stated in section II, we include in the
state-space $H=30$ future goals for both training and evaluation. These future
goals are pre-processed by a multilayer perceptron(MLP) layer before passing
into the value function and policy network to stabilize the training. Figure
2(b) shows the average percentage of waypoints hit for the 4 scenarios used in
training by the RL agents of the 4 vehicles. We note that the fire truck
waypoint following agent only solves 3 out of 4 scenarios. We assume this
follows from the fire truck being the least controllable vehicle, due to its
longer braking distance, larger turning radius and slower acceleration
compared to the other vehicle types. We note also that the accuracy of
predicted values, given in the value loss in Figure 2(a), often provides a
clear indication of how well that value function will score the feasibility of
proposed trajectories.

###  III-B Performance of planner refinement

####  III-B1 Datasets

For each vehicle type, we record four different initial trajectories generated
by executing CARLA autopilot over a fixed time interval, from a fixed
location. Using these trajectories, we warm-start our behavioral model by
executing the waypoint-following policy to the initial condition of interest,
described in II. From this position, and using the previous data along that
trajectory, we then produce an observed sequence of positions and velocities
using our behavioral model. The examples are generated in a non-trivial
setting, where the agent must complete exiting a roundabout or turn at an
intersection. This experiment is executed for four auto-pilot trajectories and
applied to four vehicles. We generate nine second trajectories given one
second of observations at ten frames per second from ITRA following the
kinematic parameter and evaluate what percentage of this route can be
completed by the pretrained controller.

####  III-B2 Evaluation Details

Following the initial burn in period described above, we sample 100
trajectories from the prior behavioral model ITRA for each vehicle type at
10Hz. We then execute the waypoint follower on these trajectories to get the
percentage of the route completed for each of vehicle type. Rejection sampling
is applied to the prior to avoid any off-road examples at evaluation time. The
average percentage of waypoints hit for this baseline is shown this “prior"
row of Table 2(c). To evaluate how effectively the value function would have
avoided infeasible routes, we calculate the importance weighted expected value
given in Table 2(c).

(a)

(b)

Vehicle Type | $\epsilon$ | Distribution | IC 1 | IC 2 | IC 3 | IC 4
---|---|---|---|---|---|---
Truck | 1.0 | Prior | 0.9 | 1.0 | 0.8 | 0.9
| 1.0 | Posterior | 1.0 | 1.0 | 1.0 | 1.0
Model3 | 1.0 | Prior | 1.0 | 1.0 | 0.9 | 0.9
| 1.0 | Posterior | 1.0 | 1.0 | 1.0 | 1.0
Jeep | 1.0 | Prior | 1.0 | 1.0 | 0.8 | 1.0
| 1.0 | Posterior | 1.0 | 1.0 | 1.0 | 1.0
Fire Truck | 1.5 | Prior | 0.7 | 0.7 | 0.6 | 0.6
| 1.5 | Posterior | 0.9 | 0.8 | 0.7 | 0.8

(c)

Figure 2: Figure 2(a) displays the value function MSE loss for 4 different
vehicle types. The value loss for the Jeep waypoint following agent fluctuates
more than the other agents but eventually converges as it is sensitive to
control inputs. Figure 2(b) displays average percentage of waypoints solved
across 4 different scenarios for all vehicle specific waypoint following
agents. The waypoint following agent for the fire truck reaches all waypoints
for 3 of 4 training scenarios but only reaches half of the waypoints for the
sharp-braking scenario. Table 2(c) shows mean percentage of waypoints hit for
the posterior $p_{e}(k_{1:T}|O_{1:T},c)$ compared to the prior $p(k_{1:T}|c)$
for different vehicle types $e$, and end conditions defined by Equation 1
given the distance $\epsilon$. Note that the Fire Truck end conditions are
loosened to improve the interpretability of our results. Initial
conditions(IC) 1 and 2 are exiting a roundabout and IC 3 and 4 are completing
a turn. The Truck is able to follow posterior trajectories versus trajectories
generated by the underlying behavior model prior across all initial
conditions. The Fire Truck is able to follow more waypoints of the posterior
than the prior even though its RL agent only solves 3 out of 4 training
scenarios. We find that the Jeep and Tesla agents were already able to reach
almost all waypoints generated from the prior for IC 1 and 2, thus the
posterior only marginally improves the percentage of waypoints hit with a
difference below significant digit level, though for IC 3 and 4, we observe
the performance difference between the posteriors and prior.

##  IV Related Work

Behavioral models predict future motion of vehicles at different levels of
abstraction, ranging from discrete, high-level commands [20] to specific
vehicle positions at each time step [21, 8, 22]. As these models are typically
learned from external observations of vehicles with unknown attributes, they
tend to disregard an explicit notion of feasibility, though some models [10,
8] incorporate constraints [23]. As shown in this work, simple constraints are
not sufficient to guarantee feasibility of predicted trajectories for varying
vehicle types. This is typically not a concern, since such models are usually
deployed to predict the behavior of other agents in order to facilitate
planning [24], but it becomes an issue when a behavioral model is used as a
planner.

Vehicles operating in the real world typically feature a specialized planner
which needs to be extremely reliable, so learned behavior models are generally
not suitable for this purpose. Nonetheless, when generating high-fidelity
simulation, it makes sense to use behavioral models as planners for non-
playable characters. However, for predicted trajectories to be useful as
plans, they must be feasible. Incorporating explicit constraints into
trajectory planning has been extensively studied in the literature [25, 26]
but it typically makes assumptions on the vehicle dynamics, and relies on
knowing problem specific constants. In our approach the constraints, and the
dynamics which produce them, are not explicitly known. Instead, we estimate
them by observing which trajectories a trained controller is able to follow.

Trajectory planning can be cast as a constrained optimization problem [2],
where the target trajectory which satisfies a known dynamics model is selected
to optimize a cost function. This cost function defined to produce desired
characteristics, such as feasibility, comfort, achieving target, or minimizing
infractions. Our framework, which uses maximum aposteriori estimates is
equivalent to using a soft constraint instead of a direct projection onto the
constraint set. While inexact, our method can easily maintain a set of diverse
trajectories, which is crucial in the generation of diverse sets of scenarios.

Value functions learned via inverse RL have been used to produce rule-based
planners with Bayesian optimization [25]. By contrast, our approach only
requires solving a single RL problem per vehicle, instead of a sequence of RL
problems as is the case in most IRL methods. Unlike [25], we target
feasibility of the trajectory, rather than human preferences.

Using learned behavioral models to create realistic simulations is a
relatively recent research area [8, 10]. Existing work uses simple 2D
simulators with easily solvable kinematic models or performs 3D simulation by
teleporting non-playable characters to their target positions at each time
step. To the best of our knowledge, this is the first paper studying the task
of incorporating realistic vehicle dynamics into simulations generated with
learned behavioral models. Notably, hand-crafted behavioral models with
corresponding controllers have been used for this purpose before [27, 28].
This includes controllers, but not behavioral models, which were trained using
recorded human trajectories [29].

##  V DISCUSSION

In this work, we identify and formalize a fundamental problem at the
intersection of foundational models of behavior and low level control. This
issue, described by the discrepancy between a behavioral model and the
physical constraints of a real (or simulated) environment, can be solved using
artifacts already available from the byproducts of the reinforcement learning
algorithm used to train low-level vehicle controllers. Our work illustrates
that these byproducts implicitly target the same likelihoods needed to score
feasibility, and in practice can be used to great effect.

There remain however, opportunities to improve upon our work. These include
investigating methods to improve the bound in Equation 12, amortized
characterization of the posterior distribution of reachable waypoint
trajectories given by Equation 15, and meta-learning a single conditional
distribution $p(k_{1_{T}}|O_{1:T},c,e)$ that automatically adapts to the given
vehicle type. We expect the approach we have proposed will generalize to
additional vehicle types but this could also be verified by further study.

##  VI ACKNOWLEDGEMENT

We acknowledge the support of the Natural Sciences and Engineering Research
Council of Canada (NSERC), the Canada CIFAR AI Chairs Program, and the Intel
Parallel Computing Centers program. Additional support was provided by UBC’s
Composites Research Network (CRN), and Data Science Institute (DSI). This
research was enabled in part by technical support and computational resources
provided by WestGrid (www.westgrid.ca), Compute Canada (www.computecanada.ca),
and Advanced Research Computing at the University of British Columbia
(arc.ubc.ca).

## References

  * [1] Sajjad Mozaffari, Omar Y. Al-Jarrah, Mehrdad Dianati, Paul Jennings, and Alexandros Mouzakitis.  Deep learning-based vehicle behavior prediction for autonomous driving applications: A review.  2022\.
  * [2] Abbas Sadat, Mengye Ren, Andrei Pokrovsky, Yen-Chen Lin, Ersin Yumer, and Raquel Urtasun.  Jointly learnable behavior and trajectory planning for self-driving vehicles.  In IROS, 2019.
  * [3] Christopher Vo, Joseph F Harrison, and Jyh-Ming Lien.  Behavior-based motion planning for group control.  In IEEE RSJ International Conference on Intelligent Robots and Systems, 2009.
  * [4] Alexander Cui, Sergio Casas, Abbas Sadat, Renjie Liao, and Raquel Urtasun.  Lookout: Diverse multi-future prediction and planning for self-driving.  In ICCV, 2021.
  * [5] Tanmay Agarwal, Hitesh Arora, Tanvir Parhar, Shubhankar Deshpande, and Jeff Schneider.  Learning to drive using waypoints.  In Machine Learning for Autonomous Driving Workshop: NeurIPS, 2019\.
  * [6] Dian Chen, Brady Zhou, Vladlen Koltun, and Philipp Krähenbühl.  Learning by Cheating.  In Conference on Robot Learning, pages 66–75. PMLR, May 2020\.  ISSN: 2640-3498.
  * [7] Wei Zhan, Liting Sun, Di Wang, Haojie Shi, Aubrey Clausse, Maximilian Naumann, Julius Kümmerle, Hendrik Königshof, Christoph Stiller, Arnaud de La Fortelle, and Masayoshi Tomizuka.  INTERACTION Dataset: An INTERnational, Adversarial and Cooperative moTION Dataset in Interactive Driving Scenarios with Semantic Maps.  arXiv preprint arXiv:1910.03088, 2019.
  * [8] Simon Suo, Sebastian Regalado, Sergio Casas, and Raquel Urtasun.  Trafficsim: Learning to simulate realistic multi-agent behaviors.  In CVPR, 2021.
  * [9] Yu Yao, Mingze Xu, Chiho Choi, David J Crandall, Ella M Atkins, and Behzad Dariush.  Egocentric vision-based future vehicle localization for intelligent driving assistance systems.  In ICRA, 2019.
  * [10] Adam Ścibior, Vasileios Lioutas, Daniele Reda, Peyman Bateni, and Frank Wood.  Imagining the road ahead: Multi-agent trajectory prediction via differentiable simulation.  In ITSC, 2021.
  * [11] Hang Zhao, Jiyang Gao, Tian Lan, Chen Sun, Benjamin Sapp, Balakrishnan Varadarajan, Yue Shen, Yi Shen, Yuning Chai, Cordelia Schmid, Congcong Li, and Dragomir Anguelov.  Tnt: Target-driven trajectory prediction.  CoRR, 2019.
  * [12] M. Zhao, D. Kathner, M. Jipp, D. Soffker, and K. Lemmer.  Modeling driver behavior at roundabouts: Results from a field study.  In IEEE Intelligent Vehicles Symposium (IV), 2017.
  * [13] Xiaoyu Chen, Jiachen Hu, Chi Jin, Lihong Li, and Liwei Wang.  Understanding domain randomization for sim-to-real transfer.  In ICLR, 2022.
  * [14] Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio Lopez, and Vladlen Koltun.  Carla: An open urban driving simulator.  In CoRL, 2017.
  * [15] Nvidia drive sim, 2021.
  * [16] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov.  Proximal policy optimization algorithms.  2017\.
  * [17] Sergey Levine.  Reinforcement learning and control as probabilistic inference: Tutorial and review.  arXiv preprint arXiv:1805.00909, 2018.
  * [18] Jonathan Wilder Lavington, Michael Teng, Mark Schmidt, and Frank Wood.  A closer look at gradient estimators with reinforcement learning as inference.  In Deep RL Workshop: NeurIPS, 2021.
  * [19] Alexandre Piché, Valentin Thomas, Cyril Ibrahim, Yoshua Bengio, and Chris Pal.  Probabilistic planning with sequential monte carlo methods.  In ICLR, 2018.
  * [20] Zlatan Ajanovic, Bakir Lacevic, Barys Shyrokau, Michael Stolz, and Martin Horn.  Search-based optimal motion planning for automated driving.  In IROS, 2018.
  * [21] Nicholas Rhinehart, Rowan McAllister, and Sergey Levine.  Deep imitative models for flexible inference, planning, and control.  In ICLR, 2020.
  * [22] Hang Zhao, Jiyang Gao, Tian Lan, Chen Sun, Benjamin Sapp, Balakrishnan Varadarajan, Yue Shen, Yi Shen, Yuning Chai, Cordelia Schmid, Congcong Li, and Dragomir Anguelov.  TNT: target-driven trajectory prediction.  CoRR, abs/2008.08294, 2020.
  * [23] Philip Polack, Florent Altché, Brigitte d’Andréa Novel, and Arnaud de La Fortelle.  The kinematic bicycle model: A consistent model for planning feasible trajectories for autonomous vehicles?  In IEEE Intelligent Vehicles Symposium (IV), 2017.
  * [24] Andrey Rudenko, Luigi Palmieri, Michael Herman, Kris M Kitani, Dariu M Gavrila, and Kai O Arras.  Human motion trajectory prediction: a survey.  The International Journal of Robotics Research, 39(8):895–935, July 2020.  Publisher: SAGE Publications Ltd STM.
  * [25] Shu Jiang, Zikang Xiong, Weiman Lin, Yu Cao, Zhongpu Xia, Jinghao Miao, and Qi Luo.  An efficient framework for reliable and personalized motion planner in autonomous driving.  IEEE Robotics and Automation Letters, page 8, 2022.
  * [26] Yu Wang, Shu Jiang, Weiman Lin, Yu Cao, Longtao Lin, Jiangtao Hu, Jinghao Miao, and Qi Luo.  A learning-based tune-free control framework for large scale autonomous driving system deployment.  CoRR, abs/2011.04250, 2020.
  * [27] Qianwen Chao, Huikun Bi, Weizi Li, Tianlu Mao, Zhaoqi Wang, Ming C Lin, and Zhigang Deng.  A survey on visual traffic simulation: Models, evaluations, and applications in autonomous driving.  In Computer Graphics Forum, volume 39, pages 287–308. Wiley Online Library, 2020.
  * [28] Arne Kesting, Martin Treiber, and Dirk Helbing.  Agents for traffic simulation.  Multi-agent systems: Simulation and applications, 5, 2009.
  * [29] Raunak Bhattacharyya, Soyeon Jung, Liam A. Kruse, Ransalu Senanayake, and Mykel J. Kochenderfer.  A hybrid rule-based and data-driven approach to driver modeling through particle filtering.  IEEE Transactions on Intelligent Transportation Systems, pages 1–14, 2021.


# Waveform Simulation in PandaX-4T

Jiafu Li  School of Physics, Sun Yat-Sen University, Guangzhou 510275, China
Abdusalam Abdukerim  School of Physics and Astronomy, Shanghai Jiao Tong
University, Key Laboratory for Particle Astrophysics and Cosmology (MoE),
Shanghai Key Laboratory for Particle Physics and Cosmology, Shanghai 200240,
China  Zihao Bo  School of Physics and Astronomy, Shanghai Jiao Tong
University, Key Laboratory for Particle Astrophysics and Cosmology (MoE),
Shanghai Key Laboratory for Particle Physics and Cosmology, Shanghai 200240,
China  Wei Chen  School of Physics and Astronomy, Shanghai Jiao Tong
University, Key Laboratory for Particle Astrophysics and Cosmology (MoE),
Shanghai Key Laboratory for Particle Physics and Cosmology, Shanghai 200240,
China  Xun Chen  School of Physics and Astronomy, Shanghai Jiao Tong
University, Key Laboratory for Particle Astrophysics and Cosmology (MoE),
Shanghai Key Laboratory for Particle Physics and Cosmology, Shanghai 200240,
China  Shanghai Jiao Tong University Sichuan Research Institute, Chengdu
610213, China  Chen Cheng  School of Physics, Sun Yat-Sen University,
Guangzhou 510275, China  Zhaokan Cheng  Sino-French Institute of Nuclear
Engineering and Technology, Sun Yat-Sen University, Zhuhai, 519082, China
Xiangyi Cui  Tsung-Dao Lee Institute, Shanghai Jiao Tong University, Shanghai,
200240, China  Yingjie Fan  Department of Physics,Yantai University, Yantai
264005, China  Deqing Fang  Key Laboratory of Nuclear Physics and Ion-beam
Application (MOE), Institute of Modern Physics, Fudan University, Shanghai
200433, China  Changbo Fu  Key Laboratory of Nuclear Physics and Ion-beam
Application (MOE), Institute of Modern Physics, Fudan University, Shanghai
200433, China  Mengting Fu  School of Physics, Peking University, Beijing
100871, China  Lisheng Geng  School of Physics, Beihang University, Beijing
102206, China  International Research Center for Nuclei and Particles in the
Cosmos & Beijing Key Laboratory of Advanced Nuclear Materials and Physics,
Beihang University, Beijing 100191, China  School of Physics and
Microelectronics, Zhengzhou University, Zhengzhou, Henan 450001, China  Karl
Giboni  School of Physics and Astronomy, Shanghai Jiao Tong University, Key
Laboratory for Particle Astrophysics and Cosmology (MoE), Shanghai Key
Laboratory for Particle Physics and Cosmology, Shanghai 200240, China  Linhui
Gu  School of Physics and Astronomy, Shanghai Jiao Tong University, Key
Laboratory for Particle Astrophysics and Cosmology (MoE), Shanghai Key
Laboratory for Particle Physics and Cosmology, Shanghai 200240, China  Xuyuan
Guo  Yalong River Hydropower Development Company, Ltd., 288 Shuanglin Road,
Chengdu 610051, China  Chencheng Han  Tsung-Dao Lee Institute, Shanghai Jiao
Tong University, Shanghai, 200240, China  Ke Han  School of Physics and
Astronomy, Shanghai Jiao Tong University, Key Laboratory for Particle
Astrophysics and Cosmology (MoE), Shanghai Key Laboratory for Particle Physics
and Cosmology, Shanghai 200240, China  Changda He  School of Physics and
Astronomy, Shanghai Jiao Tong University, Key Laboratory for Particle
Astrophysics and Cosmology (MoE), Shanghai Key Laboratory for Particle Physics
and Cosmology, Shanghai 200240, China  Jinrong He  Yalong River Hydropower
Development Company, Ltd., 288 Shuanglin Road, Chengdu 610051, China  Di Huang
School of Physics and Astronomy, Shanghai Jiao Tong University, Key Laboratory
for Particle Astrophysics and Cosmology (MoE), Shanghai Key Laboratory for
Particle Physics and Cosmology, Shanghai 200240, China  Yanlin Huang  School
of Medical Instrument and Food Engineering, University of Shanghai for Science
and Technology, Shanghai 200093, China  Junting Huang  School of Physics and
Astronomy, Shanghai Jiao Tong University, Key Laboratory for Particle
Astrophysics and Cosmology (MoE), Shanghai Key Laboratory for Particle Physics
and Cosmology, Shanghai 200240, China  Zhou Huang  School of Physics and
Astronomy, Shanghai Jiao Tong University, Key Laboratory for Particle
Astrophysics and Cosmology (MoE), Shanghai Key Laboratory for Particle Physics
and Cosmology, Shanghai 200240, China  Ruquan Hou  Shanghai Jiao Tong
University Sichuan Research Institute, Chengdu 610213, China  Yu Hou  School
of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai 200240,
China  Xiangdong Ji  Department of Physics, University of Maryland, College
Park, Maryland 20742, USA  Yonglin Ju  School of Mechanical Engineering,
Shanghai Jiao Tong University, Shanghai 200240, China  Chenxiang Li  School of
Physics and Astronomy, Shanghai Jiao Tong University, Key Laboratory for
Particle Astrophysics and Cosmology (MoE), Shanghai Key Laboratory for
Particle Physics and Cosmology, Shanghai 200240, China  Mingchuan Li  Yalong
River Hydropower Development Company, Ltd., 288 Shuanglin Road, Chengdu
610051, China  Shuaijie Li  Yalong River Hydropower Development Company, Ltd.,
288 Shuanglin Road, Chengdu 610051, China  School of Physics and Astronomy,
Shanghai Jiao Tong University, Key Laboratory for Particle Astrophysics and
Cosmology (MoE), Shanghai Key Laboratory for Particle Physics and Cosmology,
Shanghai 200240, China  Tao Li  Sino-French Institute of Nuclear Engineering
and Technology, Sun Yat-Sen University, Zhuhai, 519082, China  Qing Lin
Corresponding author<EMAIL_ADDRESS>Department of Modern Physics,
University of Science and Technology of China, Hefei 230026, China  State Key
Laboratory of Particle Detection and Electronics, University of Science and
Technology of China, Hefei 230026, China  Jianglai Liu Spokesperson:
<EMAIL_ADDRESS>School of Physics and Astronomy, Shanghai Jiao Tong
University, Key Laboratory for Particle Astrophysics and Cosmology (MoE),
Shanghai Key Laboratory for Particle Physics and Cosmology, Shanghai 200240,
China  Tsung-Dao Lee Institute, Shanghai Jiao Tong University, Shanghai,
200240, China  Shanghai Jiao Tong University Sichuan Research Institute,
Chengdu 610213, China  Congcong Lu  School of Mechanical Engineering, Shanghai
Jiao Tong University, Shanghai 200240, China  Xiaoying Lu  Research Center for
Particle Science and Technology, Institute of Frontier and Interdisciplinary
Science, Shandong University, Qingdao 266237, Shandong, China  Key Laboratory
of Particle Physics and Particle Irradiation of Ministry of Education,
Shandong University, Qingdao 266237, Shandong, China  Lingyin Luo  School of
Physics, Peking University, Beijing 100871, China  Yunyang Luo  Department of
Modern Physics, University of Science and Technology of China, Hefei 230026,
China  Wenbo Ma Corresponding author<EMAIL_ADDRESS>School of Physics
and Astronomy, Shanghai Jiao Tong University, Key Laboratory for Particle
Astrophysics and Cosmology (MoE), Shanghai Key Laboratory for Particle Physics
and Cosmology, Shanghai 200240, China  Yugang Ma  Key Laboratory of Nuclear
Physics and Ion-beam Application (MOE), Institute of Modern Physics, Fudan
University, Shanghai 200433, China  Yajun Mao  School of Physics, Peking
University, Beijing 100871, China  Yue Meng  School of Physics and Astronomy,
Shanghai Jiao Tong University, Key Laboratory for Particle Astrophysics and
Cosmology (MoE), Shanghai Key Laboratory for Particle Physics and Cosmology,
Shanghai 200240, China  Shanghai Jiao Tong University Sichuan Research
Institute, Chengdu 610213, China  Xuyang Ning  School of Physics and
Astronomy, Shanghai Jiao Tong University, Key Laboratory for Particle
Astrophysics and Cosmology (MoE), Shanghai Key Laboratory for Particle Physics
and Cosmology, Shanghai 200240, China  Ningchun Qi  Yalong River Hydropower
Development Company, Ltd., 288 Shuanglin Road, Chengdu 610051, China  Zhicheng
Qian  School of Physics and Astronomy, Shanghai Jiao Tong University, Key
Laboratory for Particle Astrophysics and Cosmology (MoE), Shanghai Key
Laboratory for Particle Physics and Cosmology, Shanghai 200240, China
Xiangxiang Ren  Research Center for Particle Science and Technology, Institute
of Frontier and Interdisciplinary Science, Shandong University, Qingdao
266237, Shandong, China  Key Laboratory of Particle Physics and Particle
Irradiation of Ministry of Education, Shandong University, Qingdao 266237,
Shandong, China  Nasir Shaheed  Research Center for Particle Science and
Technology, Institute of Frontier and Interdisciplinary Science, Shandong
University, Qingdao 266237, Shandong, China  Key Laboratory of Particle
Physics and Particle Irradiation of Ministry of Education, Shandong
University, Qingdao 266237, Shandong, China  Xiaofeng Shang  School of Physics
and Astronomy, Shanghai Jiao Tong University, Key Laboratory for Particle
Astrophysics and Cosmology (MoE), Shanghai Key Laboratory for Particle Physics
and Cosmology, Shanghai 200240, China  Xiyuan Shao  School of Physics, Nankai
University, Tianjin 300071, China  Guofang Shen  School of Physics, Beihang
University, Beijing 102206, China  Lin Si  School of Physics and Astronomy,
Shanghai Jiao Tong University, Key Laboratory for Particle Astrophysics and
Cosmology (MoE), Shanghai Key Laboratory for Particle Physics and Cosmology,
Shanghai 200240, China  Wenliang Sun  Yalong River Hydropower Development
Company, Ltd., 288 Shuanglin Road, Chengdu 610051, China  Andi Tan  Department
of Physics, University of Maryland, College Park, Maryland 20742, USA  Yi Tao
School of Physics and Astronomy, Shanghai Jiao Tong University, Key Laboratory
for Particle Astrophysics and Cosmology (MoE), Shanghai Key Laboratory for
Particle Physics and Cosmology, Shanghai 200240, China  Shanghai Jiao Tong
University Sichuan Research Institute, Chengdu 610213, China  Anqing Wang
Research Center for Particle Science and Technology, Institute of Frontier and
Interdisciplinary Science, Shandong University, Qingdao 266237, Shandong,
China  Key Laboratory of Particle Physics and Particle Irradiation of Ministry
of Education, Shandong University, Qingdao 266237, Shandong, China  Meng Wang
Research Center for Particle Science and Technology, Institute of Frontier and
Interdisciplinary Science, Shandong University, Qingdao 266237, Shandong,
China  Key Laboratory of Particle Physics and Particle Irradiation of Ministry
of Education, Shandong University, Qingdao 266237, Shandong, China  Qiuhong
Wang  Key Laboratory of Nuclear Physics and Ion-beam Application (MOE),
Institute of Modern Physics, Fudan University, Shanghai 200433, China  Shaobo
Wang  School of Physics and Astronomy, Shanghai Jiao Tong University, Key
Laboratory for Particle Astrophysics and Cosmology (MoE), Shanghai Key
Laboratory for Particle Physics and Cosmology, Shanghai 200240, China  SJTU
Paris Elite Institute of Technology, Shanghai Jiao Tong University, Shanghai,
200240, China  Siguang Wang  School of Physics, Peking University, Beijing
100871, China  Wei Wang  Sino-French Institute of Nuclear Engineering and
Technology, Sun Yat-Sen University, Zhuhai, 519082, China  School of Physics,
Sun Yat-Sen University, Guangzhou 510275, China  Xiuli Wang  School of
Mechanical Engineering, Shanghai Jiao Tong University, Shanghai 200240, China
Zhou Wang  School of Physics and Astronomy, Shanghai Jiao Tong University, Key
Laboratory for Particle Astrophysics and Cosmology (MoE), Shanghai Key
Laboratory for Particle Physics and Cosmology, Shanghai 200240, China
Shanghai Jiao Tong University Sichuan Research Institute, Chengdu 610213,
China  Tsung-Dao Lee Institute, Shanghai Jiao Tong University, Shanghai,
200240, China  Yuehuan Wei  Sino-French Institute of Nuclear Engineering and
Technology, Sun Yat-Sen University, Zhuhai, 519082, China  Mengmeng Wu  School
of Physics, Sun Yat-Sen University, Guangzhou 510275, China  Weihao Wu  School
of Physics and Astronomy, Shanghai Jiao Tong University, Key Laboratory for
Particle Astrophysics and Cosmology (MoE), Shanghai Key Laboratory for
Particle Physics and Cosmology, Shanghai 200240, China  Jingkai Xia  School of
Physics and Astronomy, Shanghai Jiao Tong University, Key Laboratory for
Particle Astrophysics and Cosmology (MoE), Shanghai Key Laboratory for
Particle Physics and Cosmology, Shanghai 200240, China  Mengjiao Xiao
Department of Physics, University of Maryland, College Park, Maryland 20742,
USA  Xiang Xiao  School of Physics, Sun Yat-Sen University, Guangzhou 510275,
China  Pengwei Xie  Tsung-Dao Lee Institute, Shanghai Jiao Tong University,
Shanghai, 200240, China  Binbin Yan  School of Physics and Astronomy, Shanghai
Jiao Tong University, Key Laboratory for Particle Astrophysics and Cosmology
(MoE), Shanghai Key Laboratory for Particle Physics and Cosmology, Shanghai
200240, China  Xiyu Yan  School of Physics and Astronomy, Sun Yat-Sen
University, Zhuhai, 519082, China  Jijun Yang  School of Physics and
Astronomy, Shanghai Jiao Tong University, Key Laboratory for Particle
Astrophysics and Cosmology (MoE), Shanghai Key Laboratory for Particle Physics
and Cosmology, Shanghai 200240, China  Yong Yang  School of Physics and
Astronomy, Shanghai Jiao Tong University, Key Laboratory for Particle
Astrophysics and Cosmology (MoE), Shanghai Key Laboratory for Particle Physics
and Cosmology, Shanghai 200240, China  Yukun Yao  School of Physics and
Astronomy, Shanghai Jiao Tong University, Key Laboratory for Particle
Astrophysics and Cosmology (MoE), Shanghai Key Laboratory for Particle Physics
and Cosmology, Shanghai 200240, China  Chunxu Yu  School of Physics, Nankai
University, Tianjin 300071, China  Ying Yuan  School of Physics and Astronomy,
Shanghai Jiao Tong University, Key Laboratory for Particle Astrophysics and
Cosmology (MoE), Shanghai Key Laboratory for Particle Physics and Cosmology,
Shanghai 200240, China  Zhe Yuan  Key Laboratory of Nuclear Physics and Ion-
beam Application (MOE), Institute of Modern Physics, Fudan University,
Shanghai 200433, China  Xinning Zeng  School of Physics and Astronomy,
Shanghai Jiao Tong University, Key Laboratory for Particle Astrophysics and
Cosmology (MoE), Shanghai Key Laboratory for Particle Physics and Cosmology,
Shanghai 200240, China  Dan Zhang  Department of Physics, University of
Maryland, College Park, Maryland 20742, USA  Minzhen Zhang  School of Physics
and Astronomy, Shanghai Jiao Tong University, Key Laboratory for Particle
Astrophysics and Cosmology (MoE), Shanghai Key Laboratory for Particle Physics
and Cosmology, Shanghai 200240, China  Peng Zhang  Yalong River Hydropower
Development Company, Ltd., 288 Shuanglin Road, Chengdu 610051, China  Shibo
Zhang  School of Physics and Astronomy, Shanghai Jiao Tong University, Key
Laboratory for Particle Astrophysics and Cosmology (MoE), Shanghai Key
Laboratory for Particle Physics and Cosmology, Shanghai 200240, China  Shu
Zhang  School of Physics, Sun Yat-Sen University, Guangzhou 510275, China  Tao
Zhang  School of Physics and Astronomy, Shanghai Jiao Tong University, Key
Laboratory for Particle Astrophysics and Cosmology (MoE), Shanghai Key
Laboratory for Particle Physics and Cosmology, Shanghai 200240, China  Wei
Zhang  Tsung-Dao Lee Institute, Shanghai Jiao Tong University, Shanghai,
200240, China  Yang Zhang  Research Center for Particle Science and
Technology, Institute of Frontier and Interdisciplinary Science, Shandong
University, Qingdao 266237, Shandong, China  Key Laboratory of Particle
Physics and Particle Irradiation of Ministry of Education, Shandong
University, Qingdao 266237, Shandong, China  Yingxin Zhang  Research Center
for Particle Science and Technology, Institute of Frontier and
Interdisciplinary Science, Shandong University, Qingdao 266237, Shandong,
China  Key Laboratory of Particle Physics and Particle Irradiation of Ministry
of Education, Shandong University, Qingdao 266237, Shandong, China  Yuanyuan
Zhang  Tsung-Dao Lee Institute, Shanghai Jiao Tong University, Shanghai,
200240, China  Li Zhao  School of Physics and Astronomy, Shanghai Jiao Tong
University, Key Laboratory for Particle Astrophysics and Cosmology (MoE),
Shanghai Key Laboratory for Particle Physics and Cosmology, Shanghai 200240,
China  Qibin Zheng  School of Medical Instrument and Food Engineering,
University of Shanghai for Science and Technology, Shanghai 200093, China
Jifang Zhou  Yalong River Hydropower Development Company, Ltd., 288 Shuanglin
Road, Chengdu 610051, China  Ning Zhou  School of Physics and Astronomy,
Shanghai Jiao Tong University, Key Laboratory for Particle Astrophysics and
Cosmology (MoE), Shanghai Key Laboratory for Particle Physics and Cosmology,
Shanghai 200240, China  Shanghai Jiao Tong University Sichuan Research
Institute, Chengdu 610213, China  Xiaopeng Zhou  School of Physics, Beihang
University, Beijing 102206, China  Yong Zhou  Yalong River Hydropower
Development Company, Ltd., 288 Shuanglin Road, Chengdu 610051, China  Yubo
Zhou  School of Physics and Astronomy, Shanghai Jiao Tong University, Key
Laboratory for Particle Astrophysics and Cosmology (MoE), Shanghai Key
Laboratory for Particle Physics and Cosmology, Shanghai 200240, China

(August 28, 2024)

###### Abstract

Signal reconstruction through software processing is a crucial component of
the background and signal models in the PandaX-4T experiment, which is a
multi-tonne dark matter direct search experiment. The accuracy of signal
reconstruction is influenced by various detector artifacts, including noise,
dark count of photomultiplier, impurity photoionization in the detector, and
other relevant considerations. In this study, we present a detailed
description of a semi-data-driven approach designed to simulate the signal
waveform. This work provides a reliable model for the efficiency and bias of
the signal reconstruction in the data analysis of PandaX-4T. By comparing
critical variables which relate to the temporal shape and hit pattern of the
signals, we demonstrate a good agreement between the simulation and data.

##  1 Introduction

The search for dark matter (DM) [1, 2, 3] is a highly active frontier in
astroparticle physics. Among the various experimental approaches [4, 5, 6, 7,
8, 9, 10, 11, 12], dual-phase xenon time projection chamber (TPC) based
experiments have been recognized as highly sensitive for detecting cold DM
particles within the mass range of approximately 10 GeV/c2 to TeV/c2.
PandaX-4T [10], located in the China Jinping Underground Laboratory (CJPL)
[13, 14], is a typical experiment of this kind. PandaX-4T employs a
cylindrical dual-phase xenon TPC with a sensitive volume measuring
approximately 1.2 meters in diameter and 1.2 meters in height. The TPC is
equipped with 368 Photomultipliers (PMTs), 169 of which are distributed at the
top PMT array and 199 at the bottom. The TPC detects and measures both the
prompt scintillation signal ($S1$) and the subsequent amplified proportional
scintillation signal ($S2$), which arises from the delayed ionization signals.
By analyzing the time difference between $S1$ and $S2$ signals, as well as the
spatial distribution of PMT hits associated with the $S2$ signal, we are
capable of reconstructing the vertical and horizontal positions of the
interaction vertex, respectively. Accurate 3-D position reconstruction, along
with the determination of the $S2/S1$ ratio, plays a crucial role in particle
discrimination within the TPC detector. This discrimination capability is of
crucial importance as it greatly reduces the effective background for DM
direct searches.

In the interpretation of the DM search results in PandaX-4T, precise models
for the low-energy background and the DM signal are of utmost importance. A
critical component of the low-energy models is the reconstruction of the $S1$
and $S2$. This reconstruction process involves various steps such as peak
identification, pulse classification, clustering, and other relevant
techniques. In order to accurately evaluate the efficiency and potential
biases inherent in the software reconstruction, it is essential to obtain pure
events that faithfully represent the desired physical signal. However,
acquiring such pure samples from the recorded reconstructed data is
challenging, as the recorded data have already been influenced by the effects
of the software signal reconstruction inefficiency and biases. To address this
challenge, PandaX-4T has developed a dedicated waveform simulation (WS)
framework that generates synthetic data waveforms. The WS framework
incorporates our best understanding of the processes involved in the
generation, collection, and reconstruction of $S1$ and $S2$ signals, as well
as the accompanying sources of noise, such as dark counts, PMT afterpulsing,
and impurity photoionization. In addition, the extensive samples generated by
the WS can also be utilized to train machine learning and deep neural network
algorithms, which can further enhance background rejection capabilities.

This manuscript presents a detailed account of the WS employed in various
scientific studies conducted using the PandaX-4T data, including the search
for Weakly Interacting Massive Particles (WIMPs) [10], the detection of solar
8B neutrinos [15], investigations into light dark matter particles [16], and
other related research endeavors [17, 18]. Section 2 of the manuscript
elucidates the details of simulating the $S1$ and $S2$ pulses, as well as
incorporating various sources of noise into the simulation process. Notably,
the comparison between simulation and real data is presented in Section 3,
wherein critical variables such as signal width, pattern, and charge are
evaluated. Following the comparison, the manuscript concludes by summarizing
the work and offering a discussion on its future perspectives. Similar works
of other experiments can be found in Ref. [19].

##  2 Waveform simulation

The simulated event waveforms in WS further undergo the software processing
and reconstruction chain that is used for data processing and analysis in
PandaX-4T, to ensure that the efficiency and bias of the simulated waveforms
align with the observed data. The WS employs a semi-data-driven approach to
simulate the $S1$ and $S2$ waveforms. It takes into account various spurious
pulses, including noise, dark counts, PMT afterpulsing, and delayed afterglow
after a large $S2$ arising from impurity ionization and electron train
effects. To illustrate, several simulated waveforms are presented in Fig. 1.
Among these examples, one demonstrates successful identification of both the
$S1$ and $S2$ signals, albeit with a slight difference in the reconstructed
charge values with respect to the true charge, due to fluctuation in the
reconstruction steps and interference from spurious pulses. In contrast,
another example exhibits complete misidentification of the $S1$ signal,
indicating a potential inefficiency in our software signal reconstruction
process.

Figure 1:  Three waveforms corresponding to events with similar drift times
are depicted. The top panel represents a recorded data waveform. In the middle
panel, a simulated waveform is shown where the true $S1$ signal is
overshadowed by a noise $S1$ signal with a slightly higher charge. The bottom
panel displays a simulated waveform where both the $S1$ and $S2$ signals are
correctly identified. The gray, green, and cyan shaded regions represent the
time windows associated with the true $S1$, misidentified $S1$, and true $S2$
signals, respectively. The reconstructed charge of the identified $S1$s and
$S2$s are displayed in the panels. For enhanced detail, the inset panels
provide zoomed-in views of the $S1$ and $S2$ signals. The hit patterns of the
main $S1$s and $S2$s are also shown in the right columns.

###  2.1 S1 pulse

The $S1$ signal represents the prompt scintillation signals detected in the
TPC detector, arising from the interaction of particles with the xenon shell
electrons or nucleus. The $S1$ signal exhibits a relatively short time scale,
typically ranging from a few tens of nanoseconds to around 100 nanoseconds
originated from light propagation in the TPC. The shape of the $S1$ pulse is
influenced by the PMT signal shaping, the decay time profile of singlet and
triplet xenon dimers, and the propagation of photons within the TPC. The ratio
between the singlet and triplet dimer decays differs slightly between
electronic recoils (ERs) and nuclear recoils (NRs). In ER events, where the
incoming particle interacts with the xenon shell electrons, this ratio tends
to be slightly higher compared to NR events [20, 21], where the interaction
occurs with the xenon nucleus. However, the difference in pulse shapes between
ER and NR events is obscured as the consequence of the photon propagation
effects, such photon Rayleigh scattering as well as the reflection on the
liquid surface and TPC wall.

The simulation of the $S1$ waveform adopts a data-driven approach. In this
approach, the $S1$ signals from 220Rn [22] and neutron calibration data,
including 241AmBe radioactive source and Deuteron-Deuteron (DD) neutron
generator, are utilized to simulate the $S1$ waveforms corresponding to ERs
and NRs, respectively. $S1$s with the charge in 2 to 200 photoelectrons (PE)
and 2 to 150 PE, respectively, are selected as the ER and NR $S1$ pools.
Within these charge levels, the $S1$ waveforms consist of single-photon hits
that are distributed among multiple PMTs and are distinguishable in time. The
time profile of a single-photon hit on a PMT is depicted in the left panel of
Fig. 2. To create a simulated $S1$ waveform that shared the same event
position as the pooled $S1$ (transverse distance $<$5 cm and vertical distance
$<$8 cm), a fraction of the hit waveforms of the pooled $S1$ are randomly
selected to form a simulated $S1$ with reduced charge. This data-driven
approach naturally takes into account the position-dependent characteristics
of the $S1$ pulse time profile resulting from light propagation. Right panel
of Fig. 2 displays the hit time distributions of $S1$ signals in the ER and NR
calibration data from various $Z$ positions. The difference in shape of
waveforms arises from the difference of spatial position distributions in ER
and NR calibration data.

Figure 2:  The single PE waveform and the time distributions of the $S1$ hits
are shown in the left and right panels, respectively. The blue, orange, and
red solid lines in the right panel represent the distributions from 220Rn,
241AmBe, and DD calibration data, respectively.

###  2.2 S2 pulse

Figure 3:  The normalized distributions of the $W^{80}_{\textrm{CDF}}$ over
the drift time for four types of data: a) the 220Rn calibration, b) the
neutron calibration using 241AmBe and DD, c) the 83mKr calibration, and d) the
$\alpha$ events originated from 222Rn impurities. The red solid lines
represent the means of the distributions. The $D_{L}$ values obtained from a
fit to the means using Eq. 1 are given in each panel.

The $S2$ signal corresponds to the detection of proportional scintillation
light emitted by the drifted electrons when they reach the gaseous xenon layer
between the anode and gate electrodes in the TPC detector. The shape of the
$S2$ pulse is predominantly determined by two factors: the diffusion of
electrons during their drift from the interaction vertex to the gaseous xenon
layer, and the subsequent travel within the gaseous layer. The longitudinal
diffusion coefficient in liquid xenon has been experimentally measured to be
approximately 50 cm2/s at an electric field of 500 V/cm [23]. To obtain
accurate in-situ values for the longitudinal diffusion coefficient,
measurements were performed using different calibration sources. Specifically,
the 41 keV gamma line from the 83mKr calibration data, and the $\alpha$ decay
events from 222Rn were utilized. Fig. 3 illustrates the 80%-CDF $S2$ width
(defined as the length of time window that covers central 80% of the $S2$
charge) over drift time distributions for these events, and the mean 80%-CDF
$S2$ widths as a function of drift time were used to calculate the respective
longitudinal diffusion coefficients. The longitudinal diffusion coefficient
$D_{L}$ can be obtained by fitting the mean 80%-CDF $S2$ widths (denoted as
$\langle W_{S2}^{80}\rangle$) as a function of drift time by the relation:

$\langle W_{S2}^{80}\rangle=f^{80}\sqrt{2D_{L}T+\sigma^{2}_{0}},$ (1)

where $T$ is the drift time and $\sigma_{0}$ represents the standard deviation
of photon hit time in a single electron (SE) waveform. $\sigma_{0}$ reflects
the travel time of the electron in the gaseous xenon layer, and depends on the
gas gap and the electric field strength in the gap. The factor $f^{80}=2.56$
is the conversion factor from the standard deviation to 80%-CDF width assuming
the $S2$ pulse shape is Gaussian. The best-fit $D_{L}$ using low-energy data
(${}^{220}\textrm{Rn}$, 241AmBe, and DD) is systematically lower than the
results obtained using high-energy data (83mKr and $\alpha$ from 222Rn). This
is due to the statistical bias of the $S2$ pulse width once the $S2$ charge is
only caused by a few electrons. The variations observed in the diffusion
coefficients obtained from high-energy $\alpha$ also suggest the potential
presence of systematic effects in high-energy samples, such as PMT saturation.
Consequently, the WS employs the diffusion coefficient from 83mKr calibration
data. This coefficient has a good agreement in the width versus drift time
distributions of the low-energy data, as illustrated by later text. The time
profile of electron travel within the gaseous xenon can be determined by
analyzing the SE waveforms, as depicted in Fig. 4. Both the electric field and
the decay characteristics of xenon dimers in the gaseous phase influence the
observed time profile.

Figure 4:  Average single electron waveforms.

The SE waveforms from the data are reassembled to generate simulated $S2$
signals in the WS. In order to ensure compatibility of the ($X$, $Y$)
positions between the simulated $S2$ signals and the data, SE waveforms with
reconstructed positions $R_{0}$ within a certain range, specifically within 4
cm from the simulated position, are selected for the assembly process. This
value is determined to efficiently match the pattern of the $S2$ signals (see
Sec. 3 for more detail). The selected SE pulses’ arrival times are shuffled to
ensure that the simulated SE arrival times adhere to the diffusion principle,
wherein the standard deviation of the arrival time, denoted as $\sigma_{T}$,
must satisfy $\sigma_{T}=\sqrt{2D_{L}T}$.

###  2.3 PMT after pulsing

When a PMT detects light signals, residual impurities present within the PMT
can become ionized due to the acceleration of photoelectrons (PEs) [24]. These
positively charged impurity ions then drift back towards the PMT’s
photocathode, causing the emission of additional PEs. As a result, small
delayed signals may appear after the main $S1$ pulse. The majority of
impurities tend to be concentrated on the first dynode of the PMT, leading to
characteristic and constant time delays for certain impurity ion in the PMT’s
response due to afterpulsing. The magnitude of the time delay is primarily
proportional to the square root of the atomic mass of the impurity.
Furthermore, the number of PEs that each impurity ion can generate is also
dependent on the type of impurity. In WS, the number of PMT afterpulsing hit
and the charge of each hit are sampled according to the average differential
probability and mean charge of PMT afterpulsing, respectively, as a function
of the delay time, shown in Fig. 5. The cumulative probability of afterpulsing
occurrence in the 3-inch PMTs utilized in the PandaX-4T experiment is
approximately 3%.

Figure 5:  The differential probability of the PMT after pulsing as a function
of delay time. The after pulsings caused by helium, nitrogen, and argon
residual gases are most visible, and highlighted by the magenta dashed lines.
There are also secondary after pulsing of nitrogen which is indicated by
magenta dashed lines as well. The red solid line give the mean charge of after
pulsing.

###  2.4 Photoionization and delayed electrons

In several LXe-TPC detectors, it has been observed that additional $S2$
signals appear subsequent to a large $S2$ signal [25, 26, 27], commonly
referred to as delayed $S2$ or delayed electrons. The appearance of these
delayed signals is attributed to various factors. One significant factor is
the photoionization of the electrode metal. The electrons resulting from this
photoionization process exhibit distinct drift times, as depicted in Fig. 6.
Delayed electrons can also arise from the photoionization of impurities
uniformly distributed within the detector. It is also speculated that
impurities present in the LXe may ”capture” and subsequently release drifting
electrons [27], leading to delayed $S2$ signals. Other processes, such as
electron trapping at the liquid-gas interface [28] and spontaneous electron
emission from the electrode [29, 30], have also been proposed as potential
contributors to the delayed $S2$ phenomenon. In the PandaX-4T experiment, a
data-driven approach is employed to model the probability of delayed electron
generation. By stacking selected $S2$ waveforms with a fixed reference time
(e.g., the start of the waveform), the resulting stacked waveform is analyzed
to give the production probability, as shown in Fig. 6. The correlation
between the probability of the delayed electron production and the
corresponding delay times is modeled empirically using a combination of two
Gaussians and two exponential distributions. The Gaussian components represent
delayed electrons originating from the gate and cathode electrodes,
respectively. The mean delay time for the gate delayed electrons, extracted
from the fit to the stacked waveform, is 3.2 $\mu$s. These values align with
the expected behavior, assuming the liquid-gas interface is 5-8 mm above the
gate. The delayed electrons from cathode are negligible compared to other
components. The two exponential distributions are found to adequately model
delayed electrons resulting from impurity photoionization and electron delays
caused by impurity or liquid surface trapping. The probabilities of generating
one delayed electron per detected $S2$ photon are determined as 0.24% and
0.15%, respectively, for the gate photoionization and other aforementioned
effects. In the WS, number of delayed electrons are sampled based on these
probabilities and the delay time distributions shown in Fig. 6. The secondary
photoionization caused by the primary photoionization is negligible and not
implemented in WS. Further adjustments to the photoionization probability are
made to account for the presence of small $S2$ signals in the $S2$ waveform
samples obtained from real data.

Figure 6:  The average $S2$ waveform is displayed in blue solid line. The red
solid line gives a fit to the average $S2$ waveform, which consists of the
gate ionization (orange solid line) and the delayed electron contribution from
the bulk of LXe (black and green solid lines). The gate photoionization is
modeled as Gaussian, and the bulk photoionization is modeled as two
exponentials. The $S2$ light of gate photoionization can also further generate
photoionization (secondary gate photoionzation), which is included in the plot
and shown as the magenta solid line.

###  2.5 Noise and dark counts

Apart from the aforementioned effects, it is important to consider the
presence of noise, including the spurious waveforms, and dark counts from
PMTs, as they can potentially overshadow the small $S1$ signals and lead to
incorrect pairing. To account for these sources of noise and dark counts, they
are incorporated into the WS by stacking real segmented waveforms on top of
the simulated waveform. These noise segments are randomly sampled from the
2-ms window preceding the identified $S1$s. The level of noise and the rate of
dark counts appear to have a correlation with the type of run being conducted.
Fig. 7 displays selected segmented waveforms from scientific runs for DM
search, ER calibration, and NR calibration, respectively.

Figure 7:  Sample noise waveforms from 220Rn calibration data (top), the
neutron calibration data (middle), and DM data (bottom).

##  3 Comparison to data

To evaluate the performance of the WS, a comparative analysis is carried out
by examining the distributions of key variables between the simulated samples
and the experimental data. These key variables are derived by subjecting the
simulated waveforms to the same data processing algorithm employed in the
PandaX-4T experiment. The variables selected for comparison can be classified
into four distinct categories, with respect to $S1$ pulse shape, $S2$ pulse
shape, pattern, and waveform “dirtiness”. The description and exact
definitions of these variables can be found in Table 1.

Category |  Symbol |  Description |  reference
---|---|---|---
$S1$-related |  $N^{\textrm{peak}}_{S1}$ |  Number of peaks of the major $S1$. |  Fig. 8 and 10
$Q_{S1}$ |  Total charge of the major $S1$.
$h_{S1}$ |  Pulse maximum height of the major $S1$.
$W_{S1}$ |  The end time minus the start time of major $S1$ pulse.
$W_{S1}^{\textrm{FWHM}}$ |  Half-height width of the major $S1$.
$W_{S1}^{\textrm{ten}}$ |  10%-height width of the major $S1$.
$N^{\textrm{hit}}_{S1}$ |  Number of hits for the major $S1$.
$N^{\textrm{cand}}_{S1}$ |  Number of candidate $S1$s in the event.
$M^{\textrm{bot}}_{S1}$ |  Charge on the most-fired bottom PMT for a major $S1$.
$S2$-related |  $N^{\textrm{peak}}_{S2}$ |  Number of peaks of the major $S2$. |  Fig. 9 and 11
$Q_{S2}$ |  Total charge of the major $S2$.
$W^{\textrm{FWHM}}_{S2}$ |  Half-height width of the major $S2$.
$W^{\textrm{ten}}_{S2}$ |  10%-height width of the major $S2$.
$h_{S2}$ |  Pulse maximum height of the major $S2$.
$R_{\textrm{pre}S2}$ |  Fraction of charge in pre-maximum-height window to total for the major $S2$.
$N^{\textrm{hit}}_{S2}$ |  Number of hits for the major $S2$.
$W^{80}_{S2}$ |  $S2$ width that contain 80% charge.
$\sigma^{\textrm{hit}}_{S2}$ |  Standard deviation of the charges of hits among PMTs for the major $S2$.
Pattern-related |  $A_{S1}$ |  Top-bottom asymmetry of the major $S1$. |  Fig. 12 and 13
$A_{S2}$ |  Top-bottom asymmetry of the major $S2$.
$F_{S1}^{\textrm{maxq}}$ |  Fraction of charge on most-fired PMT over total charge of the major $S1$.
$F_{S2}^{\textrm{maxq}}$ |  Fraction of charge on most-fired PMT over total charge of the major $S2$.
$\sigma^{ch}_{S1}$ |  Standard deviation of the charges of fired PMTs for the major $S1$.
$\sigma^{\textrm{CoG}}_{S2_{b}}$ |  Root-mean-square of the hit PMT positions to the center-of-gravity reconstructed position for the major $S2$.
Waveform “dirtiness” |  $\rho_{\textrm{preS1}}$ |  Charge density before the major $S1$. |  Fig. 14 and 15
$\rho_{S1-S2}$ |  Charge density inbetween the major $S1$ and $S2$.
$\rho_{\textrm{postS2}}$ |  Charge density after the major $S2$.
$F_{\textrm{S1-S2}}$ |  Charge fraction of the major $S1$ plus $S2$ to the total charge in event waveform.
Table 1:  List of key variables that are used in the comparison between data
and WS.

The comparison is conducted using both the 220Rn and neutron calibration data
for ER and NR, respectively. The data selections that are used for DM search
[31] are applied. To mitigate the potential influence arising from
correlations between the key variables and parameters such as $S1$ charge,
$S2$ charge, and event position, it is necessary to ensure that the
distributions of $S1$, $S2$, and position in both the WS and data samples are
compatible.

###  3.1 Comparison of $S1$ and $S2$ pulse shape related variables

Figure 8:  Comparisons of S1 key variables which are related to pulse shapes
using ER calibration data. The red error bars represent the data distribution,
and the shaded green histogram gives the distribution from the WS. The
$\chi^{2}$ values divided by the degree of freedom are shown on top of each
panel.  Figure 9:  Comparisons of S2 key variables which are related to pulse
shapes using ER calibration data. The red error bars represent the data
distribution, and the shaded green histogram gives the distribution from the
WS. The $\chi^{2}$ values divided by the degree of freedom are shown on top of
each panel.  Figure 10:  Comparisons of S1 key variables which are related to
pulse shapes using NR calibration data. The red error bars represent the data
distribution, and the shaded green histogram gives the distribution from the
WS. The $\chi^{2}$ values divided by the degree of freedom are shown on top of
each panel.  Figure 11:  Comparisons of S2 key variables which are related to
pulse shapes using NR calibration data. The red error bars represent the data
distribution, and the shaded green histogram gives the distribution from the
WS. The $\chi^{2}$ values divided by the degree of freedom are shown on top of
each panel.

It is crucial to verify the fidelity of the WS by closely matching the pulse
shapes of the simulated waveforms to those observed in real data. To validate
this aspect, a comparison is conducted between the WS and data based on a set
of selected variables, including pulse height, pulse width, number of hits,
hit charge variations, and the like. The comparisons are performed using ER
and NR calibration data, respectively, and the chi-square values are provided
in each plot as a measure of agreement. Fig. 8, 10, 9, and 11 illustrate the
comparisons for $S1$ and $S2$ pulses in both ER and NR calibration data. The
majority of variables demonstrate good agreement between the WS and data, with
only a few variables exhibiting noticeable differences, including $h_{S2}$,
$\sigma_{S2}^{\textrm{hit}}$, and $N_{S2}^{\textrm{peak}}$. This is speculated
to arise from the limited statistical significance of the calibration data
comprising the $S1$ and SE pools in the WS. In addition, a relaxed selection
criterion is employed for these $S1$ and SE events in order to augment the
statistics, potentially introducing impurities into the collected data.

###  3.2 Comparison of pattern related variables

Not only the temporal features of the simulated pulses are of great
importance, the pattern on PMTs plays also a crucial role. Fig. 12 and 13 show
the comparison of $S1$ and $S2$ pattern related variables between the WS and
calibration data, including the top-bottom asymmetries for $S1$ and $S2$, the
standard deviation of hit charges among fired PMTs, RMS of the hit PMT
positions. Most variables show good agreement with the exception for the
fraction of $S2$ charge on the most-fired PMT over total $S2$ charge
$F_{S2}^{\textrm{maxq}}$. This is attributed to the inherent uncertainty in
the reconstructed positions of the SEs within the simulation pool. It is
important to notice that the reconstructed positions of the SEs exhibit
considerable statistical uncertainty due to the low $S2$ charge, particularly
in proximity to the edge of the TPC. Given that the actual positions of the SE
blocks constituting the simulated $S2$ may not align with the expected
position, it is comprehensible that discrepancies in $S2$ pattern-related
variables between the data and WS arise.

Figure 12:  Comparisons of pattern-related key variables using ER calibration
data. The red error bars represent the data distribution, and the shaded green
histogram gives the distribution from the WS. The $\chi^{2}$ values divided by
the degree of freedom are shown on top of each panel.  Figure 13:  Comparisons
of pattern-related key variables using NR calibration data. The red error bars
represent the data distribution, and the shaded green histogram gives the
distribution from the WS. The $\chi^{2}$ values divided by the degree of
freedom are shown on top of each panel.

###  3.3 Comparison of waveform “dirtiness” related variables

The presence of spurious noise in the waveforms can introduce incompatibility
between the WS and the data. To evaluate this effect, direct comparisons are
performed for several selected variables, as summarized in Table 1 and
depicted in Fig. 14 and 15. Initial results indicate slight deviations between
the WS and the data, suggesting the need for adjustments in the production
probability of the photoionization model within the WS. Subsequent
modifications are made based on these findings, leading to a satisfactory
improvement in the matching between the WS and the data.

Figure 14:  Comparisons of waveform “dirtiness”-related key variables using ER
calibration data. The red error bars represent the data distribution, and the
shaded green histogram gives the distribution from the WS. The chi square
values divided by the degree of freedom are shown on top of each panel.
Figure 15:  Comparisons of waveform “dirtiness”-related key variables using NR
calibration data. The red error bars represent the data distribution, and the
shaded green histogram gives the distribution from the WS. The chi square
values divided by the degree of freedom are shown on top of each panel.

##  4 Summary and discussion

We have presented a detailed overview of the waveform simulation framework
developed for the PandaX-4T experiment. This WS framework has been extensively
utilized to address various research objectives that require a sufficient
statistical sample size, which may not be readily available in the
experimental data. The primary application of the WS is to generate synthetic
samples for conducting studies related to reconstruction efficiency and bias,
as referenced in [31]. Additionally, the WS serves as a valuable resource for
generating training samples for Boosted Decision Tree algorithms, as mentioned
in [15]. Through our analysis, we have demonstrated that the data-driven
waveform simulation provides a comparable description of the experimental
waveforms, particularly in terms of pulse shape, pattern, and the presence of
spurious noise. This highlights the effectiveness and reliability of the WS in
capturing important features of the observed data. With more calibration data
and understanding of the detector, we expect to further improve the WS,
especially in terms of correlations between pulse shape and pattern, as well
as its efficacy in high-energy regimes. Additionally, we envision broadening
the application of the WS to encompass the interpretation of a wider range of
physical models.

## Acknowledgements

This project is supported in part by grants from National Natural Science
Foundation of China (Nos. 12090060, 12090061, 12005131, 11905128, 11925502,
12222505, 11835005), and by the Office of Science and Technology, Shanghai
Municipal Government (grant No. 22JC1410100). We thank supports from Double
First Class Plan of the Shanghai Jiao Tong University. We also thank the
sponsorship from the Hongwen Foundation in Hong Kong, Tencent Foundation in
China, and Yangyang Development Fund. Finally, we thank the CJPL
administration and the Yalong River Hydropower Development Company Ltd. for
indispensable logistical support and other help.

## References

  * [1] Gianfranco Bertone, Dan Hooper, and Joseph Silk.  Particle dark matter: Evidence, candidates and constraints.  Physics reports, 405(5-6):279–390, 2005.
  * [2] Gerard Jungman, Marc Kamionkowski, and Kim Griest.  Supersymmetric dark matter.  Physics Reports, 267(5-6):195–373, 1996.
  * [3] Jonathan L Feng.  Dark matter candidates from particle physics and methods of detection.  Annual Review of Astronomy and Astrophysics, 48:495–545, 2010.
  * [4] Xiang Xiao et al.  Low-mass dark matter search results from full exposure of the pandax-i experiment.  Phys. Rev. D, 92(5):052004, 2015.
  * [5] Andi Tan, Mengjiao Xiao, Xiangyi Cui, Xun Chen, Yunhua Chen, Deqing Fang, Changbo Fu, Karl Giboni, Franco Giuliani, Haowei Gong, et al.  Dark matter results from first 98.7 days of data from the pandax-ii experiment.  Physical review letters, 117(12):121303, 2016.
  * [6] XENON Collaboration, E Aprile, J Aalbers, F Agostini, M Alfonsi, L Althueser, FD Amaro, M Anthony, F Arneodo, L Baudis, et al.  Dark matter search results from a one ton-year exposure of xenon1t.  Physical Review Letters, 121(11):111302, 2018.
  * [7] DS Akerib, S Alsum, HM Araújo, X Bai, AJ Bailey, J Balajthy, P Beltrame, EP Bernard, A Bernstein, TP Biesiadzinski, et al.  Results from a search for dark matter in the complete lux exposure.  Physical review letters, 118(2):021303, 2017.
  * [8] R Agnese, T Aramaki, IJ Arnquist, W Baker, D Balakishiyeva, S Banik, D Barker, R Basu Thakur, DA Bauer, T Binder, et al.  Results from the super cryogenic dark matter search experiment at soudan.  Physical review letters, 120(6):061802, 2018.
  * [9] F Petricca, G Angloher, P Bauer, A Bento, C Bucci, L Canonica, X Defay, A Erb, F v Feilitzsch, N Ferreiro Iachellini, et al.  First results on low-mass dark matter from the cresst-iii experiment.  In Journal of Physics: Conference Series, volume 1342, page 012076\. IOP Publishing, 2020.
  * [10] Yue Meng, Zhou Wang, Yi Tao, Abdusalam Abdukerim, Zihao Bo, Wei Chen, Xun Chen, Yunhua Chen, Chen Cheng, Yunshan Cheng, et al.  Dark matter search results from the pandax-4t commissioning run.  Physical Review Letters, 127(26):261802, 2021.
  * [11] E Aprile, K Abe, F Agostini, S Ahmed Maouloud, L Althueser, B Andrieu, E Angelino, JR Angevaare, VC Antochi, D Antón Martin, et al.  First dark matter search with nuclear recoils from the xenonnt experiment.  Physical Review Letters, 131(4):041003, 2023.
  * [12] LUX-ZEPLIN Collaboration et al.  First dark matter search results from the lux-zeplin (lz) experiment.  Physical Review Letters, 131(4):1–11, 2023.
  * [13] Ke-Jun Kang, Jian-Ping Cheng, Yun-Hua Chen, YJ Li, MB Shen, SY Wu, and Q Yue.  Status and prospects of a deep underground laboratory in china.  In Journal of Physics: Conference Series, volume 203, page 012028\. IOP Publishing, 2010.
  * [14] Jian-Ping Cheng, Ke-Jun Kang, Jian-Min Li, Jin Li, Yuan-Jing Li, Qian Yue, Zhi Zeng, Yun-Hua Chen, Shi-Yong Wu, Xiang-Dong Ji, et al.  The china jinping underground laboratory and its early science.  Annual Review of Nuclear and Particle Science, 67:231–251, 2017\.
  * [15] Wenbo Ma, Abdusalam Abdukerim, Chen Cheng, Zihao Bo, Wei Chen, Xun Chen, Yunhua Chen, Zhaokan Cheng, Xiangyi Cui, Yingjie Fan, et al.  Search for solar b 8 neutrinos in the pandax-4t experiment using neutrino-nucleus coherent scattering.  Physical Review Letters, 130(2):021802, 2023.
  * [16] Shuaijie Li, Mengmeng Wu, Abdusalam Abdukerim, Zihao Bo, Wei Chen, Xun Chen, Yunhua Chen, Chen Cheng, Zhaokan Cheng, Xiangyi Cui, et al.  Search for light dark matter with ionization signals in the pandax-4t experiment.  Physical Review Letters, 130(26):261001, 2023.
  * [17] Linhui Gu, Abdusalam Abdukerim, Zihao Bo, Wei Chen, Xun Chen, Yunhua Chen, Chen Cheng, Yunshan Cheng, Zhaokan Cheng, Xiangyi Cui, et al.  First search for the absorption of fermionic dark matter with the pandax-4t experiment.  Physical Review Letters, 129(16):161803, 2022.
  * [18] Dan Zhang, Abdusalam Abdukerim, Zihao Bo, Wei Chen, Xun Chen, Yunhua Chen, Chen Cheng, Zhaokan Cheng, Xiangyi Cui, Yingjie Fan, et al.  Search for light fermionic dark matter absorption on electrons in pandax-4t.  Physical Review Letters, 129(16):161804, 2022.
  * [19] XENONnT.  WFSim: Waveform simulator, 2023.
  * [20] Shinzou Kubota, Masahiko Hishida, and Jian-zhi Raun.  Evidence for a triplet state of the self-trapped exciton states in liquid argon, krypton and xenon.  Journal of Physics C: Solid State Physics, 11(12):2645, 1978.
  * [21] Akira Hitachi, Tan Takahashi, Nobutaka Funayama, Kimiaki Masuda, Jun Kikuchi, and Tadayoshi Doke.  Effect of ionization density on the time dependence of luminescence from liquid argon and xenon.  Physical Review B, 27(9):5279, 1983.
  * [22] Wenbo Ma, Abdusalam Abdukerim, Zihao Bo, Wei Chen, Xun Chen, Yunhua Chen, Chen Cheng, Xiangyi Cui, Yingjie Fan, Deqing Fang, et al.  Internal calibration of the pandax-ii detector with radon gaseous sources.  Journal of Instrumentation, 15(12):P12038, 2020.
  * [23] Oumarou Njoya, T Tsang, M Tarka, W Fairbank, KS Kumar, T Rao, Trey Wager, Soud Al Kharusi, G Anton, Isaac J Arnquist, et al.  Measurements of electron transport in liquid and gas xenon using a laser-driven photocathode.  Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, 972:163965, 2020\.
  * [24] Shaoli Li, Xun Chen, Karl L Giboni, Guodong Guo, Xiangdong Ji, Qing Lin, Jianglai Liu, Yajun Mao, Kaixuan Ni, Xiangxiang Ren, et al.  Performance of photosensors in the pandax-i experiment.  Journal of Instrumentation, 11(02):T02005, 2016.
  * [25] E Aprile, M Alfonsi, K Arisaka, F Arneodo, C Balan, L Baudis, B Bauermeister, A Behrens, Philippe Beltrame, K Bokeloh, et al.  Observation and applications of single-electron charge signals in the xenon100 experiment.  Journal of Physics G: Nuclear and Particle Physics, 41(3):035201, 2014.
  * [26] Elena Aprile, K Abe, F Agostini, S Ahmed Maouloud, M Alfonsi, L Althueser, E Angelino, JR Angevaare, Vasile C Antochi, D Antón Martin, et al.  Emission of single and few electrons in xenon1t and limits on light dark matter.  Physical Review D, 106(2):022001, 2022.
  * [27] DS Akerib, S Alsum, HM Araújo, X Bai, J Balajthy, A Baxter, EP Bernard, A Bernstein, TP Biesiadzinski, EM Boulton, et al.  Investigation of background electron emission in the lux detector.  Physical Review D, 102(9):092004, 2020.
  * [28] P Sorensen and Kate Kamdin.  Two distinct components of the delayed single electron noise in liquid xenon emission detectors.  Journal of Instrumentation, 13(02):P02032, 2018.
  * [29] A Tomás, HM Araújo, AJ Bailey, A Bayer, E Chen, B López Paredes, and TJ Sumner.  Study and mitigation of spurious electron emission from cathodic wires in noble liquid time projection chambers.  Astroparticle Physics, 103:49–61, 2018.
  * [30] Adam Bailey.  Dark Matter Searches and Study of Electrode Design in LUX and LZ. PhD thesis, Imperial College London, 2016.
  * [31] PandaX-4T collaboration.  Pandax-4t analysis paper.  To be published.