††thanks: These authors contributed equally.††thanks: These authors
contributed equally.

# Magnetic field regression using artificial neural networks for cold atom
experiments

Ziting Chen  Department of Physics, The Hong Kong University of Science and
Technology, Clear Water Bay, Kowloon, Hong Kong, China  Kin To Wong
Department of Physics, The Hong Kong University of Science and Technology,
Clear Water Bay, Kowloon, Hong Kong, China  Bojeong Seo  Department of
Physics, The Hong Kong University of Science and Technology, Clear Water Bay,
Kowloon, Hong Kong, China  Mingchen Huang  Department of Physics, The Hong
Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong
Kong, China  Mithilesh K. Parit  Department of Physics, The Hong Kong
University of Science and Technology, Clear Water Bay, Kowloon, Hong Kong,
China  Haoting Zhen  Department of Physics, The Hong Kong University of
Science and Technology, Clear Water Bay, Kowloon, Hong Kong, China  Jensen Li
Department of Physics, The Hong Kong University of Science and Technology,
Clear Water Bay, Kowloon, Hong Kong, China  Gyu-Boong Jo<EMAIL_ADDRESS>Department of Physics, The Hong Kong University of Science and Technology,
Clear Water Bay, Kowloon, Hong Kong, China

###### Abstract

Accurately measuring magnetic fields is essential for magnetic-field sensitive
experiments in fields like atomic, molecular, and optical physics, condensed
matter experiments, and other areas. However, since many experiments are
conducted in an isolated vacuum environment that is inaccessible to
experimentalists, it can be challenging to accurately determine the magnetic
field. Here, we propose an efficient method for detecting magnetic fields with
the assistance of an artificial neural network (NN). Instead of measuring the
magnetic field directly at the desired location, we detect magnetic fields at
several surrounding positions, and a trained NN can accurately predict the
magnetic field at the target location. After training, we achieve a relative
error of magnetic field magnitude (magnitude of error over the magnitude of
magnetic field) below 0.3$\%$, and we successfully apply this method to our
erbium quantum gas apparatus. This approach significantly simplifies the
process of determining magnetic fields in isolated vacuum environments and can
be applied to various research fields across a wide range of magnetic field
magnitudes.

## Introduction

Precisely calibrating the magnetic field inside a vacuum chamber is of
experimental significance to various fields. For instance, a magnetic field is
one of the common control knob in the ultracold atoms experiment, enabling
various studies on many-body physics bloch2008many  including BEC-BCS
crossover bourdel2004experimental , formation of matter-wave soliton
khaykovich2002formation ; strecker2002formation ; burger1999dark , and Efimov
states kraemer2006evidence  through the Feshbach resonance that tunes inter-
atomic interactions chin2010feshbach . However, the demanding precision of the
magnetic field imposed by these experimental control and the inaccessibility
to the vacuum chamber makes calibration of the magnetic field a difficult and
time-consuming task. Moreover, spectroscopic measurement, a typical approach
to magnetic field calibration, is mainly sensitive only to the magnitude of
the magnetic field. The magnetic field direction and its precise calibration
are of critical importance for magnetic atoms (e.g. erbium or dysprosium)
where the orientation of the magnetic dipole moment plays a critical role
lahaye2009physics ; chomaz2022dipolar .

Recent years have witnessed the great success of neural network (NN) applied
to assist experiments, including multi-parameter optimization of magneto-
optical trap tranter2018multiparameter ; seo2021maximized , optimization of
production of Bose-Einstein condensate wigley2016fast ; barker2020applying ;
vendeiro2022machine ; davletov2020machine , and recognition of hidden phase
from experimental data zhang2019machine ; zhao2021heuristic ;
zhao2022observing ; guo2021machine . Here, we introduce a novel method to
precisely determine the magnetic field vector $B=(B_{x},B_{y},B_{z})$ inside a
vacuum chamber with the assistance of a NN. Since the target position inside
the vacuum chamber is typically inaccessible, we detect magnetic fields at
several surrounding positions, which are sent to the trained NN that is able
to accurately deduce the magnetic field inside the vacuum chamber. We apply
this method to our apparatus of erbium quantum gas seo2020efficient ;
seo2023apparatus ; chen2021active , which has large magnetic dipole moment
making it particularly sensitive to magnetic field vector. aikawa2012bose ;
frisch2014quantum . We present the details of the NN-based method, including
setting up the simulation model, training process, and final performance. For
simplicity, magnetic field data for training and validation of NN is generated
by a standard finite-element simulation package from COMSOL Multiphysics
electromagnetic module pepper2017finite , instead of experimental measurement.
Moreover, we systematically investigate the impact of the number of sensors at
surrounding positions and the magnitude of the magnetic field on the
performance of the method providing a practical guide for implementation. In
contrast to previous works solin2018modeling ; nouri2015prototype ;
nouri2014systematic ; raissi2019physics ; coskun2022magnetic , which predict
the magnetic field vector across a wide experimental region, our goal in this
work is to extrapolate the magnetic field vector at a specific position within
an inaccessible region. Our approach provides a simple method for monitoring
magnetic fields without requiring any prior knowledge of solution of the
Maxwell equation.

Figure 1:  Schematic of the simulation model. (a) Overview of the model. The
main body of the model is an experimental chamber in ultra-high vacuum
environment. The control magnetic field $B_{control}$ is generated by three
orthogonal pairs of copper coils. Several magnetic field sensors surrounding
vacuum chamber are indicated in light yellow. (b) Top view of the model. An
exemplary magnetic fields (blue arrows) generated by coils are shown. A red
arrow is also added to show the Earth’s magnetic field in the model which is
set to the x direction.

## Methodology

The implemented machine learning algorithm is an artificial NN that can be
coded by common Python packages like Tensorflow and Pytorch tensorflow ;
pytorch . The magnetic field measured by sensors outside a vacuum chamber is
fed into the NN, which predicts the magnetic field at the center of the
chamber. Hence, the function of the NN is to act as a hyperplane that relates
the magnetic field at different spatial points. Regressing such a hyperplane
would require a substantial amount of data. Obtaining training data from a
real experimental setup, though it can take into account every factor, is
difficult and time-consuming. Instead, we generate training data from finite
element simulation using COMSOL Multiphysics. As long as the simulation model
takes into account important factors, simulation data would be a reliable
substitute for real data.

The simulation model is shown in Fig. 1, which contains a science chamber
commonly used in cold atoms experiments. Originally, the chamber is made of
materials including 314L, 304 stainless steel, aluminum, glass, and plastic.
However, to reduce computational time, non-magnetic parts of the chamber are
removed as they do not affect the result, and only parts made of 314L, and 304
stainless steel are kept. 314L and 304 stainless steel, under low strain and
irradiation, can be considered as linear materials with no hysteresis loop
Xu2018 . In addition, only a static magnetic field is of concern for most
experiments. Therefore, we only need to simulate the magnetic field under
time-independent conditions. In order to generate a stable magnetic field,
three pairs of copper coils with constant current are placed around the
chamber along three orthogonal directions, mimicking the coils used in
experiments. To account for the Earth magnetic field $B_{Earth}$ in the
laboratory, a constant 400 mG magnetic field is added in the x-direction. To
notice, when adopting this method, it is important to check the direction and
magnitude of $B_{Earth}$ before training the NN since they can vary in
different locations.

Figure 2: Pipeline figure for the overall procedure. First, a simulation model
which contains a science chamber and coils as magnetic field sources are set
up. Then, the model is simulated to obtain data that is passed into an
artificial NN for training. The NN has four layers: the input layer which is
the magnetic field measured by sensors outside the chamber, two fully
connected hidden layers with ReLU as activation functions, output layer which
is the target magnetic field at the chamber center. At last, the prediction of
the NN is evaluated.

Even though simulation data is much easier to obtain than real data,
simulation of each datum still requires a significant amount of time. To
diminish data acquisition time, we exploit the linearity of Maxwell’s
equations that any superposition of solutions is also a valid solution for
linear materials. By virtue of this, the whole output space can be mapped
using only three linear independent results, which can be obtained by
simulating each pair of coils in the model. Using these results, we obtain new
data that is not redundant and form a large data set.

As shown in Fig. 2, after setting up the simulation model and obtaining the
simulation data, we input the data into the artificial NN for training and
prediction testing. The implemented NN contains two fully-connected hidden
layers with ReLU as activation functions. During the training, the neurons’
parameters of the NNs are adjusted to minimize the root-mean-square error
(RMSE) loss function using the Adam optimizer zhao2022observing . This is a
common loss function for regression and can be defined as
$\mathbf{RMSE}=\sqrt{\frac{1}{3n}\sum_{i}^{n}\sum_{\alpha}^{x,y,z}(y^{\alpha}_{i}-\hat{y}^{\alpha}_{i})^{2}}$
where $n$ is the number of data, $\hat{y}$ is the actual output, $y$ is the
predicted output and $\alpha$ denotes the component of the magnetic field
data. Total number of simulation data is 1.5$\times$105, 80$\%$ of which is
used for training while 20$\%$ is reserved for validation. When the validation
loss isn’t improving for several epochs, the training is terminated to prevent
overtraining. It should be noted, however, RMSE could not reflect the
statistic of the error for each measurement.

To properly evaluate the performance of the NN-based magnetic field
regression, we define the relative prediction error which is defined as the
vector difference between the actual output and prediction over the magnitude
of the actual output:

$\text{Relative prediction error}=\frac{||y-\hat{y}||}{||\hat{y}||}$ (1)

This value is evaluated from another data set of size of 105. We calculate the
error from each datum to form a set of errors and extract the upper bound of
the relative error, below which 90$\%$ of data points would be accurately
estimated.

## Interpretation of the training process

To fully evaluate the NN’s performance, it is essential to first grasp the
purpose of training the NN or specifically, the target equation of the
regression. In theory, the hyperplane relating magnetic field at different
spatial points can be calculated using Maxwell’s equations directly. By
substituting the spatial coordinates of the target point into the general
solution, we remove its spatial degree of freedom, causing the output to
depend only on the integration constants that are determined by the boundary
conditions. However, the boundary conditions can just be the magnetic field at
other spatial points. Hence, in principle, we can extract a hyperplane from
the solution that serves the same function as the NN. Since this hyperplane is
entirely based on Maxwell’s equations, its output must always be correct.
Therefore, the goal of training the NN is to reduce the gap between the NN and
this ideal hyperplane.

To have a better understanding of the relation between Maxwell’s equations and
the NNs, we look at how the error depends on the number of sensors. As shown
in Fig. 3, the error drastically reduces when the number of sensors increases
to 6 and has a similar value onward. This dramatic reduction can be explained
by the number of boundary conditions required to identify the hyperplane from
directly calculating Maxwell’s equations.
$\nabla^{2}\mathbf{B}-\frac{1}{c^{2}}\frac{\partial^{2}\mathbf{B}}{\partial
t^{2}}=-\mu\nabla\times\mathbf{J}$ shows the form of the time-dependent
Maxwell’s equations that only depend on the magnetic field. Through this, we
can count the number of integration constants and thus, the boundary
conditions required to get a unique solution by multiplying the number of
independent variables, the order of differential equation, and the number of
components together. For the Maxwell’s equation, the obtained value is 24, but
we are only concerning time-independent cases, therefore resulting required
number of 18. Since the boundary conditions are provided by magnetic field
measured at different spatial points by sensors, the minimum number of
sensors, each has three components, needed to obtain a unique and accurate
result is equal to 6. This, together with the claim that the NN is
approximating the ideal hyperplane calculated from Maxwell’s equations,
explains the rapid reduction of error in Fig. 3. When below 6 sensors are
used, the NN does not have enough information needed for the ideal hyperplane
to calculate the unique result. Therefore, the trend in Fig. 3 indicates a
strong relation between the NN and Maxwell’s equations, strengthening the
claim that training the NN is equivalent to bringing the NN closer to the
ideal hyperplane calculated from Maxwell’s equations. However, it is crucial
to notice the NN can only approximate the ideal hyperplane but never reach it
since their mathematical forms are fundamentally different. Hence, errors
always exist from the network’s prediction. Besides, the quality of the
approximation greatly depends on and is limited by the training conditions and
data. This aspect of the network will be clearly shown in the following
result.

## NN’s performance over a range of magnetic field

In this study, we evaluate the performance of the neural network under various
magnetic fields created by the coils $B_{control}$, ranging from 0.6 G to 100
G. These values cover the typical range of magnetic fields used in experiments
involving ultracold dipolar atoms seo2023apparatus . This can be achieved by
generating multiple prediction data sets with different average magnitudes of
output. Before commencing the test, it is crucial to consider the average
magnitude of the output of the training data since it could significantly
affect the training process that determines the network’s response to
different magnetic fields.

Figure 3: Relative error of magnetic field Relative prediction error of
magnetic field versus different numbers of sensors, with the vertical axis in
logarithmic scale. The error plummets when the number of sensors increases
from 1 to 6. Afterward, the error stays in the same order of magnitude and
varies non-systematically.

Fig. 4(a) demonstrates the NN’s accuracy in predicting the magnetic field at
the center of the chamber under different magnetic field conditions. When the
NN is trained with weak magnetic field strength ($B_{control}$) of 1-5 G, the
error is minimized within that range and can be reduced to as low as
approximately 0.2$\%$. However, when the NN is trained with a strong magnetic
field strength of 10-50 G, the error is minimized at a $B_{control}$ order of
magnitude of 10 G. In addition, when the $B_{control}$ is outside of the
specified range, the error quickly surges to above 10$\%$. This issue could be
avoided if the magnetic field were completely rescalable. Unfortunately, this
is not possible due to the presence of the constant Earth magnetic field
($B_{Earth}$). As $B_{control}$ changes, so does $B_{Earth}/B_{control}$, and
these variations can be drastic, particularly when $B_{control}$ changes by an
entire order of magnitude. If the NN is trained with a weak magnetic field
strength, there is a possibility that the network would calculate the bias
based on $B_{Earth}/B_{control}$ for a weaker field even when $B_{control}$ is
much stronger, leading to a higher error. This also explains why the
applicable range of $B_{control}$ for NN trained at weaker $B_{control}$ is
narrower, as $B_{Earth}/B_{control}$ varies much faster at weaker magnetic
field strengths.

To expand the applicable range of the neural network, one option is to train
it on a larger range of $B_{control}$. However, this method has been proven to
be ineffective, as shown in Fig. 4(a). The figure demonstrates that the errors
between the neural network trained with a large range of $B_{control}$ and the
one trained with strong $B_{control}$ are very similar. This indicates that
the neural network simply ignores weaker $B_{control}$. The reason for this
lies in the use of the RMSE loss function in absolute value. Larger
$B_{control}$ usually produces larger absolute errors, causing the network’s
parameters to be tuned in a way that provides more accurate results at larger
$B_{control}$ over weaker ones.

Figure 4: Relative prediction error for various conditions Relative prediction
error versus the magnetic field generated by the coils $B_{control}$, covering
from 0.6 to 100 G range. (a) The Earth’s magnetic field $B_{Earth}$ is present
and compensated (assumed to be 400 mG). The legend shows in which range of
field the artificial NN is trained. Weak refers to a field from 1 to 5 G and
strong refers to 10 to 50 G range. The error is minimized and reduced to
around 0.2$\%$ over a range where the training field has a similar magnitude
as the testing field. However, the error increases rapidly to over 10$\%$
outside the working range. The working range of the NN trained with a stronger
field is wider (in log scale) than the one trained with a weak field. When the
NN is trained with both weak and strong fields, the result is similar to the
one produced by a NN trained with a strong field only. (b) $B_{Earth}$ (EB in
the figure) is either compensated or removed entirely as indicated by the
legend. For the one with compensated $B_{Earth}$, the NN is trained with a
weak field. The minimized region still remains but the increase in error is
way less drastic outside the range compared to having full $B_{Earth}$. The
error can be suppressed under 0.7$\%$ over the whole range. For the one
without $B_{Earth}$, the NN is trained at 2.5 G. The performance of the
network is very consistent over the whole range with errors around or below
0.3$\%$.

Here we suggest two viable ways to improve the working range of the NN. First,
the NN can simply be trained without $B_{Earth}$ in the data by adjusting the
sensors offsets until the reading is zero or compensating $B_{Earth}$ in the
laboratory. Both of the methods involve removing the effect of the $B_{Earth}$
during the training process. In Fig. 4(b), the error of the network’s
prediction can then be kept below 0.4$\%$ over the whole range when there is
no $B_{Earth}$. Therefore, this method is very effective in principle but
requires good control over the sensors.

The second method is to compensate $B_{Earth}$ using coils in the laboratory
to attenuate the effect of $B_{Earth}$. Practically, it is difficult to
completely eliminate $B_{Earth}$, but it is possible to compensate 90$\%$.
Therefore, we evaluate the effectiveness of this method by training NN with a
small bias of 40 mG, which is displayed in Fig. 4(b). The error is below
0.7$\%$ over the whole range and below 0.3$\%$ from 20 G to 100 G, indicating
a good precision and wide working range can be achieved simultaneously with
this method.

Nonetheless, even when both of the above methods are not applicable, the NN is
still useful and powerful. As long as the approximate value of the magnetic
field is known, the properly trained NN can always be picked to calculate the
magnetic field with an extremely low error. When the order of the magnetic
field is unknown, however, we can still use the NN trained with the wrong
range to find back the order of magnitude since the error, in this case, is
still over an order larger. The more accurate result can then be calculated
using the properly trained NN.

## Application to cold atoms experiments

The performance of magnetic field regression demonstrated in this work opens a
new possibility of applying this method to experiments with cold atoms. One
such system is an apparatus of dipolar erbium atoms which have a large
magnetic moment of 7$\mu_{B}$ where $\mu_{B}$ is the Bohr magneton and
contains a dense spectrum of Feshbach resonances frisch2014quantum . Even at a
low magnetic field regime, erbium atoms have multiple Feshbach resonances for
a magnetic field smaller than 3 G, where the widths of these resonances are
around 10-200 mG aikawa2012bose . The regression method could allow us to
monitor the magnetic field vector with the resolution of $\sim$10 mG in the
range of 3 G, which is accurate enough to properly calibrate the experimental
system. This is still favorable for other atomic species, such as alkali
atoms, since a maximal error of magnetic field is about 1.25 G in the range of
500 G, which is sensitive enough compared to the linewidth of most commonly
used Feshbach resonances chin2010feshbach . However, even though the
regression method can provide useful results, we suggest that its accuracy
should be verified with another method, such as radio-frequency spectroscopy.

Apart from precisely determining the magnetic field inside a vacuum chamber,
the proposed method also serves as a quick indicator for magnetic field
change. For instance, when sensors around the vacuum chamber give unexpected
values, it indicates a change in a magnetic field. If all 6 sensors are
changed along the same direction, then an external magnetic field is suddenly
introduced to the system. On the other hand, if these 6 sensors change in a
different direction, then it is likely because the position of a device within
the area is changed. The former can be solved by compensating the external
magnetic field by coils, while the latter requires to re-train the NN. The
change of sensor value signals a change in the environmental magnetic field,
which provides important information and is easy to monitor.

## Conclusion

In conclusion, a novel method based on NN to precisely determine magnetic
fields in an isolated vacuum environment is demonstrated. An artificial NN is
trained such that it outputs the magnetic field at the center of the vacuum
chamber based on the magnetic fields surrounding the chamber. The effect of
the number of sensors and the magnitude of the magnetic field on the
performance of the trained NN are evaluated.

After training, the relative error of magnetic field magnitude below 0.3$\%$
is achieved under a wide range of magnetic fields, which is sufficient to
calibrate the magnetic field in our erbium quantum gas apparatus, where many
narrow Feshbach resonances exist even in the low field regime. Besides,
experiments with other atomic species can benefit from this method.
Furthermore, as no special setup is required, the established method can be
extended to other magnetic field-sensitive experiments conducted in an
isolated environment.

### Acknowledgments

We acknowledge support from the RGC through 16306119, 16302420, 16302821,
16306321, 16306922, C6009-20G, N-HKUST636-22, and RFS2122-6S04.

## References

  * (1) I. Bloch, J. Dalibard and W. Zwerger, Reviews of modern physics 80, 885 (2008).
  * (2) T. Bourdel, L. Khaykovich, J. Cubizolles, J. Zhang, F. Chevy, M. Teichmann, L. Tarruell, S. Kokkelmans and C. Salomon, Physical Review Letters 93, 050401 (2004).
  * (3) L. Khaykovich, F. Schreck, G. Ferrari, T. Bourdel, J. Cubizolles, L. D. Carr, Y. Castin and C. Salomon, Science 296, 1290 (2002).
  * (4) K. E. Strecker, G. B. Partridge, A. G. Truscott and R. G. Hulet, Nature 417, 150 (2002).
  * (5) S. Burger, K. Bongs, S. Dettmer, W. Ertmer, K. Sengstock, A. Sanpera, G. V. Shlyapnikov and M. Lewenstein, Physical Review Letters 83, 5198 (1999).
  * (6) T. Kraemer, M. Mark, P. Waldburger, J. G. Danzl, C. Chin, B. Engeser, A. D. Lange, K. Pilch, A. Jaakkola, H.-C. Nägerl et al., Nature 440, 315 (2006).
  * (7) C. Chin, R. Grimm, P. Julienne and E. Tiesinga, Reviews of Modern Physics 82, 1225 (2010).
  * (8) T. Lahaye, C. Menotti, L. Santos, M. Lewenstein and T. Pfau, Reports on Progress in Physics 72, 126401 (2009).
  * (9) L. Chomaz, I. Ferrier-Barbut, F. Ferlaino, B. Laburthe-Tolra, B. L. Lev and T. Pfau, Reports on Progress in Physics 72, (2022).
  * (10) A. D. Tranter, H. J. Slatyer, M. R. Hush, A. C. Leung, J. L. Everett, K. V. Paul, P. Vernaz-Gris, P. K. Lam, B. C. Buchler and G. T. Campbell, Nature communications 9, 1 (2018).
  * (11) S. Seo, J. H. Lee, S.-B. Lee, S. E. Park, M. H. Seo, J. Park, T. Y. Kwon and H.-G. Hong, Optics Express 29, 35623 (2021).
  * (12) P. B. Wigley, P. J. Everitt, A. van den Hengel, J. W. Bastian, M. A. Sooriyabandara, G. D. McDonald, K. S. Hardman, C. D. Quinlivan, P. Manju, C. C. Kuhn et al., Scientific reports 6, 1 (2016).
  * (13) A. J. Barker, H. Style, K. Luksch, S. Sunami, D. Garrick, F. Hill, C. J. Foot and E. Bentine, Machine Learning: Science and Technology 1, 015007 (2020).
  * (14) Z. Vendeiro, J. Ramette, A. Rudelis, M. Chong, J. Sinclair, L. Stewart, A. Urvoy and V. Vuletić, Physical Review Research 4, 043216 (2022).
  * (15) E. Davletov, V. Tsyganok, V. Khlebnikov, D. Pershin, D. Shaykin and A. Akimov, Physical Review A 102, 011302 (2020).
  * (16) Y. Zhang, A. Mesaros, K. Fujita, S. Edkins, M. Hamidian, K. Ch’ng, H. Eisaki, S. Uchida, J. Davis, E. Khatami et al., Nature 570, 484 (2019).
  * (17) E. Zhao, J. Lee, C. He, Z. Ren, E. Hajiyev, J. Liu and G.-B. Jo, Nature communications 12, 1 (2021).
  * (18) E. Zhao, T. H. Mak, C. He, Z. Ren, K. K. Pak, Y.-J. Liu and G.-B. Jo, Optics Express 30, 37786 (2022).
  * (19) S. Guo, A. R. Fritsch, C. Greenberg, I. B. Spielman and J. P. Zwolak, Machine Learning: Science and Technology 2, 035020 (2021).
  * (20) B. Seo, P. Chen, Z. Chen, W. Yuan, M. Huang, S. Du and G.-B. Jo, Physical Review A 102, 013319 (2020).
  * (21) B. Seo, Z. Chen, M. Huang, M. K. Parit, Y. He, P. Chen and G.-B. Jo, Journal of the Korean Physical Society 102, 1–6 (2023).
  * (22) Z. Chen, B. Seo, M. Huang, M. K. Parit, P. Chen and G.-B. Jo, Review of Scientific Instruments 92, 123005 (2021).
  * (23) K. Aikawa, A. Frisch, M. Mark, S. Baier, A. Rietzler, R. Grimm and F. Ferlaino, Physical review letters 108, 210401 (2012).
  * (24) A. Frisch, M. Mark, K. Aikawa, F. Ferlaino, J. L. Bohn, C. Makrides, A. Petrov and S. Kotochigova, Nature 507, 475 (2014).
  * (25) D.W. Pepper, J.C. Heinrich, The finite element method: basic concepts and applications with MATLAB, MAPLE, and COMSOL (CRC press, 2017).
  * (26) A. Solin, M. Kok, N. Wahlström, T. B. Schön and S. Särkkä, IEEE Transactions on robotics 34, 1112 (2018).
  * (27) N. Nouri, A. Biswas, M. Brown, R. Carr, B. Filippone, C. Osthelder, B. Plaster, S. Slutsky and C. Swank, Journal of Instrumentation 10, P12003 (2015).
  * (28) N. Nouri and B. Plaster, Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment 767, 92 (2014).
  * (29) M. Raissi, P. Perdikaris and G. E. Karniadakis, Journal of Computational physics 378, 686 (2019).
  * (30) U. H. Coskun, B. Sel and B. Plaster, Scientific Reports 12, 12858 (2022).
  * (31) M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin et al., arXiv preprint arXiv:1603.04467 12, (2016).
  * (32) A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga et al., Advances in neural information processing systems 32, (2019).
  * (33) C. Xu, X. Liu, F. Xue, Y. Li, W. Qian and M. Yu, Fusion Engineering and Design 133, 125 (2018).


# On Lasso estimator for the drift function in diffusion models

Gabriela<EMAIL_ADDRESS>[ Dmytro
<EMAIL_ADDRESS>[ Mark
<EMAIL_ADDRESS>[ Department of Mathematics,
University of Luxembourg presep=, ]e1,e3  Department of Mathematics,
University or Copenhagen presep=, ]e2

###### Abstract

In this paper we study the properties of the Lasso estimator of the drift
component in the diffusion setting. More specifically, we consider a
multivariate parametric diffusion model $X$ observed continuously over the
interval $[0,T]$ and investigate drift estimation under sparsity constraints.
We allow the dimensions of the model and the parameter space to be large. We
obtain an oracle inequality for the Lasso estimator and derive an error bound
for the $L^{2}$-distance using concentration inequalities for linear
functionals of diffusion processes. The probabilistic part is based upon
elements of empirical processes theory and, in particular, on the chaining
method.

62M05,

60G15,

62H12,

62M99,

Concentration inequalities,

diffusion models,

high dimensional statistics,

Lasso,

parametric estimation,

###### keywords:

[class=AMS]

###### keywords:

, and

##  1 Introduction

During past decades numerous studies have been devoted to statistical
inference for diffusion models. Researchers typically focus on parametric and
nonparametric estimation of the drift and volatility components under various
sampling schemes. We refer to excellent monographs [24, 28, 30] for an
extensive study in high and low frequency settings.

While most papers consider a fixed dimensional framework, high dimensional
statistical problems received much less attention in the field of diffusion
processes. The only notable exceptions are recent articles [10, 14, 18, 19,
35] that investigate parametric inference for high dimensional Ornstein-
Uhlenbeck models or related processes with linear drift. Their approach is
based upon the Lasso method and Dantzig selection, which have been popularised
in the context of classical discrete linear models; see e.g. [7, 8] among many
other references. Due to a more complex probabilistic structure of diffusion
models, statistical analysis of Lasso and Dantzig estimators are far from
trivial even in the Ornstein-Uhlenbeck setting. In particular, concentration
inequalities for functionals of diffusion processes, which build a backbone
for statistical applications, turn out to pose serious challenges.

In this paper we focus on high dimensional parametric estimation of the drift
function in diffusion models. We consider a $d$-dimensional ergodic diffusion
process given as a strong solution of the stochastic differential equation

$dX_{t}=-b_{\theta}(X_{t})dt+dW_{t},\qquad t\geq 0,$ (1.1)

where $\theta\in\Theta\subseteq\mathbb{R}^{p}$ is the model parameter,
$\Theta$ is an open set, $b_{\theta}:\mathbb{R}^{d}\to\mathbb{R}^{d}$,
$W=(W_{t})_{t\geq 0}$ is a $d$-dimensional Brownian motion and the initial
value $X_{0}$ is a random vector independent of $W$ (here we suppress the
dependence of $X$ on $\theta$). The underlying observations are
$(X_{t})_{t\in[0,T]}$ and we assume that $p$, $d$ and $T$ are large. Such high
dimensional models find numerous applications in physics, economics and
biology among other sciences. For instance, high dimensional diffusions are
studied in the context of particle systems within mean field theory.
Probabilistic results in this topic can be found in the classical works [31,
32, 42] while applications of particle systems in sciences are studied in [5,
9, 17] among many others. More recently, statistical inference for large
particle systems and their mean field limits has been investigated in [2, 11,
21, 26, 39]. However, the authors only consider a finite dimensional parameter
space $\Theta$.

This work aims at investigating the statistical behaviour of the Lasso
estimator of the unknown parameter $\theta_{0}$ under the sparsity constraint:

$\|\theta_{0}\|_{0}:=\\#\\{1\leq i\leq p:\theta_{0,i}\neq 0\\}=s_{0}.$ (1.2)

Our approach is based on the penalised maximum likelihood estimation and we
derive an oracle inequality for the resulting estimator as well as statements
about the error bound for the $L^{2}$-distance in the non-asymptotic
framework. The main theoretical results heavily use empirical processes theory
and the method of generic chaining, and the upper bound is explicitly
expressed in terms of $p$, $d$, $T$ and certain entropy numbers. To the best
of our knowledge this is the first theoretical study of the Lasso method in
the setting of general drift functions and high dimensional parameter space
$\Theta$. We emphasise that the mathematical methodology is much more involved
than the approach in [10, 19] as the latter use very specific properties of
Ornstein-Uhlenbeck models; in particular, the linearity of the drift and the
quadratic form of the likelihood function are absolutely crucial in this
context. Related works include [12] that studies Lasso estimation in the
finite dimensional setting and [41] that investigates general theory for
maximum likelihood estimators with quadratic penalisations. We remark however
that their methods can not be applied in our context.

The rest of the paper is structured as follows. Section 2 introduces basic
assumptions, main examples and the classical statistical theory in the finite
dimensional setting. Section 3 is devoted to the derivation of the oracle
inequality for the Lasso estimator. In Section 4 we introduce the necessary
probabilistic background and obtain some concentration inequalities. We derive
the error bound for the Lasso estimator in Section 5 and consider results of
some numerical experiments in Section 6. Finally, Section 7 contains proofs of
the main results.

### Notation

All vectors are understood as column vectors. For a matrix $A$ we denote by
$A^{\star}$ the transpose of $A$. For $x\in\mathbb{R}^{d}$ we write
$\|x\|_{q}$, $q\in[1,\infty]$, to denote the $l_{q}$-norm of $x$; we set
$\|\cdot\|:=\|\cdot\|_{2}$. The operator norm of a matrix
$A\in\mathbb{R}^{d\times d}$ is denoted by $\|A\|_{\text{op}}$; for
$A,B\in\mathbb{R}^{d\times d}$ we write $A\geq B$ if $B-A$ is positive
semidefinite. For a set $\mathcal{S}\subset\\{1,\ldots,d\\}$ and
$x\in\mathbb{R}^{d}$ we use the notiation

$(x|_{\mathcal{S}})_{j}:=x_{j}1_{\\{j\in\mathcal{S}\\}}\qquad j=1,\ldots,d.$

For a function $f:\Theta\times\mathbb{R}^{d}\to\mathbb{R}^{l}$ we denote by
$\dot{f}$ (resp. $\nabla f$) the derivative in $\theta\in\Theta$ (resp. in
$x\in\mathbb{R}^{d}$). We say that a function
$f:\mathbb{R}^{d}\to\mathbb{R}^{l}$ has polynomial growth when $\|f(x)\|\leq
C(1+\|x\|^{q})$ for some $q,C>0$. We write $\|f\|_{\text{Lip}}$ to denote the
Lipschitz constant if $f$ is Lipschitz continuous. For a continuous martingale
$(M_{t})_{t\geq 0}$ we denote by $(\langle M\rangle_{t})_{t\geq 0}$ its
quadratic variation process.

##  2 Overview and examples

In this section we introduce the main assumptions on the model (1.1) and
briefly review classical results about maximum likelihood estimation in finite
dimension. We consider the following conditions:

$\displaystyle\text{The function
}b_{\theta}:\mathbb{R}^{d}\to\mathbb{R}^{d}\text{ is locally Lipschitz,}$
(2.1) $\displaystyle(b_{\theta}(x)-b_{\theta}(y))^{\star}(x-y)\geq
M\|x-y\|^{2}$ (2.2)

for a constant $M>0$, $x,y\in\mathbb{R}^{d}$ and $\theta\in\Theta$. These
conditions imply that equation (1.1) has a unique strong solution, which is a
homogeneous continuous Markov process (cf. [37, Theorem 12.1]). Finally, we
assume that initial value $X_{0}$ follows the invariant law such that the
process $X$ is strictly stationary.

We suppose that the complete path $(X_{t})_{t\in[0,T]}$ is observed and we are
interested in estimating the unknown parameter $\theta_{0}\in\Theta$. Let us
briefly recall the classical maximum likelihood theory when $d$ and $p$ are
fixed, and $T\to\infty$. We denote by $\mathbb{P}_{\theta}^{T}$ (resp.
$\mathbb{P}_{0}^{T}$) the law of the process (1.1) with parameter $\theta$
(resp. drift zero) restricted to $\mathcal{F}_{T}$. The (scaled) negative log-
likelihood function $\mathcal{L}_{T}(\theta)$ is explicitly computed via
Girsanov’s theorem as

$\mathcal{L}_{T}(\theta)=\log(\mathbb{P}_{\theta}^{T}/\mathbb{P}_{0}^{T})=\frac{1}{T}\int_{0}^{T}b_{\theta}^{\star}(X_{t})dX_{t}+\frac{1}{2T}\int_{0}^{T}\|b_{\theta}(X_{t})\|^{2}dt.$
(2.3)

and we define the maximum likelihood estimator by

$\widehat{\theta}_{\text{M}}=\operatorname*{argmin}_{\theta\in\Theta}\mathcal{L}_{T}(\theta).$
(2.4)

To obtain the asymptotic normality of the estimator
$\widehat{\theta}_{\text{M}}$ the following conditions are imposed:

($\mathcal{A}$): (a) We assume that for any function
$f_{\theta}:\mathbb{R}^{d}\to\mathbb{R}$ with polynomial growth and any $q>0$,
there exists $C_{q}>0$ such that

$\sup_{\theta\in\Theta}\mathbb{E}_{\theta}\left[\left|\frac{1}{T}\int_{0}^{T}\left(f_{\theta}(X_{t})-\mathbb{E}_{\theta}[f_{\theta}(X_{0})]\right)dt\right|^{q}\right]\leq
C_{q}T^{-q/2}.$

(b) The drift function $b$ is continuously differentiable in $\theta$ such
that $\dot{b}_{\theta}(\cdot)$ has polynomial growth. Furthermore, $\dot{b}$
is uniformly continuous, i.e. for any compact set $K\subset\Theta$ it holds
that

$\lim_{\delta\to 0}\sup_{\theta_{0}\in
K}\sup_{\|\theta-\theta_{0}\|<\delta}\mathbb{E}_{\theta_{0}}[\|\dot{b}_{\theta}(X_{0})-\dot{b}_{\theta_{0}}(X_{0})\|^{2}]=0$

(c) Define the information matrix

$I(\theta):=\mathbb{E}_{\theta}[\dot{b}_{\theta}(X_{0})\dot{b}_{\theta}(X_{0})^{\star}].$

We assume that $I(\theta)$ is positive definite for all $\theta\in\Theta$.
Furthermore, for any compact set $K\subset\Theta$ it holds that

$\inf_{\theta\in
K}~{}\inf_{e\in\mathbb{R}^{p}:~{}\|e\|=1}e^{\star}I(\theta)e>0$

and

$\inf_{\theta_{0}\in
K}\inf_{\|\theta-\theta_{0}\|>\delta}\mathbb{E}_{\theta_{0}}[\|b_{\theta}(X_{0})-b_{\theta_{0}}(X_{0})\|^{2}]>0$

for any $\delta>0$.

We emphasise that the last condition in ($\mathcal{A}$)(c) ensures the
identifiability of the parameter $\theta$. The following result corresponds to
[30, Theorem 2.8] adapted to the setting of the model (1.1) with $d>1$.

######  Theorem 2.1.

Consider the model (1.1) and assume that conditions (2.2), ($\mathcal{A}$)
hold. Then we obtain the central limit theorem

$\sqrt{T}(\widehat{\theta}_{\text{M}}-\theta_{0})\stackrel{{\scriptstyle\raisebox{-1.0pt}{{\tiny
d}}}}{{\longrightarrow}}\mathcal{N}_{p}\left(0,I(\theta_{0})^{-1}\right).$

To conclude this section we demonstrate some examples of multivariate
diffusion models, which are commonly used in the literature.

######  Example 2.2.

(i) (Ornstein-Uhlenbeck model) For a matrix $A\in\mathbb{R}^{d\times d}$
consider the diffusion model

$dX_{t}=-AX_{t}dt+dW_{t}.$

It is well known that this SDE possesses an ergodic strong solutions if all
eigenvalues of $A$ have strictly positive real parts. High dimensional
estimation of Ornstein-Uhlenbeck model has been discussed in [10, 14, 19].

(ii) (General linear models) We may consider extensions of the Ornstein-
Uhlenbeck model to more general linear functions. In this setting we are
interested in the model

$b_{\theta}=\phi_{0}+\sum_{j=1}^{p}\theta_{j}\phi_{j},$

where $\theta_{j}>0$ and the functions
$\phi_{j}:\mathbb{R}^{d}\to\mathbb{R}^{d}$ satisfy the condition (2.2). High
dimensional inference for linear models have been investigated in [18, 35].

(iii) (Langevin equation) Assume that there exists a potential $V_{\theta}\in
C^{1}(\mathbb{R}^{d};\mathbb{R})$ such that

$b_{\theta}=\nabla V_{\theta}.$

This type of models is frequently used in Monte Carlo simulations. According
to [3, Theorem 3.5], under condition (2.2) the diffusion process $X$ is
ergodic with unique invariant probability measure that is absolutely
continuous with respect to the Lebesgue measure and its density is given by

$\mu_{\theta}(x)=\left(\int_{\mathbb{R}^{d}}\exp(-2V(y))dy\right)^{-1}\exp(-2V(x)),\qquad
x\in\mathbb{R}^{d}.$ (2.5)

∎

##  3 Oracle inequality for the Lasso estimator

In this section we turn our attention to large $p$/large $d$/large $T$
setting. We consider the diffusion model (1.1) and assume that the unknown
parameter $\theta_{0}$ satisfies the sparsity constraint (1.2). A standard
approach to estimate $\theta$ under the sparsity constraint is the Lasso
method, which has been investigated in [10, 19] in the framework of an
Ornstein-Uhlenbeck model. We define the Lasso estimator as the penalised MLE:

$\widehat{\theta}_{L}=\operatorname*{argmin}_{\theta\in\mathbb{R}^{p}}\big{(}\mathcal{L}_{T}(\theta)+\lambda\|\theta\|_{1}\big{)},$
(3.1)

where $\lambda>0$ is a tuning parameter and the (scaled) negative log-
likelihood $\mathcal{L}_{T}(\theta)$ has been introduced at (2.3). Our first
goal is to obtain a basic inequality, which provides a basis for the oracle
inequality. For this purpose we introduce some notations. For two smooth
functions $f,g:\mathbb{R}^{d}\to\mathbb{R}^{d}$ we define a random bi-linear
form as

$\langle f,g\rangle_{T}:=\frac{1}{T}\int_{0}^{T}f^{\star}(X_{t})g(X_{t})dt$
(3.2)

and set $\|f\|_{T}:=\sqrt{\langle f,f\rangle_{T}}$. Furthermore, for $\theta$,
$\vartheta\in\mathbb{R}^{p}$ we consider the following random function:

$G(\theta,\vartheta)=\frac{1}{T}\int_{0}^{T}\big{(}b_{\theta}(X_{t})-b_{\vartheta}(X_{t})\big{)}^{\star}dW_{t}.$
(3.3)

By definition of the Lasso estimator we obtain our first result.

######  Lemma 3.1.

(Basic inequality) For any $\theta\in\mathbb{R}^{p}$ it holds that

$\|b_{\widehat{\theta}_{L}}-b_{\theta_{0}}\|_{T}^{2}\leq\|b_{\theta}-b_{\theta_{0}}\|_{T}^{2}+2G(\theta,\widehat{\theta}_{L})+2\lambda\big{(}\|\theta\|_{1}-\|\widehat{\theta}_{L}\|_{1}\big{)}.$
(3.4)

###### Proof.

The proof is rather standard. We can write

$\mathcal{L}_{T}(\theta)=\frac{1}{T}\int_{0}^{T}b_{\theta}^{\star}(X_{t})dW_{t}+\frac{1}{2}\left(\|b_{\theta}-b_{\theta_{0}}\|_{T}^{2}-\|b_{\theta_{0}}\|_{T}^{2}\right).$
(3.5)

By definition of $\widehat{\theta}_{L}$ we also have that
$\mathcal{L}_{T}(\widehat{\theta}_{L})+\lambda\|\widehat{\theta}_{L}\|_{1}\leq\mathcal{L}_{T}(\theta)+\lambda\|\theta\|_{1}$
for any $\theta\in\Theta$. This inequality together with (3.5) implies the
desired result. ∎

In the next step we would like to show an oracle inequality, which holds on
sets of high probability. As common in the literature we require a good
control of the empirical part $G(\theta,\widehat{\theta}_{L})$ as well as a
version of the restricted eigenvalue property. For this purpose we introduce
the sets $\mathcal{T}$ and $\mathcal{T}^{\prime}$ via

$\mathcal{T}:=\left\\{\left\|\sup_{\theta\in\Theta}\frac{1}{T}\int_{0}^{T}\dot{b}_{\theta}(X_{t})^{\star}dW_{t}\right\|_{\infty}\leq\lambda/2\right\\}$
(3.6)
$\mathcal{T}^{\prime}:=\left\\{\inf_{\begin{subarray}{c}\theta\in\mathbb{R}^{p}:\|\theta\|_{0}=s\\\
\vartheta\in\mathbb{R}^{p}:\theta-\vartheta\in\mathcal{C}(s,3+4/\gamma)\end{subarray}}\frac{\|b_{\theta}-b_{\vartheta}\|_{T}}{\|\theta-\vartheta\|_{2}}\geq
k\right\\}$ (3.7)

Here the set $\mathcal{C}(s,c)$ is given via

$\mathcal{C}(s,c):=\left\\{x\in\mathbb{R}^{d}\setminus\\{0\\}:~{}\|x\|_{1}\leq(1+c)\|x_{|\mathcal{I}_{s}(x)}\|_{1}\right\\},$
(3.8)

where $c>0$ and $\mathcal{I}_{s}(x)$ is a set of coordinates of $s$ largest
elements of $A$. The constant $k$ in the definition of the set
$\mathcal{T}^{\prime}$ will be chosen later, while the constant $\gamma>0$
remains arbitrary. The main result of this section is the following oracle
inequality.

######  Theorem 3.2.

Assume that $\|\theta\|_{0}=s$. On $\mathcal{T}\cap\mathcal{T}^{\prime}$ it
holds that

$\|b_{\widehat{\theta}_{L}}-b_{\theta_{0}}\|_{T}^{2}\leq(1+\gamma)\|b_{\theta}-b_{\theta_{0}}\|_{T}^{2}+4\frac{(\gamma+2)^{2}s\lambda^{2}}{\gamma
k^{2}}.$ (3.9)

###### Proof.

Applying mean value theorem we conclude that there exists a measurable
$\theta^{\prime}\in\mathbb{R}^{p}$ such that

$\displaystyle|G(\theta,\widehat{\theta}_{L})|$
$\displaystyle=\left|\left\langle\frac{1}{T}\int_{0}^{T}\dot{b}_{\theta^{\prime}}(X_{t})^{\star}dW_{t},\theta-\widehat{\theta}_{L}\right\rangle\right|$
(3.10)
$\displaystyle\leq\left\|\sup_{\theta\in\Theta}\frac{1}{T}\int_{0}^{T}\dot{b}_{\theta}(X_{t})^{\star}dW_{t}\right\|_{\infty}\|\theta-\widehat{\theta}_{L}\|_{1}.$
(3.11)

On the set $\mathcal{T}$ we conclude from the basic inequality (3.4) that

$\|b_{\widehat{\theta}_{L}}-b_{\theta_{0}}\|_{T}^{2}+\lambda\|\widehat{\theta}_{L}-\theta\|_{1}\leq\|b_{\theta}-b_{\theta_{0}}\|_{T}^{2}+2\lambda(\|\theta\|_{1}-\|\widehat{\theta}_{L}\|_{1}+\|\widehat{\theta}_{L}-\theta\|_{1}).$
(3.12)

Let $\mathcal{S}(\theta)=\text{supp }\theta$. Then

$\|\theta\|_{1}-\|\widehat{\theta}_{L}\|_{1}+\|\widehat{\theta}_{L}-\theta\|_{1}\leq
2\|\widehat{\theta}_{L}|_{\mathcal{S}(\theta)}-\theta\|_{1}$ (3.13)

and hence

$\|b_{\widehat{\theta}_{L}}-b_{\theta_{0}}\|_{T}^{2}+\lambda\|\widehat{\theta}_{L}-\theta\|_{1}\leq\|b_{\theta}-b_{\theta_{0}}\|_{T}^{2}+4\lambda\|\widehat{\theta}_{L}|_{\mathcal{S}(\theta)}-\theta\|_{1}.$
(3.14)

Now if
$4\lambda\|\widehat{\theta}_{L}|_{\mathcal{S}(\theta)}-\theta\|_{1}\leq\gamma\|b_{\theta}-b_{\theta_{0}}\|_{T}^{2}$
then
$\|b_{\widehat{\theta}_{L}}-b_{\theta_{0}}\|_{T}^{2}\leq(1+\gamma)\|b_{\theta}-b_{\theta_{0}}\|_{T}^{2}$.
Next, we consider the case

$4\lambda\|\widehat{\theta}_{L}|_{\mathcal{S}(\theta)}-\theta\|_{1}>\gamma\|b_{\theta}-b_{\theta_{0}}\|_{T}^{2}.$
(3.15)

In this setting, due to (3.14) we get
$\widehat{\theta}_{L}-\theta\in\mathcal{C}(s,3+4/\gamma)$ and by Cauchy-
Schwarz inequality

$\|b_{\widehat{\theta}_{L}}-b_{\theta_{0}}\|_{T}^{2}+\lambda\|\widehat{\theta}_{L}-\theta\|_{1}\leq\|b_{\theta}-b_{\theta_{0}}\|_{T}^{2}+4\lambda\sqrt{s}\|\widehat{\theta}_{L}-\theta\|_{2}.$
(3.16)

On $\mathcal{T}^{\prime}$ we obtain via (3.16):

$\|b_{\widehat{\theta}_{L}}-b_{\theta_{0}}\|_{T}^{2}\leq\|b_{\theta}-b_{\theta_{0}}\|_{T}^{2}+4\lambda\frac{\sqrt{s}}{k}\big{(}\|b_{\widehat{\theta}_{L}}-b_{\theta_{0}}\|_{T}+\|b_{\theta}-b_{\theta_{0}}\|_{T}\big{)}.$
(3.17)

Using the inequality

$2xy\leq\frac{x^{2}}{b}+by^{2}$ (3.18)

with $x=\lambda\sqrt{s_{0}}$,
$y=\|b_{\widehat{\theta}_{L}}-b_{\theta_{0}}\|_{T}$ or
$y=\|b_{\theta}-b_{\theta_{0}}\|_{T}$ and $b=\gamma/(\gamma+2)$ we finally
obtain that

$\|b_{\widehat{\theta}_{L}}-b_{\theta_{0}}\|_{T}^{2}\leq(1+\gamma)\|b_{\theta}-b_{\theta_{0}}\|_{T}^{2}+4\frac{(\gamma+2)^{2}s\lambda^{2}}{\gamma
k^{2}},$ (3.19)

which completes the proof of Theorem 3.2. ∎

######  Remark 3.3.

In linear models discussed in Examples (2.2)(i) and (ii) the mean value
theorem used in the previous proof is trivial, and hence the uniformity in
$\theta\in\Theta$ in the definition of the sets $\mathcal{T}$ and
$\mathcal{T}^{\prime}$ is not required. In this setting it is easier to
control the probabilities $\mathbb{P}(\mathcal{T})$ and
$\mathbb{P}(\mathcal{T}^{\prime})$ (cf. [10, 19]). In the general framework of
the model (1.1) we need more advanced techniques from empirical processes
theory. ∎

Applying the oracle inequality (3.9) to $\theta=\theta_{0}$ we immediately
conclude that

$\|b_{\widehat{\theta}_{L}}-b_{\theta_{0}}\|_{T}^{2}\leq
4\frac{(\gamma+2)^{2}s_{0}\lambda^{2}}{\gamma k^{2}}$ (3.20)

and this statement can be used to obtain bounds on $l_{1}$ and $l_{2}$ norms.
Indeed, on $\mathcal{T}^{\prime}$ it holds that
$k\|\widehat{\theta}_{L}-\theta_{0}\|_{2}\leq\|b_{\widehat{\theta}_{L}}-b_{\theta_{0}}\|_{T}$
and we deduce the bound

$\|\widehat{\theta}_{L}-\theta_{0}\|_{2}^{2}\leq
4\frac{(\gamma+2)^{2}s_{0}\lambda^{2}}{\gamma k^{4}}$ (3.21)

On the other hand, since
$\widehat{\theta}_{L}-\theta_{0}\in\mathcal{C}(s_{0},3+4/\gamma)$ we obtain
that

$\|\widehat{\theta}_{L}-\theta_{0}\|_{1}\leq\frac{4(\gamma+1)}{\gamma}\sqrt{s_{0}}\|\widehat{\theta}_{L}-\theta_{0}\|_{2}\leq\frac{8(\gamma+1)(\gamma+2)s_{0}\lambda}{\gamma^{3/2}k^{2}}.$
(3.22)

by applying Cauchy-Schwarz inequality and (3.21).

In the next section we will determine an upper bound for the probabilities
$\mathbb{P}(\mathcal{T})$ and $\mathbb{P}(\mathcal{T}^{\prime})$ with the help
of uniform concentration inequalities.

##  4 Concentration bounds

In this section we review some concentration bounds for linear functionals of
the diffusion process $X$ defined in (1.1). We will apply these theoretical
results to control the probabilities of the sets $\mathcal{T}$ and
$\mathcal{T}^{\prime}$.

###  4.1 Concentration bounds for linear functionals of the SDE

We first consider the random set $\mathcal{T}^{\prime}$ and ignore the
uniformity for the moment. From the definition of the norm $\|\cdot\|_{T}$ it
is obvious that we require concentration bounds for functionals of the form

$\frac{1}{T}\int_{0}^{T}f(X_{t})dt$ (4.1)

for functions $f:\mathbb{R}^{d}\to\mathbb{R}$. Similarly, the quadratic
variation of the martingale term
$\int_{0}^{T}\dot{b}_{\theta}(X_{t})^{\star}dW_{t}$ in the definition of
$\mathcal{T}$ has the form (4.1). In view of Bernstein type inequalities for
martingales, which crucially depend on the quadratic variation, it is evident
that deviation bounds for linear functionals of $X$ are crucial to obtain
approximations of $\mathbb{P}(\mathcal{T})$ and
$\mathbb{P}(\mathcal{T}^{\prime})$.

Concentration bounds for functionals of the form (4.1) have been investigated
in several papers, see e.g. [20, 22, 46] among others. It is important to note
that there are no minimal assumptions on $f$ and $X$ under which such
concentration bounds hold. Rather there is a certain trade-off between
assumptions on $f$ and the diffusion model $X$. Here we give a short overview
about the theory, mainly following the exposition of [20]. In the setting of
model (1.1) concentration inequalities usually take the form

$\mathbb{P}_{\theta}\left(\frac{1}{T}\int_{0}^{T}\left(f(X_{t})-\mathbb{E}_{\theta}[f(X_{0})]\right)dt>\mu\right)\leq\exp\left(-\frac{T\mu^{2}}{a_{1}+a_{2}\mu}\right)$
(4.2)

where the constants $a_{1},a_{2}>0$ depend on the properties of $f$ and $X$,
but not directly on the dimension $d$. There are three specific settings in
which the concentration bound (4.2) has been obtained: (i) $X$ is a symmetric
Markov process and the function $f$ is bounded, (ii) $X$ satisfies assumptions
as in (2.2) and $f$ is Lipschitz, (iii) $X$ is a symmetric Markov process that
satisfies a (local) Poincaré inequality and $f$ fulfils certain growth and
integrability conditions. We remark that the handling of these three settings
is quite different in nature and the user has to decide which setting is more
convenient in a particular situation. In this paper we will deal with the
second setting (although one could also try to derive similar results in the
other two settings; we note however that even in the Ornstein-Uhlenbeck model
the function $f$ is not bounded, so the setting (i) does not apply). In the
Lipschitz setting concentration inequalities are strongly related to
transportation cost-information inequalities via the famous $T_{p}(C)$
conditions; see e.g. [6, 13, 22, 46] for more details.

We now present an exponential type bound for Lipschitz functions
$f:\mathbb{R}^{d}\to\mathbb{R}$.

######  Theorem 4.1 (Theorem 3 and Section 3 in [38]).

Assume that conditions (2.2) are satisfied and let
$f:\mathbb{R}^{d}\to\mathbb{R}$ be a Lipschitz function. Then there exists a
constant $C>0$, independent of $d$, such that

$\mathbb{E}_{\theta}\left[\exp\left(\frac{\mu}{T}\int_{0}^{T}(f(X_{t})-\mathbb{E}_{\theta}[f(X_{0})])dt\right)\right]\leq\exp\left(C\mu^{2}\|f\|_{\text{\rm
Lip}}^{2}/T\right)$ (4.3)

for all $\mu\in\mathbb{R}$.

Theorem 4.1 is a basic building block for uniform bounds investigated in the
next section.

###  4.2 Generic chaining and control of the probabilities
$\mathbb{P}(\mathcal{T})$ and $\mathbb{P}(\mathcal{T}^{\prime})$

Here we apply the exponential bound introduced in Theorem 4.1 to obtain upper
bounds for the probabilities $\mathbb{P}(\mathcal{T})$ and
$\mathbb{P}(\mathcal{T}^{\prime})$. We will use the generic chaining method,
which has been introduced by Talagrand, cf. [43]. We recall the basic notions
of the theory. Let d be a distance measure on the parameter set $\Theta$ and
assume that $\max_{\theta,\vartheta\in\Theta}\text{d}(\theta,\vartheta)$ is
finite. The diameter of a set $A\subset\Theta$ with respect to d is defined as

$\Delta(A)=\Delta(A,\text{d}):=\max_{\theta,\vartheta\in
A}\text{d}(\theta,\vartheta)<\infty.$ (4.4)

We call an admissible sequence an increasing sequence
$(\mathcal{A}_{n})_{n\in\mathbb{N}}$ of partitions of $\Theta$ such that
$\text{card }\mathcal{A}_{0}=1$ and $\text{card }\mathcal{A}_{n}\leq
2^{2^{n}}$ for $n\geq 1$. Furthermore, we introduce the entropy numbers
($\alpha>0$)

$\gamma_{\alpha}(\Theta,d)=\inf\sup_{\theta\in\Theta}\sum_{n\geq
0}2^{n/\alpha}\Delta(A_{n}(\theta)),$ (4.5)

where the infimum is taken over all admissible sequences and $A_{n}(\theta)$
denotes a the unique element of $\mathcal{A}_{n}$ which contains $\theta$. The
generic chaining approach is a tool to obtain uniform probability bounds from
non-uniform ones. To sketch ideas we demonstrate a result from [15]. Consider
a one-dimensional family $\\{Z_{\theta}:~{}\theta\in\Theta\\}$ and assume that
the inequality

$\mathbb{P}(|Z_{\theta}-Z_{\vartheta}|>u\text{d}(\theta,\vartheta))\leq
2\exp(-u^{\alpha})$ (4.6)

holds for some $\alpha>0$. Then, according to [15, Theorem 3.2], there exist
constants $a_{\alpha},b_{\alpha}>0$ such that the uniform probability bound

$\mathbb{P}\left(\sup_{\theta\in\Theta}|Z_{\theta}-Z_{\theta_{0}}|>a_{\alpha}\gamma_{\alpha}(\Theta,d)+ub_{\alpha}\Delta(\Theta)\right)\leq\exp(-u^{\alpha}/\alpha)$
(4.7)

holds for any $u\geq 1$. We will use this type of uniform bounds to treat the
sets $\mathcal{T}$ and $\mathcal{T}^{\prime}$.

We introduce two distance measures on $\Theta$ that are related to definitions
of $\mathcal{T}$ and $\mathcal{T}^{\prime}$ as well as impose a set of
assumptions.

($\mathcal{B}$) (a) We denote by $\dot{b}_{\theta,j}$ the $j$th column vector
of the matrix $\dot{b}_{\theta}$, $j=1,\ldots,p$. The functions
$f_{\theta,\vartheta}^{j}(\cdot):=\|\dot{b}_{\theta,j}(\cdot)-\dot{b}_{\vartheta,j}(\cdot)\|^{2}_{2}:\mathbb{R}^{d}\to\mathbb{R}$
and
$(\dot{b}_{\theta}\dot{b}_{\theta}^{\star})_{ij}(\cdot):\mathbb{R}^{d}\to\mathbb{R}$
are assumed to be Lipschitz continuous.

(b) Define the matrix

$\mathcal{M}_{\text{Lip}}(\theta,\vartheta):=\left(\|((\dot{b}_{\theta}\dot{b}_{\theta}^{\star})_{ij}-(\dot{b}_{\vartheta}\dot{b}_{\vartheta}^{\star})_{ij})(\cdot)\|_{\text{Lip}}\right)_{i,j=1}^{d}.$
(4.8)

We introduce two distance measures on the parameter set $\Theta$

$\displaystyle d_{1}(\theta,\vartheta)$
$\displaystyle:=\max_{j=1,...,p}\left(\mathbb{E}_{\theta_{0}}\left[\|\dot{b}_{\theta,j}(X_{0})-\dot{b}_{\vartheta,j}(X_{0})\|^{2}\right]+\|f_{\theta,\vartheta}^{j}\|_{\text{\rm
Lip}}\right)^{\frac{1}{2}}$ $\displaystyle d_{2}(\theta,\vartheta)$
$\displaystyle:=\|\mathcal{M}_{\text{Lip}}(\theta,\vartheta)\|_{\text{op}}$

and assume that

$\Delta_{1}(\Theta)=\Delta(\Theta,d_{1})<\infty\qquad\text{and}\qquad\Delta_{2}(\Theta)=\Delta(\Theta,d_{2})<\infty.$
(4.9)

(c) Let $l_{\text{min}}(\theta)$ denote the smallest eigenvalue of the Fisher
information matrix $I(\theta)$. We assume that

$l_{\text{min}}:=\inf_{\theta\in\Theta}l_{\text{min}}(\theta)>0.$ (4.10)

The distance measures $d_{1}$ and $d_{2}$ will appear to be crucial to obtain
upper bounds for the probabilities $\mathbb{P}(\mathcal{T})$ and
$\mathbb{P}(\mathcal{T}^{\prime})$ via the generic chaining method. We also
remark that the assumption ($\mathcal{B}$)(c) is stronger than the
identifiability assumption in ($\mathcal{A}$)(c).

To control the probabilities of the sets $\mathcal{T}$ and
$\mathcal{T}^{\prime}$ we define the constants

$\displaystyle\lambda_{1}$
$\displaystyle:=4L\Delta_{1}(\Theta)\max{\left(\sqrt{\frac{\log{(2Lp)}+\log{(2/\epsilon)}}{2T}},\left(\frac{\frac{(2C)^{1/3}}{6}\left(\log{(2Lp)+\log{(2/\epsilon)}}\right)}{T}\right)^{3/4}\right)}$
$\displaystyle+\frac{4L\gamma_{4/3}(\Theta,d_{1})}{\sqrt{T}},$ (4.11)

and

$T_{1}:=\frac{2592(c_{0}+2)^{4}L^{2}C}{l_{\text{min}}^{2}}\left(\Delta_{2}(\Theta)\sqrt{\log{\left(21^{2s_{0}}\left(p^{2s_{0}}\wedge\frac{ep}{2s_{0}}\right)^{2s_{0}}\right)}+\log{\frac{L}{\epsilon_{0}}}}+\gamma_{2}(\Theta,d_{2})\right)^{2}.$
(4.12)

where $\epsilon>0$ is a given number, $L>0$ is defined in (7.17) and $C$ has
been introduced in Theorem 4.1. The main result of this section is the
following theorem.

######  Theorem 4.2.

Assume that conditions (2.2), ($\mathcal{B}$) are satisfied and fix
$\varepsilon\in(0,1)$. Then, for all $T\geq\frac{2C}{27}$ and
$\lambda\geq\lambda_{1}$, it holds that

$\mathbb{P}(\mathcal{T})\geq 1-\varepsilon.$

Furthermore, for any $T\geq T_{1}$ and $k=l_{\text{min}}/2$, we obtain the
bound

$\mathbb{P}(\mathcal{T}^{\prime})\geq 1-\varepsilon.$

###### Proof.

See Section 7. ∎

######  Remark 4.3.

Here we present a class of drift functions $b_{\theta}$ that satisfies
conditions (2.2) and ($\mathcal{B}$)(a) and (b). We consider a potential
$V_{\theta}:\mathbb{R}^{d}\to\mathbb{R}$ of the form

$V_{\theta}(x)=v_{1}(x)+v_{2}(\theta,x),$

where $v_{1}\in C^{2}(\mathbb{R}^{d};\mathbb{R})$ and $v_{2}\in
C^{1,2}(\Theta\times\mathbb{R}^{d};\mathbb{R})$, and set $b_{\theta}=\nabla
V_{\theta}$. This corresponds to the setting of Example 2.2(iii). We assume
that $v_{1}$ is strictly convex satisfying the condition

$\nabla^{2}v_{1}(x)\geq 2M\times\text{Id}\qquad\forall x\in\mathbb{R}^{d},$

where Id denotes the $d$-dimensional identity matrix, and

$\|\nabla^{2}v_{2}(\theta,x)\|_{\text{op}}\leq M\qquad\forall
x\in\mathbb{R}^{d},\theta\in\Theta.$

Under these assumptions we conclude the inequality
$\nabla^{2}V_{\theta}(x)\geq M\times\text{Id}$ and condition (2.2) is
automatically satisfied. Let us further assume that
$\dot{b}_{\theta}=\nabla\dot{v}_{2}$ satisfies

$\displaystyle\sup_{\theta\in\Theta,x\in\mathbb{R}^{d}}\max_{i,j}|\dot{b}_{\theta,ij}(x)|<\infty,$
$\displaystyle\sup_{\theta\in\Theta}\max_{i,j}\|\dot{b}_{\theta,ij}\|_{\text{Lip}}<\infty.$

Then, using the estimate
$\|\mathcal{M}_{\text{Lip}}(\theta,\vartheta)\|_{\text{op}}\leq\|\mathcal{M}_{\text{Lip}}(\theta,\vartheta)\|_{\text{F}}$
where $\|\cdot\|_{\text{F}}$ denotes the Frobenius norm, conditions
($\mathcal{B}$)(a) and (b) are fulfilled, and it holds that

$\Delta_{1}(\Theta)\leq Kd^{1/2}\qquad\text{and}\qquad\Delta_{2}(\Theta)\leq
Kdp^{1/2}$

for some constant $K>0$. We note however that these bounds may be far from
optimal. ∎

##  5 Error bounds for the Lasso estimator

Now we are ready to demonstrate the error bounds for the Lasso estimator
$\widehat{\theta}_{L}$.

######  Corollary 5.1.

Let $\|\theta_{0}\|_{0}=s_{0}$. Assume that conditions (2.2), ($\mathcal{B}$)
are satisfied and fix $\varepsilon\in(0,1)$. If $T\geq\max\\{2C/27,T_{1}\\}$
and $\lambda\geq\lambda_{1}$ we obtain with probability at least
$1-2\varepsilon$:

$\displaystyle\|\widehat{\theta}_{L}-\theta_{0}\|_{2}^{2}$
$\displaystyle\leq\frac{64(\gamma+2)^{2}s_{0}\lambda^{2}}{\gamma
l_{\text{min}}^{4}},$ (5.1)
$\displaystyle\|\widehat{\theta}_{L}-\theta_{0}\|_{1}$
$\displaystyle\leq\frac{32(\gamma+1)(\gamma+2)s_{0}\lambda}{\gamma^{3/2}l_{\text{min}}^{2}},$
(5.2) $\displaystyle\|\widehat{\theta}_{L}\|_{0}$
$\displaystyle\leq\frac{64\mathfrak{M}_{\infty}(\gamma+1)(\gamma+2)}{\gamma^{3/2}l_{\text{min}}^{2}}s_{0},$
(5.3)

where
$\mathfrak{M}_{\infty}=\|\sup_{\theta,\vartheta\in\Theta}\frac{1}{T}\int_{0}^{T}\dot{b}_{\theta}^{\star}(X_{t})b_{\vartheta}(X_{t})dt\|_{\infty}$.
In particular, if $\log(p)/T\to 0$ we deduce that

$\displaystyle\|\widehat{\theta}_{L}-\theta_{0}\|_{2}$
$\displaystyle=O_{\mathbb{P}}\left(\sqrt{s_{0}}\left(\Delta_{1}(\Theta)\sqrt{\frac{\log(p)}{T}}+\frac{\gamma_{4/3}(\Theta,d_{1})}{\sqrt{T}}\right)\right),$
(5.4) $\displaystyle\|\widehat{\theta}_{L}-\theta_{0}\|_{1}$
$\displaystyle=O_{\mathbb{P}}\left(s_{0}\left(\Delta_{1}(\Theta)\sqrt{\frac{\log(p)}{T}}+\frac{\gamma_{4/3}(\Theta,d_{1})}{\sqrt{T}}\right)\right).$
(5.5)

###### Proof.

First two inequalities follow from the bounds (3.20)-(3.22) and the statement
of Theorem 4.2, so we just need to show the last statement. We notice that
necessary and sufficient condition for $\widehat{\theta}_{L}$ to be the
solution of the optimisation problem (3.1) is the existence of a vector
$\vartheta$ in subdifferential of target function that satisfies

$\frac{1}{T}\int_{0}^{T}\dot{b}_{\widehat{\theta}_{L}}^{\star}(X_{t})dW_{t}+\frac{1}{T}\int_{0}^{T}\dot{b}_{\widehat{\theta}_{L}}^{\star}(X_{t})(b_{\widehat{\theta}_{L}}(X_{t})-b_{\theta_{0}}(X_{t}))dt+\lambda\vartheta=0.$
(5.6)

Furthermore $\widehat{\theta}_{L}^{i}\neq 0$ implies that
$\vartheta^{i}=\text{sign }{\widehat{\theta}_{L}^{i}}$ and we conclude that

$\displaystyle\mathfrak{M}_{\infty}\|\widehat{\theta}_{L}-\theta_{0}\|_{1}\geq\|\frac{1}{T}\int_{0}^{T}\dot{b}_{\widehat{\theta}_{L}}^{\star}(X_{t})(b_{\widehat{\theta}_{L}}(X_{t})-b_{\theta_{0}}(X_{t}))dt\|_{1}$
$\displaystyle=\|\frac{1}{T}\int_{0}^{T}\dot{b}_{\widehat{\theta}_{L}}^{\star}(X_{t})dW_{t}+\vartheta\|_{1}$
$\displaystyle\geq\sum_{i:\widehat{\theta}_{L}^{i}\neq
0}|\lambda\vartheta^{i}-|(\frac{1}{T}\int_{0}^{T}\dot{b}_{\widehat{\theta}_{L}}^{\star}(X_{t})dW_{t})^{i}||$
$\displaystyle\geq\|\widehat{\theta}_{L}\|_{0}\frac{\lambda}{2}.$

This completes the proof of Corollary 5.1.

∎

######  Remark 5.2.

As mentioned in Remark 3.3 in linear drift models of Example 2.2(ii) uniform
bounds are not required and the bounds simplify to

$\|\widehat{\theta}_{L}-\theta_{0}\|_{2}=O_{\mathbb{P}}\left(\sqrt{s_{0}\frac{\log(p)}{T}}\right)\qquad\text{and}\qquad\|\widehat{\theta}_{L}-\theta_{0}\|_{1}=O_{\mathbb{P}}\left(s_{0}\sqrt{\frac{\log(p)}{T}}\right).$

These are rather classical bounds, which have also being obtained in the
setting of Ornstein-Uhlenbeck models in [10]. They are known to be minimax
optimal up to $\log(p)$ factor. Similar optimality statements seem to be out
of reach in the general setting. ∎

The quantity $\gamma_{4/3}(\Theta,d_{1})$ appearing in Corollary 5.1 is hard
to compute explicitly, but it can be upper bounded using covering numbers. To
demonstrate ideas we introduce the parametric class

$F:=\left\\{\dot{b}_{\theta}:~{}\theta\in\Theta\right\\}$

and denote by $N(F,d_{1},\varepsilon)$ the covering number of $F$ with respect
to $d_{1}$-distance, i.e. the smallest number of balls of radius $\varepsilon$
needed to cover the set $F$. Then one can always approximate

$\gamma_{\alpha}(\Theta,d_{1})\leq c_{\alpha}\int_{0}^{\infty}\left(\log
N(F,d_{1},\varepsilon)\right)^{1/\alpha}d\varepsilon,$

for some positive constant $c_{\alpha}$, see e.g. [43]. The order of covering
numbers has been computed for numerous examples and these studies can be
applied to control the entropy constant $\gamma_{4/3}(\Theta,d_{1})$. Assume,
for instance, that $\Theta\subset\mathbb{R}^{p}$ is a bounded set, in which
case $N(\Theta,\|\cdot\|_{2},\varepsilon)\sim\varepsilon^{-p}$. When the map
$\theta\mapsto\dot{b}_{\theta}$ is Lipschitz with Lipschitz constant $R>0$,
then we deduce the inequality $N(F,d_{1},\varepsilon)\leq
N(\Theta,\|\cdot\|_{2},\varepsilon/R)$. In this case we obtain

$\gamma_{4/3}(\Theta,d_{1})\leq c_{4/3,R}~{}p^{3/4}.$

This bound may be far from optimal, but it can be useful if $p<<d$. On the
other hand, when the dimension $d$ of the process $X$ is constant (or
significantly smaller than $p$), the covering number $N(F,d_{1},\varepsilon)$
can be approximated ignoring the dependence on the parameter space $\Theta$.
To demonstrate ideas we make the additional assumption that $X_{t}\in Q$ for
all $t\geq 0$ $\mathbb{P}$-almost surely and
$\text{supp}(\dot{b}_{\theta})\subset Q$ for all $\theta\in\Theta$, where
$Q\subset\mathbb{R}^{d}$ is a compact set, and $\dot{b}_{\theta}(\cdot)\in
C^{\beta}_{Q}(\mathbb{R}^{d};\mathbb{R}^{d})$ for some $\beta>1$ (class of
$\beta$-smooth functions with support $Q$). We observe that
$d_{1}(\theta,\vartheta)\leq d^{\prime}(\dot{b}_{\theta},\dot{b}_{\vartheta})$
where

$d^{\prime}(\dot{b}_{\theta},\dot{b}_{\vartheta}):=\max_{j=1,...,p}\left(\sup_{x\in
Q}\|\dot{b}_{\theta,j}(x)-\dot{b}_{\vartheta,j}(x)\|^{2}+\|f_{\theta,\vartheta}^{j}\|_{\text{\rm
Lip}}\right)^{\frac{1}{2}}$

Now, we can use the estimate

$N(F,d_{1},\varepsilon)\leq
N(C^{\beta}_{Q}(\mathbb{R}^{d};\mathbb{R}^{d}),d^{\prime},\varepsilon),$

which effectively ignores the parametric structure of the set $F$. The latter
covering number can be computed in a similar fashion as demonstrated in [45,
Section 2.7], and thus we may obtain an explicit upper bound for the quantity
$\gamma_{4/3}(\Theta,d_{1})$.

######  Remark 5.3.

For the purpose of model selection it is a common practice to consider the
adaptive Lasso. If we denote by $\widehat{\theta}_{\text{o}}$ an initial Lasso
estimator of $\theta_{0}$, the adaptive Lasso is defined as

$\widehat{\theta}_{\text{adap}}=\operatorname*{argmin}_{\theta\in\mathbb{R}^{p}}\left(\mathcal{L}_{T}(\theta)+\lambda\sum_{j=1}^{p}\frac{\theta_{j}}{|\widehat{\theta}_{\text{o}}|^{\alpha}}\right)$

for some $\alpha>0$. The support recovery property for adaptive Lasso
estimator has been shown in [19] for the setting of Ornstein-Uhlenbeck models
with fixed $d$ and $p$; we also refer to [35] for the study of linear drift
models. However, showing this property for general drift functions and large
$d$ and $p$ is not a trivial task, and we leave it for future research. ∎

##  6 Numerical experiments

Our Lasso estimator is based on continuous observations of the underlying
process, which have to be discretised for numerical simulations. We will use
time of observation $T=20$ with 100 discretisation points over each unit
interval. Further refinement of the grid does not lead to a significant
improvement of simulation, and consequently such approximation is sufficient
for our purposes.

For $(x,k)\in\mathbb{R}^{2}$ we introduce the function

$h(x,k)=\begin{cases}x^{2}\sin{\frac{k}{x}},\text{ if }x\neq 0\\\ 0,\text{ if
}x=0\end{cases}$ (6.7)

and for a positive-definite $d\times d$-matrix $A$ we define a function
$b_{A}:\mathbb{R}^{d}\to\mathbb{R}^{d}$ as

$b_{A}^{(i)}(x)=\sum_{j=1}^{d}h(x_{j},A_{ij}),\text{ where
}x\in\mathbb{R}^{d}\text{ and }i\in\\{1,...,d\\}.$ (6.8)

As an example, we are going to consider a diffusion process (1.1) with drift
function $b_{A_{0}}$ with some sparse parameter
$\theta_{0}=\text{vect}(A_{0})$. We set sparsity of $\theta_{0}$ at 35%.

The selection of a tuning parameter $\lambda$ is usually performed by cross-
validation algorithm. We use 80% of our observations as training set and last
10% as validation set:

$\lambda_{0}=\operatorname*{argmin}_{\lambda>0}\mathcal{L}_{[0.8T,T]}\hat{A}_{L}(\lambda),$
(6.9)

where

$\hat{A}_{L}(\lambda)=\operatorname*{argmin}_{A\in R^{d\times
d}}(\mathcal{L}_{[0,0.8T]}(A)+\lambda\|A\|_{1}).$ (6.10)

Figure 1 below demonstrates an example of the parameter matrix
$A_{0}\in\mathbb{R}^{d\times d}$ and the corresponding maximum likelihood and
Lasso estimators. We use a colour code to highlight the sparsity of given
matrices. We can see that performance of Lasso estimator, especially in terms
of support recovery, is much superior to that one of MLE, even for a
relatively small dimension of true parameter.

(a) True parameter

(b) MLE

(c) Lasso

Figure 1: Comparison of the sparsed true parameter matrix with maximum
likelihood and Lasso estimators.

##  7 Proof of Theorem 4.2

We divide the proof of Theorem 4.2 into several steps.

###  7.1 Treatment of the set $\mathcal{T}$

We consider univariate martingale terms

$M_{T}^{j}(\theta):=\frac{1}{\sqrt{T}}\int_{0}^{T}\big{(}\dot{b}_{\theta,j}(X_{t}))^{\star}dW_{t},\qquad
j=1,\ldots,p.$ (7.1)

The first result determines bounds for exponential moments of
$M_{T}^{j}(\theta)$.

######  Proposition 7.1.

Assume that conditions (2.2), ($\mathcal{B}$) are satisfied. Then it holds
that

$\mathbb{E}_{\theta_{0}}\left[\exp\left(\mu\left(M_{T}^{j}(\theta)-M_{T}^{j}(\vartheta)\right)\right)\right]\leq\exp\left(\mu^{2}d_{1}^{2}(\theta,\vartheta)+\frac{2C}{T}\mu^{4}d_{1}^{4}(\theta,\vartheta)\right)$
(7.2)

for any $\mu\in\mathbb{R}$.

###### Proof.

We first recall that due to the property of an exponential martingale we have
that

$\mathbb{E}_{\theta_{0}}\left[a(M_{T}^{j}(\theta)-M_{T}^{j}(\vartheta))-\frac{a^{2}}{2}\langle
M^{j}(\theta)-M^{j}(\vartheta)\rangle_{T}\right]=1$

for any $a\in\mathbb{R}$. Next, using Cauchy-Schwarz inequality, we deduce
that

$\displaystyle\mathbb{E}_{\theta_{0}}\left[\exp\left(\mu\left(M_{T}^{j}(\theta)-M_{T}^{j}(\vartheta)\right)\right)\right]$
$\displaystyle=\mathbb{E}_{\theta_{0}}\left[\exp\left(\mu\left(M_{T}^{j}(\theta)-M_{T}^{j}(\vartheta)\right)-\mu^{2}\langle
M^{j}(\theta)-M^{j}(\vartheta)\rangle_{T}\right)\right.$ (7.3)
$\displaystyle\times\left.\exp\left(\mu^{2}\langle
M^{j}(\theta)-M^{j}(\vartheta)\rangle_{T}\right)\right]$ (7.4)
$\displaystyle\leq\mathbb{E}_{\theta_{0}}\left[\exp\left(2\mu^{2}\langle
M^{j}(\theta)-M^{j}(\vartheta)\rangle_{T}\right)\right]^{1/2}.$ (7.5)

Hence, we conclude from Theorem 4.1 that

$\mathbb{E}_{\theta_{0}}\left[\exp\left(\mu\left(M_{T}^{j}(\theta)-M_{T}^{j}(\vartheta)\right)\right)\right]\leq\exp\left(\mu^{2}d_{1}^{2}(\theta,\vartheta)+\frac{2C}{T}\mu^{4}d_{1}^{4}(\theta,\vartheta)\right),$

which completes the proof. ∎

An immediate consequence of Proposition 7.1 is the following concentration
inequality.

######  Corollary 7.2.

Assume that conditions (2.2), ($\mathcal{B}$) are satisfied. Then we obtain
the inequality

$\mathbb{P}_{\theta_{0}}\left(|M_{T}^{j}(\theta)-M_{T}^{j}(\vartheta)|>ud_{1}(\theta,\vartheta)\right)\leq\begin{cases}2\exp{\left(-\frac{u^{2}}{32}\right)},\quad
0<u\leq 8\sqrt{\frac{T}{2C}}\\\
2\exp{\left(-\frac{3}{8}\frac{1}{(2C)^{1/3}}T^{1/3}u^{4/3}\right)},\quad
u>8\sqrt{\frac{T}{2C}}\end{cases}.$ (7.6)

###### Proof.

Due to Chebyshev’s inequality and (7.2) we can bound

$\mathbb{P}_{\theta_{0}}\left(|M_{T}^{j}(\theta)-M_{T}^{j}(\vartheta)|>ud_{1}(\theta,\vartheta)\right)\leq\exp\left(\mu^{2}d_{1}^{2}(\theta,\vartheta)+\frac{2C}{T}\mu^{4}d_{1}^{4}(\theta,\vartheta)-u\mu
d_{1}(\theta,\vartheta)\right)$ (7.7)

for all $\mu>0$. In particular, for
$0<\mu\leq\sqrt{\frac{T}{2C}}\frac{1}{d_{1}(\theta,\vartheta)}$ the minimum of
right side of (7.7) is obtained for $\mu=\frac{u}{4d_{1}(\theta,\vartheta)}$
and, consequently,

$\mathbb{P}_{\theta_{0}}\left(|M_{T}^{j}(\theta)-M_{T}^{j}(\vartheta)|>ud_{1}(\theta,\vartheta)\right)\leq\exp\left(-\frac{u^{2}}{8}\right),\quad\text{if}\quad
u\leq 4\sqrt{\frac{T}{2C}}.$ (7.8)

Similarly, for $\mu>\sqrt{\frac{T}{2C}}\frac{1}{d_{1}(\theta,\vartheta)}$ the
minimum of right side of (7.7) is obtained for
$\mu=\left(\frac{T}{16C}\frac{u}{d_{1}^{3}(\theta,\vartheta)}\right)^{1/3}$
and

$\mathbb{P}_{\theta_{0}}\left(|M_{T}^{j}(\theta)-M_{T}^{j}(\vartheta)|>ud_{1}(\theta,\vartheta)\right)\leq\exp{\left(-\frac{3}{8}\frac{1}{(2C)^{1/3}}T^{1/3}u^{4/3}\right)},\quad\text{if}\quad
u\geq 8\sqrt{\frac{T}{2C}}.$ (7.9)

Finally, rescaling (7.8) for
$u\in\left(4\sqrt{\frac{T}{2C}},8\sqrt{\frac{T}{2C}}\right)$ and combining it
with (7.9) finishes the proof. ∎

We now use the generic chaining method to obtain a uniform concentration
inequality for the martingale part via Corollary 7.2.

######  Proposition 7.3.

Assume that conditions (2.2), ($\mathcal{B}$) are satisfied. For all
$T\geq\frac{2C}{27}$ we deduce the inequality

$\displaystyle\mathbb{P}_{\theta_{0}}\bigg{(}\sup_{\theta\in\Theta}|M_{T}^{j}(\theta)-M_{T}^{j}(\theta_{0})|\geq
L\left(\gamma_{4/3}(\Theta,d_{1})+u\Delta_{1}(\Theta)\right)\bigg{)}$ (7.10)
$\displaystyle\leq
L\exp{\left(-2u^{2}\right)}+L\exp{\left(-\frac{6}{(2C)^{1/3}}T^{1/3}u^{4/3}\right)},$
(7.11)

where $L>0$ is a constant independent of $p$, $d$ and $T$.

###### Proof.

We apply the generic chaining technique to obtain uniform bounds. Consider an
admissible sequence $(\mathcal{A}_{n})$ of partitions such that

$\sup_{\theta\in\Theta}\sum_{n\geq 0}2^{3n/4}\Delta(A_{n}(\theta),d_{1})\leq
2\gamma_{4/3}(\Theta,d_{1}).$ (7.12)

For each $n\geq 0$ consider a set $\Theta_{n}$ with $\text{card
}{\Theta_{n}}\leq 2^{2^{n}}$ such that every element $A$ of $\mathcal{A}_{n}$
meets $\Theta_{n}$ and define $\Psi_{n}=\cup_{i\leq n}\Theta_{n}$. For $u>0$
consider the random event

$\displaystyle\Omega(u)=\big{\\{}\forall n\geq
1,\quad\forall\theta,\vartheta\in\Psi_{n},\quad|M_{T}^{j}(\theta)-M_{T}^{j}(\vartheta)|\leq
8(2^{3n/4}+u)d_{1}(\theta,\vartheta)\big{\\}}.$

Then, using the union bound, we can estimate

$\displaystyle\mathbb{P}_{\theta_{0}}(\Omega^{c}(u))\leq\sum_{n\geq
1}(\text{card
}\Psi_{n})^{2}\mathbb{P}\left(|M_{T}^{j}(\theta)-M_{T}^{j}(\vartheta)|>8(2^{3n/4}+u)d_{1}(\theta,\vartheta)\right)$
(7.13) $\displaystyle\leq 8\sum_{n\geq
1}\exp{\left(2^{n}\log{2}-2^{3n/2+1}\right)}\exp{\left(-2u^{2}\right)}$ (7.14)
$\displaystyle+8\sum_{n\geq
1}\exp{\left(2^{n}\log{2}-\frac{6}{(2C)^{1/3}}T^{1/3}2^{n}\right)}\exp{\left(-\frac{6}{(2C)^{1/3}}T^{1/3}u^{4/3}\right)}$
(7.15) $\displaystyle\leq
L_{0}\left(\exp{\left(-2u^{2}\right)}+\exp{\left(-\frac{6}{(2C)^{1/3}}T^{1/3}u^{4/3}\right)}\right),$
(7.16)

where

$L_{0}:=8\sum_{n\geq 1}\exp{\left(2^{n}\log{2}-2^{3n/2+1}\right)}$ (7.17)

and the last inequality holds if $T\geq\frac{2C}{27}$. Now, we will follow the
approach similar to proof of [44, Theorem 2.2.27], to show that when
$\Omega(u)$ occurs, we have

$\sup_{\theta\in\Theta}|M_{T}^{j}(\theta)-M_{T}^{j}(\theta_{0})|\leq
L\left(\gamma_{4/3}(\Theta,d_{1})+u\Delta_{1}(\Theta)\right).$ (7.18)

Consider $\vartheta\in\Theta$ and define by induction over $q\geq 0$ integers
$n(\vartheta,q)$ as $n(\vartheta,0)=0$ and for $q\geq 1$

$n(\vartheta,q)=\inf\\{n\text{ }|\text{ }n\geq n(\vartheta,q-1)\text{ and
}d_{1}(\vartheta,\Psi_{n})\leq\frac{1}{2}d_{1}(\vartheta,\Psi_{n(\vartheta,q-1)})\\}.$
(7.19)

We then consider $\pi_{q}(\vartheta)\in\Psi_{n(\vartheta,q)}$ with
$d_{1}(\vartheta,\pi_{q}(\vartheta))=d_{1}(\vartheta,\Psi_{n(\vartheta,q)})$.
Thus by induction it holds that

$d(\vartheta,\pi_{q}(\vartheta))\leq 2^{-q}\Delta_{1}(\Theta).$ (7.20)

Also when $\Omega(u)$ occurs we obtain

$|M_{T}^{j}(\pi_{q}(\vartheta))-M_{T}^{j}(\pi_{q-1}(\vartheta))|\leq
8(2^{3n(\vartheta,q)/4}+u)d_{1}(\pi_{q}(\vartheta),\pi_{q-1}(\vartheta)).$
(7.21)

and

$\displaystyle|M_{T}^{j}(\theta)-M_{T}^{j}(\theta_{0})|$
$\displaystyle\leq\sum_{q\geq
1}|M_{T}^{j}(\pi_{q}(\vartheta))-M_{T}^{j}(\pi_{q-1}(\vartheta))|$
$\displaystyle\leq\sum_{q\geq
1}8(2^{3n(\vartheta,q)/4}+u)d_{1}(\pi_{q}(\vartheta),\pi_{q-1}(\vartheta))$
$\displaystyle\leq\sum_{q\geq
1}8(2^{3n(\vartheta,q)/4}+u)d_{1}(\vartheta,\pi_{q}(\vartheta))$
$\displaystyle+\sum_{q\geq
1}8(2^{3n(\vartheta,q)/4}+u)d_{1}(\vartheta,\pi_{q-1}(\vartheta))$

We now control each summation separately. First

$\displaystyle\sum_{q\geq
1}2^{3n(\vartheta,q)/4+3}d_{1}(\vartheta,\pi_{q}(\vartheta))$
$\displaystyle\leq\sum_{q\geq
1}2^{3n(\vartheta,q)/4+3}d_{1}(\vartheta,\Theta_{n(\vartheta,q)})\leq
16\gamma_{4/3}{(\Theta,d_{1})}.$

Now by the definition

$d_{1}(\vartheta,\pi_{q-1}(\vartheta))=d_{1}(\vartheta,\Psi_{n(\vartheta,q-1)})\leq
2d_{1}(\vartheta,\Psi_{n(\vartheta,q)-1})$ (7.22)

so that

$\sum_{q\geq
1}2^{3n(\vartheta,q)/4+3}d_{1}(\vartheta,\pi_{q-1}(\vartheta))\leq\sum_{q\geq
1}2^{3n(\vartheta,q)/4+4}d_{1}(\vartheta,\Psi_{n(\vartheta,q)-1})\leq
2^{23/4}\gamma_{4/3}{(\Theta,d_{1})}.$ (7.23)

Finally, we can notice that (7.20) implies

$\sum_{q\geq
1}8(d_{1}(\vartheta,\pi_{q}(\vartheta))+d_{1}(\vartheta,\pi_{q-1}(\vartheta)))\leq
24\Delta(\Theta),$ (7.24)

which finishes the proof for $L=\max(16+2^{23/4},L_{0})=16+2^{23/4}$.

∎

Now, we apply the statement of Proposition to control
$\mathbb{P}(\mathcal{T})$. We recall the definition of $\lambda_{1}$ at
(4.11). Due to previous proposition, we deduce via the union bound that

$\displaystyle\mathbb{P}_{\theta_{0}}\left(\left\|\sup_{\theta\in\Theta}\frac{1}{T}\int_{0}^{T}\big{(}\dot{b}_{\theta}(X_{t}))^{\star}dW_{t}\right\|_{\infty}\geq\frac{\lambda}{2}\right)\leq\sum_{j=1}^{p}\mathbb{P}_{\theta_{0}}\left(\left|\sup_{\theta\in\Theta}\frac{1}{\sqrt{T}}\int_{0}^{T}\big{(}\dot{b}_{\theta,j}(X_{t}))^{\star}dW_{t}\right|\geq\frac{\lambda}{2}\sqrt{T}\right)$
(7.25)
$\displaystyle\leq\sum_{j=1}^{p}\left(\mathbb{P}_{\theta_{0}}\left(\sup_{\theta\in\Theta}|M_{T}^{j}(\theta)-M_{T}^{j}(\theta_{0})|)\geq\frac{\lambda}{4}\sqrt{T}\right)+\mathbb{P}_{\theta_{0}}\left(|M_{T}^{j}(\theta_{0})|\geq\frac{\lambda}{4}\sqrt{T}\right)\right)$
(7.26) $\displaystyle\leq
2pL\exp{\left(-2\left(\frac{\lambda\sqrt{T}}{4L\Delta_{1}(\Theta)}-\frac{\gamma_{4/3}(\Theta,d_{1})}{\Delta_{1}(\Theta)}\right)^{2}\right)}$
(7.27)
$\displaystyle+2pL\exp{\left(-\frac{6}{(2C)^{1/3}}T^{1/3}\left(\frac{\lambda\sqrt{T}}{4L\Delta_{1}(\Theta)}-\frac{\gamma_{4/3}(\Theta,d_{1})}{\Delta_{1}(\Theta)}\right)^{4/3}\right)}\leq\epsilon,$
(7.28)

where the last inequality holds since $\lambda\geq\lambda_{1}$. This completes
the proof of the first part of Theorem 4.2.

###  7.2 Treatment of the set $\mathcal{T}^{\prime}$

We start by defining the random function ($u\in\mathbb{R}^{p}$ with
$\|u\|_{2}=1$)

$H_{u}(\theta):=\frac{u^{\star}}{T}\int_{0}^{T}\left(\dot{b}_{\theta}(X_{t})\dot{b}_{\theta}(X_{t})^{\star}-\mathbb{E}_{\theta_{0}}[\dot{b}_{\theta}(X_{t})\dot{b}_{\theta}(X_{t})^{\star}]\right)dt\times
u.$ (7.29)

We first show a uniform probability bound for the process $H$.

######  Lemma 7.4.

Assume that conditions (2.2), ($\mathcal{B}$) are satisfied. Then we obtain
the bound

$\sup_{u\in\Theta}\mathbb{P}_{\theta_{0}}\bigg{(}\sup_{\xi\in\Theta}\frac{|H_{u}(\theta_{0})-H_{u}(\xi)|}{\|u\|_{2}^{2}}\geq
y\bigg{)}\leq
L\exp\left(-\frac{1}{\Delta^{2}_{2}(\Theta)}\left(\frac{y}{L}\sqrt{\frac{T}{2C}}-\gamma_{2}(\Theta,d_{2})\right)^{2}\right)$
(7.30)

where $L$ has been introduced at (7.17).

###### Proof.

For $u\in\Theta$, $\|u\|_{2}=1$ lets consider functional
$F_{u,\theta,\xi}:C([0,T],\mathbb{R}^{d})\to\mathbb{R}$ such that

$F_{u,\theta,\xi}(X[0,T])=|H_{u}(\theta)-H_{u}(\xi)|$ (7.31)

Then by definition the following estimate holds

$\|F_{u,\theta,\xi}\|_{\text{Lip}}\leq\frac{d_{2}(\theta,\xi)}{T}.$ (7.32)

Then once again according to Theorem 4.1 we deduce that

$\mathbb{P}_{\theta_{0}}\left(\frac{|H_{u}(\theta)-H_{u}(\xi)|}{\|u\|_{2}^{2}}\geq
yd_{2}(\theta,\xi)\right)\leq 2\exp\left(-\frac{T}{4C}y^{2}\right).$ (7.33)

Applying Theorem 2.2.27 in [44] we obtain for all $u\in\Theta$:

$\mathbb{P}_{\theta_{0}}\bigg{(}\sup_{\xi\in\Theta}\frac{|H_{u}(\theta_{0})-H_{u}(\xi)|}{\|u\|_{2}^{2}}\geq\sqrt{\frac{2C}{T}}L\left(\gamma_{2}(\Theta,\Lambda)+y\Delta(\Theta,\Lambda)\right)\bigg{)}\leq
L\exp{(-y^{2})}$ (7.34)

This finishes the proof. ∎

Now we proceed with estimation of the probability
$\mathbb{P}(\mathcal{T}^{\prime})$. We observe that

$\displaystyle\frac{\|b_{\theta}-b_{\vartheta}\|_{T}^{2}}{\|\theta-\vartheta\|_{2}^{2}}=\frac{\mathbb{E}[\|b_{\theta}-b_{\vartheta}\|_{T}^{2}]}{\|\theta-\vartheta\|_{2}^{2}}-\frac{\mathbb{E}[\|b_{\theta}-b_{\vartheta}\|_{T}^{2}]-\|b_{\theta}-b_{\vartheta}\|_{T}^{2}}{\|\theta-\vartheta\|_{2}^{2}}\geq$
$\displaystyle
l_{\text{min}}-\frac{\mathbb{E}[\|b_{\theta}-b_{\vartheta}\|_{T}^{2}]-\|b_{\theta}-b_{\vartheta}\|_{T}^{2}}{\|\theta-\vartheta\|_{2}^{2}}$

where the constant $l_{\text{min}}$ has been defined in (4.10). Consequently,
we deduce the inequality

$\displaystyle\mathbb{P}_{\theta_{0}}\bigg{(}\inf_{\begin{subarray}{c}\theta\neq\vartheta\in\Theta:\\\
\theta-\vartheta\in\mathcal{C}(s,3+4/\gamma)\end{subarray}}\frac{[\|b_{\theta}-b_{\vartheta}\|_{T}^{2}}{\|\theta-\vartheta\|_{2}^{2}}\geq\frac{l_{\text{min}}}{2}\bigg{)}\geq\mathbb{P}\bigg{(}\sup_{\begin{subarray}{c}\theta\neq\vartheta\in\Theta:\\\
\theta-\vartheta\in\mathcal{C}(s,3+4/\gamma)\end{subarray}}\frac{\mathbb{E}[\|b_{\theta}-b_{\vartheta}\|_{T}^{2}]-\|b_{\theta}-b_{\vartheta}\|_{T}^{2}}{\|\theta-\vartheta\|_{2}^{2}}\leq\frac{l_{\text{min}}}{2}\bigg{)}.$

Applying the mean value theorem we can rewrite for some $\xi\in\Theta$

$\mathbb{E}_{\theta_{0}}[\|b_{\theta}-b_{\vartheta}\|_{T}^{2}]-\|b_{\theta}-b_{\vartheta}\|_{T}^{2}=-H_{\theta-\vartheta}(\xi).$
(7.35)

To summarize, we obtain that

$\displaystyle\mathbb{P}_{\theta_{0}}\bigg{(}\inf_{\begin{subarray}{c}\theta\neq\vartheta\in\Theta:\\\
\theta-\vartheta\in\mathcal{C}(s,3+4/\gamma)\end{subarray}}\frac{[\|b_{\theta}-b_{\vartheta}\|_{T}^{2}}{\|\theta-\vartheta\|_{2}^{2}}\geq\frac{l_{\text{min}}}{2}\bigg{)}\geq
1-\mathbb{P}_{\theta_{0}}\bigg{(}\sup_{u\in\mathcal{C}(s,3+4/\gamma)}\sup_{\xi\in\Theta}\left(-\frac{H_{u}(\xi)}{\|u\|_{2}^{2}}\right)\geq\frac{l_{\text{min}}}{2}\bigg{)}\geq$
$\displaystyle
1-\mathbb{P}_{\theta_{0}}\bigg{(}\sup_{u\in\mathcal{C}(s,3+4/\gamma)}\sup_{\xi\in\Theta}\frac{|H_{u}(\theta_{0})-H_{u}(\xi)|}{\|u\|_{2}^{2}}\geq\frac{l_{\text{min}}}{4}\bigg{)}-\mathbb{P}_{\theta_{0}}\bigg{(}\sup_{u\in\mathcal{C}(s,3+4/\gamma)}\frac{|H_{u}(\theta_{0})|}{\|u\|_{2}^{2}}\geq\frac{l_{\text{min}}}{4}\bigg{)}.$

The second probability is easy to analyse, so we will concentrate on bounding
the first one. Similarly to Lemmas F.1 and F.3 from the supplementary material
of [1] one can show that

$\displaystyle\mathbb{P}_{\theta_{0}}\bigg{(}\sup_{u\in\mathcal{C}(s,3+4/\gamma)}\sup_{\xi\in\Theta}\frac{|H_{u}(\theta_{0})-H_{u}(\xi)|}{\|u\|_{2}^{2}}\geq\frac{l_{\text{min}}}{4}\bigg{)}\leq$
$\displaystyle\mathbb{P}_{\theta_{0}}\bigg{(}3(c_{0}+2)^{2}\sup_{u\in\mathcal{K}(2s)}\sup_{\xi\in\Theta}\frac{|H_{u}(\theta_{0})-H_{u}(\xi)|}{\|u\|_{2}^{2}}\geq\frac{l_{\text{min}}}{4}\bigg{)}\leq$
$\displaystyle
21^{2s}\left(p^{2s}\wedge\left(\frac{ep}{2s}\right)^{2s}\right)\sup_{u\in\Theta}\mathbb{P}_{\theta_{0}}\bigg{(}\sup_{\xi\in\Theta}\frac{|H_{u}(\theta_{0})-H_{u}(\xi)|}{\|u\|_{2}^{2}}\geq\frac{l_{\text{min}}}{36(c_{0}+2)^{2}}\bigg{)}$

Consequently, the second part of Theorem 4.2 follows from Lemma 7.4. This
completes the proof for $L=16+2^{23/4}$.

[Acknowledgments] The authors would like to thank Claudia Strauch, Richard
Nickl, Yuri Kutoyants and Jon Wellner for useful comments.

The authors gratefully acknowledge financial support of ERC Consolidator Grant
815703 “STAMFORD: Statistical Methods for High Dimensional Diffusions”.

## References

  * [1] Basu, S. and Michailidis, G. (2015). Regularized estimation in sparse high-dimensional time series models. Annals of Statistics 43(4), 1535–1567.
  * [2] Belomestny, D. , Pilipauskaite, V. and Podolskij, M. (2022). Semiparametric estimation of McKean-Vlasov SDEs. To appear in Annales de l’Institut Henri Poincaré.
  * [3] Bhattacharya, R. N. (1978). Criteria for recurrence and existence of invariant measures for multidimensional diffusions. Ann. Probab. 6 (4), 541–-553.
  * [4] Bickel, P.J., Ritov, Y. and Tsybakov, A.B. (2009). Simultaneous analysis of Lasso and Dantzig selector. Annals of Statistics 37, 1705–1732.
  * [5] Bolley, F., Ca$\tilde{\text{n}}$izo, J.A. and Carrillo, J.A. (2011): Stochastic mean-field limit: non-Lipschitz forces and swarming. Mathematical Models and Methods in Applied Sciences 21(11), 2179–2210.
  * [6] Bobkov, S.G. and Götze, F. (1999). Exponential integrability and transportation cost related to logarithmic Sobolev inequalities. Journal of Functional Analysis 163, 1–28.
  * [7] Bühlmann, P. and van de Geer, S. (2011). Statistics for high-dimensional data. Springer Series in Statistics, Springer.
  * [8] Candes, E. and Tao, T. (2007). The Dantzig selector: Statistical estimation when $p$ is much larger than $n$. Annals of Statistics 35(6), 2313–2351.
  * [9] Carmona, R. and Zhu, X. (2016). A probabilistic approach to mean field games with major and minor players. Annals of Applied Probability 26(3), 1535–1580.
  * [10] Ciolek, G., Marushkevych, D. and Podolskij, M. (2020). On Dantzig and Lasso estimators of the drift in a high dimensional Ornstein-Uhlenbeck model. Electronic Journal of Statistics 14(2), 4395–4420.
  * [11] Della Maestra, L. and Hoffmann, M. (2021). Nonparametric estimation for interacting particle systems: Mckean-Vlasov models. To appear inProbability Theory and Related Fields.
  * [12] De Gregorio, A. and Iacus, S. (2012). Adaptive Lasso-type estimation for multivariate diffusion processes. Econometric Theory 28, 838–860.
  * [13] Djellout, H., Guillin, A. and Wu, L. (2004). Transportation cost-information inequalities and applications to random dynamical systems and diffusions. Annals of Probability 32, 2702–2732.
  * [14] Dexheimer, N. and Strauch, C. (2022). On Lasso and Slope drift estimators for Lévy-driven Ornstein-Uhlenbeck processes. Working paper.
  * [15] T. Dirksen (2015): Tail bounds via generic chaining. Electronic Journal of Probability 20, 1–29.
  * [16] Dicker, L., Li, Y. and Zhao, SD (2014): The Dantzig selector for censored linear regression models. Statistica Sinica 24(1), 251–275
  * [17] Faugeras, O., Touboul, J. and Cessac,B. (2009). A constructive mean-field analysis of multi-population neural networks with random synaptic weights and stochastic inputs. Frontiers in Computational Neuroscience 3, 1–28.
  * [18] Fujimori, K. (2019). The Dantzig selector for a linear model of diffusion processes. Statistical Inference for Stochastic Processes 22, 475–498.
  * [19] Gaïffas, S. and Matulewicz, G. (2019). Sparse inference of the drift of a high-dimensional Ornstein-Uhlenbeck process. Journal of Multivariate Analysis 169, 1–20.
  * [20] Gao, F., Guillin, A. and Wu, L. (2014). Bernstein-type concentration inequalities for symmetric Markov processes. Probability Theory and Its Applications 58(3), 358–382.
  * [21] Genon-Catalot, V. and Laredo, C. (2020). Parametric inference for small variance and long time horizon McKean-Vlasov diffusion models. Working paper.
  * [22] Guillin, A., Leonard, C., Wu, L.  and Yao, N. (2009). Transportation-information inequalities for Markov processes. Probability Theory and Related Fields 144, 669–695.
  * [23] Jackson, M. (2008). Social and economic networks. Princeton, NJ: Princeton University Press.
  * [24] Jacod, J. and Protter, P. (2012). Discretization of processes. Stochastic Modelling and Applied Probability, Springer.
  * [25] James, G., Radchenko, P., and Lv, J. (2009). Dasso: connections between the Dantzig selector and Lasso. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 71(1), 127–142
  * [26] Kasonga, R.A. (1990). Maximum likelihood theory for large interacting systems. SIAM Mathematics 50(3), 865–875.
  * [27] Kessler, M. and Rahbek, A. (2001). Asymptotic likelihood based inference for co-integrated homogenous Gaussian diffusions. Scandinavian Journal of Statistics 28, 455–470.
  * [28] Küchler, U. and Sørensen, M. (1997). Exponential families of stochastic processes. Springer Series in Statistics, Springer.
  * [29] Küchler, U. and Sørensen, M. (1999). A note on limit theorems for multivariate martingales. Bernoulli 5(3), 483–493.
  * [30] Kutoyants, Y. A. (2004). Statistical inference for ergodic diffusion processes. Springer Series in Statistics, Springer.
  * [31] McKean, H.P. (1966). Speed of approach to equilibrium for Kac’s caricature of a Maxwellian gas. Archive for Rational Mechanics and Analysis 21(5), 343–367.
  * [32] McKean, H.P. (1967). Propagation of chaos for a class of non-linear parabolic equations. In Stochastic Differential Equations (Lecture Series in Differential Equations, Session 7, Catholic University, 41–57. Air Force Office of Scientific Research, Arlington.
  * [33] Nourdin, I. and Viens, F.G. (2009). Density formula and concentration inequalities with Malliavin calculus. Electronic Journal of Probability 14, 2287–2309.
  * [34] Nualart, D. (2006). The Malliavin calculus and related topics. 2nd edition, Probability and Its Applications, Springer.
  * [35] Periera, J.B.A. and Ibrahimi, M. (2014). Support recovery for the drift coefficient of high-dimensional diffusions. IEEE Trnasactions of Information Theory 60(7), 4026–4049.
  * [36] Revuz, D. and Yor, M. (2005): Continuous martingales and Brownian motion. 3rd edition, A Series of Comprehensive Studies in Mathematics, Springer.
  * [37] Rogers, L.C.G. and Williams, D. (1987). Diffusions, Markov processes, and martingales. Vol. 2. Itô calculus.  Wiley Series in Probability and Mathematical Statistics, New York.
  * [38] Saussereau, B. (2012). Transportation inequalities for stochastic differential equations driven by a fractional Brownian motion. Bernoulli 18(1), 1–23.
  * [39] Sharrock, L., Kantas, N., Parpas, P., Pavliotis, G.A. (2021). Parameter estimation for the McKean-Vlasov stochastic differential equation. Working paper.
  * [40] Spokoiny, V. (2012). Parametric estimation. Finite sample theory. Annals of Statistics 40(6), 2877–2909.
  * [41] Spokoiny, V. (2017). Penalized maximum likelihood estimation and effective dimension. Annales de l’Institut Henri Poincaré 53(1), 1389–429.
  * [42] Sznitman, A.-S. (1991). Topics in propagation of chaos. In P.-L. Hennequin, editor, École d’Été de Probabilités de Saint Flour XIX - 1989, volume 1464 of Lecture Notes in Mathematics, Springer, Berlin, 165–251.
  * [43] Talagrand, M. (2005). The generic chaining. Springer Monographs in Mathematics, Springer.
  * [44] Talagrand, M. (2014). Upper and lower bounds for stochastic processes. Springer, Berlin Heidelberg.
  * [45] van der Vaart, A. and J.A. Wellner (1996). Weak convergence and empirical processes. Springer Series in Statistics, Springer.
  * [46] Varvenne, M. (2019). Concentration inequalities for stochastic differential equations with additive fractional noise. Electronic Journal of Probability 24, 1–22.
  * [47] Vershynin, R. (2009). Lectures in Geometric Functional Analysis. available at http://www-personal.umich.edu/ romanv/papers/GFA-book/GFA-book.pdf.