$S=((\xi^{i})_{i=1,\dots,N_{\text{Batch}}}$ where $x^{i}$ are i.i.d. drawn
from $\nu$

Compute the gradient $g_{m}=\nabla J^{S}(\theta_{m})$

Update biased first moment estimate:
$\bar{M}_{m}=\beta_{1}\bar{M}_{m-1}+(1-\beta_{1})g_{m}$

Update biased second moment estimate:
$\bar{V}_{m}=\beta_{2}\bar{V}_{m-1}+(1-\beta_{2})g_{m}^{2}$

Compute biased-corrected first moment estimate:
$\hat{M}_{m}=\bar{M}_{m}/(1-\beta_{1}^{m})$

Compute biased-corrected second moment estimate:
$\hat{V}_{m}=\bar{V}_{m}/(1-\beta_{2}^{m})$

Set $\theta_{m+1}=\theta_{m}-\alpha\hat{M}_{m}/(\sqrt{\hat{V}_{m}}+\epsilon)$

end for

Return $\theta_{M}$

Algorithm 2 ADAM: Adaptive Moment Estimation

##  Appendix B Preliminaries on SDDE

In this section, we provide some background on stochastic differential delay
equations. Let $\mathscr{C}_{-\delta}=C([-\delta,0],\mathbb{R}^{d})$ be the
Banach space of all $\mathbb{R}^{d}$-valued continuous functions defined on
$[-\delta,0]$ endowed with the supremum norm

$\|y\|_{\mathscr{C}_{-\delta}}=\sup_{-\delta\leq s\leq 0}|y_{s}|,\;\forall
y\in\mathscr{C}_{-\delta}.$ (B.1)

This space is represents portions of trajectories. The drift $b$ and
volatility $\sigma$ coefficients, and the running $f$ and terminal costs $g$
are deterministic functionals

$(b,\sigma,f):[0,T]\times\mathscr{C}_{-\delta}\times\mathcal{A}\to(\mathbb{R}^{d},\mathbb{R}^{d\times
m},\mathbb{R});\quad g:\mathscr{C}_{-\delta}\to\mathbb{R}.$ (B.2)

Denote by $L^{2}(\Omega,\mathscr{C}_{-\delta})$ the space of all
$\mathcal{F}$-measurable stochastic processes, _i.e._ ,

$\Omega\ni\omega\to X(\omega)\in\mathscr{C}_{-\delta}\text{ is in
}L^{2}(\Omega,\mathscr{C}_{-\delta}),\text{ iff.
}\int_{\Omega}\|X(\omega)\|_{\mathscr{C}_{-\delta}}^{2}\,\mathrm{d}\mathbb{P}(\omega)<\infty.$
(B.3)

The space $L^{2}(\Omega,\mathscr{C}_{-\delta})$ endowed with the semi-norm
$\|X\|_{L^{2}(\Omega,\mathscr{C}_{-\delta})}=[\int_{\Omega}\|X(\omega)\|_{\mathscr{C}_{-\delta}}^{2}\,\mathrm{d}\mathbb{P}(\omega)]^{1/2}$
is a complete space. We assume that the initial path $\underline{\varphi}\in
L^{2}(\Omega,\mathscr{C}_{-\delta})$ and is independent of the Brownian motion
$W$, and the existence of a solution $X$ to the SDDE (2.28) is considered in
$L^{2}(\Omega,C([-\delta,T],\mathbb{R}^{d}))$. Let $(\mathcal{F}_{t})_{t\geq
0}$ be the filtration supporting $W$ and $\varphi$, and let
$C([0,T],L^{2}(\Omega,\mathscr{C}_{-\delta}))$ be the space of all
$L^{2}$-continuous $\mathscr{C}_{-\delta}$-valued $\mathcal{F}_{t}$-adapted
processes $P:[0,T]\ni t\to\underline{P}_{t}\in
L^{2}(\Omega,\mathscr{C}_{-\delta})$ endowed with the semi-norm

$\|P\|_{C([0,T],L^{2}(\Omega,\mathscr{C}_{-\delta}))}=\sup_{0\leq t\leq
T}\|\underline{P}_{t}\|_{L^{2}(\Omega,\mathscr{C}_{-\delta})}.$ (B.4)

The trajectory $\underline{X}_{t}$ solving the SDDE (2.28) is considered in
$C([0,T],L^{2}(\Omega,\mathscr{C}_{-\delta}))$.

Usually, one requires uniform Lipschitz conditions in the second variable of
$b$ and $\sigma$ to ensure the existence and uniqueness of strong solutions to
the SDDE (2.28), that is,

$\|(b,\sigma)(t,y_{1},\alpha)-(b,\sigma)(t,y_{2},\alpha)\|_{L^{2}}\leq
L\|y_{1}-y_{2}\|_{L^{2}(\Omega,\mathscr{C}_{-\delta})},\;\forall
t\in[0,T]\text{ and }y_{1},y_{2}\in L^{2}(\Omega,\mathscr{C}_{-\delta}).$
(B.5)

See detailed analysis in Mohammed’s monographs [257, 258]. Assumptions on $f$
and $g$ will ensure the expected cost (2.29) is finite.

##  Appendix C Pseudo-codes of Algorithms

Algorithm 3 Deep Fictitious Play for Finding Open-loop Nash Equilibrium

0: $N$ = # of players, $N_{T}$ = # of subintervals on $[0,T]$, $M$ = # of
training paths, $M^{\prime}$ = # of out-of-sample paths for final evaluation,
$\bm{\alpha}^{0}=\\{\alpha_{t)n}^{i,0}\in\mathcal{A}\subset\mathbb{R}^{k},i\in\mathcal{I}\\}_{n=0}^{N_{T}-1}$
= initial belief,
$\bm{X}_{0}=\\{x_{0}^{i}\in\mathbb{R}^{d},i\in\mathcal{I}\\}$ = initial states

1: Create $N$ separated deep neural networks as described in Eq. (3.7)

2: Generate $M$ discrete sample path of BM:
$\bm{W}=\\{W_{t_{n}}^{i}\in\mathbb{R}^{m},i\in\mathcal{I}\cup\\{0\\}\\}_{n=1}^{N_{T}}$

3: $\mathtt{k}\leftarrow 0$

4: repeat

5: for all $i\in\mathcal{I}$ do in parallel

6: $\mathtt{k}\leftarrow\mathtt{k}+1$

7: (Continue to) Train $i^{th}$ NN with data
$\\{\bm{X}_{0},\bm{\alpha}^{-i,\mathtt{k}-1}=\\{\alpha_{t_{n}}^{j,\mathtt{k}-1},j\in\mathcal{I}\setminus\\{i\\}\\}_{n=0}^{N_{T}-1},\bm{W}\\}$

8: Obtain the approximated optimal strategy $\alpha^{i,\mathtt{k}}$ and cost
$J^{i}(\alpha^{i,\mathtt{k}};\bm{\alpha}^{-i,\mathtt{k}-1})$

9: end for

10: Collect optimal policies at stage $\mathtt{k}$:
$\bm{\alpha}^{\mathtt{k}}\leftarrow(\alpha^{1,\mathtt{k}},\ldots,\alpha^{N,\mathtt{k}})$

11: Compute relative change of cost $\displaystyle
err^{\mathtt{k}}=\max_{i\in\mathcal{I}}\left\\{\frac{\left|J^{i}(\alpha^{i,\mathtt{k}};\bm{\alpha}^{-i,\mathtt{k}-1})-J^{i}(\alpha^{i,\mathtt{k}-1};\bm{\alpha}^{-i,\mathtt{k}-2})\right|}{J^{i}(\alpha^{i,\mathtt{k}-1};\bm{\alpha}^{-i,\mathtt{k}-2})}\right\\}$

12: until $err^{\mathtt{k}}$ go below a threshold

13: Generate $M^{\prime}$ out-of-sample paths of BM for final evaluation

14: $\mathtt{k}^{\prime}\leftarrow 0$

15: repeat

16: $\mathtt{k}^{\prime}\leftarrow\mathtt{k}^{\prime}+1$

17: Evaluate $i^{th}$ NN with {$\bm{X}_{0}$,
$\bm{\alpha}^{-i,\mathtt{k}^{\prime}-1}$, out-of-sample paths}, $\forall
i\in\mathcal{I}$

18: Obtain $\alpha^{i,\mathtt{k}^{\prime}}$ and
$J^{i,\mathtt{k}^{\prime}}=J^{i}(\alpha^{i,\mathtt{k}^{\prime}};\bm{\alpha}^{-i,\mathtt{k}^{\prime}-1})$
$\forall i\in\mathcal{I}$

19: until $J^{i,\mathtt{k}^{\prime}}$ converges in $\mathtt{k}^{\prime}$,
$\forall i\in\mathcal{I}$

20: return The optimal policy $\alpha^{i,\mathtt{k}^{\prime}}$, and the final
cost for each player $J^{i,\mathtt{k}^{\prime}}$

Algorithm 4 Deep Fictitious Play for Finding Markovian Nash Equilibrium

0: $N$ = # of players, $N_{T}$ = # of subintervals on $[0,T]$, $\mathtt{K}$ =
# of total stages in fictitious play, $N_{\text{sample}}$ = # of sample paths
generated for each player at each stage of fictitious play,
$N_{\text{SGD\\_per\\_stage}}$ = # of SGD steps for each player at each stage,
$N_{\text{batch}}$ = batch size per SGD update, $\bm{\alpha}^{0}\colon$ the
initial policies that are smooth enough

1: Initialize $N$ deep neural networks to represent $u^{i,0},i\in\mathcal{I}$

2: for $\mathtt{k}\leftarrow 1$ to $\mathtt{K}$ do

3: for all $i\in\mathcal{I}$ do in parallel

4: Generate $N_{\text{sample}}$ sample paths
$\\{\check{\bm{X}}_{t_{n}}^{i}\\}_{n=0}^{N_{T}}$ according to (3.24) and the
realized optimal policies
$\bm{\alpha}^{-i,\mathtt{k}-1}(t_{n},\check{\bm{X}}_{t_{n}}^{i})$

5: for $\ell\leftarrow 1$ to $N_{\text{SGD}\\_\text{per}\\_\text{stage}}$ do

6: Update the parameters of the $i^{th}$ neural network one step with
$N_{\text{batch}}$ paths using the SGD algorithm (or its variant), based on
the loss function (3.23)

7: end for

8: Obtain the approximate optimal policy $\alpha^{i,\mathtt{k}}$ according to
(3.19)

9: end for

10: Collect the optimal policies at stage $\mathtt{k}$:
$\bm{\alpha}^{\mathtt{k}}\leftarrow(\alpha^{1,\mathtt{k}},\ldots,\alpha^{N,\mathtt{k}})$

11: end for

12: return The optimal policy $\bm{\alpha}^{\mathtt{K}}$

Algorithm 5 The Sig-DFP Algorithm

Input: $b,\sigma,\sigma_{0},f,g,\iota$ and
$X_{0}(\omega_{i}),\\{W_{t_{n}}(\omega_{i})\\}_{n=0}^{N_{T}},\\{W^{0}_{t_{n}}(\omega_{i})\\}_{n=0}^{N_{T}}$
for $i=1,2,\dots,N$; $\mathtt{K}$: rounds for FP; $B$: minibatch size;
$N_{\text{batch}}$: number of minibatches.

Compute the signatures of $\hat{W}^{0}_{t_{n}}(\omega_{i})$ for $i=1,\dots,N$,
$n=1,\dots,N_{T}$;

Initialize $\hat{\nu}^{(0)}$, $\theta$;

for $\mathtt{k}=1$ to $\mathtt{K}$ do

for $r=1$ to $N_{\text{batch}}$ do

Simulate the $r^{th}$ minibatch of $X^{(\mathtt{k})}(\omega_{i})$ using
$\hat{\nu}^{(\mathtt{k}-1)}$ and compute
$J_{B}(\theta,\hat{\nu}^{(\mathtt{k}-1)})$;

Minimize $J_{B}(\theta,\hat{\nu}^{(\mathtt{k}-1)})$ over $\theta$, then update
$\alpha(\cdot;\theta)$;

end for

Simulate $X^{(\mathtt{k})}(\omega_{i})$ with the optimized
$\alpha(\cdot;\theta^{\ast})$, for $i=1,\dots,N$;

Regress
$\iota(X^{(\mathtt{k})}_{0}(\omega_{i}),\alpha^{(\mathtt{k})}_{0}(\omega_{i}))$,
$\iota(X^{(\mathtt{k})}_{T/2}(\omega_{i}),\alpha^{(\mathtt{k})}_{T/2}(\omega_{i}))$,
$\iota(X^{(\mathtt{k})}_{T}(\omega_{i}),\alpha^{(\mathtt{k})}_{T}(\omega_{i}))$
on $S^{M}(\hat{W}^{0}_{0}{\omega_{i}})$,
$S^{M}(\hat{W}^{0}_{t_{N_{T}/2}}(\omega_{i}))$,
$S^{M}(\hat{W}^{0}_{T}(\omega_{i}))$ to get $\tilde{l}^{(\mathtt{k})}$;

Update
$\hat{l}^{(\mathtt{k})}=\frac{\mathtt{k}-1}{\mathtt{k}}\hat{l}^{(\mathtt{k}-1)}+\frac{1}{\mathtt{k}}\tilde{l}^{(\mathtt{k})}$;

Compute $\hat{\nu}^{(\mathtt{k})}$ by
$\hat{\nu}^{(\mathtt{k})}_{t_{n}}(\omega^{i})=\langle\hat{l}^{(\mathtt{k})},S^{M}(\hat{W}^{0}_{t_{n}}(\omega_{i}))\rangle$,
for $i=1,2,\dots,N,n=1,\dots,N_{T}$;

end for

Output: the optimized $\alpha_{\varphi}^{\ast}$ and $\hat{l}^{(\mathtt{K})}$.

##  Appendix D List of Acronyms

ADAM | Adaptive Moments Gradient Descent
---|---
BSDE | Backward Stochastic Differential Equation
2BSDE | Second Order Backward Stochastic Differential Equation
CDC | Centers for Disease Control and Prevention
DNN | Deep Neural Networks
DBDP | Deep Backward Dynamic Programming
Deep BSDE | Deep Backward Stochastic Differential Equation
DDPG | Deep Deterministic Policy Gradient
DGM | Deep Galerkin Method
DFP | Deep Fictitious Play
DPG | Deterministic Policy Gradient
DPP | Dynamic Programming Principle
FBSDE | Forward Backward Stochastic Differential Equation
FNN | Feedforward Neural Network
FP | Fokker-Planck
GRU | Gated Recurrent Unit
HJB | Hamilton-Jacobi-Bellman
KFP | Kolmogorov-Fokker-Planck
LQ | Linear-Quadratic
LSTM | Long Short-term Memory
MARL | Multi-agent Reinforcement Learning
MC | Monte Carlo
MDP | Markov Decision Process
MFC | Mean-Field Control
MFG | Mean-Field Game
MFMDP | Mean-Field Markov Decision Process
MKV | McKean-Vlasov
MKV FBSDE | McKean-Vlasov Forward Backward Stochastic Differential Equation
NE | Nash Equilibrium
NN | Neural Network
ODE | Ordinary Differential Equation
PDE | Partial Differential Equations
PINN | Physics Informed Neural Network
ReLu | Rectified Linear Unit
RL | Reinforcement Learning
RNN | Recurrent Neural Network
SC | Stochastic Control
SDE | Stochastic Differential Equation
SDDE | Stochastic Differential Delay Equation
SDFP | Scaled Deep Fictitious Play
SGD | Stochastic Gradient Descent
Sig-DFP | Signatured Deep Fictitious Play
TD | Temporal-Difference

##  Appendix E List of Frequently Used Notations

Deep learning related notations
---
$\theta$ | neural network parameters
$\varphi^{\theta}$ | neural network with parameters $\theta$
$J(\theta)$ | optimization function for training the parameters $\theta$
Optimal control related notations
$x$ | state variable
$(X_{t})_{t\in[0,T]}$ | state process
$(Y_{t},Z_{t})_{t\in[0,T]}$ | backward and adjoint processes
$(\alpha_{t})_{t\in[0,T]}$, $(\beta_{t})_{t\in[0,T]}$ | control process
$(W_{t})_{t\in[0,T]}$, $(W^{0}_{t})_{t\in[0,T]}$ | Wiener process
$d$ | dimension of the state
$k$ | dimension of the action
$m$ | dimension of the noise
$b(t,x,\alpha)$ | drift coefficient of the state process
$\sigma(t,x,\alpha)$ | diffusion coefficient of the state process
$f(t,x,\alpha)$ | instantaneous cost function
$g(x)$ | terminal cost function
$J(\alpha)$ | total cost function associated with the control $\alpha$
$\check{J}(\theta)$ | total cost function associated with the discretized stochastic control problem
| with neural network parameters $\theta$
$(\overline{X}_{t})_{t\in[0,T]}$ | empirical average of $N$-players’ states
$\underline{X}_{t}=\bigl{(}\underline{X}_{t}(s)\bigr{)}_{s\in[-\delta,0]}$ | trajectory of $X_{t}$ from time $t-\delta$ to $t$
$T$ | time horizon
$\mathrm{Hess}_{x}u(t,x)$ | Hessian matrix of $u$ with respect to $x$
$H$ | Hamiltonian function
$u(t,x)$ | value function
Stochastic games related notations
$X^{i}_{t}$ | state process for player $i$
$\alpha^{i}_{t}$ | action function for player $i$
$\bm{\alpha}=[\alpha^{1},\alpha^{2},\ldots,\alpha^{N}]$ | a collection of all players’ strategy profiles
$\bm{\alpha}^{-i}=[\alpha^{1},\ldots,\alpha^{i-1},\alpha^{i+1},\ldots,\alpha^{N}]$ | the strategy profiles excluding player $i$’s
$\bm{W}=[W^{0},W^{1},\ldots,W^{N}]$ |  $(N+1)$-vector of $m$-dimensional independent Brownian motions
$\mathbb{F}=\\{\mathcal{F}_{t},0\leq t\leq T\\}$ | the augmented filtration generated by $\bm{W}$
$J^{i}(\boldsymbol{\alpha})$ | cost function for player $i$
$\mathrm{k}$ | stage index in deep fictitious play
$m$ | mean process (or conditional mean if there is common noise)
$\mu$ | state distribution process
$\mu^{N}$ | empirical state distribution process
$\nu$ | state-action distribution process
$\nu^{N}$ | empirical state-action distribution process

## References

  * [1] B. Acciaio, J. Backhoff-Veraguas, and R. Carmona.  Extended mean field control problems: stochastic maximum principle and transport perspective.  SIAM journal on Control and Optimization, 57(6):3666–3693, 2019\.
  * [2] Y. Achdou, F. Camilli, and I. Capuzzo-Dolcetta.  Mean field games: numerical methods for the planning problem.  SIAM Journal on Control and Optimization, 50(1):77–109, 2012.
  * [3] Y. Achdou, F. Camilli, and I. Capuzzo-Dolcetta.  Mean field games: convergence of a finite difference method.  SIAM J. Numer. Anal., 51(5):2585–2612, 2013.
  * [4] Y. Achdou and I. Capuzzo-Dolcetta.  Mean field games: numerical methods.  SIAM J. Numer. Anal., 48(3):1136–1162, 2010.
  * [5] Y. Achdou and I. Capuzzo-Dolcetta.  Mean field games: numerical methods.  SIAM Journal on Numerical Analysis, 48(3):1136–1162, 2010.
  * [6] Y. Achdou and Z. Kobeissi.  Mean field games of controls: Finite difference approximations.  arXiv preprint arXiv:2003.03968, 2020.
  * [7] Y. Achdou and M. Laurière.  On the system of partial differential equations arising in mean field type control.  Discrete & Continuous Dynamical Systems, 35(9):3879, 2015.
  * [8] Y. Achdou and M. Laurière.  Mean Field Type Control with Congestion (II): An augmented Lagrangian method.  Appl. Math. Optim., 74(3):535–578, 2016.
  * [9] Y. Achdou and M. Laurière.  Mean field games and applications: Numerical aspects.  Mean field games, pages 249–307, 2020.
  * [10] Y. Achdou, M. Lauriere, and P.-L. Lions.  Optimal control of conditioned processes with feedback controls.  Journal de Mathématiques Pures et Appliquées, 2020.
  * [11] N. Agram, A. Bakdi, and B. Oksendal.  Deep learning and stochastic mean-field control for a neural network model.  SSRM preprint ssrn.3683722, 2020.
  * [12] A. Al-Aradi, A. Correia, G. Jardim, D. de Freitas Naiff, and Y. Saporito.  Extensions of the deep galerkin method.  Applied Mathematics and Computation, 430:127287, 2022.
  * [13] N. Almulla, R. Ferreira, and D. Gomes.  Two numerical approaches to stationary mean-field games.  Dynamic Games and Applications, 7(4):657–682, 2017.
  * [14] B. Anahtarci, C. D. Kariksiz, and N. Saldi.  Q-learning in regularized mean-field games.  arXiv preprint arXiv:2003.12151, 2020.
  * [15] D. Andersson and B. Djehiche.  A maximum principle for SDEs of mean-field type.  Appl. Math. Optim., 63(3):341–356, 2011.
  * [16] R. Andreev.  Preconditioning the augmented Lagrangian method for instationary mean field games with diffusion.  SIAM J. Sci. Comput., 39(6):A2763–A2783, 2017.
  * [17] A. Angiuli, J.-P. Fouque, and M. Laurière.  Unified reinforcement Q-learning for mean field game and control problems.  To appear in Mathematics of Control, Signals, and Systems (MCSS). arXiv:2006.13912, 2020.
  * [18] A. Angiuli, J.-P. Fouque, and M. Laurière.  Unified reinforcement Q-learning for mean field game and control problems.  arXiv preprint arXiv:2006.13912, 2020.
  * [19] A. Angiuli, C. V. Graves, H. Li, J.-F. Chassagneux, F. Delarue, and R. Carmona.  Cemracs 2017: numerical probabilistic approach to MFG.  ESAIM: Proceedings and Surveys, 65:84–113, 2019.
  * [20] P. K. Asea and P. J. Zak.  Time-to-build and cycles.  Journal of Economic Dynamics and Control, 23(8):1155–1175, 1999\.
  * [21] M. Assouli and B. Missaoui.  Deep learning for Mean Field Games with non-separable Hamiltonians.  Chaos, Solitons & Fractals, 174:113802, 2023.
  * [22] A. Aurell, R. Carmona, G. Dayanıklı, and M. Laurière.  Finite state graphon games with applications to epidemics.  Dynamic Games and Applications, pages 1–33, 2022.
  * [23] A. Aurell, R. Carmona, G. Dayanikli, and M. Lauriere.  Optimal incentives to mitigate epidemics: a stackelberg mean field game approach.  SIAM Journal on Control and Optimization, 60:S294–S322, 2022.
  * [24] A. Bachouch, C. Huré, N. Langrené, and H. Pham.  Deep neural networks algorithms for stochastic control problems on finite horizon: numerical applications.  Methodology and Computing in Applied Probability, pages 1–36, 2021\.
  * [25] A. Balata, C. Huré, M. Laurière, H. Pham, and I. Pimentel.  A class of finite-dimensional numerically solvable McKean-Vlasov control problems.  ESAIM: Proceedings and Surveys, 65:114–144, 2019.
  * [26] V. Bally and G. Pages.  Error analysis of the optimal quantization algorithm for obstacle problems.  Stochastic processes and their applications, 106(1):1–40, 2003\.
  * [27] V. Bally, J. Printems, et al.  A stochastic quantization method for nonlinear problems.  Monte Carlo Methods and Applications, 7(1-2):21–34, 2001.
  * [28] M. Bardi, M. Falcone, and P. Soravia.  Numerical methods for pursuit-evasion games via viscosity solutions.  In Stochastic and differential games: theory and numerical methods, pages 105–175. Springer, 1999.
  * [29] M. Bardi, T. Raghavan, and T. Parthasarathy.  Stochastic and differential games: theory and numerical methods, volume 4.  Springer Science & Business Media, 1999.
  * [30] M. Barnett, W. Brock, L. P. Hansen, R. Hu, and J. Huang.  A deep learning analysis of climate change, innovation, and uncertainty.  arXiv preprint arXiv:2310.13200, 2023.
  * [31] C. Barrera-Esteve, F. Bergeret, C. Dossal, E. Gobet, A. Meziou, R. Munos, and D. Reboul-Salze.  Numerical methods for the pricing of swing options: a stochastic control approach.  Methodology and computing in applied probability, 8:517–540, 2006\.
  * [32] T. Başar.  A tutorial on dynamic and differential games.  Dynamic games and applications in economics, pages 1–25, 1986.
  * [33] H. Bauer and U. Rieder.  Stochastic control problems with delay.  Mathematical Methods of Operations Research, 62(3):411–427, 2005\.
  * [34] E. Bayraktar, A. Budhiraja, and A. Cohen.  A numerical scheme for a mean field game in some queueing systems based on markov chain approximation method.  SIAM Journal on Control and Optimization, 56(6):4017–4044, 2018\.
  * [35] E. Bayraktar, N. Bauerle, and A. D. Kara.  Finite approximations and Q learning for Mean Field Type Multi Agent Control.  arXiv preprint arXiv:2211.09633, 2022.
  * [36] E. Bayraktar, Q. Feng, and Z. Zhang.  Deep signature algorithm for path-dependent american option pricing.  arXiv preprint arXiv:2211.11691, 2022.
  * [37] R. W. Beard, G. N. Saridis, and J. T. Wen.  Galerkin approximations of the generalized hamilton-jacobi-bellman equation.  Automatica, 33(12):2159–2177, 1997.
  * [38] C. Beck, W. E, and A. Jentzen.  Machine learning approximation algorithms for high-dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equations.  Journal of Nonlinear Science, 29(4):1563–1619, 2019.
  * [39] S. Becker, P. Cheridito, and A. Jentzen.  Deep optimal stopping.  Journal of Machine Learning Research, 20:74, 2019.
  * [40] S. Becker, P. Cheridito, and A. Jentzen.  Pricing and hedging American-style options with deep learning.  Journal of Risk and Financial Management, 13(7):158, 2020.
  * [41] R. Bellman.  A Markovian decision process.  Journal of mathematics and mechanics, pages 679–684, 1957.
  * [42] J.-D. Benamou and G. Carlier.  Augmented Lagrangian methods for transport optimization, mean field games and degenerate elliptic equations.  J. Optim. Theory Appl., 167(1):1–26, 2015.
  * [43] C. Bender, N. Schweizer, and J. Zhuo.  A primal–dual algorithm for BSDEs.  Mathematical Finance, 27(3):866–901, 2017.
  * [44] C. Bender and J. Steiner.  Least-squares monte carlo for backward sdes.  Springer, 2012.
  * [45] C. Bender and J. Steiner.  A posteriori estimates for backward SDEs.  SIAM/ASA Journal on Uncertainty Quantification, 1(1):139–163, 2013\.
  * [46] A. Bensoussan.  Estimation and control of dynamical systems, volume 48.  Springer, 2018.
  * [47] A. Bensoussan, J. Frehse, and S. C. P. Yam.  Mean field games and mean field type control theory.  Springer Briefs in Mathematics. Springer, New York, 2013.
  * [48] A. Bensoussan, J. Frehse, and S. C. P. Yam.  The master equation in mean field theory.  J. Math. Pures Appl. (9), 103(6):1441–1474, 2015.
  * [49] J. Berner, P. Grohs, G. Kutyniok, and P. Petersen.  The Modern Mathematics of Deep Learning, page 1–111.  Cambridge University Press, 2022.
  * [50] D. P. Bertsekas and S. E. Shreve.  Stochastic optimal control: the discrete-time case, volume 5.  Athena Scientific, 1996.
  * [51] D. Bloembergen, K. Tuyls, D. Hennes, and M. Kaisers.  Evolutionary dynamics of multi-agent learning: A survey.  Journal of Artificial Intelligence Research, 53:659–697, 2015.
  * [52] H. Boedihardjo, X. Geng, T. Lyons, and D. Yang.  The signature of a rough path: uniqueness.  Advances in Mathematics, 293:720–737, 2016.
  * [53] J. F. Bonnans and H. Zidani.  Consistency of generalized finite difference schemes for the stochastic hjb equation.  SIAM Journal on Numerical Analysis, 41(3):1008–1021, 2003.
  * [54] P. Bonnier, P. Kidger, I. P. Arribas, C. Salvi, and T. Lyons.  Deep signature transforms.  In Advances in Neural Information Processing Systems 32 (NeurIPS), 2019.
  * [55] B. Bouchard, R. Elie, and N. Touzi.  Discrete-time approximation of bsdes and probabilistic schemes for fully nonlinear pdes.  Advanced financial modelling, 8:91–124, 2009.
  * [56] B. Bouchard and N. Touzi.  Discrete-time approximation and Monte-Carlo simulation of backward stochastic differential equations.  Stochastic Processes and their applications, 111(2):175–206, 2004\.
  * [57] M. Boulbrachene and M. Haiour.  The finite element approximation of hamilton-jacobi-bellman equations.  Computers & Mathematics with Applications, 41(7-8):993–1007, 2001\.
  * [58] A. Briani and P. Cardaliaguet.  Stable solutions in potential mean field game systems.  Nonlinear Differential Equations and Applications, 25(1):1, 2018\.
  * [59] L. M. Briceño Arias, D. Kalise, Z. Kobeissi, M. Laurière, A. Mateos González, and F. J. Silva.  On the implementation of a primal-dual algorithm for second order time-dependent mean field games with local couplings.  ESAIM: ProcS, 65:330–348, 2019.
  * [60] L. M. Briceño Arias, D. Kalise, and F. J. Silva.  Proximal methods for stationary mean field games with local couplings.  SIAM J. Control Optim., 56(2):801–836, 2018.
  * [61] G. W. Brown.  Some notes on computation of games solutions.  Technical report, Rand Corp Santa Monica CA, 1949.
  * [62] G. W. Brown.  Iterative solution of games by fictitious play.  Activity Analysis of Production and Allocation, 13(1):374–376, 1951\.
  * [63] A. Budhiraja and K. Ross.  Convergent numerical scheme for singular stochastic control with state constraints in a portfolio selection problem.  SIAM Journal on Control and Optimization, 45(6):2169–2206, 2007\.
  * [64] L. Busoniu, R. Babuska, and B. De Schutter.  A comprehensive survey of multiagent reinforcement learning.  IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 38(2):156–172, 2008.
  * [65] F. Camilli, S. Duisembay, and Q. Tang.  Approximation of an optimal control problem for the time-fractional fokker-planck equation.  Journal of Dynamics & Games, 8(4), 2021.
  * [66] H. Cao, X. Guo, and M. Laurière.  Connecting GANs, mean-field games, and optimal transport.  arXiv preprint arXiv:2002.04112, 2020.
  * [67] P. Cardaliaguet.  Notes on mean field games, 2013.
  * [68] P. Cardaliaguet, F. Delarue, J.-M. Lasry, and P.-L. Lions.  The master equation and the convergence problem in mean field games.  Princeton University Press, 2019.
  * [69] P. Cardaliaguet and S. Hadikhanloo.  Learning in mean field games: the fictitious play.  ESAIM: Control, Optimisation and Calculus of Variations, 23(2):569–591, 2017.
  * [70] P. Cardaliaguet and C.-A. Lehalle.  Mean field game of controls and an application to trade crowding.  Mathematics and Financial Economics, 12(3):335–363, 2018.
  * [71] P. Cardaliaguet and C. Rainer.  On the (in) efficiency of mfg equilibria.  SIAM Journal on Control and Optimization, 57(4):2292–2314, 2019\.
  * [72] E. Carlini and F. J. Silva.  A fully discrete semi-Lagrangian scheme for a first order mean field game problem.  SIAM J. Numer. Anal., 52(1):45–67, 2014.
  * [73] E. Carlini and F. J. Silva.  A fully discrete semi-Lagrangian scheme for a first order mean field game problem.  SIAM Journal on Numerical Analysis, 52(1):45–67, 2014.
  * [74] E. Carlini and F. J. Silva.  A semi-Lagrangian scheme for a degenerate second order mean field game system.  Discrete Contin. Dyn. Syst., 35(9):4269–4292, 2015.
  * [75] E. Carlini and F. J. Silva.  On the discretization of some nonlinear Fokker–Planck–Kolmogorov equations and applications.  SIAM Journal on Numerical Analysis, 56(4):2148–2177, 2018.
  * [76] E. Carlini and F. J. Silva.  On the discretization of some nonlinear Fokker-Planck-Kolmogorov equations and applications.  SIAM J. Numer. Anal., 56(4):2148–2177, 2018.
  * [77] R. Carmona.  Lectures on BSDEs, Stochastic Control, and Stochastic Differential Games with Financial Applications, volume 1.  SIAM, 2016.
  * [78] R. Carmona and F. Delarue.  Probabilistic analysis of mean-field games.  SIAM Journal on Control and Optimization, 51(4):2705–2734, 2013\.
  * [79] R. Carmona and F. Delarue.  Probabilistic Theory of Mean Field Games with Applications II.  Springer, 2017.
  * [80] R. Carmona, J.-P. Fouque, and L.-H. Sun.  Mean field games and systemic risk.  Communications in Mathematical Sciences, 13(4):911–933, 2015.
  * [81] R. Carmona.  Lectures on BSDEs, stochastic control, and stochastic differential games with financial applications.  SIAM, 2016.
  * [82] R. Carmona, F. Delarue, and A. Lachapelle.  Control of McKean-Vlasov dynamics versus mean field games.  Math. Financ. Econ., 7(2):131–166, 2013.
  * [83] R. Carmona and F. Delarue.  The master equation for large population equilibriums.  In Stochastic analysis and applications 2014, pages 77–128. Springer, 2014.
  * [84] R. Carmona and F. Delarue.  Probabilistic Theory of Mean Field Games with Applications I.  Springer, 2018.
  * [85] R. Carmona and F. Delarue.  Probabilistic Theory of Mean Field Games with Applications II.  Springer, 2018.
  * [86] R. Carmona, J.-P. Fouque, and L.-H. Sun.  Mean field games and systemic risk.  Commun. Math. Sci., 13(4):911–933, 2015.
  * [87] R. Carmona, C. V. Graves, and Z. Tan.  Price of anarchy for mean field games.  In CEMRACS 2017—numerical methods for stochastic models: control, uncertainty quantification, mean-field, volume 65 of ESAIM Proc. Surveys, pages 349–383. EDP Sci., Les Ulis, 2019.
  * [88] R. Carmona and D. Lacker.  A probabilistic weak formulation of mean field games and applications.  Ann. Appl. Probab., 25(3):1189–1231, 2015.
  * [89] R. Carmona and M. Laurière.  Convergence analysis of machine learning algorithms for the numerical solution of mean field control and games: I–the ergodic case.  To appear in SIAM Journal on Numerical Analysis. arXiv preprint arXiv:1907.05980, 2019.
  * [90] R. Carmona and M. Laurière.  Convergence analysis of machine learning algorithms for the numerical solution of mean field control and games: II–the finite horizon case.  arXiv preprint arXiv:1908.01613. To appear in Annals of Applied Probability, 2019.
  * [91] R. Carmona and M. Laurière.  Deep learning for mean field games and mean field control with applications to finance.  Machine Learning in Financial Markets: A guide to contemporary practises, editors: A. Capponi and C.-A. Lehalle, Cambridge University Press (Preprint arXiv:2107.04568), 2022.
  * [92] R. Carmona, M. Laurière, and Z. Tan.  Linear-quadratic mean-field reinforcement learning: convergence of policy gradient methods.  arXiv preprint arXiv:1910.04295, 2019.
  * [93] R. Carmona, M. Laurière, and Z. Tan.  Model-free mean-field reinforcement learning: mean-field mdp and mean-field q-learning.  The Annals of Applied Probability, 33(6B):5334–5381, 2023.
  * [94] R. Carmona and L. Leal.  Optimal execution with quadratic variation inventories.  Technical report, Princeton University, 2021.
  * [95] R. Carmona and K. Webster.  The self-financing equation in high frequency markets.  Finance & Stochastics, 23:729 – 759, 2019.
  * [96] A. Cartea and S. Jaimungal.  Incorporating order-flow into optimal execution.  Math. Financ. Econ., 10(3):339–364, 2016.
  * [97] P. Casgrain, B. Ning, and S. Jaimungal.  Deep Q-learning for Nash equilibria: Nash-DQN.  arXiv preprint arXiv:1904.10554, 2019.
  * [98] S. Cen, C. Cheng, Y. Chen, Y. Wei, and Y. Chi.  Fast global convergence of natural policy gradient methods with entropy regularization.  Operations Research, 70(4):2563–2578, 2022.
  * [99] Q. Chan-Wai-Nam, J. Mikael, and X. Warin.  Machine learning for semi linear PDEs.  Journal of Scientific Computing, 79(3):1667–1712, 2019.
  * [100] J.-F. Chassagneux, D. Crisan, and F. Delarue.  A probabilistic approach to classical solutions of the master equation for large population equilibria.  Forthcoming in Memoirs of the AMS, arXiv:1411.3009, 2014.
  * [101] J.-F. Chassagneux.  Linear multistep schemes for bsdes.  SIAM Journal on Numerical Analysis, 52(6):2815–2836, 2014.
  * [102] J.-F. Chassagneux, J. Chen, and N. Frikha.  Deep runge-kutta schemes for bsdes.  arXiv preprint arXiv:2212.14372, 2022.
  * [103] J.-F. Chassagneux, D. Crisan, and F. Delarue.  Numerical method for FBSDEs of McKean–Vlasov type.  The Annals of Applied Probability, 29(3):1640–1684, 2019.
  * [104] T. Chen, Z. O. Wang, I. Exarchos, and E. Theodorou.  Large-scale multi-agent deep FBSDEs.  In International Conference on Machine Learning, pages 1740–1748. PMLR, 2021.
  * [105] Z. Chen and P. A. Forsyth.  A semi-Lagrangian approach for natural gas storage valuation and optimal operation.  SIAM Journal on Scientific Computing, 30(1):339–368, 2008.
  * [106] P. Cheridito, H. M. Soner, N. Touzi, and N. Victoir.  Second-order backward stochastic differential equations and fully nonlinear parabolic PDEs.  Communications on Pure and Applied Mathematics: A Journal Issued by the Courant Institute of Mathematical Sciences, 60(7):1081–1110, 2007.
  * [107] J. Chessari, R. Kawai, Y. Shinozaki, and T. Yamada.  Numerical methods for backward stochastic differential equations: A survey.  Probability Surveys, 20:486–567, 2023.
  * [108] D. Chevance.  Numerical methods for backward stochastic differential equations.  Numerical methods in finance, 232, 1997.
  * [109] K. Cho, B. Van Merriënboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y. Bengio.  Learning phrase representations using RNN encoder-decoder for statistical machine translation.  arXiv preprint arXiv:1406.1078, 2014.
  * [110] R. Cont and W. Xiong.  Dynamics of market making algorithms in dealer markets: Learning and tacit collusion.  Mathematical Finance, 2022.
  * [111] K. Cui and H. Koeppl.  Approximately solving mean field games via entropy-regularized deep reinforcement learning.  In proc. of AISTATS, 2021.
  * [112] J. Cvitanić, D. Possamaï, and N. Touzi.  Dynamic programming approach to principal-agent problems.  Finance Stoch., 22(1):1–37, 2018.
  * [113] J. Cvitanić and J. Zhang.  Contract theory in continuous-time models.  Springer Finance. Springer, Heidelberg, 2013.
  * [114] G. Cybenko.  Approximation by superpositions of a sigmoidal function.  Mathematics of control, signals and systems, 2(4):303–314, 1989\.
  * [115] C. Daskalakis, P. W. Goldberg, and C. H. Papadimitriou.  The complexity of computing a Nash equilibrium.  SIAM Journal on Computing, 39:195–259, 2009.
  * [116] A. Davey and H. Zheng.  Deep learning for constrained utility maximisation.  arXiv preprint arXiv:2008.11757, 2020.
  * [117] T. De Ryck, A. D. Jagtap, and S. Mishra.  Error estimates for physics informed neural networks approximating the navier-stokes equations.  arXiv preprint arXiv:2203.09346, 2022.
  * [118] T. De Ryck and S. Mishra.  Error analysis for physics-informed neural networks (PINNs) approximating Kolmogorov PDEs.  Advances in Computational Mathematics, 48(6):1–40, 2022.
  * [119] T. De Ryck and S. Mishra.  Generic bounds on the approximation error for physics-informed (and) operator learning.  arXiv preprint arXiv:2205.11393, 2022.
  * [120] K. Debrabant and E. R. Jakobsen.  Semi-lagrangian schemes for linear and fully non-linear diffusion equations.  Math. Comput., 82(283):1433–1462, 2013.
  * [121] T. Degris, M. White, and R. S. Sutton.  Off-policy actor-critic.  arXiv preprint arXiv:1205.4839, 2012.
  * [122] F. Delarue and A. Vasileiadis.  Exploration noise for learning linear-quadratic mean field games.  arXiv preprint arXiv:2107.00839, 2021.
  * [123] V. Duarte, D. Duarte, and D. Silva.  Machine learning for continuous-time finance.  2024\.
  * [124] W. E and B. Yu.  The deep Ritz method: a deep learning-based numerical algorithm for solving variational problems.  Communications in Mathematics and Statistics, 6(1):1–12, 2018.
  * [125] W. E, J. Han, and A. Jentzen.  Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations.  Commun. Math. Stat., 5(4):349–380, 2017.
  * [126] W. E, M. Hutzenthaler, A. Jentzen, and T. Kruse.  On multilevel Picard numerical approximations for high-dimensional nonlinear parabolic partial differential equations and high-dimensional nonlinear backward stochastic differential equations.  Journal of Scientific Computing, 79(3):1534–1571, 2019.
  * [127] R. Elie, J. Pérolat, M. Laurière, M. Geist, and O. Pietquin.  On the convergence of model free learning in mean field games.  In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 7143–7150, 2020.
  * [128] I. Elsanosi and B. Larssen.  Optimal consumption under partial observations for a stochastic system with delay.  Preprint series. Pure mathematics http://urn. nb. no/URN: NBN: no-8076, 2001.
  * [129] I. Exarchos, E. Theodorou, and P. Tsiotras.  Stochastic differential games: A sampling approach via fbsdes.  Dynamic Games and Applications, 9:486–505, 2019.
  * [130] M. Falcone.  Numerical methods for differential games based on partial differential equations.  International Game Theory Review, 8(02):231–272, 2006.
  * [131] M. Falcone and R. Ferretti.  Numerical methods for hamilton–jacobi type equations.  In Handbook of Numerical Analysis, volume 17, pages 603–626. Elsevier, 2016.
  * [132] A.-m. Farahmand, C. Szepesvári, and R. Munos.  Error propagation for approximate policy and value iteration.  Advances in Neural Information Processing Systems, 23, 2010.
  * [133] M. Fazel, R. Ge, S. Kakade, and M. Mesbahi.  Global convergence of policy gradient methods for the linear quadratic regulator.  In International conference on machine learning, pages 1467–1476. PMLR, 2018.
  * [134] S. Federico.  A stochastic control problem with delay arising in a pension fund model.  Finance and Stochastics, 15(3):421–459, 2011.
  * [135] X. Feng, R. Glowinski, and M. Neilan.  Recent developments in numerical methods for fully nonlinear second order partial differential equations.  siam REVIEW, 55(2):205–267, 2013.
  * [136] D. Firoozi and S. Jaimungal.  Exploratory LQG mean field games with entropy regularization.  Automatica, 139:110177, 2022.
  * [137] P. A. Forsyth and G. Labahn.  Numerical methods for controlled Hamilton-Jacobi-Bellman PDEs in finance.  Journal of Computational Finance, 11(2):1, 2007.
  * [138] J.-P. Fouque and Z. Zhang.  Deep learning methods for mean field control problems with delay.  Frontiers in Applied Mathematics and Statistics, 6(11), 2020.
  * [139] C. Gao, S. Gao, R. Hu, and Z. Zhu.  Convergence of the backward deep BSDE method with applications to optimal stopping problems.  arXiv preprint arXiv:2210.04118, 2022.
  * [140] N. Gast, B. Gaujal, and J.-Y. Le Boudec.  Mean field for Markov decision processes: from discrete to continuous optimization.  IEEE Transactions on Automatic Control, 57(9):2266–2280, 2012.
  * [141] M. Germain, M. Laurière, H. Pham, and X. Warin.  DeepSets and their derivative networks for solving symmetric PDEs.  Journal of Scientific Computing, 91(2):1–33, 2022.
  * [142] M. Germain, J. Mikael, and X. Warin.  Numerical resolution of McKean-Vlasov FBSDEs using neural networks.  Methodology and Computing in Applied Probability, pages 1–30, 2022\.
  * [143] M. Germain, H. Pham, and X. Warin.  Neural networks-based algorithms for stochastic control and PDEs in finance.  arXiv preprint arXiv:2101.08068, 2021.
  * [144] F. A. Gers, N. N. Schraudolph, and J. Schmidhuber.  Learning precise timing with LSTM recurrent networks.  Journal of machine learning research, 3(Aug):115–143, 2002.
  * [145] E. Gobet, J.-P. Lemor, and X. Warin.  A regression-based monte carlo method to solve backward stochastic differential equations.  Annals of Applied Probability, pages 2172–2202, 2005.
  * [146] E. Gobet and R. Munos.  Sensitivity analysis using Itô-Malliavin calculus and martingales, and application to stochastic optimal control.  SIAM J. Control Optim., 43(5):1676–1713, 2005.
  * [147] D. A. Gomes, S. Patrizi, and V. Voskanyan.  On the existence of classical solutions for stationary extended mean field games.  Nonlinear Analysis: Theory, Methods & Applications, 99:49–79, 2014\.
  * [148] D. A. Gomes and V. K. Voskanyan.  Extended deterministic mean-field games.  SIAM Journal on Control and Optimization, 54(2):1030–1055, 2016\.
  * [149] D. Gomes, J. Gutiérrez, and M. Laurière.  Machine learning architectures for price formation models.  arXiv preprint arXiv:2204.03968, 2022.
  * [150] I. Goodfellow, Y. Bengio, and A. Courville.  Deep learning.  MIT press, 2016.
  * [151] F. Gozzi, S. di Roma, and C. Marinelli.  Stochastic optimal control of delay equations arising in advertising models.  Stochastic Partial Differential Equations and Applications-VII, pages 133–148, 2005.
  * [152] F. Gozzi, C. Marinelli, and S. Savin.  On controlled linear diffusions with delay in a model of optimal advertising under uncertainty with memory effects.  Journal of Optimization Theory and Applications, 142(2):291–321, 2009.
  * [153] P. J. Graber.  Linear quadratic mean field type control and mean field games with common noise, with application to production of an exhaustible resource.  Applied Mathematics & Optimization, 74(3):459–486, 2016.
  * [154] A. Graves.  Generating sequences with recurrent neural networks.  arXiv preprint arXiv:1308.0850, 2013.
  * [155] A. Graves, A.-r. Mohamed, and G. Hinton.  Speech recognition with deep recurrent neural networks.  In 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, pages 6645–6649. IEEE, 2013.
  * [156] A. Graves and J. Schmidhuber.  Offline handwriting recognition with multidimensional recurrent neural networks.  In Advances in Neural Information Processing Systems, pages 545–552, 2009.
  * [157] P. Grohs, S. Ibragimov, A. Jentzen, and S. Koppensteiner.  Lower bounds for artificial neural network approximations: A proof that shallow neural networks fail to overcome the curse of dimensionality.  arXiv preprint arXiv:2103.04488, 2021.
  * [158] S. Gronauer and K. Diepold.  Multi-agent deep reinforcement learning: a survey.  Artificial Intelligence Review, pages 1–49, 2021.
  * [159] H. Gu, X. Guo, X. Wei, and R. Xu.  Dynamic programming principles for mean-field controls with learning.  arXiv preprint arXiv:1911.07314, 2019.
  * [160] H. Gu, X. Guo, X. Wei, and R. Xu.  Mean-field controls with Q-learning for cooperative MARL: convergence and complexity analysis.  SIAM Journal on Mathematics of Data Science, 3(4):1168–1196, 2021\.
  * [161] H. Gu, X. Guo, X. Wei, and R. Xu.  Mean-field multi-agent reinforcement learning: A decentralized network approach.  arXiv preprint arXiv:2108.02731, 2021.
  * [162] X. Guo, A. Hu, R. Xu, and J. Zhang.  Learning mean-field games.  In Advances in Neural Information Processing Systems 32 (NeurIPS), 2019.
  * [163] X. Guo, R. Xu, and T. Zariphopoulou.  Entropy regularization for mean field games with learning.  Mathematics of Operations Research, 2022.
  * [164] S. Hadikhanloo.  Ph. d. paris-dauphine thesis defence document.  2018\.
  * [165] S. Hadikhanloo and F. J. Silva.  Finite mean field games: fictitious play and convergence to a first order continuous mean field game.  Journal de Mathématiques Pures et Appliquées, 132:369–397, 2019.
  * [166] B. Hambly, R. Xu, and H. Yang.  Policy gradient methods for the noisy linear quadratic regulator over a finite horizon.  SIAM Journal on Control and Optimization, 59(5):3359–3391, 2021\.
  * [167] B. Hambly, R. Xu, and H. Yang.  Recent advances in reinforcement learning in finance.  arXiv preprint arXiv:2112.04553, 2021.
  * [168] J. Han, R. Hu, and J. Long.  Convergence of deep fictitious play for stochastic differential games.  Frontiers of Mathematical Finance, 1(2):287–319, 2022.
  * [169] J. Han, R. Hu, and J. Long.  A class of dimensionality-free metrics for the convergence of empirical measures.  Stochastic Processes and their Applications, 164:242–287, 2023\.
  * [170] J. Han, R. Hu, and J. Long.  Learning high-dimensional McKean-Vlasov forward-backward stochastic differential equations with general distribution dependence.  SIAM Journal on Numerical Analysis, accepted, 2023.
  * [171] J. Han and W. E.  Deep learning approximation for stochastic control problems.  Deep Reinforcement Learning Workshop, NIPS, arXiv preprint arXiv:1611.07422, 2016.
  * [172] J. Han and R. Hu.  Deep fictitious play for finding Markovian Nash equilibrium in multi-agent games.  In Mathematical and Scientific Machine Learning (MSML), volume 107, pages 221–245. PMLR, 2020.
  * [173] J. Han and R. Hu.  Recurrent neural networks for stochastic control problems with delay.  Mathematics of Control, Signals, and Systems, 33(4):775–795, 2021\.
  * [174] J. Han, A. Jentzen, and W. E.  Solving high-dimensional partial differential equations using deep learning.  Proceedings of the National Academy of Sciences, 115(34):8505–8510, 2018.
  * [175] J. Han and J. Long.  Convergence of the deep BSDE method for coupled FBSDEs.  Probability, Uncertainty and Quantitative Risk, 5(1):1–33, 2020\.
  * [176] J. Han, Y. Yang, and W. E.  DeepHAM: A global solution method for heterogeneous agent models with aggregate shocks.  arXiv preprint arXiv:2112.14377, 2021.
  * [177] B. Hanin.  Universal function approximation by deep neural nets with bounded width and ReLu activations.  Mathematics, 7(10):992, 2019.
  * [178] B. Hanin and D. Rolnick.  Deep ReLu networks have surprisingly few activation patterns.  Advances in neural information processing systems, 32, 2019.
  * [179] B. Hanin and M. Sellke.  Approximating continuous functions by ReLu nets of minimal width.  arXiv preprint arXiv:1710.11278, 2017.
  * [180] P. Henry-Labordere.  Deep primal-dual algorithm for BSDEs: Applications of machine learning to CVA and IM.  Available at SSRN 3071506, 2017.
  * [181] P. Henry-Labordere, C. Litterer, and Z. Ren.  A dual algorithm for stochastic control problems: Applications to uncertain volatility models and CVA.  SIAM Journal on Financial Mathematics, 7(1):159–182, 2016.
  * [182] C. F. Higham and D. J. Higham.  Deep learning: An introduction for applied mathematicians.  Siam review, 61(4):860–891, 2019.
  * [183] S. Hochreiter and J. Schmidhuber.  Long short-term memory.  Neural computation, 9(8):1735–1780, 1997.
  * [184] K. Hornik.  Approximation capabilities of multilayer feedforward networks.  Neural networks, 4(2):251–257, 1991.
  * [185] K. Hornik, M. Stinchcombe, and H. White.  Multilayer feedforward networks are universal approximators.  Neural networks, 2(5):359–366, 1989.
  * [186] J. Hu and M. P. Wellman.  Nash Q-learning for general-sum stochastic games.  Journal of machine learning research, 4(Nov):1039–1069, 2003.
  * [187] R. Hu and M. Ludkovski.  Sequential design for ranking response surfaces.  SIAM/ASA Journal on Uncertainty Quantification, 5(1):212–239, 2017\.
  * [188] R. Hu.  Deep learning for ranking response surfaces with applications to optimal stopping problems.  Quantitative Finance, 20(9):1567–1581, 2020.
  * [189] R. Hu.  Deep fictitious play for stochastic differential games.  Communications in Mathematical Sciences, 19(2):325–353, 2021.
  * [190] R. Hu and T. Zariphopoulou.  $N$-player and mean-field games in Itô-diffusion markets with competitive or homophilous interaction.  In Stochastic Analysis, Filtering, and Stochastic Optimization: A Commemorative Volume to Honor Mark HA Davis’s Contributions, pages 209–237. Springer, 2022.
  * [191] M. Huang, P. E. Caines, and R. P. Malhamé.  Large-population cost-coupled LQG problems with nonuniform agents: individual-mass behavior and decentralized $\epsilon$-Nash equilibria.  IEEE Transactions on Automatic Control, 52(9):1560–1571, 2007.
  * [192] M. Huang, R. P. Malhamé, and P. E. Caines.  Large population stochastic dynamic games: closed-loop McKean-Vlasov systems and the Nash certainty equivalence principle.  Communications in Information and Systems, 6(3):221–252, 2006.
  * [193] K. J. Hunt, D. Sbarbaro, R. Żbikowski, and P. J. Gawthrop.  Neural networks for control systems—a survey.  Automatica, 28(6):1083–1112, 1992.
  * [194] C. Huré, H. Pham, A. Bachouch, and N. Langrené.  Deep neural networks algorithms for stochastic control problems on finite horizon: convergence analysis.  SIAM Journal on Numerical Analysis, 59(1):525–557, 2021.
  * [195] C. Huré, H. Pham, and X. Warin.  Deep backward schemes for high-dimensional nonlinear PDEs.  Mathematics of Computation, 89(324):1547–1579, 2020.
  * [196] M. Hutzenthaler, A. Jentzen, T. Kruse, and T. A. Nguyen.  A proof that rectified deep neural networks overcome the curse of dimensionality in the numerical approximation of semilinear heat equations.  SN partial differential equations and applications, 1:1–34, 2020\.
  * [197] R. Isaacs.  Differential Games: A Mathematical Theory with Applications to Warfare and Pursuit, Control and Optimization.  London: John Wiley and Sons, 1965.
  * [198] R. Isaacs.  Differential games: a mathematical theory with applications to warfare and pursuit, control and optimization.  Courier Corporation, 1999.
  * [199] S. Jaimungal, S. M. Pesenti, Y. S. Wang, and H. Tatsat.  Robust risk-aware reinforcement learning.  SIAM Journal on Financial Mathematics, 13(1):213–226, 2022.
  * [200] M. Jensen and I. Smears.  On the convergence of finite element methods for hamilton–jacobi–bellman equations.  SIAM Journal on Numerical Analysis, 51(1):137–162, 2013.
  * [201] A. Jentzen, D. Salimova, and T. Welti.  A proof that deep artificial neural networks overcome the curse of dimensionality in the numerical approximation of Kolmogorov partial differential equations with constant diffusion and nonlinear drift coefficients.  Communications in Mathematical Sciences, 19(5):1167–1205, 2021\.
  * [202] S. Ji, S. Peng, Y. Peng, and X. Zhang.  Three algorithms for solving high-dimensional fully coupled FBSDEs through deep learning.  IEEE Intelligent Systems, 35(3):71–84, 2020.
  * [203] Y. Jia and X. Y. Zhou.  Policy evaluation and temporal-difference learning in continuous time and space: A martingale approach.  Available at SSRN 3905379, 2021.
  * [204] Y. Jia and X. Y. Zhou.  Policy gradient and actor-critic learning in continuous time and space: Theory and algorithms.  arXiv preprint arXiv:2111.11232, 2021.
  * [205] Z. Jin, M. Qiu, K. Q. Tran, and G. Yin.  A survey of numerical solutions for stochastic control problems: Some recent progress.  Numerical Algebra, Control and Optimization, 12(2):213–253, 2022\.
  * [206] L. P. Kaelbling, M. L. Littman, and A. W. Moore.  Reinforcement learning: A survey.  Journal of artificial intelligence research, 4:237–285, 1996.
  * [207] N. Keriven, A. Bietti, and S. Vaiter.  On the universality of graph neural networks on large random graphs.  Advances in Neural Information Processing Systems, 34:6960–6971, 2021.
  * [208] J. Kierzenka and L. F. Shampine.  A BVP solver based on residual control and the Maltab PSE.  ACM Transactions on Mathematical Software (TOMS), 27(3):299–316, 2001.
  * [209] D. P. Kingma and J. Ba.  Adam: A method for stochastic optimization.  arXiv preprint arXiv:1412.6980, 2014.
  * [210] A. C. Kizilkale and R. P. Malhame.  Collective target tracking mean field control for Markovian jump-driven models of electric water heating loads.  IFAC Proceedings Volumes, 47(3):1867–1872, 2014.  19th IFAC World Congress.
  * [211] Z. Kobeissi.  On classical solutions to the mean field game system of controls.  Communications in Partial Differential Equations, 47(3):453–488, 2022.
  * [212] Z. Kobeissi and F. Bach.  On a variance reduction correction of the temporal difference for policy evaluation in the stochastic continuous setting.  arXiv preprint arXiv:2202.07960, 2022.
  * [213] M. Kohler, A. Krzyżak, and N. Todorovic.  Pricing of high-dimensional American options by neural networks.  Mathematical Finance: An International Journal of Mathematics, Statistics and Financial Economics, 20(3):383–410, 2010.
  * [214] M. Kohlmann and X. Y. Zhou.  Relationship between backward stochastic differential equations and stochastic controls: a linear-quadratic approach.  SIAM J. Control Optim., 38(5):1392–1407, 2000.
  * [215] V. B. Kolmanovskiĭ and L. E. Shaĭkhet.  Control of Systems with Aftereffect, volume 157.  American Mathematical Society, 1996.
  * [216] V. N. Kolokoltsov and A. Bensoussan.  Mean-field-game model for botnet defense in cyber-security.  Appl. Math. Optim., 74(3):669–692, 2016.
  * [217] H. Kushner and P. Dupuis.  Numerical Methods for Stochastic Control Problems in Continuous Time.  Stochastic Modelling and Applied Probability. Springer New York, 2013\.
  * [218] H. J. Kushner.  Numerical methods for stochastic control problems in continuous time.  SIAM Journal on Control and Optimization, 28(5):999–1048, 1990\.
  * [219] H. J. Kushner.  Numerical approximations for stochastic differential games.  SIAM journal on control and optimization, 41(2):457–486, 2002.
  * [220] H. J. Kushner.  Numerical approximations for stochastic differential games: the ergodic case.  SIAM journal on control and optimization, 42(6):1911–1933, 2004\.
  * [221] H. J. Kushner.  Numerical approximations for nonzero-sum stochastic differential games.  SIAM journal on control and optimization, 46(6):1942–1971, 2007\.
  * [222] F. E. Kydland and E. C. Prescott.  Time to build and aggregate fluctuations.  Econometrica: Journal of the Econometric Society, pages 1345–1370, 1982.
  * [223] D. Lacker.  Limit theory for controlled McKean–Vlasov dynamics.  SIAM Journal on Control and Optimization, 55(3):1641–1672, 2017\.
  * [224] D. Lacker and A. Soret.  Many-player games of optimal consumption and investment under relative performance criteria.  Mathematics and Financial Economics, 14(2):263–281, 2020.
  * [225] D. Lacker and T. Zariphopoulou.  Mean field and n-agent games for optimal investment under relative performance criteria.  Mathematical Finance, 29(4):1003–1038, 2019.
  * [226] M. Lanctot, V. Zambaldi, A. Gruslys, A. Lazaridou, K. Tuyls, J. Pérolat, D. Silver, and T. Graepel.  A unified game-theoretic approach to multiagent reinforcement learning.  Advances in neural information processing systems, 30, 2017.
  * [227] B. Lapeyre and J. Lelong.  Neural network regression for Bermudan option pricing.  Monte Carlo Methods and Applications, 27(3):227–247, 2021.
  * [228] J.-M. Lasry and P.-L. Lions.  Jeux à champ moyen. I. Le cas stationnaire.  C. R. Math. Acad. Sci. Paris, 9:619–625, 2006.
  * [229] J.-M. Lasry and P.-L. Lions.  Jeux à champ moyen. II. Horizon fini et contrôle optimal.  C. R. Math. Acad. Sci. Paris, 10:679–684, 2006.
  * [230] J.-M. Lasry and P.-L. Lions.  Mean field games.  Japanese Journal of Mathematics, 2:229–260, 2007.
  * [231] J.-M. Lasry and P.-L. Lions.  Mean field games.  Jpn. J. Math., 2(1):229–260, 2007.
  * [232] M. Laurière.  On numerical methods for mean field games and mean field type control.  Proc. AMS Short Course, 2020.
  * [233] M. Laurière, S. Perrin, M. Geist, and O. Pietquin.  Learning mean field games: A survey.  arXiv preprint arXiv:2205.12944, 2022.
  * [234] M. Laurière, S. Perrin, S. Girgin, P. Muller, A. Jain, T. Cabannes, G. Piliouras, J. Pérolat, R. Élie, O. Pietquin, et al.  Scalable deep reinforcement learning algorithms for mean field games.  arXiv preprint arXiv:2203.11973, 2022.
  * [235] M. Laurière and O. Pironneau.  Dynamic programming for mean-field type control.  C. R. Math. Acad. Sci. Paris, 352(9):707–713, 2014.
  * [236] M. Lauriere and L. Tangpi.  Convergence of large population games to mean field games with interaction through the controls.  SIAM Journal on Mathematical Analysis, 54(3):3535–3574, 2022.
  * [237] L. Leal, M. Laurière, and C.-A. Lehalle.  Learning a functional control for high-frequency finance.  Preprint, arXiv:2006.09611, 2020.
  * [238] W. Lefebvre and E. Miller.  Linear-quadratic stochastic delayed control and deep learning resolution.  Journal of Optimization Theory and Applications, 191(1):134–168, 2021.
  * [239] C.-A. Lehalle and R. Azencott.  Piecewise affine neural networks and nonlinear control.  In International Conference on Artificial Neural Networks, pages 633–638. Springer, 1998.
  * [240] M. Leshno, V. Y. Lin, A. Pinkus, and S. Schocken.  Multilayer feedforward networks with a nonpolynomial activation function can approximate any function.  Neural networks, 6(6):861–867, 1993.
  * [241] K. Li and J. Liu.  Portfolio selection under time delays: A piecewise dynamic programming approach.  Available at SSRN 2916481, 2018.
  * [242] Y. Li.  Deep reinforcement learning: An overview.  arXiv preprint arXiv:1701.07274, 2017.
  * [243] T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra.  Continuous control with deep reinforcement learning.  In Proceedings of the International Conference on Learning Representations (ICLR 2016), 2016.
  * [244] A. T. Lin, S. W. Fung, W. Li, L. Nurbekyan, and S. J. Osher.  APAC-Net: Alternating the population and agent control via two neural networks to solve high-dimensional stochastic mean field games.  arXiv preprint arXiv:2002.10113, 2020.
  * [245] P.-L. Lions.  Cours du Collège de France.  https://www.college-de-france.fr/site/en-pierre-louis-lions/_course.htm, 2007-2011.
  * [246] Z. Lu, H. Pu, F. Wang, Z. Hu, and L. Wang.  The expressive power of neural networks: A view from the width.  Advances in neural information processing systems, 30, 2017.
  * [247] J. Luo and H. Zheng.  Deep neural network solution for finite state mean field game with error estimation.  Dynamic Games and Applications, 13(3):859–896, 2023.
  * [248] T. Lyons and Z. Qian.  System control and rough paths.  Oxford University Press, 2002.
  * [249] T. J. Lyons, M. Caruana, and T. Lévy.  Differential equations driven by rough paths.  Springer, 2007.
  * [250] J. Ma, P. Protter, J. San Martín, and S. Torres.  Numerical method for backward stochastic differential equations.  Annals of Applied Probability, pages 302–316, 2002.
  * [251] J. L. Mathieu, S. Koch, and D. S. Callaway.  State estimation and control of electric loads to manage real-time energy imbalance.  IEEE Transactions on Power Systems, 28:430–440, 2013.
  * [252] M. Min and R. Hu.  Signatured deep fictitious play for mean field games with common noise.  In International Conference on Machine Learning (ICML), pages 7736–7747. PMLR 139, 2021.
  * [253] M. Min, R. Hu, and T. Ichiba.  Directed chain generative adversarial networks.  In International Conference on Machine Learning (ICML), pages 24812–24830. PMLR 202, 2023.
  * [254] S. Mishra and R. Molinaro.  Estimates on the generalization error of physics-informed neural networks for approximating PDEs.  IMA Journal of Numerical Analysis, 2022.
  * [255] S. Mishra and R. Molinaro.  Estimates on the generalization error of physics-informed neural networks for approximating a class of inverse problems for PDEs.  IMA Journal of Numerical Analysis, 42(2):981–1022, 2022.
  * [256] V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, S. Petersen, C. Beattie, A. Sadik, I. Antonoglou, H. King, D. Kumaran, D. Wierstra, S. Legg, and D. Hassabis.  Human-level control through deep reinforcement learning.  Nature, 518(7540):529–533, 2015.
  * [257] S.-E. A. Mohammed.  Stochastic Functional Differential Equations, volume 99.  Pitman Advanced Publishing Program, 1984.
  * [258] S.-E. A. Mohammed.  Stochastic differential systems with memory: theory, examples and applications.  In Stochastic Analysis and Related Topics VI, pages 1–77. Springer, 1998.
  * [259] M. Motte and H. Pham.  Mean-field Markov decision processes with common noise and open-loop controls.  To appear in Annals of Applied Probability (arXiv preprint arXiv:1912.07883), 2019.
  * [260] R. Munos.  Policy gradient in continuous time.  Journal of Machine Learning Research, 7:771–791, 2006.
  * [261] J. Nash.  Non-cooperative games.  Annals of mathematics, pages 286–295, 1951.
  * [262] M. Nutz and Y. Zhang.  Conditional optimal stopping: a time-inconsistent optimization.  Ann. Appl. Probab., 30(4):1669–1692, 2020.
  * [263] A. M. Oberman.  Convergent difference schemes for degenerate elliptic and parabolic equations: Hamilton–jacobi equations and free boundary problems.  SIAM Journal on Numerical Analysis, 44(2):879–895, 2006.
  * [264] B. Øksendal and A. Sulem.  A maximum principle for optimal control of stochastic systems with delay, with applications to finance.  Preprint series. Pure mathematics http://urn. nb. no/URN: NBN: no-8076, 2000.
  * [265] E. Pardoux and S. Peng.  Adapted solution of a backward stochastic differential equation.  Systems & Control Letters, 14(1):55–61, 1990.
  * [266] S. Park, C. Yun, J. Lee, and J. Shin.  Minimum width for universal approximation.  In International Conference on Learning Representations, 2020.
  * [267] S. Peng.  Stochastic Hamilton–Jacobi–Bellman equations.  SIAM Journal on Control and Optimization, 30(2):284–304, 1992.
  * [268] S. Perrin, M. Laurière, J. Pérolat, R. Élie, M. Geist, and O. Pietquin.  Generalization in mean field games by learning master policies.  AAAI’22 (arXiv preprint arXiv:2109.09717), 2021.
  * [269] S. Perrin, M. Laurière, J. Pérolat, M. Geist, R. Élie, and O. Pietquin.  Mean field games flock! The reinforcement learning way.  In proc. of IJCAI, 2021.
  * [270] H. Pham.  Continuous-time stochastic control and optimization with financial applications, volume 61.  Springer Science & Business Media, 2009.
  * [271] H. Pham.  On some recent aspects of stochastic control and their applications.  Probability Surveys, 2:506–549, 2005.
  * [272] H. Pham, X. Warin, and M. Germain.  Neural networks-based backward scheme for fully nonlinear PDEs.  SN Partial Differential Equations and Applications, 2(1):16, 2021\.
  * [273] H. Pham and X. Wei.  Dynamic programming for optimal control of stochastic McKean-Vlasov dynamics.  SIAM J. Control Optim., 55(2):1069–1101, 2017.
  * [274] A. Pinkus.  Approximation theory of the MLP model in neural networks.  Acta Numerica, 8:143–195, 1999.
  * [275] W. B. Powell.  Approximate Dynamic Programming: Solving the curses of dimensionality, volume 703.  John Wiley & Sons, 2007.
  * [276] D. Psaltis, A. Sideris, and A. A. Yamamura.  A multilayered neural network controller.  IEEE control systems magazine, 8(2):17–21, 1988.
  * [277] M. Raissi, P. Perdikaris, and G. E. Karniadakis.  Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations.  Journal of Computational Physics, 378:686–707, 2019.
  * [278] C. Reisinger, W. Stockinger, and Y. Zhang.  A fast iterative PDE-based algorithm for feedback controls of nonsmooth mean-field control problems.  arXiv preprint arXiv:2108.06740, 2021.
  * [279] C. Reisinger, W. Stockinger, and Y. Zhang.  A posteriori error estimates for fully coupled McKean-Vlasov forward-backward SDEs.  IMA Journal of Numerical Analysis, page drad060, 09 2023.
  * [280] A. M. Reppen, H. M. Soner, and V. Tissot-Daguette.  Deep stochastic optimization in finance.  arXiv preprint arXiv:2205.04604, 2022.
  * [281] A. M. Reppen, H. M. Soner, and V. Tissot-Daguette.  Neural optimal stopping boundary.  arXiv preprint arXiv:2205.04595, 2022.
  * [282] D. E. Rumelhart, G. E. Hinton, and R. J. Williams.  Learning representations by back-propagating errors.  Nature, 323(6088):533–536, 1986.
  * [283] L. Ruthotto, S. J. Osher, W. Li, L. Nurbekyan, and S. W. Fung.  A machine learning framework for solving high-dimensional mean field game and mean field control problems.  Proceedings of the National Academy of Sciences, 117(17):9183–9193, 2020.
  * [284] Y. F. Saporito and Z. Zhang.  Path-dependent deep Galerkin method: A neural network approach to solve path-dependent partial differential equations.  SIAM Journal on Financial Mathematics, 12(3):912–940, 2021.
  * [285] A. M. Schäfer and H. G. Zimmermann.  Recurrent neural networks are universal approximators.  In Artificial Neural Networks–ICANN 2006: 16th International Conference, Athens, Greece, September 10-14, 2006. Proceedings, Part I 16, pages 632–640. Springer, 2006.
  * [286] B. Scherrer, M. Ghavamzadeh, V. Gabillon, B. Lesner, and M. Geist.  Approximate modified policy iteration and its application to the game of tetris.  J. Mach. Learn. Res., 16(49):1629–1676, 2015.
  * [287] S. Shalev-Shwartz, S. Shammah, and A. Shashua.  Safe, multi-agent, reinforcement learning for autonomous driving.  arXiv preprint arXiv:1610.03295, 2016.
  * [288] D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. Van Den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot, et al.  Mastering the game of Go with deep neural networks and tree search.  Nature, 529(7587), 2016.
  * [289] D. Silver, G. Lever, N. Heess, T. Degris, D. Wierstra, and M. Riedmiller.  Deterministic policy gradient algorithms.  In Proceedings of the 31st International Conference on Machine Learning, volume 32 of Proceedings of Machine Learning Research, pages 387–395. PMLR, 2014.
  * [290] J. Sirignano and K. Spiliopoulos.  DGM: A deep learning algorithm for solving partial differential equations.  Journal of computational physics, 375:1339–1364, 2018.
  * [291] I. H. Sloan and H. Woźniakowski.  When are quasi-monte carlo algorithms efficient for high dimensional integrals?  Journal of Complexity, 14(1):1–33, 1998.
  * [292] J. Subramanian and A. Mahajan.  Reinforcement learning in stationary mean-field games.  In in proc. of AAMAS, 2019.
  * [293] R. S. Sutton and A. G. Barto.  Reinforcement learning: An introduction.  MIT press, 2018.
  * [294] A.-S. Sznitman.  Topics in propagation of chaos.  In Ecole d’été de probabilités de Saint-Flour XIX—1989, pages 165–251. Springer, 1991.
  * [295] K. Tuyls and G. Weiss.  Multiagent learning: Basics, challenges, and prospects.  Ai Magazine, 33(3):41–41, 2012.
  * [296] R. van der Meer, C. W. Oosterlee, and A. Borovykh.  Optimally weighted loss functions for solving PDEs with neural networks.  Journal of Computational and Applied Mathematics, 405:113887, 2022\.
  * [297] O. Vinyals, I. Babuschkin, W. M. Czarnecki, M. Mathieu, A. Dudzik, J. Chung, D. H. Choi, R. Powell, T. Ewalds, P. Georgiev, et al.  Grandmaster level in StarCraft II using multi-agent reinforcement learning.  Nature, 575(7782), 2019.
  * [298] H. Wang, T. Zariphopoulou, and X. Y. Zhou.  Reinforcement learning in continuous time and space: A stochastic control approach.  Journal of Machine Learning Research, 21(198):1–34, 2020.
  * [299] H. Wang and X. Y. Zhou.  Continuous-time mean–variance portfolio selection: A reinforcement learning framework.  Mathematical Finance, 30(4):1273–1308, 2020.
  * [300] L. Wang, Q. Cai, Z. Yang, and Z. Wang.  Neural policy gradient methods: Global optimality and rates of convergence.  In International Conference on Learning Representations, 2019.
  * [301] R. J. Williams.  Simple statistical gradient-following algorithms for connectionist reinforcement learning.  Reinforcement learning, pages 5–32, 1992.
  * [302] Q. Xie, Z. Yang, Z. Wang, and A. Minca.  Learning while playing in mean-field games: Convergence and optimality.  In International Conference on Machine Learning, pages 11436–11447. PMLR, 2021.
  * [303] Y. Xuan, R. Balkin, J. Han, R. Hu, and H. D. Ceniceros.  Optimal policies for a pandemic: A stochastic game approach and a deep learning algorithm.  In Mathematical and Scientific Machine Learning, pages 145:987–1012. PMLR, 2022.
  * [304] Y. Xuan, R. Balkin, J. Han, R. Hu, and H. D. Ceniceros.  Pandemic control, game theory and machine learning.  Notices of the AMS, 69(11):1878–1887, December 2022.
  * [305] Y. Yang and J. Wang.  An overview of multi-agent reinforcement learning from game theoretical perspective.  arXiv preprint arXiv:2011.00583, 2020.
  * [306] J. Yong.  Differential games: a concise introduction.  World scientific, 2014.
  * [307] Y. Zang, J. Long, X. Zhang, W. Hu, E. Weinan, and J. Han.  A machine learning enhanced algorithm for the optimal landing problem.  In Mathematical and Scientific Machine Learning, pages 319–334. PMLR, 2022.
  * [308] J. Zhang.  A numerical scheme for BSDEs.  The annals of applied probability, 14(1):459–488, 2004.
  * [309] K. Zhang, Z. Yang, and T. Başar.  Multi-agent reinforcement learning: A selective overview of theories and algorithms.  Handbook of Reinforcement Learning and Control, pages 321–384, 2021\.
  * [310] W. Zhao, L. Chen, and S. Peng.  A new kind of accurate numerical method for backward stochastic differential equations.  SIAM Journal on Scientific Computing, 28(4):1563–1581, 2006.
  * [311] W. Zhao, G. Zhang, and L. Ju.  A stable multistep scheme for solving backward stochastic differential equations.  SIAM Journal on Numerical Analysis, 48(4):1369–1394, 2010.
  * [312] D.-X. Zhou.  Universality of deep convolutional neural networks.  Applied and computational harmonic analysis, 48(2):787–794, 2020\.