A parsimonious neural network approach to solve portfolio optimization
  problems without using dynamic programming
We present a parsimonious neural network approach, which does not rely on
dynamic programming techniques, to solve dynamic portfolio optimization
problems subject to multiple investment constraints. The number of parameters
of the (potentially deep) neural network remains independent of the number of
portfolio rebalancing events, and in contrast to, for example, reinforcement
learning, the approach avoids the computation of high-dimensional conditional
expectations. As a result, the approach remains practical even when considering
large numbers of underlying assets, long investment time horizons or very
frequent rebalancing events. We prove convergence of the numerical solution to
the theoretical optimal solution of a large class of problems under fairly
general conditions, and present ground truth analyses for a number of popular
formulations, including mean-variance and mean-conditional value-at-risk
problems. We also show that it is feasible to solve Sortino ratio-inspired
objectives (penalizing only the variance of wealth outcomes below the mean) in
dynamic trading settings with the proposed approach. Using numerical
experiments, we demonstrate that if the investment objective functional is
separable in the sense of dynamic programming, the correct time-consistent
optimal investment strategy is recovered, otherwise we obtain the correct
pre-commitment (time-inconsistent) investment strategy. The proposed approach
remains agnostic as to the underlying data generating assumptions, and results
are illustrated using (i) parametric models for underlying asset returns, (ii)
stationary block bootstrap resampling of empirical returns, and (iii)
generative adversarial network (GAN)-generated synthetic asset returns.
Pieter M. van Staden, Peter A. Forsyth, Yuying Li
A parsimonious neural network approach to solve portfolio optimization problems without using dynamic programming

###### Abstract

We present a parsimonious neural network approach, which does not rely on dynamic programming techniques, to solve dynamic portfolio optimization problems subject to multiple investment constraints. The number of parameters of the (potentially deep) neural network remains independent of the number of portfolio rebalancing events, and in contrast to, for example, reinforcement learning, the approach avoids the computation of high-dimensional conditional expectations. As a result, the approach remains practical even when considering large numbers of underlying assets, long investment time horizons or very frequent rebalancing events. We prove convergence of the numerical solution to the theoretical optimal solution of a large class of problems under fairly general conditions, and present ground truth analyses for a number of popular formulations, including mean-variance and mean-conditional value-at-risk problems. We also show that it is feasible to solve Sortino ratio-inspired objectives (penalizing only the variance of wealth outcomes below the mean) in dynamic trading settings with the proposed approach. Using numerical experiments, we demonstrate that if the investment objective functional is separable in the sense of dynamic programming, the correct time-consistent optimal investment strategy is recovered, otherwise we obtain the correct pre-commitment (time-inconsistent) investment strategy. The proposed approach remains agnostic as to the underlying data generating assumptions, and results are illustrated using (i) parametric models for underlying asset returns, (ii) stationary block bootstrap resampling of empirical returns, and (iii) generative adversarial network (GAN)-generated synthetic asset returns.

**Keywords**: Asset allocation, portfolio optimization, neural network, dynamic programming

**JEL classification**: G11, C61

## 1 Introduction

We present, and analyze the convergence of, a parsimonious and flexible neural network approach to obtain the numerical solution of a large class of dynamic (i.e. multi-period) portfolio optimization problems that can be expressed in the following form,

\[\inf_{\xi\in\mathbb{R}}\inf_{\mathcal{P}\in\mathcal{A}}\left\{\,E_{\mathcal{P} }^{t_{0},w_{0}}\left[F\left(W\left(T\right),\xi\right)+G\left(W\left(T\right), E_{\mathcal{P}}^{t_{0},w_{0}}\left[W\left(T\right)\right],w_{0},\xi\right)\, \right]\,\right\}. \tag{1.1}\]

While rigorous definitions and assumptions are discussed in subsequent sections, here we simply note that in general, \(F:\mathbb{R}^{2}\rightarrow\mathbb{R}\) and \(G:\mathbb{R}^{4}\rightarrow\mathbb{R}\) denote some continuous functions and \(\xi\in\mathbb{R}\) some auxiliary variable, with \(T>0\) denoting the investment time horizon, \(W\left(t\right),t\in[t_{0},T]\), the controlled wealth process, and \(\mathcal{P}\) representing the investment strategy (or control) implemented over \([t_{0},T]\). Typically, \(\mathcal{P}\) specifies the amount or fraction of wealth to invest in each of (a potentially large number of) the underlying assets at each portfolio rebalancing event, which in practice occurs at some discrete subset of rebalancing times in \([t_{0},T]\). \(\mathcal{A}\) denotes the set of admissible investment strategies encoding the (possibly multiple) investment constraints faced by the investor. Finally, \(E_{\mathcal{P}}^{t_{0},w_{0}}\left[\cdot\right]\) denotes the expectation given control \(\mathcal{P}\) and initial wealth \(W\left(t_{0}\right)=w_{0}\).
Although (1.1) is written for objective functions involving the terminal portfolio wealth \(W\left(T\right)\), the approach and convergence analysis could be generalized without difficulty to objective functions that are wealth path-dependent, i.e. functions of \(\left\{W\left(t\right):t\in\mathcal{T}\right\}\) for some subset \(\mathcal{T}\subseteq\left[t_{0}\text{,}T\right]\) - see Forsyth et al. (2022); Van Staden et al. (2022a) for examples. However, since a sufficiently rich class of problems are of the form (1.1), this will remain the main focus of this paper.

The proposed approach does not rely on the separability of the objective functional in (1.1) in the sense of dynamic programming, remains agnostic as to the underlying data generation assumptions, and is sufficiently flexible such that practical considerations such as multiple investment constraints and discrete portfolio rebalancing can be incorporated without difficulty.

Leaving the more formal treatment for subsequent sections, for introductory purposes we highlight some specific examples of problems of the form (1.1):

* Utility maximization (see for example Vigna (2014)), in which case there is no outer optimization problem and \(G\equiv 0\), while \(w\to U\left(w\right)\) denotes the investor's utility function, so that (1.1) therefore reduces to \[\sup_{\mathcal{P}\in\mathcal{A}}\left\{E_{\mathcal{P}}^{t_{0},w_{0}}\left[U \left(W\left(T\right)\right)\right]\right\}.\] (1.2)
* Mean-variance (MV) optimization (see e.g. Li and Ng (2000); Zhou and Li (2000)), with \(\rho>0\) denoting the scalarization (or risk aversion) parameter, where the problem \[\sup_{\mathcal{P}\in\mathcal{A}}\left\{E_{\mathcal{P}}^{t_{0},w_{0}}\left[W \left(T\right)\right]-\rho\cdot Var_{\mathcal{P}}^{t_{0},w_{0}}\left[W\left(T \right)\right]\right\},\] (1.3) can also be written in the general form (1.1).
* Mean-Conditional Value-at-Risk (CVaR) optimization, in which case we do have both an inner and an outer optimization problems (see e.g. Forsyth (2020); Miller and Yang (2017), resulting in a problem of the form \[\inf_{\xi\in\mathbb{R}}\inf_{\mathcal{P}\in\mathcal{A}}\left\{E_{\mathcal{P}}^ {t_{0},w_{0}}\left[F\left(W\left(T\right),\xi\right)\right]\right\},\] (1.4) for a particular choice of the function \(F\) (see (2.15) below).
* To illustrate the flexibility and generality of the proposed approach, we also consider a "mean semi-variance" portfolio optimization problem that is inspired by the popular Sortino ratio (Bodie et al. (2014)) in the case of one-period portfolio analysis, where only the variance of downside outcomes relative to the mean is penalized. In the case of dynamic trading strategies, this suggests an objective function of the form \[\sup_{\mathcal{P}\in\mathcal{A}}\left\{E_{\mathcal{P}}^{t_{0},w_{0}}\left[W \left(T\right)-\rho\cdot\left(\min\left\{W\left(T\right)-E_{\mathcal{P}}^{t_ {0},w_{0}}\left[W\left(T\right)\right],0\right\}\right)^{2}\right]\right\},\] (1.5) where, as in the case of (1.3), the parameter \(\rho>0\) encodes the trade-off between risk and return. Note that (1.5) is not separable in the sense of dynamic programming, and in the absence of embedding results (analogous to those of Li and Ng (2000); Zhou and Li (2000) in the case of MV optimization (1.3)), problem (1.5) cannot be solved using traditional dynamic programming-based methods.

However, we emphasize that (1.2)-(1.5) are only a selection of examples, and the proposed approach and theoretical analysis remains applicable to problems that can be expressed in the general form (1.1).

Portfolio optimization problems of the form (1.1) can give rise to investment strategies that are not time-consistent due to the presence of the (possibly non-linear) function \(G\) (Bjork et al. (2021)). Since the objective in (1.1) is therefore potentially not separable in the sense of dynamic programming (see for example (1.3) or (1.5)). This gives rise to two related problems: (i) Since (1.1) cannot be solved using a dynamic programming-based approach, some other solution methodology has to be implemented, or some re-interpretation of the problem or the concept of "optimality" might be required (see for example Bjork and Murgoci (2014); Vigna (2022)), (ii) if the investment strategies are time-inconsistent, this can raise questions as to whether these strategies are feasible to implement as practical investment strategies.

We make the following general observations:

* It may be desirable to avoid using dynamic programming (DP) even if (1.1) _can_ be solved using DP techniques, such as in the special case where \(G\equiv 0\) in (1.1) and the investment strategies are time-consistent. For example, it is well known that DP has an associated "curse of dimensionality", in that as the
number state variables increases linearly, the computational burden increases exponentially (Fernandez-Villaverde et al. (2020); Han and Weinan (2016)). In addition, since DP techniques necessarily incur estimation errors at each time step, significant error amplification can occur which is further exacerbated in high-dimensional settings (see for example Li et al. (2020); Tsang and Wong (2020); Wang and Foster (2020)).

However, instead of relying on DP-based techniques and attempting to address the challenges of dimensionality using machine learning techniques (see for example Bachouch et al. (2022); Dixon et al. (2020); Fernandez-Villaverde et al. (2020); Gao et al. (2020); Henry-Labordere (2017); Hure et al. (2021); Lucarelli and Borrotti (2020); Park et al. (2020)), the proposed method fundamentally avoids DP techniques altogether. This is especially relevant in our setting, since we have shown that in the case of portfolio optimization problems specifically, DP can be _unnecessarily_ high-dimensional even in simple settings (see Van Staden et al. (2022b)). This occurs since the objective functional (or performance criteria (Oksendal and Sulem (2019)) ) is typically high-dimensional while the optimal investment strategy (the fundamental quantity of concern) remains relatively low-dimensional. The proposed method therefore forms part of the significant recent interest in developing machine learning techniques to solve multi-period portfolio optimization problems that avoids using DP techniques altogether (see for example Li and Forsyth (2019); Ni et al. (2022); Tsang and Wong (2020); Van Staden et al. (2022b)).
* Time-inconsistent problems naturally arise in financial applications (see Bjork et al. (2021) for numerous examples), and as a result their solution is often an area of active research due to the unique challenges involved in solving these problems without resorting to DP techniques. Examples include the mean-variance problem, which remained an open problem for decades until the solution using the embedding technique of Li and Ng (2000); Zhou and Li (2000). As a result, being able to obtain a numerical solution to problems of the form (1.1) directly is potentially very valuable for research. The solution of time-inconsistent problems is also practical interest, since in many cases, there exists an induced time consistent objective function (Forsyth (2020); Strub et al. (2019a,b)). The optimal policy for this induced time consistent objective function is identical to the pre-commitment policy at time zero. The induced time consistent strategy is, of course implementable (Forsyth (2020)), in the sense that the investor has no incentive to deviate from the strategy determined at time zero, at later times. An alternative approach to handling time-inconsistent problems is to search for the equilibrium control (Bjork et al. (2021)). A fascinating result obtained in Bjork and Murgoci (2010) is that for every equilibrium control, there exists a standard, time consistent problem which has the same control, under a different objective function. This essentially means that the question of time-consistency is a often matter of perspective, since there may be alternative objective functions which give rise to the same pre-commitment control, yet are time-consistent. In fact, other subtle issues arise in comparing pre-commitment and time consistent controls, see Vigna (2020, 2022) for further discussion. Furthermore, over very short time horizons such as those encountered in optimal trade execution, time consistency or its absence may not be of much concern to the investor or market participant (see for example Forsyth et al. (2011); Tse et al. (2013)). In addition, as noted by Bernard and Vanduffel (2014), if the strategy is realized in an investment product sold to a retail investor, then the optimal policy from the investor's point of view is in fact of pre-commitment type, since the retail client does not herself trade in the underlying assets during the lifetime of the contract.

As a result of these observations, we will consider problem (1.1) in its general form. Our method builds on and formalizes the initial results described in Li and Forsyth (2019) where a shallow NN was applied to a portfolio optimization problem with an objective that is separable in the sense of DP. The contributions of this paper are as follows:

* We present a flexible neural network (NN) approach to solve problems of the form (1.1) that does not rely on DP techniques. Our approach only requires the solution of a _single_ optimization problem, and therefore avoids the error amplification problems associated with the time-recursion in DP-based techniques, including for example Reinforcement Learning algorithms such as Q-learning (see for example Dixon et al. (2020); Gao et al. (2020); Park et al. (2020)) or other algorithms relying at some level on the DP principle
for a time-stepping backward recursion (see for example Bachouch et al. (2022); Van Heeswijk and Poutre (2019)). Perhaps the best descriptor of our approach is Policy Function Approximation, in the taxonomy in Powell (2023).

We make very limited assumptions regarding the underlying asset dynamics. In particular, if underlying asset (and by extension wealth) dynamics are specified, this can be incorporated as easily as the case where the underlying dynamics can only be observed without any parametric assumptions.

The proposed solution methodology is _parsimonious_, in that the number of parameters does not scale with the number of rebalancing events. This contrasts the proposed methodology with for example that of Han and Weinan (2016); Hure et al. (2021); Tsang and Wong (2020), and ensures that our approach remains feasible even for problems with very long time horizons (for example the accumulation phase of a pension fund - see Forsyth et al. (2019)) or with shorter time horizon but with frequent trading/rebalancing (for example the trade execution problems encountered in Forsyth et al. (2011)). The solution approach only places very weak requirements on the form of the investment objective in (1.1). In addition, we find that using relatively shallow neural networks (at most two hidden layers) in our approach achieve very accurate results in ground truth testing, thereby ensuring that the resulting NN in the proposed approach is relatively easy and efficient to train since it is less likely to be susceptible to problems of vanishing or exploding gradients associated with very deep neural networks (Goodfellow et al. (2016)).
2. We analyze the convergence of the proposed approach, and show that the theoretical optimal investment strategy of (1.1), provided it exists, can be attained by the numerical solution.
3. Finally, we present ground truth analyses confirming that the proposed approach is very effective in solving portfolio optimization problems of the form (1.1). The results illustrate numerically that if (1.1) is not separable in the sense of DP, our approach recovers the correct pre-commitment (time-inconsistent) optimal control, otherwise it recovers the correct time-consistent optimal control. To emphasize that the approach remains agnostic to the underlying data generation assumptions, results are illustrated using (i) parametric models for asset dynamics, (ii) stationary block bootstrap resampling of empirical asset returns, and (ii) generative adversarial network (GAN)-generated synthetic asset returns.

The remainder of the paper is organized as follows: Section 2 presents the problem formulation, while Section 3 provides a summary of the proposed approach, with additional technical and practical details provided in Appendix A and Appendix B. Section 4 presents the convergence analysis of the proposed approach. Finally, Section 5 provides ground truth analyses, with Section 6 concluding the paper and discussing possible avenues for future research.

## 2 Problem formulation

We start by formulating portfolio optimization problems of the form (1.1) more rigorously in a setting of discrete portfolio rebalancing and multiple investment constraints. Throughout, we work on filtered probability space \(\left(\Omega,\mathcal{F},\left\{\mathcal{F}\left(t\right)\right\}_{t\in[t_{0 },T]},\mathbb{P}\right)\) satisfying the usual conditions, with \(\mathbb{P}\) denoting the actual (and not the risk-neutral) probability measure.

Let \(\mathcal{T}\) denote the set of \(N_{rb}\) discrete portfolio rebalancing times in \(\left[t_{0}=0,T\right]\), which we assume to be equally-spaced to lighten notation,

\[\mathcal{T} = \left\{\left.t_{m}=m\Delta t\right|m=0,...,N_{rb}-1\right\}, \qquad\Delta t=T/N_{rb}, \tag{2.1}\]

where we observe that the last rebalancing event occurs at time \(t_{N_{rb}-1}=T-\Delta t\).

At each rebalancing time \(t_{m}\in\mathcal{T}\), the investor observes the \(\mathcal{F}\left(t_{m}\right)\)-measurable vector \(\boldsymbol{X}\left(t_{m}\right)=\left(X_{i}\left(t_{m}\right):i=1,...,\eta_ {X}\right)\in\mathbb{R}^{n_{X}}\), which can be interpreted informally as the information taken into account by the investor in reaching their asset allocation decision. As a concrete example, we assume below that \(\boldsymbol{X}\left(t_{m}\right)\) includes at least the wealth available for investment, an assumption which can be rigorously justified using analytical results (see for example Van Staden et al. (2022)).

Given \(\boldsymbol{X}\left(t_{m}\right)\), the investor then rebalances a portfolio of \(N_{a}\) assets to new positions given by the vector

\[\boldsymbol{p}_{m}\left(t_{m},\boldsymbol{X}\left(t_{m}\right)\right) = \left(p_{m,i}\left(t_{m},\boldsymbol{X}\left(t_{m}\right)\right) :i=1,..,N_{a}\right)\in\mathbb{R}^{N_{a}}, \tag{2.2}\]
where \(p_{m,i}\left(t_{m},\mathbf{X}\left(t_{m}\right)\right)\) denotes the fraction of wealth \(W\left(t_{m}\right)\) invested in the \(i\)th asset at rebalancing time \(t_{m}\). The subscript "\(m\)" in the notation \(\mathbf{p}_{m}\) emphasizes that in general, each rebalancing time \(t_{m}\in\mathcal{T}\) could be associated with potentially a different function \(\mathbf{p}_{m}:\mathbb{R}^{n_{X}+1}\rightarrow\mathbb{R}^{N_{n}}\), while the subscript is removed below when we consider a single function that is simply evaluated at different times, in which case we will write \(\mathbf{p}:\mathbb{R}^{n_{X}+1}\rightarrow\mathbb{R}^{N_{n}}\).

For purposes of concreteness, we assume that the investor is subject to the constraints of (i) no short-selling and (ii) no leverage being allowed, although the proposed methodology can be adjusted without difficulty to treat different constraint formulations1. For illustrative purposes, we therefore assume that each allocation (2.2) is only allowed to take values in \(\left(N_{a}-1\right)\)-dimensional probability simplex \(\mathcal{Z}\),

Footnote 1: As discussed in Section 3 and Appendix A, adjustments to the output layer of the neural network may be required.

\[\mathcal{Z} = \left\{\left(y_{1},...,y_{N_{a}}\right)\in\mathbb{R}^{N_{a}}: \sum_{i=1}^{N_{a}}y_{i}=1\text{ and }y_{i}\geq 0\text{ for all }i=1,...,N_{a}\right\}. \tag{2.3}\]

In this setting, an investment strategy or control \(\mathcal{P}\) applicable to \(\left[t_{0},T\right]\) is therefore of the form,

\[\mathcal{P} = \left\{\mathbf{p}_{m}\left(t_{m},\mathbf{X}\left(t_{m}\right)\right)= \left(p_{m,i}\left(t_{m},\mathbf{X}\left(t_{m}\right)\right):i=1,..,N_{a}\right):t _{m}\in\mathcal{T}\right\}, \tag{2.4}\]

while the set of admissible controls \(\mathcal{A}\) is defined by

\[\mathcal{A} = \left\{\left.\mathcal{P}=\left\{\mathbf{p}_{m}\left(t_{m},\mathbf{X} \left(t_{m}\right)\right):t_{m}\in\mathcal{T}\right\}\right|\mathbf{p}_{m}\left(t _{m},\mathbf{X}\left(t_{m}\right)\right)\in\mathcal{Z},\forall t_{m}\in\mathcal{T }\right\}. \tag{2.5}\]

The randomness in the system is introduced through the returns of the underlying assets. Specifically, let \(R_{i}\left(t_{m}\right)\) denote the \(\mathcal{F}\left(t_{m+1}\right)\)-measurable return observed on asset \(i\) over the interval \(\left[t_{m},t_{m+1}\right]\). We make no assumptions regarding the underlying asset dynamics, but at a minimum, we do require \(\left(\mathbb{P}\right)\) integrability, i.e. \(\mathbb{E}\left|R_{i}\left(t_{m}\right)\right|<\infty\) for all \(i\in\left\{1,...,N_{a}\right\}\) and \(m\in\left\{0,...,N_{rb}-1\right\}\). Informally, we will refer to the set

\[\mathbf{Y} = \left\{\left(Y_{i}\left(t_{m}\right):=1+R_{i}\left(t_{m}\right):i= 1,...,N_{a}\right)^{\top}:m\in\left\{0,...,N_{rb}-1\right\}\right\} \tag{2.6}\]

as the _path_ of (joint) asset returns over the investment time horizon \(\left[t_{0},T\right]\).

To clarify the subsequent notation, for any functional \(\psi\left(t\right),t\in\left[t_{0},T\right]\) we will use the notation \(\psi\left(t^{-}\right)\) and \(\psi\left(t^{+}\right)\) as shorthand for the one-sided limits \(\psi\left(t^{-}\right)=\lim_{\epsilon\downarrow 0}\psi\left(t-\epsilon\right)\) and \(\psi\left(t^{+}\right)=\lim_{\epsilon\downarrow 0}\psi\left(t+\epsilon\right)\), respectively.

Given control \(\mathcal{P}\in\mathcal{A}\), asset returns \(\mathbf{Y}\), initial wealth \(W\left(t_{0}^{-}\right)\coloneqq w_{0}>0\) and a (non-random) cash contribution schedule \(\left\{q\left(t_{m}\right):t_{m}\in\mathcal{T}\right\}\), the portfolio wealth dynamics for \(m=0,...,N_{rb}-1\) are given by the general recursion

\[W\left(t_{m+1}^{-};\mathcal{P},\mathbf{Y}\right) = \left[W\left(t_{m}^{-};\mathcal{P},\mathbf{Y}\right)+q\left(t_{m} \right)\right]\cdot\sum_{i=1}^{N_{a}}p_{m,i}\left(t_{m},\mathbf{X}\left(t_{m} \right)\right)\cdot Y_{i}\left(t_{m}\right). \tag{2.7}\]

Note that we write \(W\left(u\right)=W\left(u;\mathcal{P},\mathbf{Y}\right)\) to emphasize the dependence of wealth on the control \(\mathcal{P}\) and the (random) path of asset returns in \(\mathbf{Y}\) that relates to the time period \(t\in\left[t_{0},u\right]\). In other words, despite using \(\mathbf{Y}\) in the notation for simplicity, \(W\left(u;\mathcal{P},\mathbf{Y}\right)\) is \(\mathcal{F}\left(u\right)\)-measurable. Finally, since there are no contributions or rebalancing at maturity, we simply have \(W\left(t_{N_{rb}}^{-}\right)=W\left(T^{-}\right)=W\left(T\right)=W\left(T; \mathcal{P},\mathbf{Y}\right)\).

### Investment objectives

Given this general investment setting and wealth dynamics (2.7), our goal is to solve dynamic portfolio optimization problems of the general form

\[\inf_{\xi\in\mathbb{R}}\inf_{\mathcal{P}\in\mathcal{A}}J\left(\mathcal{P},\xi;t _{0},w_{0}\right), \tag{2.8}\]

where, for some given continuous functions \(F:\mathbb{R}^{2}\rightarrow\mathbb{R}\) and \(G:\mathbb{R}^{3}\rightarrow\mathbb{R}\), the objective functional \(J\) is given by

\[J\left(\mathcal{P},\xi;t_{0},w_{0}\right) = E_{\mathcal{P}}^{t_{0},w_{0}}\left[F\left(W\left(T;\mathcal{P}, \mathbf{Y}\right),\xi\right)+G\left(W\left(T;\mathcal{P},\mathbf{Y}\right),E_{ \mathcal{P}}^{t_{0},w_{0}}\left[W\left(T;\mathcal{P},\mathbf{Y}\right)\right],w_{0 },\xi\right)\right]. \tag{2.9}\]

Note that the expectations \(E^{t_{0},w_{0}}\left[\cdot\right]\) in (2.9) are taken over \(\mathbf{Y}\), given initial wealth \(W\left(t_{0}^{-}\right)=w_{0}\), control \(\mathcal{P}\in\mathcal{A}\) and auxiliary variable \(\xi\in\mathbb{R}\). In addition to the assumption of continuity of \(F\) and \(G\), we will make only the minimal assumptions regarding the exact properties of \(J\), including that \(\xi\to F\left(\cdot,\xi\right)\) and \(\xi\to G\left(\cdot,\cdot,w_{0},\xi\right)\) are
convex for all admissible controls \(\mathcal{P}\in\mathcal{A}\), and the standard assumption (see for example Bjork et al. (2021)) that an optimal control \(\mathcal{P}^{*}\in\mathcal{A}\) exists.

For illustrative and ground truth analysis purposes, we consider a number of examples of problems of the form (2.8)-(2.9).

As noted in the Introduction, the simplest examples of problems of the form (2.8) arise in the special case where \(G\equiv 0\) and there is no outer optimization problem over \(\xi\), such as in the case of standard utility maximization problems. As concrete examples of this class of objective functions, we will consider the quadratic target minimization (or quadratic utility) described in for example Vigna (2014); Zhou and Li (2000),

\[\left(DSQ\left(\gamma\right)\right):\qquad\inf_{\mathcal{P}\in\mathcal{A}} \left\{E^{t_{0},w_{0}}\left[\left(W\left(T;\mathcal{P},\boldsymbol{Y}\right)- \gamma\right)^{2}\right]\right\},\qquad\gamma>0, \tag{2.10}\]

as well as the (closely-related) one-sided quadratic loss minimization used in for example Dang and Forsyth (2016); Li and Forsyth (2019),

\[\left(OSQ\left(\gamma\right)\right):\qquad\inf_{\mathcal{P}\in\mathcal{A}} \left\{E^{t_{0},w_{0}}\left[\left(\min\left\{W\left(T;\mathcal{P}, \boldsymbol{Y}\right)-\gamma,0\right\}\right)^{2}-\epsilon\cdot W\left(T; \mathcal{P},\boldsymbol{Y}\right)\right]\right\},\qquad\gamma>0. \tag{2.11}\]

The term \(\epsilon W(\cdot)\) in equation (2.11) ensures that the problem remains well-posed2 in the event that \(W(t)\gg\gamma\). Observe that problems of the form (2.10) or (2.11) are separable in the sense of dynamic programming, so that the resulting optimal control is therefore time-consistent.

Footnote 2: Although this is a mathematical necessity (see e.g. (Li and Forsyth, 2019)), in practice, if we use a very small value of \(\epsilon\), then this has no perceptible effect on the summary statistics. In the numerical results of Section 5, we use \(\epsilon=10^{-6}\); see Appendix B for a discussion.

As a classical example of the case where \(G\) is nonlinear and the objective functional (2.9) is not separable in the sense of dynamic programming, we consider the mean-variance (MV) objective with scalarization or risk-aversion parameter \(\rho>0\) (see for example Bjork et al. (2017)),

\[\left(MV\left(\rho\right)\right): \sup_{\mathcal{P}\in\mathcal{A}}\left\{E^{t_{0},w_{0}}\left[W \left(T;\mathcal{P},\boldsymbol{Y}\right)\right]-\rho\cdot Var^{t_{0},w_{0}} \left[W\left(T;\mathcal{P},\boldsymbol{Y}\right)\right]\right\},\qquad\rho>0. \tag{2.12}\] \[= \sup_{\mathcal{P}\in\mathcal{A}}E_{\mathcal{P}}^{t_{0},w_{0}} \left[W\left(T;\mathcal{P},\boldsymbol{Y}\right)-\rho\cdot\left(W\left(T; \mathcal{P},\boldsymbol{Y}\right)-E_{\mathcal{P}}^{t_{0},w_{0}}\left[W\left(T ;\mathcal{P},\boldsymbol{Y}\right)\right]\right)^{2}\right].\]

Note that issues relating to the time-inconsistency of the optimal control of (2.12) are discussed in Remark 2.1 below, along with the relationship between (2.10) and (2.12).

As an example of a problem involving both the inner and outer optimization in (2.8), we consider the Mean - Conditional Value-at-Risk (or Mean-CVaR) problem, subsequently simply abbreviated the MCV problem. First, as a measure of tail risk, the CVaR at level \(\alpha\), or \(\alpha\)-CVaR, is the expected value of the worst \(\alpha\) percent of wealth outcomes, with typical values being \(\alpha\in\{1\%,5\%\}\). As in Forsyth (2020), a _larger_ value of the CVaR is preferable to smaller value, since our definition of \(\alpha\)-CVaR is formulated in terms of the terminal _wealth_, not in terms of the _loss_. Informally, if the distribution of terminal wealth \(W\left(T\right)\) is continuous with PDF \(\hat{\psi}\), then the \(\alpha\)-CVaR in this case is given by

\[\text{CVAR}_{\alpha} = \frac{1}{\alpha}\int_{-\infty}^{w_{\alpha}^{*}}W\left(T\right) \cdot\hat{\psi}\left(W\left(T\right)\right)\cdot dW\left(T\right), \tag{2.13}\]

where \(w_{\alpha}^{*}\) is the corresponding Value-at-Risk (VaR) at level \(\alpha\) defined such that \(\int_{-\infty}^{w_{\alpha}^{*}}\hat{\psi}\left(W\left(T\right)\right)dW\left(T \right)=\alpha\). We follow for example Forsyth (2020) in defining the MCV problem with scalarization parameter \(\rho>0\) formally as

\[\sup_{\mathcal{P}\in\mathcal{A}}\left\{\rho\cdot E^{t_{0},w_{0}}\left[W\left(T \right)\right]+\text{CVAR}_{\alpha}\right\},\qquad\rho>0. \tag{2.14}\]

However, instead of (2.13), we use the definition of CVaR from Rockafellar and Uryasev (2002) that is applicable to more general terminal wealth distributions, so that the MCV problem definition used subsequently aligns with the definition given in Forsyth (2020); Miller and Yang (2017)),

\[\left(MCV\left(\rho\right)\right):\qquad\inf_{\xi\in\mathbb{R}}\inf_{\mathcal{P }\in\mathcal{A}}E^{t_{0},w_{0}}\left[-\rho\cdot W\left(T;\mathcal{P}, \boldsymbol{Y}\right)-\xi+\frac{1}{\alpha}\max\left(\xi-W\left(T;\mathcal{P}, \boldsymbol{Y}\right),0\right)\right],\quad\rho>0. \tag{2.15}\]

Finally, as noted in the Introduction, we apply the ideas underlying the Sortino ratio where the variance of
returns below the mean are penalized, to formulate the following objective function for dynamic trading,

\[\left(MSemiV\left(\rho\right)\right):\sup_{\mathcal{P}\in\mathcal{A}}\left\{E_{ \mathcal{P}}^{t_{0},w_{0}}\left[W\left(T;\mathcal{P},\mathbf{Y}\right)-\rho\cdot \left(\min\left\{W\left(T;\mathcal{P},\mathbf{Y}\right)-E_{\mathcal{P}}^{t_{0},w_{0 }}\left[W\left(T;\mathcal{P},\mathbf{Y}\right)\right],0\right\}\right)^{2}\right] \right\}, \tag{2.16}\]

which we refer to as the "Mean- Semi-variance" problem, with scalarization (or risk-aversion) parameter \(\rho>0\).3

Footnote 3: In continuous time, the unconstrained Mean-Semi-variance problem is ill-posed (Jin et al. (2005)). However, we will impose bounded leverage constraints, which is, of course, a realistic condition. This makes problem \(\left(MSemiV\left(\rho\right)\right)\) well posed.

The following remark discusses issues relating to the possible time-inconsistency of the optimal controls of (2.12), (2.15) and (2.16).

**Remark 2.1**.: (Time-_in_consistency and induced time-consistency) Formally, the optimal controls for problems \(MV\left(\rho\right)\), \(MCV\left(\rho\right)\) and \(MSemiV\left(\rho\right)\) are not time-consistent, but instead are of the pre-commitment type (see Basak and Chabakauri (2010); Bjork and Murgoci (2014); Forsyth (2020)). However, in many cases, there exists an induced time consistent problem formulation which has the same controls at time zero as the pre-commitment problem (see Forsyth (2020); Strub et al. (2019, 2019)).

As a concrete example of induced time-consistency, the embedding result of Li and Ng (2000); Zhou and Li (2000) establishes that the \(DSQ\left(\gamma\right)\) objective is the induced time-consistent objective function associated with the \(MV\left(\rho\right)\) problem, which is a result that we exploit for ground truth analysis purposes in Section 5.

Similarly, there is an induced time consistent objective function for the Mean-CVAR problem \(MCV\left(\rho\right)\) in (2.15) - see Forsyth (2020).

Consequently, when we refer to a strategy as optimal, for either the Mean-CVAR \(\left(MCV\left(\rho\right)\right)\) or Mean-Variance \(\left(MV\left(\rho\right)\right)\) problems, this will be understood to mean that at any \(t>t_{0}\), the investor follows the associated induced time-consistent strategy rather than a pre-commitment strategy.

In the Mean-Semi-variance \(\left(MSemiV\left(\rho\right)\right)\) case as per (2.16), there is no obvious induced time consistent objective function. In this case, we seek the pre-commitment policy.

For a detailed discussion of the many subtle issues involved in the case of time-inconsistency, induced time-consistency, and equilibrium controls, see for example Bjork et al. (2021); Bjork and Murgoci (2014); Forsyth (2020); Strub et al. (2019, 2019); Vigna (2020, 2022). 

## 3 Neural network approach

In this section, we provide an overview of the neural network (NN) approach. Additional technical details and practical considerations are discussed in Appendices A and B, while the theoretical justification via convergence analysis will be discussed in Section 4 (and Appendix B).

Recall from (2.2) that \(\mathbf{X}\left(t_{m}\right)\in\mathbb{R}^{\eta_{X}}\) denotes the information taken into account in determining the investment strategy (2.2) at rebalancing time \(t_{m}\). Using the initial experimental results of Li and Forsyth (2019) and the analytical results of Van Staden et al. (2022b) applied to this setting, we assume that \(\mathbf{X}\left(t_{m}\right)\) includes at least the wealth available for investment at time \(t_{m}\), so that

\[W\left(t_{m}^{+};\mathcal{P},\mathbf{Y}\right)\coloneqq W\left(t_{m}^{-};\mathcal{ P},\mathbf{Y}\right)+q\left(t_{m}\right)\ \ \in\ \ \mathbf{X}\left(t_{m}\right),\qquad\forall t_{m}\in\mathcal{T}. \tag{3.1}\]

However, we emphasize that \(\mathbf{X}\left(t_{m}\right)\) may include additional variables in different settings. For example, in non-Markovian settings or in the case of certain solution approaches involving auxiliary variables, it is natural to "lift the state space" by including additional quantities in \(\mathbf{X}\) such as relevant historical quantities related to market variables, or other auxiliary variables - see for example Forsyth (2020); Miller and Yang (2017); Tsang and Wong (2020).

Let \(\mathcal{D}_{\mathbf{\phi}}\subseteq\mathbb{R}^{\eta_{X}+1}\) be the set such that \(\left(t_{m},\mathbf{X}\left(t_{m}\right)\right)\in\mathcal{D}_{\mathbf{\phi}}\) for all \(t_{m}\in\mathcal{T}\). Let \(C\left(\mathcal{D}_{\mathbf{\phi}},\mathcal{Z}\right)\) denote the set of all continuous functions from \(\mathcal{D}_{\mathbf{\phi}}\) to \(\mathcal{Z}\subset\mathbb{R}^{N_{\mathbf{\ast}}}\) (see (2.3)). We will use the notation \(\mathbf{X}^{\ast}\) to denote the information taken into account by the optimal control, since in the simplest case implied by (3.1), we simply have \(\mathbf{X}^{\ast}=W^{\ast}\), where \(W^{\ast}\) denotes the wealth under the optimal strategy. We make the following assumption.

**Assumption 3.1**.: _(Properties of the optimal control) Considering the general form of the problem (2.8), we assume that there exists an optimal feedback control \(\mathcal{P}^{\ast}\in\mathcal{A}\). Specifically, we assume that at each rebalancing time \(t_{m}\in\mathcal{T}\), the time \(t_{m}\) itself together with the information vector under optimal behavior \(\mathbf{X}^{\ast}\left(t_{m}\right)\), which includes at least the wealth \(W^{\ast}\left(t_{m}^{+}\right)\) available for investment (see (3.1)), are sufficient to fully determine the optimal asset allocation \(\mathbf{p}_{m}^{\ast}\left(t_{m},\mathbf{X}^{\ast}\left(t_{m}\right)\right)\)._
_Furthermore, we assume that there exists a continuous function \(\mathbf{p}^{*}\in C\left(\mathcal{D}_{\mathbf{\phi}},\mathcal{Z}\right)\) such that \(\mathbf{p}^{*}_{m}\left(t_{m},\mathbf{X}^{*}\left(t_{m}\right)\right)=\mathbf{p}^{*}\left(t _{m},\mathbf{X}^{*}\left(t_{m}\right)\right)\) for all \(t_{m}\in\mathcal{T}\), so that the optimal control \(\mathcal{P}^{*}\) can be expressed as_

\[\mathcal{P}^{*} = \left\{\mathbf{p}^{*}\left(t_{m},\mathbf{X}^{*}\left(t_{m}\right)\right) :\forall t_{m}\in\mathcal{T}\right\},\quad\text{where}\quad\mathbf{p}^{*}\in C \left(\mathcal{D}_{\mathbf{\phi}},\mathcal{Z}\right). \tag{3.2}\]

We make the following observations regarding Assumption 3.1:

* Continuity of \(\mathbf{p}^{*}\) in space _and time_: While assuming the optimal control is a continuous map in the state space \(\mathbf{X}\) is fairly standard in the literature, especially in the context of using neural network approximations (see for example Han and Weinan (2016); Hure et al. (2021); Tsang and Wong (2020)), the assumption of continuity in time in (3.2) is therefore worth emphasizing. This assumption enforces the requirement that in the limit of continuous rebalancing (i.e. when \(\Delta t\to 0\)), the control remains a continuous function of time, which is a practical requirement for any reasonable investment policy. In particular, this ensures that the asset allocation retains its smooth behavior as the number of rebalancing events in \(\left[0,T\right]\) is increased, which we consider a fundamental requirement ensuring that the resulting investment strategy is reasonable. In addition, in Section 5 we demonstrate how the known theoretical solution to a problem assuming continuous rebalancing (\(\Delta t\to 0\)) can be approximated very well using \(\Delta t\gg 0\) in the NN approach, even though the resulting NN approximation is only truly optimal in the case of \(\Delta t\gg 0\).
* The control is a _single_ function for _all_ rebalancing times; note that the function \(\mathbf{p}^{*}\) is not subscripted by time. If the portfolio is rebalanced only at discrete time intervals, the investment strategy can be found (as suggested in (3.2)) by evaluating this continuous function at discrete time intervals, i.e. \(\left(t_{m},\mathbf{X}\left(t_{m}\right)\right)\rightarrow\mathbf{p}^{*}\left(t_{m}, \mathbf{X}\left(t_{m}\right)\right)=\left(p_{1}^{*}\left(t_{m},\mathbf{X}\left(t_{m} \right)\right):i=1,...,N_{a}\right)\), for all \(t_{m}\in\mathcal{T}\). We discuss below how we solve for this (single) function directly, without resorting to dynamic programming, which avoids not only the challenge with error propagation due to value iteration over multiple timesteps, but also avoids solving for the high-dimensional conditional expectation (also termed the performance criteria by Oksendal and Sulem (2019)) if we are only interested in the relatively low-dimensional optimal control (see for example Van Staden et al. (2022b)).

These observations ultimately suggest the NN approach discussed below, while the soundness of Assumption 3.1 is experimentally confirmed in the ground truth results presented in Section 5.

Given Assumption 3.1 and in particular (3.2), we therefore limit our consideration to controls of the form

\[\mathcal{P} = \left\{\mathbf{p}\left(t_{m},\mathbf{X}\left(t_{m}\right)\right):\forall t _{m}\in\mathcal{T}\right\},\quad\text{for some}\quad\mathbf{p}\in C\left( \mathcal{D}_{\mathbf{\phi}},\mathcal{Z}\right). \tag{3.3}\]

To simplify notation, we identify an arbitrary control \(\mathcal{P}\) of the form (3.3) with its associated function \(\mathbf{p}=\left(p_{i}:i=1,...,N_{a}\right)\in C\left(\mathcal{D}_{\mathbf{\phi}}, \mathcal{Z}\right)\), so that the objective functional (2.9) is written as

\[J\left(\mathbf{p},\xi;t_{0},w_{0}\right) = E^{t_{0},w_{0}}\left[F\left(W\left(T;\mathbf{p},\mathbf{Y}\right),\xi \right)+G\left(W\left(T\right),E^{t_{0},w_{0}}\left[W\left(T;\mathbf{p},\mathbf{Y} \right)\right],w_{0},\xi\right)\right]. \tag{3.4}\]

In (3.4), \(W\left(\cdot;\mathbf{p},\mathbf{Y}\right)\) denotes the controlled wealth process using a control of the form (3.3), so that the wealth dynamics (2.7) for \(t_{m}\in\mathcal{T}\) (recall \(t_{N_{\mu}}^{-}=T\)) now becomes

\[W\left(t_{m+1}^{-};\mathbf{p},\mathbf{Y}\right) = \left[W\left(t_{m}^{-};\mathbf{p},\mathbf{Y}\right)+q\left(t_{m}\right) \right]\cdot\sum_{i=1}^{N_{a}}p_{i}\left(t_{m},\mathbf{X}\left(t_{m}\right)\right) \cdot Y_{i}\left(t_{m}\right). \tag{3.5}\]

Therefore, using Assumption 3.1 and (3.4)-(3.5), problem (2.8) is therefore expressed as

\[V\left(t_{0},w_{0}\right) = \inf_{\xi\in\mathbb{R}}\inf_{\mathbf{p}\in C\left(\mathcal{D}_{\mathbf{ \phi}},\mathcal{Z}\right)}J\left(\mathbf{p},\xi;t_{0},w_{0}\right). \tag{3.6}\]

We now provide a brief overview of the proposed methodology to solve problems of the form (3.6). This consists of two steps discussed in the following subsections, namely (i) the NN approximation to the control, and (ii) computational estimate of the optimal control.
### Step 1: NN approximation to control

Let \(n\in\mathbb{N}\). Consider a fully-connected, feedforward NN \(\boldsymbol{f}_{n}\) with parameter vector \(\boldsymbol{\theta}_{n}\in\mathbb{R}^{\nu_{n}}\) and a fixed number \(\mathcal{L}^{h}\geq 1\) of hidden layers, where each hidden layer contains \(\hbar\left(n\right)\in\mathbb{N}\) nodes. The NN has \(\left(\eta_{X}+1\right)\) input nodes, mapping feature (input) vectors of the form \(\boldsymbol{\phi}\left(t\right)=\left(t,\boldsymbol{X}\left(t\right)\right) \in\mathcal{D}_{\boldsymbol{\phi}}\) to \(N_{a}\) output nodes. For a more detailed introduction to neural networks, see for example Goodfellow et al. (2016).

Additional technical and practical details can be found in Appendices A and B. For this discussion, we simply note that the index \(n\in\mathbb{N}\) is used for the purposes of the analytical results and convergence analysis, where we fix a choice of \(\mathcal{L}^{h}\geq 1\) while \(\hbar\left(n\right),n\in\mathbb{N}\) is assumed to be a monotonically increasing sequence such that \(\lim_{n\rightarrow\infty}\hbar\left(n\right)=\infty\) (see Section 4 and Appendix A). However, for practical implementation, a fixed value of \(\hbar\left(n\right)\in\mathbb{N}\) is chosen (along with \(\mathcal{L}^{h}\geq 1\)) to ensure the NN has sufficient depth and complexity to solve the problem under consideration (see Appendix B).

Any NN considered is constructed such that \(\boldsymbol{f}_{n}:\mathcal{D}_{\boldsymbol{\phi}}\rightarrow\mathcal{Z}\subset \mathbb{R}^{N_{a}}\). In other words, the values of the \(N_{a}\) outputs are automatically in the set \(\mathcal{Z}\) defined in (2.3) for any \(\boldsymbol{\phi}\in\mathcal{D}_{\boldsymbol{\phi}}\),

\[\boldsymbol{f}_{n}\left(\boldsymbol{\phi}\left(t\right); \boldsymbol{\theta}_{n}\right) = \left(f_{n,i}\left(\boldsymbol{\phi}\left(t\right);\boldsymbol{ \theta}_{n}\right):i=1,...,N_{a}\right)\in\mathcal{Z}. \tag{3.7}\]

As a result, the outputs of the NN \(\boldsymbol{f}_{n}\) in (3.7) can be interpreted as portfolio weights satisfying the required investment constraints. While a more detailed discussion of the structure can be found in Assumption A.1 in Appendix A, we summarize some key aspects of the NN structure illustrated in Figure 3.1:

1. We emphasize that the rebalancing time is an _input_ into the NN as per the feature vector \(\boldsymbol{\phi}\left(t\right)=\left(t,\boldsymbol{X}\left(t\right)\right) \in\mathcal{D}_{\boldsymbol{\phi}}\), so that the NN parameter vector \(\boldsymbol{\theta}_{n}\) itself does not depend on time.
2. While we assume sigmoid activations for the hidden nodes for concreteness and convenience (see Assumption A.1), any of the commonly-used activation functions can be implemented with only minor modifications to the technical results presented in Section 4.
3. Since we are illustrating the approach using the particular form of \(\mathcal{Z}\) in (2.3) because of its wide applicability (no short-selling and no leverage), a softmax output layer is used to ensure the NN output remains in \(\mathcal{Z}\subset\mathbb{R}^{N_{a}}\) for any \(\boldsymbol{\phi}\left(t\right)\) (see (3.7)). However, different admissible control set formulations can be handled without difficulty4. Footnote 4: For example, position limits and limited leverage can be introduced using minor modifications to the output layer. Perhaps the only substantial challenge is offered by unrealistic investment scenarios, such as insisting that trading should continue in the event of bankruptcy, in which case consideration should be given to the possibility of wealth being identically zero or negative.

For some fixed value of the index \(n\in\mathbb{N}\), let \(\mathcal{N}_{n}\) denote the set of NNs constructed in the same way as \(\boldsymbol{f}_{n}\) for the fixed and given values of \(\mathcal{L}^{h}\) and \(\hbar\left(n\right)\). While a formal definition of the set \(\mathcal{N}_{n}\) is provided in Appendix A, here we simply note that each NN \(\boldsymbol{f}_{n}\left(\cdot;\boldsymbol{\theta}_{n}\right)\in\mathcal{N}_{n}\) only differs in terms of the parameter values constituting its parameter vector \(\boldsymbol{\theta}_{n}\) (i.e. for a fixed \(n\), each \(\boldsymbol{f}_{n}\in\mathcal{N}_{n}\) has the same number of hidden layers \(\mathcal{L}^{h}\), hidden nodes \(\hbar\left(n\right)\), activation functions etc.).

Figure 3.1: Illustration of the structure of the NN as per (3.7). Additional construction and implementation details can be found in Appendix A.
Observing that \(\mathcal{N}_{n}\subset C\left(\mathcal{D}_{\mathbf{\phi}},\mathcal{Z}\right)\), our first step is to approximate (3.6) by performing the optimization over \(\mathbf{f}_{n}\left(\cdot;\mathbf{\theta}_{n}\right)\in\mathcal{N}_{n}\) instead. In other words, we approximate the control \(\mathbf{p}\) by a neural network \(\mathbf{f}_{n}\in\mathcal{N}_{n}\),

\[\mathbf{p}\left(\mathbf{\phi}\left(t\right)\right) \simeq \mathbf{f}_{n}\left(\mathbf{\phi}\left(t\right);\mathbf{\theta}_{n}\right), \qquad\text{where}\quad\mathbf{\phi}\left(t\right)=\left(t,\mathbf{X}\left(t\right) \right),\mathbf{p}\in C\left(\mathcal{D}_{\mathbf{\phi}},\mathcal{Z}\right),\mathbf{f}_{n }\in\mathcal{N}_{n}. \tag{3.8}\]

We identify the NN \(\mathbf{f}_{n}\left(\cdot;\mathbf{\theta}_{n}\right)\) with its parameter vector \(\mathbf{\theta}_{n}\), so that the (approximate) objective functional using approximation (3.8) is written as

\[J_{n}\left(\mathbf{\theta}_{n},\xi;t_{0},w_{0}\right) = E^{t_{0},w_{0}}\left[F\left(W\left(T;\mathbf{\theta}_{n},\mathbf{Y} \right),\xi\right)+G\left(W\left(T;\mathbf{\theta}_{n},\mathbf{Y}\right),E^{t_{0},w_{ 0}}\left[W\left(T;\mathbf{\theta}_{n},\mathbf{Y}\right)\right],w_{0},\xi\right)\, \right]. \tag{3.9}\]

Combining (3.7) and (3.8), the wealth dynamics (3.5) is expressed as

\[W\left(t_{m+1}^{-};\mathbf{\theta}_{n},\mathbf{Y}\right) = \left[W\left(t_{m}^{-};\mathbf{\theta}_{n},\mathbf{Y}\right)+q\left(t_{m }\right)\right]\cdot\sum_{i=1}^{N_{a}}f_{n,i}\left(\mathbf{\phi}\left(t_{m} \right);\mathbf{\theta}_{n}\right)\cdot Y_{i}\left(t_{m}\right),\quad m=0,...,N_{ rb}-1. \tag{3.10}\]

Using (3.8) and (3.9), for fixed and given values of \(\mathcal{L}^{h}\) and \(\hbar\left(n\right)\), we therefore approximate problem (3.6) by

\[V_{n}\left(t_{0},w_{0}\right) = \inf_{\xi\in\mathbf{\xi}}\inf_{\mathbf{f}_{n}\left(\cdot;\mathbf{\theta}_{n} \right)\in\mathcal{N}_{n}}J_{n}\left(\mathbf{\theta}_{n},\xi;t_{0},w_{0}\right) \tag{3.11}\] \[= \inf_{\xi\in\mathbf{\xi}}\inf_{\mathbf{\theta}_{n}\in\mathbb{R}^{n-}}J_{ n}\left(\mathbf{\theta}_{n},\xi;t_{0},w_{0}\right)\] (3.12) \[= \inf_{\left(\mathbf{\theta}_{n},\xi\right)\in\mathbb{R}^{n+1}}J_{n} \left(\mathbf{\theta}_{n},\xi;t_{0},w_{0}\right).\]

We highlight that the optimization in (3.12) is unconstrained since, by construction, each NN \(\mathbf{f}_{n}\left(\cdot;\mathbf{\theta}_{n}\right)\in\mathcal{N}_{n}\) always generates outputs in \(\mathcal{Z}\).

The notation \(\left(\mathbf{\theta}_{n}^{*},\xi^{*}\right)\) and the associated NN \(\mathbf{f}_{n}^{*}\left(\cdot;\mathbf{\theta}_{n}^{*}\right)\in\mathcal{N}_{n}\) are subsequently used to denote the values achieving the optimum in (3.12) for given values of \(\mathcal{L}^{h}\) and \(\hbar\left(n\right)\). Note however that we do _not_ assume that the optimal control \(\mathbf{p}^{*}\in C\left(\mathcal{D}_{\mathbf{\phi}},\mathcal{Z}\right)\) satisfying Assumption 3.1 is also a NN in \(\mathcal{N}_{n}\), since by the universal approximation results (see for example Hornik et al. (1989)), we would expect that the error in approximating (3.6) by (3.12) can be made arbitrarily small for sufficiently large \(\hbar\left(n\right)\). These claims are rigorously confirmed in Section 4 below, where we consider a sequence of NNs \(\mathbf{f}_{n}\left(\cdot;\mathbf{\theta}_{n}\right)\in\mathcal{N}_{n}\) obtained by letting \(\hbar\left(n\right)\rightarrow\infty\) as \(n\rightarrow\infty\) (for any fixed value of \(\mathcal{L}^{h}\geq 1\)).

### Step 2 : Computational estimate of the optimal control

In order to solve the approximation (3.12) to problem (3.6), we require estimates of the expectations in (3.9). For computational purposes, suppose we take as given a set \(\mathcal{Y}_{n}\in\mathbb{R}^{n\times N_{a}\times N_{rb}}\), consisting of \(n\in\mathbb{N}\) independent realizations of the paths of joint asset returns \(\mathbf{Y}\),

\[\mathcal{Y}_{n} = \left\{\mathbf{Y}^{\left(j\right)}:j=1,...,n\right\}. \tag{3.13}\]

We highlight that each entry \(\mathbf{Y}^{\left(j\right)}\in\mathcal{Y}_{n}\) consists of a _path_ of joint asset returns (see (2.6)), and we assume that the paths are independent, we do _not_ assume that the asset returns constituting each path are independent. In particular, both cross-correlations and autocorrelation structures within each path of returns are permitted.

Constructing the set \(\mathcal{Y}_{n}\) in practical applications is further discussed in Appendix B. In the numerical examples in Section 5, we use examples where \(\mathcal{Y}_{n}\) is generated using (i) Monte Carlo simulation of parametric asset dynamics, (ii) stationary block bootstrap resampling of empirical asset returns, (Anarkulova et al. (2022)) and (iii) generative adversarial network (GAN)-generated synthetic asset returns (Yoon et al. (2019)). While we let \(n\rightarrow\infty\) in (3.13) for convergence analysis purposes, in practical applications (e.g. the results of Section 5) we simply choose \(n\) sufficiently large such that we are reasonably confident that reliable numerical estimates of the expectations in (3.9) are obtained.

Given a NN \(\mathbf{f}_{n}\left(\cdot;\mathbf{\theta}_{n}\right)\in\mathcal{N}_{n}\) and set \(\mathcal{Y}_{n}\), the wealth dynamics (3.10) along path \(\mathbf{Y}^{\left(j\right)}\in\mathcal{Y}_{n}\) is given by

\[W^{\left(j\right)}\left(t_{m+1}^{-};\mathbf{\theta}_{n},\mathcal{Y}_{n}\right) = \left[W^{\left(j\right)}\left(t_{m}^{-};\mathbf{\theta}_{n},\mathcal{Y} _{n}\right)+q\left(t_{m}\right)\right]\cdot\sum_{i=1}^{N_{a}}f_{n,i}\left(\mathbf{ \phi}^{\left(j\right)}\left(t_{m}\right);\mathbf{\theta}_{n}\right)\cdot Y_{i}^{ \left(j\right)}\left(t_{m}\right), \tag{3.14}\]
for \(m=0,...,N_{rb}-1\). We introduce the superscript \(\left(j\right)\) to emphasize that the quantities are obtained along the \(j\)th entry of (3.13).

The computational estimate of \(J_{n}\left(\mathbf{\theta}_{n},\xi;t_{0},w_{0}\right)\) in (3.9) is then given by

\[\hat{J}_{n}\left(\mathbf{\theta}_{n},\xi;t_{0},w_{0},\mathcal{Y}_{n}\right) = \frac{1}{n}\sum_{j=1}^{n}F\left(W^{\left(j\right)}\left(T;\mathbf{ \theta}_{n},\mathcal{Y}_{n}\right),\xi\right) \tag{3.15}\] \[+\frac{1}{n}\sum_{j=1}^{n}G\left(W^{\left(j\right)}\left(T;\mathbf{ \theta}_{n},\mathcal{Y}_{n}\right),\frac{1}{n}\sum_{k=1}^{n}W^{\left(k\right)} \left(T;\mathbf{\theta}_{n},\mathcal{Y}_{n}\right),w_{0},\xi\right),\]

so that we approximate problem (3.12) by

\[\hat{V}_{n}\left(t_{0},w_{0};\mathcal{Y}_{n}\right) = \inf_{\left(\mathbf{\theta}_{n},\xi\right)\in\mathbb{R}^{n+1}}\hat{ J}_{n}\left(\mathbf{\theta}_{n},\xi;t_{0},w_{0},\mathcal{Y}_{n}\right). \tag{3.16}\]

The numerical solution of (3.16) can then proceed using standard (stochastic) gradient descent techniques (see Appendix B). For subsequent reference, let \(\left(\hat{\mathbf{\theta}}_{n}^{*},\hat{\xi}_{n}^{*}\right)\) denote the optimal point in (3.16) relative to the training data set \(\mathcal{Y}_{n}\) in (3.16).

In the case of sufficiently large datasets (3.13), in other words as \(n\rightarrow\infty\), we would expect that the error in approximating (3.12) by (3.16) can be made arbitrarily small. However, as noted above, as \(n\rightarrow\infty\) and the number of hidden nodes \(\hbar\left(n\right)\rightarrow\infty\) (for any fixed \(\mathcal{L}^{h}\geq 1\)), (3.12) is also expected to approximate (3.6) more accurately. As a result, we obtain the necessary intuition for establishing the convergence of (3.16) to (3.6) under suitable conditions, which is indeed confirmed in the results of Section 4.

Note that since \(\mathcal{Y}_{n}\) is used in (3.16) to obtain the optimal NN parameter vector \(\hat{\mathbf{\theta}}_{n}^{*}\), it is usually referred to as the NN "training" dataset (see for example Goodfellow et al. (2016)). Naturally, we can also construct a "testing" dataset \(\mathcal{Y}_{\hat{n}}^{test}\), that is of a similar structure as (3.13), but typically based on a different implied distribution of \(\mathbf{Y}\) as a result of different data generation assumptions. For example, \(\mathcal{Y}_{\hat{n}}^{test}\) can be obtained using a different time period of historical data for its construction, or different process parameters if there are parametric asset dynamics specified. The resulting approximation \(\mathbf{f}_{n}^{*}\left(\cdot;\hat{\mathbf{\theta}}_{n}^{*}\right)\in\mathcal{N}_{n}\) to the optimal control \(\mathbf{p}^{*}\in C\left(\mathcal{D}_{\hat{\mathbf{\phi}}},\mathcal{Z}\right)\) obtained using the training dataset in (3.16) can then be implemented on the testing dataset for out-of-sample testing or scenario analysis. This is discussed in more detail in Appendix B.

**Remark 3.1**.: (Extension to wealth path-dependent objectives) As noted in the Introduction, the NN approach as well as the convergence analysis of Section 4 can be extended to objective functions that depend on the entire wealth path \(\left\{W\left(t\right):t\in\mathcal{T}\right\}\) instead of just the terminal wealth \(W\left(T\right)\). This is achieved by simply modifying (3.15) appropriately and ensuring the wealth is assessed at the desired intervals using (3.14). 

### Advantages of the NN approach

The following observations highlight some advantages of the proposed NN approach:

* semi-variance problem (2.16) in Section 5.
* The proposed methodology is parsimonious, in the sense that the NN parameter vector remains independent of number of rebalancing events. Specifically, we observe that the NN parameter vector \(\mathbf{\theta}_{n}\in\mathbb{R}^{n_{n}}\) of the NN does _not_ depend on the rebalancing time \(t_{m}\in\mathcal{T}\) or on the sample path \(j\). This contrasts our
approach with the approaches of for example Han and Weinan (2016); Tsang and Wong (2020),5 where the number of parameters scale with the number of rebalancing events. As a result, the NN approach presented here can lead to potentially significant computational advantages in the cases of (i) long investment time horizons or (ii) short trading time horizons with a frequent number of portfolio rebalancing events.

Footnote 5: Tsang and Wong (2020) use a stacked NN approach, with a different NN at each rebalancing time.

A natural question might be whether the NNs in the proposed approach are required to be very deep, thus potentially exposing the training of the NN in (3.16) to problem of vanishing or exploding gradients (see for example Goodfellow et al. (2016)). However, the ground truth results presented in Section 5 demonstrate that we obtain very accurate results with relatively shallow NNs (at most two hidden layers). We suspect this might be due to the optimal control being relatively low-dimensional compared to the high-dimensional objective functionals in portfolio optimization problems with discrete rebalancing (see Van Staden et al. (2022b) for a rigorous analysis), while in this NN approach approach the optimal control is obtained directly without requiring the solution of the (high-dimensional) objective functional at rebalancing times.

Note that these advantages also contrast the NN approach with Reinforcement Learning-based algorithms to solve portfolio optimization problems, as the following remark discusses.

**Remark 3.2**.: (Contrast of NN approach to Reinforcement Learning). Reinforcement learning (RL) algorithms (for example, Q-learning) relies fundamentally on the DP principle for the numerical solution of the portfolio optimization problem (see for example Gao et al. (2020); Lucarelli and Borrotti (2020); Park et al. (2020)). This requires, at each value iteration step, the approximation of a (high-dimensional) conditional expectation. As a result, RL is associated with standard DP-related concerns related to error amplification and the curse of dimensionality discussed above, and also cannot solve general problems of the form (1.1) without relying on for example an embedding approach to obtain an associated problem that can be solved using DP methods. 

## 4 Convergence analysis

In this section, we present the theoretical justification of the proposed NN approach as outlined in Section 3. We confirm that the numerical solution of (3.16) can be used to approximate the theoretical solution of (3.6) arbitrarily well (in probability) under suitable conditions. This section only summarizes the key convergence results which are among the main contributions of this paper, while additional technical details and proofs are provided in Appendix A.

We start with Theorem 4.1, which confirms the validity of Step 1 (Subsection 3.1), namely using a NN \(\boldsymbol{f}_{n}\left(\cdot;\boldsymbol{\theta}_{n}\right)\in\mathcal{N}_{n}\) to approximate the control. Note that Theorem 4.1 relies on two assumptions, presented in Appendix A.2: We emphasize that Assumption A.3 is purely made for purposes of convenience, since its requirements can easily be relaxed with only minor modifications to the proofs (as discussed in Remark A.1), but at the cost of significant notational complexity and no additional insights. In contrast, Assumption A.2 is critical to establish the result of Theorem 4.1, and requires that the optimal investment strategy (or control) satisfies Assumption 3.1, places some basic requirements on \(F\) and \(G\), and assumes that the sequence of NNs \(\left\{\boldsymbol{f}_{n}\left(\cdot;\boldsymbol{\theta}_{n}\right),n\in \mathbb{N}\right\}\) is constructed such that the number of nodes in each hidden layer \(\hbar\left(n\right)\rightarrow\infty\) as \(n\rightarrow\infty\) (no assumptions are yet required regarding the exact form of \(n\rightarrow\hbar\left(n\right)\)).

**Theorem 4.1**.: _(Validity of NN approximation) We assume that Assumption A.2 holds, and for ease of exposition, we also assume that Assumption A.3 holds. Then the NN approximation to the control in (3.8) is valid, in the sense that \(V\left(t_{0},w_{0}\right)\) in (3.6) can be approximated arbitrarily well by \(V_{n}\left(t_{0},w_{0}\right)\) in (3.12) for sufficiently large \(n\), since_

\[\lim_{n\rightarrow\infty}\left|V_{n}\left(t_{0},w_{0}\right)-V \left(t_{0},w_{0}\right)\right| = \lim_{n\rightarrow\infty}\left|\inf_{\left(\boldsymbol{\theta}_{n },\xi\right)\in\mathbb{R}^{m+1}}J_{n}\left(\boldsymbol{\theta}_{n},\xi;t_{0},w _{0}\right)-\inf_{\xi\in\mathbb{R}}\inf_{\boldsymbol{p}\in C\left(\mathcal{D}_ {\boldsymbol{\phi}},\mathcal{Z}\right)}J\left(\boldsymbol{p},\xi;t_{0},w_{0} \right)\right| \tag{4.1}\] \[= 0.\]

Proof.: See Appendix A.3. 

Having justified Step 1 of the approach, Theorem 4.2 now confirms the validity of Step 2 of the NN approach (see Subsection 3.2), namely using the computational estimate \(\boldsymbol{f}_{n}^{\star}\left(\cdot;\hat{\boldsymbol{\theta}}_{n}^{\star} \right)\in\mathcal{N}_{n}\) from (3.16) as an approximation
of the true optimal control \(\mathbf{p}^{*}\in C\left(\mathcal{D}_{\mathbf{\phi}},\mathcal{Z}\right)\). Note that in addition to the assumptions of Theorem 4.1, Theorem 4.2 also requires Assumption A.4, which by necessity includes computational considerations such as the structure of the training dataset \(\mathcal{Y}_{n}\), the rate of divergence of the number of hidden nodes \(\hbar\left(n\right)\rightarrow\infty\) as \(n\rightarrow\infty\), and assumptions regarding the optimization algorithm used in solving problem (3.16).

**Theorem 4.2**.: _(Validity of computational estimate) We assume that Assumption A.2, Assumption A.3 and Assumption A.4 hold. Then the computational estimate to the optimal control (3.2) obtained using (3.8) and (3.16) is valid, in the sense that the value function \(V\left(t_{0},w_{0}\right)\) in (3.6) can be approximated arbitrarily well in probability by \(\hat{V}_{n}\left(t_{0},w_{0};\mathcal{Y}_{n}\right)\) in (3.16) for sufficiently large \(n\), since_

\[\left|\hat{V}_{n}\left(t_{0},w_{0};\mathcal{Y}_{n}\right)-V\left(t _{0},w_{0}\right)\right| = \left|\inf_{\left(\mathbf{\theta}_{n},\xi\right)\in\mathbb{R}^{n+1} }\hat{J}_{n}\left(\mathbf{\theta}_{n},\xi;t_{0},w_{0},\mathcal{Y}_{n}\right)-\inf _{\xi\in\mathbb{R}}\inf_{\mathbf{p}\in C\left(\mathcal{D}_{\mathbf{\phi}},\mathcal{Z} \right)}J\left(\mathbf{p},\xi;t_{0},w_{0}\right)\right| \tag{4.2}\] \[\overset{P}{\longrightarrow} 0,\qquad\text{as }n\rightarrow\infty.\]

Proof.: See Appendix A.3. 

Taken together, Theorem 4.1 and Theorem 4.2 establish the theoretical validity of the NN approach to solve problems of the form (1.1).

## 5 Numerical results

In this section, we present numerical results obtained by implementing the NN approach described in Section 3. For illustrative purposes, the examples focus on investment objectives as outlined in Subsection 2.1, and we use three different data generation techniques for obtaining the training data set \(\mathcal{Y}_{n}\) of the NN: (i) parametric models for underlying asset returns, (ii) stationary block bootstrap resampling of empirical returns, and (ii) generative adversarial network (GAN)-generated synthetic asset returns.

### Closed-form solution: \(DSQ\left(\gamma\right)\) with continuous rebalancing

Under certain conditions, some of the optimization problems in Subsection 2.1 can be solved analytically. In this subsection, we demonstrate how a closed-form solution of problem \(DSQ\left(\gamma\right)\) in (2.10), assuming _continuous_ rebalancing (i.e. if we let \(\Delta t\to 0\) in (2.1)), can be approximated very accurately using a very simple NN (1 hidden layer, only 3 hidden nodes) using discrete rebalancing with \(\Delta t\gg 0\) in (2.1). This simultaneously illustrates how parsimonious the NN approach is, as well as how useful the imposition of time-continuity is in ensuring the smooth behavior of the (approximate) optimal control.

In this subsection as well as in Subsection 5.2, we assume parametric dynamics for the underlying assets. For concreteness, we consider the scenario of two assets, \(N_{a}=2\), with unit values \(S_{i},i=1,2\), evolving according to the following dynamics,

\[\frac{dS_{i}\left(t\right)}{S_{i}\left(t^{-}\right)} = \left(\mu_{i}-\lambda_{i}\kappa_{i}^{\left(1\right)}\right)\cdot dt +\sigma_{i}\cdot dZ_{i}\left(t\right)+d\left(\sum_{k=1}^{\pi_{i}\left(t \right)}\left(\vartheta_{i}^{\left(k\right)}-1\right)\right),\qquad i=1,2. \tag{5.1}\]

Note that (5.1) takes the form of the standard jump diffusion models in finance - see e.g. Kou (2002); Merton (1976) for more information. For each asset \(i\) in (5.1), \(\mu_{i}\) and \(\sigma_{i}\) denote the (actual, not risk-neutral) drift and volatility, respectively, \(Z_{i}\) denotes a standard Brownian motion, \(\pi_{i}\left(t\right)\) denotes a Poisson process with intensity \(\lambda_{i}\geq 0\), and \(\vartheta_{i}^{\left(k\right)}\) are i.i.d. random variables with the same distribution as \(\vartheta_{i}\), which represents the jump multiplier of the \(i\)th risky asset with \(\kappa_{i}^{\left(1\right)}=\mathbb{E}\left[\vartheta_{i}-1\right]\) and \(\kappa_{i}^{\left(2\right)}=\mathbb{E}\left[\left(\vartheta_{i}-1\right)^{2}\right]\). While the Brownian motions can be correlated with \(dZ_{1}\left(t\right)dZ_{2}\left(t\right)=\rho_{1,2}\cdot dt\), we make the standard assumption that the jump components are independent (see for example Forsyth and Vetzal (2022)).

For this subsection only, we treat the first asset (\(i=1\) in (5.1)) as a "risk-free" asset, and set \(\mu_{1}=r>0\) where \(r\) is the risk-free rate, so that we have \(\lambda_{1}=0\), \(\sigma_{1j}=0\)\(\forall j\), and \(Z_{1}\equiv 0\), while the second asset (\(i=2\) in (5.1)) is assumed to be a broad equity market index (the "risky asset"). In this scenario, if problem \(DSQ\left(\gamma\right)\) in (2.10) is solved subject to dynamics (5.1) together with the assumptions of costless continuous trading,
infinite leverage, and uninterrupted trading in the event of insolvency, then the \(DSQ\left(\gamma\right)\)-optimal control can be obtained analytically as

\[\boldsymbol{p}^{*}\left(t,W^{*}\left(t\right)\right)=[1-p_{2}^{*}\left(t,W^{*} \left(t\right)\right),p_{2}^{*}\left(t,W^{*}\left(t\right)\right)]\in\mathbb{R} ^{2}, \tag{5.2}\]

where the fraction of wealth in the broad stock market index (asset \(i=2\)) is given by (Zweng and Li (2011))

\[p_{2}^{*}\left(t,W^{*}\left(t\right)\right)=\frac{\mu_{2}-r}{\sigma_{2}^{2}+ \lambda_{2}\kappa_{2}^{(2)}}\cdot\left[\frac{\gamma e^{-r\left(T-t\right)}-W^{ *}\left(t\right)}{W^{*}\left(t\right)}\right],\qquad w_{0}<\gamma e^{-r\left(T -t\right)}. \tag{5.3}\]

By design, the NN approach is not constructed to solve problems with unrealistic assumptions such as continuous trading, infinite leverage and short-selling, or trading in the event of bankruptcy, all of which are required to derive (5.3). However, if the implicit quadratic wealth target for the DSQ problem (i.e. the value of \(\gamma\), see Vigna (2014)) is not too aggressive, the analytical solution (5.3) does not require significant leverage or lead to a large probability of insolvency. In such a scenario, we can use the NN approach to approximate (5.3).

We select \(w_{0}=100\), \(T=1\) year and \(\gamma=138.33\), and simulate \(n=2.56\times 10^{6}\) paths of the underlying assets using (5.1) and parameters as in Table C.1 (Appendix C). On this set of paths, the true analytical solution (5.3) is implemented using 7,200 time steps. In contrast, for the NN approach, we use only 4 rebalancing events in \([0,T=1]\), and therefore aggregate the simulated returns in quarterly time intervals to construct the training data set \(\mathcal{Y}_{n}\). We consider only a very shallow NN, consisting of a single hidden layer and only 3 hidden nodes.

Figure 5.1 compares the resulting optimal investment strategies by illustrating the optimal proportion of wealth invested in the the broad equity market index (asset \(i=2\)) as a function of time and wealth. We emphasize that the NN strategy in Figure 5.1(b) is not expected to be exactly identical to the analytical solution in Figure 5.1(a), since it is based on fundamentally different assumptions such as discrete rebalancing and investment constraints (2.5).

However, requiring that the NN feature vector includes time in the proposed NN approach, together with a NN parameter vector that does not depend on time, we guarantee the smooth behavior in time of the NN approximation observed in Figure 5.1(b). As a result, Table 5.1 shows that the shallow NN strategy trained with \(\Delta t\gg 0\) results in a remarkably accurate and parsimonious approximation to the true analytical solution where \(\Delta t\to 0\), since we obtain nearly identical optimal terminal wealth distributions.

### Ground truth: Problem \(MCV\left(\rho\right)\)

In the case of the Mean-CVaR problem \(MCV\left(\rho\right)\) in (2.15), Forsyth and Vetzal (2022) obtain an MCV-optimal investment strategy subject to the same investment constraints as in Section 2 (namely discrete rebalancing, no short-selling or leverage allowed, and no trading in insolvency) using the partial (integro-)differential equation (PDE) approach of Forsyth (2020).

Figure 5.1: Closed-form solution - \(DSQ\left(\gamma\right)\) with continuous rebalancing: Optimal proportion of wealth invested in the broad equity market index as a function of time and wealth. The NN approximation is obtained for a specific initial wealth of \(w_{0}=\)100, and only four rebalancing events in \([0,T]\).
For ground truth analysis purposes, we therefore consider the same investment scenario as in Forsyth and Vetzal (2022), where two underlying assets are considered, namely 30-day US T-bills and a broad equity market index (the CRSP VWD index) - see Appendix C for definitions. However, in contrast to the preceding section where one asset was taken as the risk-free asset, both assets are now assumed to evolve according to dynamics of the form (5.1), using the double-exponential Kou (2002) formulation for the jump distributions. The NN training data set is therefore constructed by simulating the same underlying dynamics. While further details regarding the context and motivation for the investment scenario can be found in Forsyth and Vetzal (2022), here we simply note that the scenario involves \(T=5\) years, quarterly rebalancing, a set of admissible strategies satisfying (2.5), and parameters for (5.1) as in Table C.2.

As discussed in Appendix B, the inherently higher complexity of the Mean-CVaR optimal control requires the NN to be deeper than in the case of the problem considered in Subsection 5.1. As a result, we consider approximating NNs with two hidden layers, each with 8 hidden nodes, while relatively large mini-batches of 2,000 paths were used in the stochastic gradient descent algorithm (see Appendix B) to ensure sufficiently accurate sampling of the tail of the returns distribution in selecting the descent direction at each step. Note that despite using a deeper NN, this NN structure is still very parsimonious and relatively shallow compared to the rebalancing time-dependent structures considered in for example Han and Weinan (2016), where a new set of parameters is introduced at each rebalancing event.

Table 5.2 compares the PDE results reported in Forsyth and Vetzal (2022) with the corresponding NN results. Note that the PDE optimal control was determined by solving a Hamilton-Jacobi-Bellman PDE numerically. The statistics for the PDE generated control were computed using \(n=2.56\times 10^{6}\) Monte Carlo simulations of the joint underlying asset dynamics in order to calculate the results of Table 5.2, while the NN was trained on \(n=2.56\times 10^{6}\) paths of the same underlying asset dynamics but which were independently simulated. While some variability of the results are therefore to be expected due to the underlying samples, the results in Table 5.2 demonstrate the robustness of the proposed NN approach.

### Ground truth: Problems \(MV\left(\rho\right)\) and \(DSQ\left(\gamma\right)\)

In this subsection, we demonstrate that if the investment objective (1.1) is separable in the sense of dynamic programming, the correct time-consistent optimal investment strategy is recovered, otherwise we obtain the correct pre-commitment (time-inconsistent) investment strategy.

To demonstrate this, the theoretical embedding result of Li and Ng (2000); Zhou and Li (2000), which establishes the equivalence of problems \(MV\left(\rho\right)\) and \(DSQ\left(\gamma\right)\) under fairly general conditions, can be exploited for ground truth analysis purposes as follows. Suppose we solved problems \(MV\left(\rho\right)\) and \(DSQ\left(\gamma\right)\) on the same

\begin{table}
\begin{tabular}{|c|c|c|c|c|c|c|} \cline{3-7} \multicolumn{1}{c}{} & \multicolumn{5}{|c|}{\(W\left(T\right)\) percentiles} \\ \hline Solution approach & Rebalancing & 5th & 20th & 50th & 80th & 95th \\ \hline \hline Closed-form solution & Continuous, \(\Delta t\)\(\rightarrow\) 0 & 86.81 & 98.02 & 106.35 & 112.82 & 118.15 \\ \hline Shallow NN approximation & Discrete, \(\Delta t=0.25\), total of \(N_{rb}=4\) only & 86.62 & 97.30 & 105.67 & 112.54 & 118.85 \\ \hline \end{tabular}
\end{table}
Table 5.1: Closed-form solution - \(DSQ\left(\gamma\right)\) with continuous rebalancing: Percentiles of the simulated ( \(n=2.56\times 10^{6}\)) terminal wealth distributions obtained by implementing the optimal strategies in Figure 5.1. In both cases, a mean terminal wealth of 105 is obtained. Note that the NN approximation was obtained under the assumption of quarterly rebalancing only, no leverage or short-selling, and therefore no trading in insolwebers.

\begin{table}
\begin{tabular}{|c|c|c|c|c|c|c|c|} \hline \(\rho\) & \multicolumn{2}{|c|}{5\% CVaR} & \multicolumn{2}{|c|}{\(E^{q_{0},w_{0}}\left[W\left(T\right)\right]\)} & \multicolumn{2}{|c|}{Value function} & \multicolumn{2}{|c|}{\%} \\ \cline{2-7}  & PDE & NN & PDE & NN & PDE & NN & \\ \hline
0.10 & 940.60 & 940.55 & 1069.19 & 1062.97 & 1047.52 & 1046.85 & -0.06\% \\ \hline
0.25 & 936.23 & 937.39 & 1090.89 & 1081.99 & 1208.95 & 1207.88 & -0.09\% \\ \hline
1.00 & 697.56 & 690.11 & 1437.73 & 1444.16 & 2135.29 & 2134.27 & -0.05\% \\ \hline
1.50 & 614.92 & 611.65 & 1508.10 & 1510.07 & 2877.07 & 2876.76 & -0.01\% \\ \hline \end{tabular}
\end{table}
Table 5.2: Ground truth - problem \(MCV\left(\rho\right)\): The PDE results are obtained from Forsyth and Vetzal (2022) for selected points on the Mean-CVaR “efficient frontier”. The “Value function” column reports the value of the objective function (2.14) under the corresponding optimal control, while “% difference” reports the percentage difference in the reported value functions for the NN solution compared to the PDE solution.
underlying training data set. We remind the reader that in the proposed NN approach, problem \(MV\left(\rho\right)\) can indeed be solved directly without difficulty, which is not possible in dynamic programming-based approaches. Then, considering the numerical results, there should be values of parameters \(\rho\equiv\tilde{\rho}\) and \(\gamma\equiv\tilde{\gamma}\) such that the optimal strategy of \(MV\left(\rho\equiv\tilde{\rho}\right)\) corresponds exactly to the optimal strategy of \(DSQ\left(\gamma\equiv\tilde{\gamma}\right)\), with a specific relationship holding between \(\tilde{\rho}\) and \(\tilde{\gamma}\). The NN approach can therefore enable us to numerically demonstrate the embedding result of Li and Ng (2000); Zhou and Li (2000) in a setting where the underlying asset dynamics are not explicitly specified and where multiple investment constraints are present. We start by recalling the embedding result.

**Proposition 5.1**.: _(Embedding result of Li and Ng (2000); Zhou and Li (2000)) Fix a value \(\tilde{\rho}>0\). If \(\mathcal{P}^{*}\in\mathcal{A}\) is the optimal control of problem \(MV\left(\rho\equiv\tilde{\rho}\right)\) in (2.12), then \(\mathcal{P}^{*}\) is also the optimal control for problem \(DSQ\left(\gamma=\tilde{\gamma}\right)\) in (2.10), provided that_

\[\tilde{\gamma} = \frac{1}{2\tilde{\rho}}+E^{t_{0},w_{0}}\left[W^{*}\left(T; \mathcal{P}^{*},\mathbf{Y}\right)\right]. \tag{5.4}\]

Proof.: See Li and Ng (2000); Zhou and Li (2000). We also highlight the alternative proof provided in Dang and Forsyth (2016), which shows that this result is valid for any admissible control set \(\mathcal{A}\). 

Since (5.4) is valid for any admissible control set \(\mathcal{A}\), we consider a factor investing scenario where portfolios are constructed using popular long-only investable equity factor indices (Momentum, Value, Low Volatility, Size), a broad equity market index (the CRSP VWD index), 30-day T-bills and 10-year Treasury bonds (see Appendix C for definitions). For illustrative purposes in the case of an investor primarily concerned with long-run factor portfolio performance, we use a horizon of \(T=10\) years, \(w_{0}=120\), annual contributions of \(q\left(t_{m}\right)=12\), and annual rebalancing.

Given historical returns data for the underlying assets, we construct training and testing (out-of-sample) data sets for the NN, \(\mathcal{Y}_{n}\) and \(\mathcal{Y}_{n}^{test}\), respectively, using stationary block bootstrap resampling of empirical historical asset returns (see Appendix C), which is popular with practitioners (Anarkulova et al. (2022); Cavaglia et al. (2022); Cogneau and Zakalmouline (2013); Dichtl et al. (2016); Scott and Cavaglia (2017); Simonian and Martirosyan (2022)) and is designed to handle weakly stationary time series with serial dependence. See Ni et al. (2022) for a discussion concerning the probability of obtaining a repeated path in block bootstrap resampling (which is negligible for any realistic number of samples). Due to availability of historical data we use inflation-adjusted monthly empirical returns from 1963:07 to 2020:12. The training data set (\(n=10^{6}\)) is obtained using an expected block size of 6 months of joint returns from 1963:07 to 2009:12, while the testing data set (\(n=10^{6}\)) uses an expected block size of 3 months and returns from 2010:01 to 2020:12. We consider NNs with two hidden layers, each with only eight hidden nodes.

Choosing two values of \(\tilde{\rho}>0\) to illustrate different levels of risk aversion (see Table 5.3), we solve problem \(MV\left(\rho=\tilde{\rho}\right)\) in (2.12) directly using the proposed approach to obtain the optimal investment strategy \(\mathbf{f}\left(;\hat{\mathbf{\theta}}_{me}^{*}\right)\). Note that since we consider a fixed NN structure in this setting rather than a sequence of NNs, we drop the subscript "\(n\)" in the notation \(\mathbf{f}\left(;\hat{\mathbf{\theta}}_{me}^{*}\right)\). Using this result together with (5.4), we can approximate the associated value of \(\tilde{\gamma}\) by

\[\tilde{\gamma} \simeq \frac{1}{2\tilde{\rho}}+\frac{1}{n}\sum_{j=1}^{n}W^{*\left(j \right)}\left(T;\hat{\mathbf{\theta}}_{mev}^{*},,\mathcal{Y}_{n}\right), \tag{5.5}\]

and solve problem \(DSQ\left(\gamma=\tilde{\gamma}\right)\) independently using the proposed approach on the same training data set \(\mathcal{Y}_{n}\).

According to Proposition 5.1, the resulting investment strategy \(\mathbf{f}\left(;\hat{\mathbf{\theta}}_{dsq}^{*}\right)\) should be (approximately) identical to the strategy \(\mathbf{f}\left(\cdot;\hat{\mathbf{\theta}}_{mv}^{*}\right)\) if the proposed approach works as required. Note that the parameter vectors are expected to be different (i.e. \(\hat{\mathbf{\theta}}_{dsq}^{*}\neq\hat{\mathbf{\theta}}_{mv}^{*}\)) due to a variety of reasons (multiple local minima, optimization using SGD, etc.), but the resulting wealth distributions and asset allocation should agree, i.e. \(\mathbf{f}\left(\cdot;\hat{\mathbf{\theta}}_{dsq}^{*}\right)\simeq\mathbf{f}\left(\cdot; \hat{\mathbf{\theta}}_{mv}^{*}\right)\).

Figure 5.2 demonstrates the investment strategies \(\mathbf{f}\left(\cdot;\hat{\mathbf{\theta}}_{mv}^{*}\right)\) and \(\mathbf{f}\left(\cdot;\hat{\mathbf{\theta}}_{sq}^{*}\right)\) obtained by training the NNs on the same training data set using values of \(\tilde{\rho}=0.017\) and \(\tilde{\gamma}=429.647\), respectively. Note that the values \(\tilde{\rho}\) and \(\tilde{\gamma}\) are rounded to three decimal places, and Figure 5.2 corresponds to Results set 1 in Table 5.3. In this example, only four of the underlying candidate assets have non-zero investments, which is to be expected due to the high correlation between long-only equity factor indices.
Table 5.3 confirms that the associated optimal terminal wealth distributions of \(MV\left(\rho=\tilde{\rho}\right)\) and \(DSQ\left(\gamma=\tilde{\gamma}\right)\) indeed correspond, both in-sample (training data set) and out-of-sample (testing data set).

The proposed NN approach therefore clearly works as expected, in that we demonstrated that the result of Proposition 5.1 in a completely model-independent way in a portfolio optimization setting where no known analytical solutions exist. In particular, we emphasize that no assumptions were made regarding parametric underlying asset dynamics, the results are entirely data-driven. As a result, we can interpret the preceding results as showing that the approach correctly recovers the time-inconsistent (or pre-commitment) strategy without difficulty if the objective is not separable in the sense of dynamic programming, such as in the case of the \(MV\left(\rho\right)\) problem, whereas if the objective is separable in the sense of dynamic programming, such as in the case of the \(DSQ\left(\gamma\right)\) problem, the approach correctly recovers the associated time-consistent strategy.

Figure 5.2: Ground truth - problems \(MV\left(\rho=\tilde{\rho}\right)\) and \(DSQ\left(\gamma=\tilde{\gamma}\right)\): investment strategies \(\boldsymbol{f}\left(\cdot;\hat{\boldsymbol{\theta}}_{mv}^{*}\right)\) and \(\boldsymbol{f}\left(\cdot;\hat{\boldsymbol{\theta}}_{dst}^{*}\right)\) obtained by training the NNs using values of \(\tilde{\rho}=0.017\) and \(\tilde{\gamma}=429.647\) (rounded to three decimal places), respectively. Each figure shows the proportion of wealth invested in the asset as a function of the minimal NN features, namely time and available wealth. Zero investment under the optimal strategies in the broad market index and the Size factor.

\begin{table}
\begin{tabular}{|c||c|c|c||c|c|c|c|} \cline{2-9} \multicolumn{1}{c||}{} & \multicolumn{2}{c||}{Results set 1: \(\tilde{\rho}=0.017\), \(\tilde{\gamma}=429.647\)} & \multicolumn{3}{c|}{Results set 2: \(\tilde{\rho}=0.0097\), \(\tilde{\gamma}=493.196\)} \\ \hline \(W\left(T\right)\) & \multicolumn{2}{c|}{Training data} & \multicolumn{2}{c||}{Testing data} & \multicolumn{2}{c|}{Training data} & \multicolumn{2}{c|}{Testing data} \\ \cline{2-9} distribution & MV & DSQ & MV & DSQ & MV & DSQ & MV & DSQ \\ \hline \hline Mean & 400.2 & 400.3 & 391.2 & 391.6 & 441.5 & 441.8 & 441.8 & 441.5 \\ \hline Stdev & 55.4 & 55.4 & 26.2 & 25.7 & 79.6 & 79.7 & 39.4 & 39.5 \\ \hline
5th percentile & 276.5 & 276.4 & 346.6 & 347.5 & 255.2 & 254.6 & 367.8 & 367.1 \\ \hline
25th percentile & 391.8 & 392.3 & 382.4 & 382.8 & 422.4 & 423.6 & 430.9 & 430.7 \\ \hline
50th percentile & 416.1 & 416.3 & 396.5 & 396.8 & 469.8 & 470.1 & 451.3 & 451.2 \\ \hline
75th percentile & 429.9 & 429.8 & 406.4 & 406.7 & 487.7 & 489.6 & 465.0 & 464.8 \\ \hline
95th percentile & 452.1 & 452.1 & 418.9 & 419.0 & 516.1 & 516.5 & 480.9 & 480.2 \\ \hline \end{tabular}
\end{table}
Table 5.3: Ground truth - problems \(MV\left(\rho=\tilde{\rho}\right)\) and \(DSQ\left(\gamma=\tilde{\gamma}\right)\): Terminal wealth results obtained using \(n=10^{6}\) joint paths for the underlying assets. Note that the values of \(\tilde{\rho}\) and \(\tilde{\gamma}\) are rounded to three decimal places,.
### Mean - Semi-variance strategies

Having demonstrated the reliability of the results obtained using the proposed NN approach with the preceding ground truth analyses, we now consider the solution of the Mean - Semi-variance problem (2.16). To provide the necessary context to interpret the \(MSemiV\left(\rho\right)\)-optimal results, we compare the results of the optimal solutions of the \(MCV\left(\rho=\rho_{mcv}\right)\), \(MSemiV\left(\rho=\rho_{msv}\right)\), and \(OSQ\left(\gamma=\gamma_{osq}\right)\) problems, where the values of \(\rho_{mcv}\), \(\rho_{msv}\) and \(\gamma_{osq}\) are selected to obtain the same expected value of terminal wealth on the NN training data set. This is done since the MCV- and OSQ-optimal strategies have been analyzed in great detail (Dang and Forsyth (2016); Forsyth (2020)), and are therefore well understood. Note that since all three strategies are related to the maximization of the mean terminal wealth and while simultaneously minimizing some risk measure (which is implicitly done in the case of the OSQ problem, see Dang and Forsyth (2016)), it is natural to compare the strategies on the basis of equal expectation of terminal wealth.

To highlight the main qualitative features of the \(MSemiV\left(\rho\right)\)-optimal results, we consider a simple investment scenario of two assets, namely 30-day T-bills and a broad equity market index (the VWD index) - see Appendix C for definitions. We choose \(T=\)5 years, \(w_{0}=1000\), and zero contributions to demonstrate a lump sum investment scenario with quarterly rebalancing.

To illustrate the flexibility of the NN approach to underlying data generating assumptions, the NN training data sets are constructed using generative adversarial network (GAN)-generated synthetic asset returns obtained by implementing the TimeGAN algorithm proposed by Yoon et al. (2019). In more detail, using empirical monthly asset returns from 1926:01 to 2019:12 for the underlying assets (data sources are specified in Appendix C), the TimeGAN is trained with default parameters as in Yoon et al. (2019) using block sizes of 6 months to capture both correlation and serial correlation aspects of the (joint) time series.6 Once trained, the TimeGAN is then used to generate a set of \(n=10^{6}\) paths of synthetic asset returns, which is used as the training data set to train the NNs corresponding to the MCV, MSemiV and OSQ-optimal investment strategies.

Footnote 6: It appears that the actual code in Yoon et al. (2019) implements the following steps: (i) takes as input actual price data, (ii) forms rolling blocks of price data and (iii) forms a single synthetic price path (which is the same length as the original path) by randomly sampling (without replacement) from the set of rolling blocks. Step (iii) corresponds to the non-overlapping block bootstrap using a fixed block size. This should be contrasted with stationary block bootstrap resampling of Politis and Romano (1994). Step (i) does not make sense as input to a bootstrap technique, since the data set is about 10 years long, with an initial price of $50 and a final price of $1200. We therefore changed Step (i), so that all data was converted to returns prior to being used as input.

Figure 5.3 illustrates the resulting optimal investment strategies, and we observe that the MSemiV-optimal strategy is fundamentally different from the MCV and OSQ-optimal strategies, while featuring elements of both. Specifically, Figure 5.4, which illustrates the resulting optimal terminal wealth distributions (with the same expectation), demonstrates that the MSemiV strategy, like the MCV strategy, can offer better downside protection than the OSQ strategy, while the MSemiV strategy retains some of the qualitative elements of the OSQ distribution such as the left skew.

Having illustrated that the MSemiV problem can be solved in a dynamic trading setting using the proposed NN approach to obtain investment strategies that offer potentially valuable characteristics, we leave a more in-depth investigation of the properties and applications of MSemiV-optimal strategies for future work.

Figure 5.3: Optimal investment strategies for the \(MCV\left(\rho=\rho_{mcv}\right)\), \(MSemiV\left(\rho=\rho_{msv}\right)\), and \(OSQ\left(\gamma=\gamma_{osq}\right)\) strategies, obtaining identical expectation of terminal wealth on the training data set. Each figure shows the proportion of wealth invested in the broad equity market index as a function of the minimal NN features, namely time and available wealth.
## 6 Conclusion

In this paper, we presented a flexible NN approach, which does not rely on dynamic programming techniques, to solve a large class of dynamic portfolio optimization problems. In the proposed approach, a single optimization problem is solved, issues of instability and error propagation involved in estimating high-dimensional conditional expectations are avoided, and the resulting NN is parsimonious in the sense that the number of parameters does not scale with the number of rebalancing events.

We also presented theoretical convergence analysis results which show that the numerical solution obtained using the proposed approach can recover the optimal investment strategy, provided it exists, regardless of whether the resulting optimal investment strategy is time-consistent or (formally) time-inconsistent.

Numerical results confirmed the advantages of the NN approach, and showed that accurate results can be obtained in ground truth analyses in a variety of settings. The numerical results also highlighted that the approach remains agnostic as to the underlying data generating assumptions, so that for example empirical asset returns or synthetic asset returns can be used without difficulty.

We conclude by noting that the NN approach is not necessarily limited to portfolio optimization problems such a those encountered during the accumulation phase of pension funds, and could be extended to address the significantly more challenging problems encountered during the accumulation phase of defined contribution pension funds (see for example Forsyth (2022)). We leave this extension for future work.

## 7 Declarations

The authors have no competing interests to declare that are relevant to the content of this article. P.A. Forsyth's work was supported by the Natural Sciences and Engineering Research Council of Canada (NSERC) grant RGPIN-2017-03760.
