# Forward and Reverse mode gradients in TFF


```python
#@title Licensed under the Apache License, Version 2.0 (the "License"); { display-mode: "form" }
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
```


```python
import tensorflow as tf
import functools
import tf_quant_finance.math.gradient as tff_gradient
import tf_quant_finance.math.optimizer as tff_optimizer
import numpy as np
```

Consider a simple function in two variables $x$ and $y$ but in three dimensions such that the function along each axes is [$x^2$, $y^2$, $x y$]

$
\begin{align}
& f = [f_1, f_2, f_3] \\
& where \\
\end{align}
$

$
\begin{align}
f_1 &= x^2 \\
f_2 &= y^2 \\
f_3 &= x + y \\
\end{align}
$


```python
def func_grad(x):
    func = tf.stack([x[0]**2, x[1]**2, x[0]*x[1]])
    grad = tf.stack([2.0*x[0]+x[1], 2.0*x[1]+x[0]])
    return func, grad

def func(x):
    func = tf.stack([x[0]**2, x[1]**2, x[0]*x[1]])
    return func

start = tf.constant([1,1], dtype=tf.float64)

```

# Reverse mode

$
\begin{align}
\frac {\partial f}{\partial x} &= u_1 \frac{\partial f_1}{\partial x} + u_2 \frac{\partial f_2}{\partial x} + u_3 \frac{\partial f_3}{\partial x} \\ \\
&= u_1 2 x + u_3 y\\ \\
\frac {\partial f}{\partial y} &= u_1 \frac{\partial f_1}{\partial y} + u_2 \frac{\partial f_2}{\partial y} + u_3 \frac{\partial f_3}{\partial y} \\ \\
&= u_1 2 y + u_3 x
\end{align}
$

In Tensorflow, [$u_1$, $u_2$, $u_3$] is normally set to [1, 1, 1]

Setting [$x$, $y$] to [1,1]

Reverse mode returns the gradients summed up by components


```python
func_grad(start)[1]
```




    <tf.Tensor: shape=(2,), dtype=float64, numpy=array([3., 3.])>



# Forward mode

$
\begin{align}
{\partial f_1} &= w_1 \frac{\partial f_1}{\partial x} + w_2 \frac{\partial f_1}{\partial y} \\ \\
&= w_1 2 x \\ \\
{\partial f_2} &= w_1 \frac{\partial f_2}{\partial x} + w_2 \frac{\partial f_2}{\partial y} \\ \\
&= w_2 2 y \\ \\
{\partial f_3} &= w_1 \frac{\partial f_3}{\partial x} + w_2 \frac{\partial f_3}{\partial y} \\ \\
&= w_1 x + w_2 y \\ \\
\end{align}
$

In Tensorflow, [$w_1$, $w_2$] is normally set to [1, 1]

Setting [$x$, $y$] to [1,1]

Forward mode returns the gradients by components


```python
tff_gradient.fwd_gradient(func, start)
```




    <tf.Tensor: shape=(3,), dtype=float64, numpy=array([2., 2., 2.])>



Remember, Tensorflow is the tool used in Machine Learning. In Machine Learning, the aim is to minmise the scalar loss function, the loss function being the sum of the gradient with respect to the feature set. This lowest loss is the loss summed up over all training examples.

However, lets take the use case where we are valuing a set of Options, say ten, against a single spot price $S_0$. We now have ten price functions and we need their gradients against spot $S_0$ (ten delta's).

Using the forward gradients with respect to $S_0$ would give us the ten delta's in a single pass.

Using the backward gradients would result in the sum of the ten delta's, which may not be that useful.

It is useful to note that varying the weights would also give you individual components of the gradients (in other words [1,0] and [0,1] as values of [$w_1$, $w_2$], instead of the default [1,1], similarly for backward. This is, of course, at the expense of more compute.


```python

```

69ad084cece3ae088df8a7b8375b02d9ae612512
ipynb
Jupyter Notebook
tf_quant_finance/examples/jupyter_notebooks/Forward_Backward_Diff.ipynb
rajflume/tf-quant-finance
5cb9474f6f2e74617735d38ef26aaef28ce69aff
Apache-2.0
tf_quant_finance/examples/jupyter_notebooks/Forward_Backward_Diff.ipynb
rajflume/tf-quant-finance
5cb9474f6f2e74617735d38ef26aaef28ce69aff
Apache-2.0
tf_quant_finance/examples/jupyter_notebooks/Forward_Backward_Diff.ipynb
rajflume/tf-quant-finance
5cb9474f6f2e74617735d38ef26aaef28ce69aff
Apache-2.0
Qwen/Qwen-72B
1. YES
2. YES