# Regret Optimization

Regret optimization is a technique to optimize a portfolio given that one is uncertain about future prospects. Since one must eventually settle for a single portfolio out of all the possible scenarios, the best portfolio in this case is determine by the one that will have the *least* regret. This will be elaborated further

In general, this is a 2-stage optimization. The algorithm works as such,

1. Generate several possible scenarios in the future
    * For each scenario, give it a discrete probability of occurrence
    * These probabilities must sum to 1 across all scenarios!
2. In the first stage, derive the optimal weights for the portfolio for each scenario
    * If there are 5 scenarios, there will be 5 sets of these weights
    * Thus a model with 5 scenarios and 3 assets will yield a 5 x 3 matrix
3. In the second stage, solve for the minimal regret portfolio, these time using the previous sets optimal weights
    * You will receive the final set of of weights here (a 1 x 3 vector)

## Regret

Regret is defined as the cost of choosing one portfolio (which is optimal for a scenario) when another different scenario occurs instead. Functionally, the simplest mathematical formuation is

$$
    D(R(w_s) - R(w_o))
$$

where $D$ is a distance function, $R$ is the profit function, $w_s$ is the optimal weights for scenario $s$ and $w_o$ is the "optimal" weights that was chosen. Thus to solve for the minimal regret function, the exact problem formulation is as listed below

$$ 
    \underset{w}{\min} \sum_s^S p_s D(R(w_s) - R(w))
$$

where $p_s$ is the probability of scenario $s$ occurring.

### Distance function

The distance function could be anything that makes sense. Some common examples include a linear function, absolute function or quadratic function. Different functions will penalise regret differently and lead to different outcomes. For example, if a portfolio that does not have a wide swings in terms of objectives is desired, a quadratic function will work better than a linear function. 

$$
    \begin{align}
    \text{Assuming} \quad R(w) &\geq 1 \quad \forall w \in W \\
    [R(w_s) - R(w_o)]^2 &\geq R(w_s) - R(w_o)
    \end{align}
$$

### Linear Approximations

Suppose we have convex objective and constraints functions for the first stage, we alter the second stage optimization a little. The outcome will be similar but it will give another nice interpretation of the results which will be explained later. Ideally these functions should be **strictly linear**. However, in practice, the differences are usually negligible.

Suppose we want to maximize the returns of the portfolio subject to some CVaR constraints. For simplicity, the returns function will be $R$ and CVaR constraint functions will be $C$. Thus our first stage optimization will be

For every single scenario $s$ in $S$
$$ 
    \underset{w_s}{\max} R(w_s) \\
    \text{subject to} \\
    C_s(w_s) \geq 0 \\
    \sum_i w_{s, i} = 1 \\
    0 \leq w \leq 1 \quad \forall w \in w_s
$$

From this we would get 

$$
    W = 
    \begin{bmatrix}
        w_{1, 1} & w_{1, 2} & \dots  & w_{1, n} \\
        \vdots   & \vdots   & \ddots & \vdots   \\
        w_{s, 1} & w_{s, 2} & \dots  & w_{s, n}
    \end{bmatrix}
$$

where $s$ is the number of scenarios adn $n$ is the number of assets. We would then tweak our second (Regret) optimization to

$$
    \underset{a}{\min} \sum_s^S p_s D(R(w_s) - R(W \cdot a)) \\
    \text{subject to} \\
    \sum_s^S a_s = 1
$$

The solution of the problem, $a$, will represent the proportion of importance that is taken from each scenario. Suppose there are 3 scenarios - **X**, **Y**, **Z** and that the final proportion derived is `[0.2, 0.3, 0.5]`. This means that **20%** of the weights are taken from **X**, **30%** from **Y** and **50%** from **Z**. In essence, it weights the importance of each scenario for the final outcome.

To get the final weights, simply do a dot product of $W \cdot a$.

## Example

We will run through the Regret Optimization using both the `RegretOptimizer` and `PortfolioRegretOptimizer` classes. The `PortfolioRegretOptimizer` is a helper class that has several common built-in problems within itself. Underneath the hood, it uses the `RegretOptimizer` for the same operations. The `RegretOptimizer` is the more flexible tool that is useful for modelling more exotic scenarios.


```python
import numpy as np
from muarch.calibrate import calibrate_data

from allopy import OptData, RegretOptimizer
from allopy.datasets import load_monte_carlo
```


```python
# Generate different scenarios
num_assets = 7
num_scenarios = 4
scenario_probability = [0.57, 0.1, 0.14, 0.19]

main_adjustments = np.array([
    [ 0.0061,  0.0601,  0.0466,  0.0051, -0.0066, -0.0013, -0.0026],
    [ 0.0642,  0.0537,  0.0818,  0.0713,  0.0177,  0.0099,  0.0116],
    [-0.0219, -0.0381, -0.0059,  0.0242, -0.0153,  0.0164, -0.001 ],
    [-0.0333, -0.0617, -0.0405, -0.0251,  0.0084,  0.0054, -0.0035]
])

cvar_adjustments = np.array([
    [-0.0436,  0.0586, -0.0081,  0.0051, -0.0078,  0.0135, -0.0081],
    [ 0.0662,  0.079 ,  0.0896,  0.0501, -0.0265, -0.0572,  0.0025],
    [-0.1192, -0.1701, -0.1143,  0.0736, -0.0831,  0.0134, -0.0047],
    [-0.1728, -0.257 , -0.1933, -0.1432,  0.0342,  0.0079,  0.003 ]
])


def make_scenarios(adjustments, truncate=False):
    scenarios = []
    for adj in adjustments:
        data = OptData(load_monte_carlo()[..., :num_assets], 'quarterly')
    
        if truncate:  # cut for CVaR
            data = data.cut_by_horizon(3)

        scenarios.append(data.calibrate_data(adj))
    
    return scenarios

main_scenarios = make_scenarios(main_adjustments)
cvar_scenarios = make_scenarios(cvar_adjustments, True)
```


```python
# objective and constraint functions

def make_max_returns_obj_fun(cube: OptData):
    def obj_fun(w):
        return 1e2 * cube.expected_return(w, True)

    return obj_fun


def make_cvar_constraint_fun(cube: OptData, limit: float):
    def cvar_fun(w):
        return 1e3 * (limit - cube.cvar(w, True, 5.0))

    return cvar_fun


# limits and bounds
lb = [0, 0, 0.13, 0.11, 0, 0.05, 0.04]
ub = [1, 0.18, 0.13, 0.11, 1, 0.05, 0.04]

cvar_limit = [-0.34, -0.253, -0.501, -0.562]
```


```python
# optimization model formulation and execution
opt = RegretOptimizer(num_assets, num_scenarios, scenario_probability, sum_to_1=True)
opt.set_bounds(lb, ub)

obj_funcs = []
constraint_funcs = []
for m, c, limit in zip(main_scenarios, cvar_scenarios, cvar_limit):
    obj_funcs.append(make_max_returns_obj_fun(m))
    constraint_funcs.append(make_cvar_constraint_fun(c, limit))

opt.set_max_objective(obj_funcs)
opt.add_inequality_constraint(constraint_funcs)

final_weights = opt.optimize()
```

You can get the summary of the results. The first table show the optimal weight for each scenario. The second shows the proportion of each scenario and is only available when the approx option is set to `True`. The final table shows the *final* optimal weights.


```python
opt.summary()
```




<table class="simpletable">
<tr>
     <td></td>     <th>Scenario_1</th> <th>Scenario_2</th> <th>Scenario_3</th> <th>Scenario_4</th>
</tr>
<tr>
  <th>Asset_1</th>   <td>0.2499</td>     <td>0.3495</td>     <td>0.1003</td>     <td>0.0000</td>  
</tr>
<tr>
  <th>Asset_2</th>   <td>0.1800</td>     <td>0.0946</td>     <td>0.0000</td>     <td>0.0000</td>  
</tr>
<tr>
  <th>Asset_3</th>   <td>0.1300</td>     <td>0.1300</td>     <td>0.1300</td>     <td>0.1300</td>  
</tr>
<tr>
  <th>Asset_4</th>   <td>0.1100</td>     <td>0.1100</td>     <td>0.1100</td>     <td>0.1100</td>  
</tr>
<tr>
  <th>Asset_5</th>   <td>0.2401</td>     <td>0.2259</td>     <td>0.5697</td>     <td>0.6700</td>  
</tr>
<tr>
  <th>Asset_6</th>   <td>0.0500</td>     <td>0.0500</td>     <td>0.0500</td>     <td>0.0500</td>  
</tr>
<tr>
  <th>Asset_7</th>   <td>0.0400</td>     <td>0.0400</td>     <td>0.0400</td>     <td>0.0400</td>  
</tr>
</table>
<table class="simpletable">
<tr>
  <td></td>   <th>Scenario</th>  <th>Proportion (%)</th>
</tr>
<tr>
  <th>0</th> <td>Scenario_1</td>     <td>68.5900</td>   
</tr>
<tr>
  <th>1</th> <td>Scenario_2</td>     <td>0.0000</td>    
</tr>
<tr>
  <th>2</th> <td>Scenario_3</td>     <td>0.0000</td>    
</tr>
<tr>
  <th>3</th> <td>Scenario_4</td>     <td>47.5900</td>   
</tr>
</table>
<table class="simpletable">
<tr>
     <td></td>     <th>Weight</th>
</tr>
<tr>
  <th>Asset_1</th> <td>0.1714</td>
</tr>
<tr>
  <th>Asset_2</th> <td>0.1235</td>
</tr>
<tr>
  <th>Asset_3</th> <td>0.1510</td>
</tr>
<tr>
  <th>Asset_4</th> <td>0.1278</td>
</tr>
<tr>
  <th>Asset_5</th> <td>0.4835</td>
</tr>
<tr>
  <th>Asset_6</th> <td>0.0581</td>
</tr>
<tr>
  <th>Asset_7</th> <td>0.0465</td>
</tr>
</table>



The `PortfolioRegretOptimizer` contains a number of common optimization regimes with respect to regret optimization. We can apply the same optimization we did with the `maximize_returns()` method.


```python
from allopy import PortfolioRegretOptimizer

opt = PortfolioRegretOptimizer(main_scenarios,
                               cvar_scenarios,
                               scenario_probability,
                               rebalance=True,
                               sum_to_1=True,
                               time_unit="quarterly")

opt.set_bounds(lb, ub)
opt.maximize_returns(max_cvar=cvar_limit)
opt.summary()
```




<table class="simpletable">
<tr>
     <td></td>     <th>Scenario_1</th> <th>Scenario_2</th> <th>Scenario_3</th> <th>Scenario_4</th>
</tr>
<tr>
  <th>Asset_1</th>   <td>0.2499</td>     <td>0.3494</td>     <td>0.1003</td>     <td>0.0000</td>  
</tr>
<tr>
  <th>Asset_2</th>   <td>0.1800</td>     <td>0.0947</td>     <td>0.0000</td>     <td>0.0000</td>  
</tr>
<tr>
  <th>Asset_3</th>   <td>0.1300</td>     <td>0.1300</td>     <td>0.1300</td>     <td>0.1300</td>  
</tr>
<tr>
  <th>Asset_4</th>   <td>0.1100</td>     <td>0.1100</td>     <td>0.1100</td>     <td>0.1100</td>  
</tr>
<tr>
  <th>Asset_5</th>   <td>0.2401</td>     <td>0.2259</td>     <td>0.5697</td>     <td>0.6700</td>  
</tr>
<tr>
  <th>Asset_6</th>   <td>0.0500</td>     <td>0.0500</td>     <td>0.0500</td>     <td>0.0500</td>  
</tr>
<tr>
  <th>Asset_7</th>   <td>0.0400</td>     <td>0.0400</td>     <td>0.0400</td>     <td>0.0400</td>  
</tr>
</table>
<table class="simpletable">
<tr>
  <td></td>   <th>Scenario</th>  <th>Proportion (%)</th>
</tr>
<tr>
  <th>0</th> <td>Scenario_1</td>     <td>68.5900</td>   
</tr>
<tr>
  <th>1</th> <td>Scenario_2</td>     <td>0.0000</td>    
</tr>
<tr>
  <th>2</th> <td>Scenario_3</td>     <td>0.0000</td>    
</tr>
<tr>
  <th>3</th> <td>Scenario_4</td>     <td>47.5900</td>   
</tr>
</table>
<table class="simpletable">
<tr>
     <td></td>     <th>Weight</th>
</tr>
<tr>
  <th>Asset_1</th> <td>0.1714</td>
</tr>
<tr>
  <th>Asset_2</th> <td>0.1235</td>
</tr>
<tr>
  <th>Asset_3</th> <td>0.1510</td>
</tr>
<tr>
  <th>Asset_4</th> <td>0.1278</td>
</tr>
<tr>
  <th>Asset_5</th> <td>0.4835</td>
</tr>
<tr>
  <th>Asset_6</th> <td>0.0581</td>
</tr>
<tr>
  <th>Asset_7</th> <td>0.0465</td>
</tr>
</table>



4361c1594c7487cd70be0d2886a5d13d235a50e9
ipynb
Jupyter Notebook
examples/Regret Optimization.ipynb
wangcj05/allopy
0d97127e5132df1449283198143994b45fb11214
MIT
2021-04-06T04:33:03.000Z
2021-04-06T04:33:03.000Z
examples/Regret Optimization.ipynb
wangcj05/allopy
0d97127e5132df1449283198143994b45fb11214
MIT
examples/Regret Optimization.ipynb
wangcj05/allopy
0d97127e5132df1449283198143994b45fb11214
MIT
Qwen/Qwen-72B
1. YES
2. YES