# Chap2: An Asset Selling Problem

The chapter to answer basic question: **When to sell an asset** 

In this case, we use a simple class of policies known as:  *policy function approximations*.

**Mathematical Model**

* **State Variable**: $S_t$ that captures all the information we need ...($R_t$, $I_t$, $B_t$)
* **Decsion Variable** : $x_t$ where $x_0$ is the design decision, while $x_t$ for t>0 represents control varilables.
* **Exogenous Information**: The variable $W_t$ captures information that first becomes available between $t-1$ and $t$ from outside of our system.
* **Transition Fucntion**:
$$
S_{t+1} = S^M(S_t, x_t,W_{t+1})
$$
* **Objective Function**: When there are elements of the *problem* that depend on the state variables, we represent costs/contribution as:

$$
C(S_{t},x_{t}) \ or \ C(S_t,x_t, W_{t+1})
$$

**Set of learning problems** : where we are trying to learn about a function which is not itself a function of state variable, it means:

This means the state variable consists purely of belief state $B_t$.

## Narrative

We are holding a block of shares of stock, looking for oppurtunity to sell.If we sell at time t, we receive a price that varies according to some random process over time. Once we sell the stock, the process stops.

## Basic Model

### State Variable:

Two state variables:
* The **Physical State** which says whether or not we are still holding the assest
* The **Information State** the price of stock 

* The physical State:

$$
  R_t=\left\{
  \begin{array}{@{}ll@{}}
    1, & \text{if we are holding the stock at time t}\  \\
    0, & \text{if we are not longer holding the stock at time t}
  \end{array}\right. 
$$

If we sell the stock, we receive the price per share $p_t$. This means our state variable is:
$$
S_t=(R_t, p_t)
$$

### Decision Variable:

$$
\begin{equation}
  x_t=\left\{
  \begin{array}{@{}ll@{}}
    1, & \text{if we sell the stock at time t}\  \\
    0, & \text{if we do not sell the stock at time t}
  \end{array}\right.
\end{equation} 
$$
we only can sell stock if we hold it,
$$
x_t < R_t
$$

#### Policy:

What is the policy: <br> **which is going to define how we make decision**

$$
X^{\pi}(S_t)
$$

Example policy is we might be to sell if the price drops the below some limit point, thus we could write:

$$
\begin{equation}
  X^{sell-low}(S_t|\theta^{low})=\left\{
  \begin{array}{@{}ll@{}}
    1, & \text{if}\ p_t<\theta^{low}  \ \text{and}\ R_t=1  \\
    1, & \text{if}\ t= T\ \text{and} \ R_t = 1 \\
    0, & \text{otherwise}
  \end{array}\right.
\end{equation} 
$$

### Exogenous Function

The only random process in our basic model is the change in price. Alternatively, naturally we simply write $W_t$ be the new price, in which we would write:

$$
W_{t+1} = (p_{t+1})
$$

### Transition Function

Is the equation that describe how the states evolves. The transition equation is:

$$
R_{t+1} = R_{t} - x_{t}
$$

How the price evolves over time. 

$$p_{t+1} = p_t  + \hat{p}_{t+1}$$

**Transition Function**

$$
S_{t+1} = S^M(S_t, X^{\pi}(S_t), W_{t+1})
$$

If we use our policy $X^{\pi}(S_t)$, and if we choose sample path $\omega$, that determine the sequence $W_1,W_2,W_3,...,W_T$...than the simulation of process can be written as:

$$
(S_0,x_0 = X^{\pi}(S_0)), W_1(\omega), S_1, x_1 = X^{\pi}(S_1), W_(\omega),..., x_{T-1}, W_T(\omega),S_T)
$$

### Objective Function
The performance metric is how much we earn from selling our stock.

$$
C(S_t,x_t) = p_tx_t
$$

We now can formulate the optimizatio problem:

$$
max \ \sum_{t=0}^{T-1} p_tx_t \\
x_0,x_1,...,x_{T-1}
$$

Constraints of the Optimization:

$$
\sum_{t=0}^{T-1} x_t = 1, \\
x_t \leq 1, \\
x_t \geq 0,
$$

**Including Uncertainty**:

We are simulating a policy following a sample path $\omega$ of price $p_1(\omega), p_2(\omega),...$

$$
S_{t+1}(\omega) = S^{M}(S_t(\omega), X^{\pi}(S_t(\omega)), W_{t+1}(\omega))
$$

If we follow policy $\pi$ along this sample path, we can compute the performance:

$$
\hat{F}^{\pi}(\omega) = \sum_{t=0}^{T-1} p_t(\omega)X^{\pi}(S_t(\omega))
$$

This is for one sample path . We can simulate over sample of $N$ sampels ($\omega^{1},...,\omega^n,...\omega^N$), and take average:

$$
\overline{F}^{\pi} = \frac{1}{N} \sum_{n=1}^{N}\hat{F}^{\pi}(\omega^n)
$$

Finally, we write out optimization problem in terms of finding the best policy, which we can write:

$$
max \ \ \overline{F}^{\pi} \\
x \in \Pi
$$

## Designing Policy

### Sell_low Policy:
In this policy we *sell* at that the price lower than, $\theta^{sell-low}$.

$$
  X^{sell-low}(S_t|\theta^{low}) =\left\{
  \begin{array}{@{}ll@{}}
    1  & if \ p_t < \theta^{low} \ and \ R_t=1  \\
    1, & t=T \ R_t=1 \\
    0, & otherwise
  \end{array}\right.
$$

### High_Low policy:

In this policy , we will sell the asset if the price jumps *too high** or ** too low**:

$$
  X^{high-low}(S_t|\theta^{high-low}) =\left\{
  \begin{array}{@{}ll@{}}
    1  & if \ p_t < \theta^{low} \ or \ p_t > \theta^{high}   \\
    1, & t=T \ and \ R_t=1 \\
    0, & otherwise
  \end{array}\right.
$$

### Track Policy:

Perhaps we just want to sell when the stock rises above a tracking signal. Tod this, first create a smoothed estimate of price:

$$\overline{p}_t = (1- \alpha)\overline{p}_{t-1} + \alpha \hat{p}_t$$

Now consider a tracking policy that we might write:

$$
  X^{track}(S_t | \theta^{track}) =\left\{
  \begin{array}{@{}ll@{}}
    1  & if \ p_t > \overline{p}_t + \theta^{track} \    \\
    1, & t=T \ and \ R_t=1 \\
    0, & otherwise
  \end{array}\right.
$$

For this policy, we are going to need to tweak our model, because we now $\overline{p}_{t}$ in order to make decision. This means we would now write our state as:

$$
S_t = (R_t, p_t, \overline{p}_t)
$$

**Classes of Policies**:

$F = \{"sell-low","high-low","track"\}$

**Parameter for each policy**:

$\theta^f \in F$

**Search over policies**:

$\pi \in \Pi$

## Coding Part:

### Import libraries


```python
import pandas as pd
import numpy as np
from collections import namedtuple
import matplotlib.pyplot as plt
from copy import copy
import math
import seaborn as sns
sns.set_theme(style="darkgrid")
sns.set(rc={'figure.figsize':(12,8)})
```

### Change the directory to to get module


```python
import os
os.chdir("/home/peyman/Documents/PhD_UiS/seqdec_powell_repo/Chap2_Assett_Selling/function")
os.getcwd()
```




    '/home/peyman/Documents/PhD_UiS/seqdec_powell_repo/Chap2_Assett_Selling/function'



### Import modules from function


```python
from AssetSellingModel import AssetSellingModel
from AssetSellingPolicy import AssetSellingPolicy
```

### Change the directory to to get parameters


```python
import os
os.chdir("/home/peyman/Documents/PhD_UiS/seqdec_powell_repo/Chap2_Assett_Selling/data")
os.getcwd()
```




    '/home/peyman/Documents/PhD_UiS/seqdec_powell_repo/Chap2_Assett_Selling/data'



### Policy Parameters:


```python
sheet1= pd.read_excel("asset_selling_policy_parameters_edit.xlsx", sheet_name="Sheet1", usecols=["policy", "param1","param2"])
```


```python
sheet1
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>policy</th>
      <th>param1</th>
      <th>param2</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>sell_low</td>
      <td>2</td>
      <td>NaN</td>
    </tr>
    <tr>
      <th>1</th>
      <td>high_low</td>
      <td>4</td>
      <td>10.0</td>
    </tr>
    <tr>
      <th>2</th>
      <td>track</td>
      <td>0</td>
      <td>4.0</td>
    </tr>
  </tbody>
</table>
</div>




```python
params = zip(sheet1["param1"], sheet1["param2"])
param_list = list(params)
param_list
```




    [(2, nan), (4, 10.0), (0, 4.0)]



### Full grid Policy Parameters:


```python
sheet2 = pd.read_excel("asset_selling_policy_parameters_edit.xlsx", sheet_name="Sheet2")
```


```python
sheet2.dtypes
```




    low_min             int64
    low_max             int64
    high_min            int64
    high_max            int64
    increment_size    float64
    dtype: object




```python
dtype(sheet2)
```

### Parameters of the Dynamic Model: 


```python
sheet3 = pd.read_excel("asset_selling_policy_parameters_edit.xlsx", sheet_name="Sheet3")
sheet3
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>Policy</th>
      <th>TimeHorizon</th>
      <th>DiscountFactor</th>
      <th>InitialPrice</th>
      <th>InitialBias</th>
      <th>UpStep</th>
      <th>DownStep</th>
      <th>Variance</th>
      <th>Iterations</th>
      <th>PrintStep</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>track</td>
      <td>40</td>
      <td>0.99</td>
      <td>16</td>
      <td>Up</td>
      <td>1</td>
      <td>-1</td>
      <td>2</td>
      <td>10</td>
      <td>40</td>
    </tr>
  </tbody>
</table>
</div>



### Bias Term:


```python
biasdf = pd.read_excel("asset_selling_policy_parameters_edit.xlsx", sheet_name="Sheet4")
biasdf_edit=biasdf.set_index('additional')
biasdf_edit
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>Up</th>
      <th>Neutral</th>
      <th>Down</th>
    </tr>
    <tr>
      <th>additional</th>
      <th></th>
      <th></th>
      <th></th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>Up</th>
      <td>0.9</td>
      <td>0.1</td>
      <td>0.0</td>
    </tr>
    <tr>
      <th>Neutral</th>
      <td>0.2</td>
      <td>0.6</td>
      <td>0.2</td>
    </tr>
    <tr>
      <th>Down</th>
      <td>0.0</td>
      <td>0.1</td>
      <td>0.9</td>
    </tr>
  </tbody>
</table>
</div>



### Track Policy:


```python
policy_selected = sheet3['Policy'][0]
print("The selected policy is: {}".format(policy_selected))

T = sheet3['TimeHorizon'][0]
print("The T value is: {}".format(T))

initPrice = sheet3['InitialPrice'][0]
print("The initial price is: {}".format(initPrice))

initBias = sheet3['InitialBias'][0]
print("The initial Bias is: {}".format(initBias))    

exog_params = {'UpStep':sheet3['UpStep'][0],'DownStep':sheet3['DownStep'][0],'Variance':sheet3['Variance'][0],'biasdf'
               :biasdf_edit}
print("The exog_params is: {}".format(exog_params))    
    
nIterations = sheet3['Iterations'][0]
print("The nIteration is: {}".format(nIterations))    

printStep = sheet3['PrintStep'][0]
print("The printStep is: {}".format(printStep))    

printIterations = [0]
print("The printIterations is: {}".format(printIterations))    

printIterations.extend(list(reversed(range(nIterations-1,0,-printStep))))
```

    The selected policy is: track
    The T value is: 40
    The initial price is: 16
    The initial Bias is: Up
    The exog_params is: {'UpStep': 1, 'DownStep': -1, 'Variance': 2, 'biasdf':              Up  Neutral  Down
    additional                    
    Up          0.9      0.1   0.0
    Neutral     0.2      0.6   0.2
    Down        0.0      0.1   0.9}
    The nIteration is: 10
    The printStep is: 40
    The printIterations is: [0]



```python
policy_names = ['sell_low', 'high_low', 'track']
print("policy_names are: {}".format(policy_names))    

state_names = ['price', 'resource','bias']
print("state_names are: {}".format(state_names))    

init_state = {'price': initPrice, 'resource': 1,'bias':initBias}
print("init_state are: {}".format(init_state))    

decision_names = ['sell', 'hold']
print("decision_names are: {}".format(decision_names))    

```

    policy_names are: ['sell_low', 'high_low', 'track']
    state_names are: ['price', 'resource', 'bias']
    init_state are: {'price': 16, 'resource': 1, 'bias': 'Up'}
    decision_names are: ['sell', 'hold']



```python
biasdf = exog_params["biasdf"]
print("the biasdf_edit is \n .{}".format(biasdf))
print(type(biasdf))
biasdf= biasdf.cumsum(axis =1)
print("the biasdf_edit is \n .{}".format(biasdf))

```

    the biasdf_edit is 
     .             Up  Neutral  Down
    additional                    
    Up          0.9      0.1   0.0
    Neutral     0.2      0.6   0.2
    Down        0.0      0.1   0.9
    <class 'pandas.core.frame.DataFrame'>
    the biasdf_edit is 
     .             Up  Neutral  Down
    additional                    
    Up          0.9      1.0   1.0
    Neutral     0.2      0.8   1.0
    Down        0.0      0.1   1.0



```python
M = AssetSellingModel(state_names, decision_names, init_state,exog_params,T)
```


```python
P = AssetSellingPolicy(M, policy_names)
```

### Policy Evaluation


```python
t = 0
prev_price = init_state['price']
```


```python
policy_info = {'sell_low': param_list[0],
                   'high_low': param_list[1],
                   'track': param_list[2] + (prev_price,)}
```


```python
policy_info
```




    {'sell_low': (2, nan), 'high_low': (4, 10.0), 'track': (0, 4.0, 16)}




```python
policy_selected
```




    'track'




```python
#if (not policy_selected =='full_grid'):
print("Selected policy {}, time horizon {}, initial price {} and number of iterations {}".format(policy_selected,T,initPrice,nIterations))
contribution_iterations=[P.run_policy(param_list, policy_info, policy_selected, t) for ite in list(range(nIterations))]

contribution_iterations = pd.Series(contribution_iterations)
print("Contribution per iteration: ")
print(contribution_iterations)
cum_avg_contrib = contribution_iterations.expanding().mean()
print("Cumulative average contribution per iteration: ")
print(cum_avg_contrib)
        
#plotting the results

#fig, axsubs = plt.subplots(1,2,sharex=True,sharey=True)
#fig.suptitle("Asset selling using policy {} with parameters {} and T {}".format(policy_selected,policy_info[policy_selected],T) )
#i = np.arange(0, nIterations, 1)
        
#axsubs[0].plot(i, cum_avg_contrib, 'g')
#axsubs[0].set_title('Cumulative average contribution')
          
#axsubs[1].plot(i, contribution_iterations, 'g')
#axsubs[1].set_title('Contribution per iteration')
        
    
# Create a big subplot
#ax = fig.add_subplot(111, frameon=False)
# hide tick and tick label of the big axes
#plt.tick_params(labelcolor='none', top=False, bottom=False, left=False, right=False)

#ax.set_ylabel('USD', labelpad=0) # Use argument `labelpad` to move label downwards.
#ax.set_xlabel('Iterations', labelpad=10)
        
#plt.show()        

```

    Selected policy track, time horizon 40, initial price 16 and number of iterations 10
    time=0, obj=0.0, s.resource=1, s.price=16, x=Decision(sell=0, hold=1)
    coin  0.4139840589481404  curr_bias  Up  new_bias  Up
    smoothed_price = 19.275851851363697
    time=1, obj=0.0, s.resource=1, s.price=16.818962962840924, x=Decision(sell=0, hold=1)
    coin  0.8487800961190239  curr_bias  Up  new_bias  Up
    smoothed_price = 12.919965111987779
    time=2, obj=0.0, s.resource=1, s.price=15.844213500127637, x=Decision(sell=1, hold=0)
    coin  0.45203620488154816  curr_bias  Up  new_bias  Up
    obj=15.844213500127637, state.resource=0
    time=0, obj=0.0, s.resource=1, s.price=16, x=Decision(sell=0, hold=1)
    coin  0.2720800519673423  curr_bias  Up  new_bias  Up
    smoothed_price = 29.300677434463253
    time=1, obj=0.0, s.resource=1, s.price=19.325169358615813, x=Decision(sell=0, hold=1)
    coin  0.905534061916374  curr_bias  Up  new_bias  Neutral
    smoothed_price = 11.678950072957633
    time=2, obj=0.0, s.resource=1, s.price=17.41361453720127, x=Decision(sell=1, hold=0)
    coin  0.3755268704762992  curr_bias  Neutral  new_bias  Neutral
    obj=17.41361453720127, state.resource=0
    time=0, obj=0.0, s.resource=1, s.price=16, x=Decision(sell=0, hold=1)
    coin  0.09168318648059992  curr_bias  Up  new_bias  Up
    smoothed_price = 30.17968207409804
    time=1, obj=0.0, s.resource=1, s.price=19.54492051852451, x=Decision(sell=0, hold=1)
    coin  0.9746533545172668  curr_bias  Up  new_bias  Neutral
    smoothed_price = 9.10715798582386
    time=2, obj=0.0, s.resource=1, s.price=16.93547988534935, x=Decision(sell=1, hold=0)
    coin  0.845463715621411  curr_bias  Neutral  new_bias  Down
    obj=16.93547988534935, state.resource=0
    time=0, obj=0.0, s.resource=1, s.price=16, x=Decision(sell=0, hold=1)
    coin  0.7718855570743978  curr_bias  Up  new_bias  Up
    smoothed_price = 13.590645854568095
    time=1, obj=0.0, s.resource=1, s.price=15.397661463642024, x=Decision(sell=1, hold=0)
    coin  0.058599417274914245  curr_bias  Up  new_bias  Up
    obj=15.397661463642024, state.resource=0
    time=0, obj=0.0, s.resource=1, s.price=16, x=Decision(sell=0, hold=1)
    coin  0.5630457710972723  curr_bias  Up  new_bias  Up
    smoothed_price = 14.404641398215574
    time=1, obj=0.0, s.resource=1, s.price=15.601160349553894, x=Decision(sell=1, hold=0)
    coin  0.6337684851009756  curr_bias  Up  new_bias  Up
    obj=15.601160349553894, state.resource=0
    time=0, obj=0.0, s.resource=1, s.price=16, x=Decision(sell=0, hold=1)
    coin  0.32637382885748367  curr_bias  Up  new_bias  Up
    smoothed_price = 29.23727426734112
    time=1, obj=0.0, s.resource=1, s.price=19.30931856683528, x=Decision(sell=0, hold=1)
    coin  0.010789716774533442  curr_bias  Up  new_bias  Up
    smoothed_price = 24.356969142318995
    time=2, obj=0.0, s.resource=1, s.price=20.57123121070621, x=Decision(sell=0, hold=1)
    coin  0.10655404328873319  curr_bias  Up  new_bias  Up
    smoothed_price = 32.917325150498336
    time=3, obj=0.0, s.resource=1, s.price=23.65775469565424, x=Decision(sell=0, hold=1)
    coin  0.2744609047476718  curr_bias  Up  new_bias  Up
    smoothed_price = 20.209678891873452
    time=4, obj=0.0, s.resource=1, s.price=22.795735744709045, x=Decision(sell=1, hold=0)
    coin  0.7876352836985036  curr_bias  Up  new_bias  Up
    obj=22.795735744709045, state.resource=0
    time=0, obj=0.0, s.resource=1, s.price=16, x=Decision(sell=0, hold=1)
    coin  0.5732916894272274  curr_bias  Up  new_bias  Up
    smoothed_price = 27.318028871373315
    time=1, obj=0.0, s.resource=1, s.price=18.82950721784333, x=Decision(sell=0, hold=1)
    coin  0.25230738226248894  curr_bias  Up  new_bias  Up
    smoothed_price = 32.205300235020715
    time=2, obj=0.0, s.resource=1, s.price=22.173455472137675, x=Decision(sell=0, hold=1)
    coin  0.34703831077049807  curr_bias  Up  new_bias  Up
    smoothed_price = 22.184929865392462
    time=3, obj=0.0, s.resource=1, s.price=22.176324070451372, x=Decision(sell=0, hold=1)
    coin  0.020311694923608736  curr_bias  Up  new_bias  Up
    smoothed_price = 36.75498055259952
    time=4, obj=0.0, s.resource=1, s.price=25.82098819098841, x=Decision(sell=0, hold=1)
    coin  0.5773741485181105  curr_bias  Up  new_bias  Up
    smoothed_price = 19.512647218984497
    time=5, obj=0.0, s.resource=1, s.price=24.243902947987433, x=Decision(sell=1, hold=0)
    coin  0.9296106478220897  curr_bias  Up  new_bias  Neutral
    obj=24.243902947987433, state.resource=0
    time=0, obj=0.0, s.resource=1, s.price=16, x=Decision(sell=0, hold=1)
    coin  0.4326408307147147  curr_bias  Up  new_bias  Up
    smoothed_price = 29.78638009471284
    time=1, obj=0.0, s.resource=1, s.price=19.44659502367821, x=Decision(sell=0, hold=1)
    coin  0.9844689697230611  curr_bias  Up  new_bias  Neutral
    smoothed_price = 9.047636235056132
    time=2, obj=0.0, s.resource=1, s.price=16.84685532652269, x=Decision(sell=1, hold=0)
    coin  0.8262653217404742  curr_bias  Neutral  new_bias  Down
    obj=16.84685532652269, state.resource=0
    time=0, obj=0.0, s.resource=1, s.price=16, x=Decision(sell=0, hold=1)
    coin  0.4433318705856356  curr_bias  Up  new_bias  Up
    smoothed_price = 4.415644002958089
    time=1, obj=0.0, s.resource=1, s.price=13.103911000739522, x=Decision(sell=1, hold=0)
    coin  0.6753561026946597  curr_bias  Up  new_bias  Up
    obj=13.103911000739522, state.resource=0
    time=0, obj=0.0, s.resource=1, s.price=16, x=Decision(sell=0, hold=1)
    coin  0.8662383958568906  curr_bias  Up  new_bias  Up
    smoothed_price = 41.35203449737756
    time=1, obj=0.0, s.resource=1, s.price=22.33800862434439, x=Decision(sell=0, hold=1)
    coin  0.9282091767826981  curr_bias  Up  new_bias  Neutral
    smoothed_price = 24.822358317891954
    time=2, obj=0.0, s.resource=1, s.price=22.959096047731283, x=Decision(sell=0, hold=1)
    coin  0.3886605993398763  curr_bias  Neutral  new_bias  Neutral
    smoothed_price = 28.44497845663615
    time=3, obj=0.0, s.resource=1, s.price=24.3305666499575, x=Decision(sell=0, hold=1)
    coin  0.7922183625790814  curr_bias  Neutral  new_bias  Neutral
    smoothed_price = 8.076789104517587
    time=4, obj=0.0, s.resource=1, s.price=20.26712226359752, x=Decision(sell=1, hold=0)
    coin  0.7101424377623707  curr_bias  Neutral  new_bias  Neutral
    obj=20.26712226359752, state.resource=0
    Contribution per iteration: 
    0    15.844214
    1    17.413615
    2    16.935480
    3    15.397661
    4    15.601160
    5    22.795736
    6    24.243903
    7    16.846855
    8    13.103911
    9    20.267122
    dtype: float64
    Cumulative average contribution per iteration: 
    0    15.844214
    1    16.628914
    2    16.731103
    3    16.397742
    4    16.238426
    5    17.331311
    6    18.318824
    7    18.134828
    8    17.575837
    9    17.844966
    dtype: float64


## Plot Results


```python
df_cont_per_ite = pd.DataFrame({"iteration_number": list(range(nIterations)), "contribution_iterations":contribution_iterations})
sns.lineplot(data=df_cont_per_ite, x= "iteration_number", y="contribution_iterations")
```

## Full grid Search


```python
policy_selected =='full_grid'
# obtain the theta values to carry out a full grid search
grid_search_theta_values = P.grid_search_theta_values(sheet2['low_min'], sheet2['low_max'], sheet2['high_min'], sheet2['high_max'], sheet2['increment_size'])
# use those theta values to calculate corresponding contribution values
```


```python
len(grid_search_theta_values[0])
```




    25




```python
%%capture
policy_selected =='full_grid'
# obtain the theta values to carry out a full grid search
grid_search_theta_values = P.grid_search_theta_values(sheet2['low_min'], sheet2['low_max'], sheet2['high_min'], sheet2['high_max'], sheet2['increment_size'])
# use those theta values to calculate corresponding contribution values
        
contribution_iterations = [P.vary_theta(param_list, policy_info, "high_low", t, grid_search_theta_values[0]) for ite in list(range(nIterations))]

contribution_iterations_arr = np.array(contribution_iterations)
cum_sum_contrib = contribution_iterations_arr.cumsum(axis=0)
nElem = np.arange(1,cum_sum_contrib.shape[0]+1).reshape((cum_sum_contrib.shape[0],1))
cum_avg_contrib=cum_sum_contrib/nElem
#print("cum_avg_contrib")
#print(cum_avg_contrib)
    
        # plot those contribution values on a heat map
P.plot_heat_map_many(cum_avg_contrib, grid_search_theta_values[1], grid_search_theta_values[2], printIterations)
```


```python
print("cum_avg_contrib")
print(cum_avg_contrib)
```

    cum_avg_contrib
    [[18.47367058 20.02044278 23.71558344 24.04911983 25.64664546 18.46354228
      19.9971443  23.4944788   8.82900535 27.39145963 18.41058556 20.46051754
      21.80163486 25.30872441 24.51162131 18.97575447 13.24130896 13.83837746
      14.65914641 24.51858946 22.05931859 19.54746608 23.29884974 13.64768737
      26.03581103]
     [19.69100318 20.37305553 23.23858313 23.76467669 25.84529101 18.23380197
      16.47730129 17.92318102 10.78275225 26.04158751 15.6912722  20.2448214
      17.87449738 19.47026892 24.74667952 16.78197067 16.76903764 14.10385934
      18.63771033 24.42103201 22.06444139 17.12797049 23.73444868 14.77986095
      20.09595631]
     [20.69994569 16.83950464 22.74294447 19.56126436 20.52459656 18.47919587
      15.11744387 16.20921065 15.00940661 25.51281981 13.80659895 20.25218014
      19.14019475 17.30568466 21.05307615 17.63382212 18.0583466  13.60198246
      20.07523909 21.21275022 20.9849638  16.32976729 20.95614325 15.05129774
      17.81028782]
     [20.18013059 17.80921563 22.83681797 20.66946341 21.67762855 18.64562133
      16.51153134 15.06668557 13.78071309 25.56287196 13.84389485 20.76422187
      19.73595235 18.96932916 21.87421549 18.05258783 18.52434386 15.50891778
      18.55859893 22.63139844 19.56249686 16.15022656 19.50527058 15.13432515
      17.22395803]
     [20.20517103 18.45097802 22.52135779 18.84335523 19.48817699 18.84645425
      17.12043862 16.58986652 15.60289554 25.29226768 14.71355315 19.09216357
      20.18432852 17.72401005 20.26431727 18.53066645 18.82637145 15.34413496
      19.59040908 21.06536502 19.48544474 16.04541057 20.42913257 14.70689882
      16.94170926]
     [19.9262659  18.67286269 20.75302403 19.71022848 18.21367716 19.19238006
      17.51795722 17.68624384 16.78180395 22.78350727 15.48395009 19.30591146
      19.01314863 17.08034736 20.95328276 18.66106818 18.10879285 14.98659596
      20.39322653 20.0443103  19.9037171  16.94296788 19.57782471 14.52283041
      18.59472501]
     [19.8137429  18.9086271  20.84268636 18.39076378 17.23701861 19.36730219
      18.20099241 18.23276394 17.66079438 21.17920167 16.1442936  18.43430402
      19.43508475 17.9552651  21.50933229 18.57660873 18.42222031 16.25785103
      21.01294484 19.29514058 19.22370624 16.51033466 18.79126479 14.48088114
      17.95088124]
     [19.62653733 17.82007762 20.94883455 17.34353318 18.63876886 19.81500259
      17.46284492 18.58482107 16.70445351 20.04914113 16.41550734 18.8476955
      19.63813038 17.28721213 20.53508951 18.99281685 17.86891367 17.53757442
      20.159934   20.00713271 18.72468824 16.88411792 18.1474659  14.48123896
      18.91527785]
     [19.63648073 18.1478174  21.24734214 16.45992462 19.69289125 19.82953391
      16.79495559 19.30543893 16.28062686 19.00743174 16.61822087 18.12487532
      19.85294926 16.80365162 19.69257256 19.14099563 18.57933051 18.03082425
      20.63793609 19.24262662 18.34680726 16.33745161 17.85696122 14.5706945
      18.48843037]
     [19.63661504 18.36680492 21.42992756 17.12582509 20.30051109 19.84620636
      17.08335876 19.55338491 15.79131993 18.31310674 17.06211989 18.52285707
      20.03049054 17.48311031 20.27369434 19.18396337 18.20648271 18.56249637
      19.88403096 18.75795701 18.08166846 16.30207016 18.1781855  14.65780872
      19.2417764 ]]



```python
P.plot_heat_map_many(cum_avg_contrib, grid_search_theta_values[1], grid_search_theta_values[2], printIterations)
```


```python
type(cum_avg_contrib)
```




    numpy.ndarray




```python
grid_search_theta_values[1]
```




    array([[12.],
           [13.],
           [14.],
           [15.],
           [16.]])




```python
ax = sns.heatmap(cum_avg_contrib)
```


```python
len(cum_avg_contrib)
```




    10



ed422414ceefaf6f7a4d1b5ec4fa26a44789c7b1
ipynb
Jupyter Notebook
Chap2_Assett_Selling/An_Asset_Selling_Problem.ipynb
Peymankor/seqdec_powell_repo
d3a03399ca7762821e80988d112f98bad5adefd8
MIT
2021-04-21T19:21:53.000Z
2021-04-21T19:21:53.000Z
Chap2_Assett_Selling/An_Asset_Selling_Problem.ipynb
Efsilvaa/seqdec_powell_repo
d3a03399ca7762821e80988d112f98bad5adefd8
MIT
Chap2_Assett_Selling/An_Asset_Selling_Problem.ipynb
Efsilvaa/seqdec_powell_repo
d3a03399ca7762821e80988d112f98bad5adefd8
MIT
2021-04-21T19:21:45.000Z
2021-06-24T18:19:17.000Z
Qwen/Qwen-72B
1. YES
2. YES