# Tutorial: optimal binning with continuous target

## Basic

To get us started, let's load a well-known dataset from the UCI repository and transform the data into a ``pandas.DataFrame``.


```python
import pandas as pd
from sklearn.datasets import load_boston
```


```python
data = load_boston()
df = pd.DataFrame(data.data, columns=data.feature_names)
```

We choose a variable to discretize and the continuous target.


```python
variable = "LSTAT"
x = df[variable].values
y = data.target
```

Import and instantiate an ``ContinuousOptimalBinning`` object class. We pass the variable name and its data type.


```python
from optbinning import ContinuousOptimalBinning
```


```python
optb = ContinuousOptimalBinning(name=variable, dtype="numerical")
```

We fit the optimal binning object with arrays ``x`` and ``y``.


```python
optb.fit(x, y)
```




    ContinuousOptimalBinning(cat_cutoff=None, dtype='numerical', max_bin_size=None,
                             max_n_bins=None, max_n_prebins=20, max_pvalue=None,
                             max_pvalue_policy='consecutive', min_bin_size=None,
                             min_mean_diff=0, min_n_bins=None, min_prebin_size=0.05,
                             monotonic_trend='auto', name='LSTAT',
                             outlier_detector=None, outlier_params=None,
                             prebinning_method='cart', special_codes=None,
                             split_digits=None, time_limit=100, user_splits=None,
                             user_splits_fixed=None, verbose=False)



You can check if an optimal solution has been found via the ``status`` attribute:


```python
optb.status
```




    'OPTIMAL'



You can also retrieve the optimal split points via the ``splits`` attribute:


```python
optb.splits
```




    array([ 4.6500001 ,  5.49499989,  6.86500001,  9.7249999 , 11.67499971,
           13.0999999 , 16.08500004, 19.89999962, 23.31500053])



#### The binning table

The optimal binning algorithms return a binning table; a binning table displays the binned data and several metrics for each bin. Class ``ContinuousOptimalBinning`` returns an object ``ContinuousBinningTable`` via the ``binning_table`` attribute.


```python
binning_table = optb.binning_table
```


```python
type(binning_table)
```




    optbinning.binning.binning_statistics.ContinuousBinningTable



The `binning_table` is instantiated, but not built. Therefore, the first step is to call the method `build`, which returns a ``pandas.DataFrame``.


```python
binning_table.build()
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>Bin</th>
      <th>Count</th>
      <th>Count (%)</th>
      <th>Sum</th>
      <th>Std</th>
      <th>Mean</th>
      <th>Min</th>
      <th>Max</th>
      <th>Zeros count</th>
      <th>WoE</th>
      <th>IV</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>[-inf, 4.65)</td>
      <td>50</td>
      <td>0.098814</td>
      <td>1985.9</td>
      <td>8.19865</td>
      <td>39.718000</td>
      <td>22.8</td>
      <td>50.0</td>
      <td>0</td>
      <td>17.1852</td>
      <td>1.698142</td>
    </tr>
    <tr>
      <th>1</th>
      <td>[4.65, 5.49)</td>
      <td>28</td>
      <td>0.055336</td>
      <td>853.2</td>
      <td>6.12354</td>
      <td>30.471429</td>
      <td>21.9</td>
      <td>50.0</td>
      <td>0</td>
      <td>7.93862</td>
      <td>0.439291</td>
    </tr>
    <tr>
      <th>2</th>
      <td>[5.49, 6.87)</td>
      <td>45</td>
      <td>0.088933</td>
      <td>1188.6</td>
      <td>5.13626</td>
      <td>26.413333</td>
      <td>20.6</td>
      <td>48.8</td>
      <td>0</td>
      <td>3.88053</td>
      <td>0.345106</td>
    </tr>
    <tr>
      <th>3</th>
      <td>[6.87, 9.72)</td>
      <td>89</td>
      <td>0.175889</td>
      <td>2274.9</td>
      <td>6.84525</td>
      <td>25.560674</td>
      <td>11.9</td>
      <td>50.0</td>
      <td>0</td>
      <td>3.02787</td>
      <td>0.532570</td>
    </tr>
    <tr>
      <th>4</th>
      <td>[9.72, 11.67)</td>
      <td>49</td>
      <td>0.096838</td>
      <td>1057.9</td>
      <td>2.99484</td>
      <td>21.589796</td>
      <td>15.0</td>
      <td>31.0</td>
      <td>0</td>
      <td>-0.94301</td>
      <td>0.091319</td>
    </tr>
    <tr>
      <th>5</th>
      <td>[11.67, 13.10)</td>
      <td>35</td>
      <td>0.069170</td>
      <td>697.5</td>
      <td>2.59214</td>
      <td>19.928571</td>
      <td>14.5</td>
      <td>27.9</td>
      <td>0</td>
      <td>-2.60423</td>
      <td>0.180135</td>
    </tr>
    <tr>
      <th>6</th>
      <td>[13.10, 16.09)</td>
      <td>66</td>
      <td>0.130435</td>
      <td>1289.9</td>
      <td>3.54171</td>
      <td>19.543939</td>
      <td>10.2</td>
      <td>30.7</td>
      <td>0</td>
      <td>-2.98887</td>
      <td>0.389852</td>
    </tr>
    <tr>
      <th>7</th>
      <td>[16.09, 19.90)</td>
      <td>69</td>
      <td>0.136364</td>
      <td>1129.3</td>
      <td>3.60795</td>
      <td>16.366667</td>
      <td>8.3</td>
      <td>27.5</td>
      <td>0</td>
      <td>-6.16614</td>
      <td>0.840837</td>
    </tr>
    <tr>
      <th>8</th>
      <td>[19.90, 23.32)</td>
      <td>28</td>
      <td>0.055336</td>
      <td>368.4</td>
      <td>3.91284</td>
      <td>13.157143</td>
      <td>5.0</td>
      <td>21.7</td>
      <td>0</td>
      <td>-9.37566</td>
      <td>0.518811</td>
    </tr>
    <tr>
      <th>9</th>
      <td>[23.32, inf)</td>
      <td>47</td>
      <td>0.092885</td>
      <td>556.0</td>
      <td>4.00659</td>
      <td>11.829787</td>
      <td>5.0</td>
      <td>23.7</td>
      <td>0</td>
      <td>-10.703</td>
      <td>0.994154</td>
    </tr>
    <tr>
      <th>10</th>
      <td>Special</td>
      <td>0</td>
      <td>0.000000</td>
      <td>0.0</td>
      <td>NaN</td>
      <td>0.000000</td>
      <td>NaN</td>
      <td>NaN</td>
      <td>0</td>
      <td>-22.5328</td>
      <td>0.000000</td>
    </tr>
    <tr>
      <th>11</th>
      <td>Missing</td>
      <td>0</td>
      <td>0.000000</td>
      <td>0.0</td>
      <td>NaN</td>
      <td>0.000000</td>
      <td>NaN</td>
      <td>NaN</td>
      <td>0</td>
      <td>-22.5328</td>
      <td>0.000000</td>
    </tr>
    <tr>
      <th>Totals</th>
      <td></td>
      <td>506</td>
      <td>1.000000</td>
      <td>11401.6</td>
      <td></td>
      <td>22.532806</td>
      <td>5.0</td>
      <td>50.0</td>
      <td>0</td>
      <td></td>
      <td>6.030218</td>
    </tr>
  </tbody>
</table>
</div>



Let's describe the columns of this binning table:

- Bin: the intervals delimited by the optimal split points.
- Count: the number of records for each bin.
- Count (%): the percentage of records for each bin.
- Sum: the target sum for each bin.
- Std: the target std for each bin.
- Mean: the target mean for each bin.
- Min: the target min value for each bin.
- Max: the target max value for each bin.
- Zeros count: the number of zeros for each bin.
- WoE: Surrogate Weight-of-Evidence for each bin.
- IV: Surrogate IV for each bin.

The WoE IV for a continuous target is computed as follows:

\begin{equation}
IV = \sum_{i=1}^n \text{WoE}_i \frac{r_i}{r_T}, \quad \text{WoE}_i = |U_i - \mu|,
\end{equation}

where $U_i$ is the target mean value for each bin, $\mu$ is the total target mean, $r_i$ is the number of records for each bin, and $r_T$ is the total number of records.

The last row shows the total number of records, sum and mean.

You can use the method ``plot`` to visualize the histogram and mean curve. Note that the Bin ID corresponds to the binning table index.


```python
binning_table.plot()
```

Optionally, you can show the binning plot with the actual bin widths.


```python
binning_table.plot(style="actual", add_special=False, add_missing=False)
```

##### Mean transformation

Now that we have checked the binned data, we can transform our original data into mean values. You can check the correctness of the transformation using pandas ``value_counts`` method, for instance.


```python
x_transform_mean = optb.transform(x, metric="bins")
```


```python
pd.Series(x_transform_mean).value_counts()
```




    [6.87, 9.72)      89
    [16.09, 19.90)    69
    [13.10, 16.09)    66
    [-inf, 4.65)      50
    [9.72, 11.67)     49
    [23.32, inf)      47
    [5.49, 6.87)      45
    [11.67, 13.10)    35
    [4.65, 5.49)      28
    [19.90, 23.32)    28
    dtype: int64



## Advanced

Many of the advanced options have been covered in the previous tutorials with a binary target. **Check it out!** In this section, we focus on the mean monotonicity trend and the mean difference between bins.

#### Binning table statistical analysis

The ``analysis`` method performs a statistical analysis of the binning table, computing the Information Value (IV) and Herfindahl-Hirschman Index (HHI). Additionally, several statistical significance tests between consecutive bins of the contingency table are performed using the Student's t-test.


```python
binning_table.analysis()
```

    -------------------------------------------------
    OptimalBinning: Continuous Binning Table Analysis
    -------------------------------------------------
    
      General metrics
    
        IV                       6.03021763
        HHI                      0.11313253
        HHI (normalized)         0.03250821
        Quality score            0.21464367
    
      Monotonic trend            descending
    
      Significance tests
    
        Bin A  Bin B  t-statistic       p-value
            0      1     5.644492  3.313748e-07
            1      2     2.924528  5.175586e-03
            2      3     0.808313  4.206096e-01
            3      4     4.714124  6.127108e-06
            4      5     2.712699  8.189719e-03
            5      6     0.622294  5.353397e-01
            6      7     5.162973  8.663417e-07
            7      8     3.742511  4.988278e-04
            8      9     1.408305  1.643801e-01
    


#### Mean monotonicity

The monotonic_trend option permits forcing a monotonic trend to the mean curve. The default setting “auto” should be the preferred option, however, some business constraints might require to impose different trends. The default setting “auto” chooses the monotonic trend most likely to minimize the L1-norm from the options “ascending”, “descending”, “peak” and “valley” using a machine-learning-based classifier.


```python
variable = "INDUS"
x = df[variable].values
y = data.target
```


```python
optb = ContinuousOptimalBinning(name=variable, dtype="numerical",
                                monotonic_trend="auto")
optb.fit(x, y)
```




    ContinuousOptimalBinning(cat_cutoff=None, dtype='numerical', max_bin_size=None,
                             max_n_bins=None, max_n_prebins=20, max_pvalue=None,
                             max_pvalue_policy='consecutive', min_bin_size=None,
                             min_mean_diff=0, min_n_bins=None, min_prebin_size=0.05,
                             monotonic_trend='auto', name='INDUS',
                             outlier_detector=None, outlier_params=None,
                             prebinning_method='cart', special_codes=None,
                             split_digits=None, time_limit=100, user_splits=None,
                             user_splits_fixed=None, verbose=False)




```python
binning_table = optb.binning_table
binning_table.build()
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>Bin</th>
      <th>Count</th>
      <th>Count (%)</th>
      <th>Sum</th>
      <th>Std</th>
      <th>Mean</th>
      <th>Min</th>
      <th>Max</th>
      <th>Zeros count</th>
      <th>WoE</th>
      <th>IV</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>[-inf, 3.35)</td>
      <td>63</td>
      <td>0.124506</td>
      <td>1994.0</td>
      <td>8.56984</td>
      <td>31.650794</td>
      <td>16.5</td>
      <td>50.0</td>
      <td>0</td>
      <td>9.11799</td>
      <td>1.135243</td>
    </tr>
    <tr>
      <th>1</th>
      <td>[3.35, 5.04)</td>
      <td>57</td>
      <td>0.112648</td>
      <td>1615.2</td>
      <td>8.07271</td>
      <td>28.336842</td>
      <td>17.2</td>
      <td>50.0</td>
      <td>0</td>
      <td>5.80404</td>
      <td>0.653814</td>
    </tr>
    <tr>
      <th>2</th>
      <td>[5.04, 6.66)</td>
      <td>66</td>
      <td>0.130435</td>
      <td>1723.7</td>
      <td>7.87908</td>
      <td>26.116667</td>
      <td>16.0</td>
      <td>50.0</td>
      <td>0</td>
      <td>3.58386</td>
      <td>0.467460</td>
    </tr>
    <tr>
      <th>3</th>
      <td>[6.66, 8.01)</td>
      <td>31</td>
      <td>0.061265</td>
      <td>692.0</td>
      <td>4.60489</td>
      <td>22.322581</td>
      <td>14.4</td>
      <td>35.2</td>
      <td>0</td>
      <td>-0.210226</td>
      <td>0.012879</td>
    </tr>
    <tr>
      <th>4</th>
      <td>[8.01, 16.57)</td>
      <td>100</td>
      <td>0.197628</td>
      <td>2045.5</td>
      <td>3.54735</td>
      <td>20.455000</td>
      <td>11.9</td>
      <td>28.7</td>
      <td>0</td>
      <td>-2.07781</td>
      <td>0.410634</td>
    </tr>
    <tr>
      <th>5</th>
      <td>[16.57, 18.84)</td>
      <td>132</td>
      <td>0.260870</td>
      <td>2165.3</td>
      <td>8.50734</td>
      <td>16.403788</td>
      <td>5.0</td>
      <td>50.0</td>
      <td>0</td>
      <td>-6.12902</td>
      <td>1.598874</td>
    </tr>
    <tr>
      <th>6</th>
      <td>[18.84, inf)</td>
      <td>57</td>
      <td>0.112648</td>
      <td>1165.9</td>
      <td>9.51909</td>
      <td>20.454386</td>
      <td>7.0</td>
      <td>50.0</td>
      <td>0</td>
      <td>-2.07842</td>
      <td>0.234130</td>
    </tr>
    <tr>
      <th>7</th>
      <td>Special</td>
      <td>0</td>
      <td>0.000000</td>
      <td>0.0</td>
      <td>NaN</td>
      <td>0.000000</td>
      <td>NaN</td>
      <td>NaN</td>
      <td>0</td>
      <td>-22.5328</td>
      <td>0.000000</td>
    </tr>
    <tr>
      <th>8</th>
      <td>Missing</td>
      <td>0</td>
      <td>0.000000</td>
      <td>0.0</td>
      <td>NaN</td>
      <td>0.000000</td>
      <td>NaN</td>
      <td>NaN</td>
      <td>0</td>
      <td>-22.5328</td>
      <td>0.000000</td>
    </tr>
    <tr>
      <th>Totals</th>
      <td></td>
      <td>506</td>
      <td>1.000000</td>
      <td>11401.6</td>
      <td></td>
      <td>22.532806</td>
      <td>5.0</td>
      <td>50.0</td>
      <td>0</td>
      <td></td>
      <td>4.513036</td>
    </tr>
  </tbody>
</table>
</div>




```python
binning_table.plot()
```


```python
binning_table.analysis()
```

    -------------------------------------------------
    OptimalBinning: Continuous Binning Table Analysis
    -------------------------------------------------
    
      General metrics
    
        IV                       4.51303567
        HHI                      0.16875752
        HHI (normalized)         0.06485221
        Quality score            0.74831364
    
      Monotonic trend                valley
    
      Significance tests
    
        Bin A  Bin B  t-statistic   p-value
            0      1     2.180865  0.031181
            1      2     1.537968  0.126745
            2      3     2.976660  0.003739
            3      4     2.075258  0.044180
            4      5     4.934158  0.000002
            5      6    -2.770230  0.006721
    


A smoother curve, keeping the valley monotonicity, can be achieved by using ``monotonic_trend="convex"``.


```python
optb = ContinuousOptimalBinning(name=variable, dtype="numerical",
                                monotonic_trend="convex")
optb.fit(x, y)
```




    ContinuousOptimalBinning(cat_cutoff=None, dtype='numerical', max_bin_size=None,
                             max_n_bins=None, max_n_prebins=20, max_pvalue=None,
                             max_pvalue_policy='consecutive', min_bin_size=None,
                             min_mean_diff=0, min_n_bins=None, min_prebin_size=0.05,
                             monotonic_trend='convex', name='INDUS',
                             outlier_detector=None, outlier_params=None,
                             prebinning_method='cart', special_codes=None,
                             split_digits=None, time_limit=100, user_splits=None,
                             user_splits_fixed=None, verbose=False)




```python
binning_table = optb.binning_table
binning_table.build()
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>Bin</th>
      <th>Count</th>
      <th>Count (%)</th>
      <th>Sum</th>
      <th>Std</th>
      <th>Mean</th>
      <th>Min</th>
      <th>Max</th>
      <th>Zeros count</th>
      <th>WoE</th>
      <th>IV</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>[-inf, 3.99)</td>
      <td>92</td>
      <td>0.181818</td>
      <td>2932.6</td>
      <td>8.6887</td>
      <td>31.876087</td>
      <td>16.5</td>
      <td>50.0</td>
      <td>0</td>
      <td>9.34328</td>
      <td>1.698778</td>
    </tr>
    <tr>
      <th>1</th>
      <td>[3.99, 5.04)</td>
      <td>28</td>
      <td>0.055336</td>
      <td>676.6</td>
      <td>4.0002</td>
      <td>24.164286</td>
      <td>17.2</td>
      <td>37.3</td>
      <td>0</td>
      <td>1.63148</td>
      <td>0.090279</td>
    </tr>
    <tr>
      <th>2</th>
      <td>[5.04, 6.08)</td>
      <td>40</td>
      <td>0.079051</td>
      <td>878.6</td>
      <td>4.40503</td>
      <td>21.965000</td>
      <td>16.0</td>
      <td>42.8</td>
      <td>0</td>
      <td>-0.567806</td>
      <td>0.044886</td>
    </tr>
    <tr>
      <th>3</th>
      <td>[6.08, 18.84)</td>
      <td>289</td>
      <td>0.571146</td>
      <td>5747.9</td>
      <td>8.10754</td>
      <td>19.888927</td>
      <td>5.0</td>
      <td>50.0</td>
      <td>0</td>
      <td>-2.64388</td>
      <td>1.510042</td>
    </tr>
    <tr>
      <th>4</th>
      <td>[18.84, inf)</td>
      <td>57</td>
      <td>0.112648</td>
      <td>1165.9</td>
      <td>9.51909</td>
      <td>20.454386</td>
      <td>7.0</td>
      <td>50.0</td>
      <td>0</td>
      <td>-2.07842</td>
      <td>0.234130</td>
    </tr>
    <tr>
      <th>5</th>
      <td>Special</td>
      <td>0</td>
      <td>0.000000</td>
      <td>0.0</td>
      <td>NaN</td>
      <td>0.000000</td>
      <td>NaN</td>
      <td>NaN</td>
      <td>0</td>
      <td>-22.5328</td>
      <td>0.000000</td>
    </tr>
    <tr>
      <th>6</th>
      <td>Missing</td>
      <td>0</td>
      <td>0.000000</td>
      <td>0.0</td>
      <td>NaN</td>
      <td>0.000000</td>
      <td>NaN</td>
      <td>NaN</td>
      <td>0</td>
      <td>-22.5328</td>
      <td>0.000000</td>
    </tr>
    <tr>
      <th>Totals</th>
      <td></td>
      <td>506</td>
      <td>1.000000</td>
      <td>11401.6</td>
      <td></td>
      <td>22.532806</td>
      <td>5.0</td>
      <td>50.0</td>
      <td>0</td>
      <td></td>
      <td>3.578116</td>
    </tr>
  </tbody>
</table>
</div>




```python
binning_table.plot()
```


```python
binning_table.analysis()
```

    -------------------------------------------------
    OptimalBinning: Continuous Binning Table Analysis
    -------------------------------------------------
    
      General metrics
    
        IV                       3.57811558
        HHI                      0.38126670
        HHI (normalized)         0.27814448
        Quality score            0.22169448
    
      Monotonic trend       valley (convex)
    
      Significance tests
    
        Bin A  Bin B  t-statistic       p-value
            0      1     6.536206  2.732420e-09
            1      2     2.139577  3.636038e-02
            2      3     2.459425  1.602640e-02
            3      4    -0.419474  6.761033e-01
    


For example, we can force the variable INDUS (proportion of non-retail business acres per town) to be monotonically descending with respect to the house-price.


```python
optb = ContinuousOptimalBinning(name=variable, dtype="numerical",
                                monotonic_trend="descending")
optb.fit(x, y)
```




    ContinuousOptimalBinning(cat_cutoff=None, dtype='numerical', max_bin_size=None,
                             max_n_bins=None, max_n_prebins=20, max_pvalue=None,
                             max_pvalue_policy='consecutive', min_bin_size=None,
                             min_mean_diff=0, min_n_bins=None, min_prebin_size=0.05,
                             monotonic_trend='descending', name='INDUS',
                             outlier_detector=None, outlier_params=None,
                             prebinning_method='cart', special_codes=None,
                             split_digits=None, time_limit=100, user_splits=None,
                             user_splits_fixed=None, verbose=False)




```python
binning_table = optb.binning_table
binning_table.build()
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>Bin</th>
      <th>Count</th>
      <th>Count (%)</th>
      <th>Sum</th>
      <th>Std</th>
      <th>Mean</th>
      <th>Min</th>
      <th>Max</th>
      <th>Zeros count</th>
      <th>WoE</th>
      <th>IV</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>[-inf, 3.35)</td>
      <td>63</td>
      <td>0.124506</td>
      <td>1994.0</td>
      <td>8.56984</td>
      <td>31.650794</td>
      <td>16.5</td>
      <td>50.0</td>
      <td>0</td>
      <td>9.11799</td>
      <td>1.135243</td>
    </tr>
    <tr>
      <th>1</th>
      <td>[3.35, 5.04)</td>
      <td>57</td>
      <td>0.112648</td>
      <td>1615.2</td>
      <td>8.07271</td>
      <td>28.336842</td>
      <td>17.2</td>
      <td>50.0</td>
      <td>0</td>
      <td>5.80404</td>
      <td>0.653814</td>
    </tr>
    <tr>
      <th>2</th>
      <td>[5.04, 6.66)</td>
      <td>66</td>
      <td>0.130435</td>
      <td>1723.7</td>
      <td>7.87908</td>
      <td>26.116667</td>
      <td>16.0</td>
      <td>50.0</td>
      <td>0</td>
      <td>3.58386</td>
      <td>0.467460</td>
    </tr>
    <tr>
      <th>3</th>
      <td>[6.66, 8.01)</td>
      <td>31</td>
      <td>0.061265</td>
      <td>692.0</td>
      <td>4.60489</td>
      <td>22.322581</td>
      <td>14.4</td>
      <td>35.2</td>
      <td>0</td>
      <td>-0.210226</td>
      <td>0.012879</td>
    </tr>
    <tr>
      <th>4</th>
      <td>[8.01, 16.57)</td>
      <td>100</td>
      <td>0.197628</td>
      <td>2045.5</td>
      <td>3.54735</td>
      <td>20.455000</td>
      <td>11.9</td>
      <td>28.7</td>
      <td>0</td>
      <td>-2.07781</td>
      <td>0.410634</td>
    </tr>
    <tr>
      <th>5</th>
      <td>[16.57, 20.73)</td>
      <td>162</td>
      <td>0.320158</td>
      <td>2874.8</td>
      <td>9.61845</td>
      <td>17.745679</td>
      <td>5.0</td>
      <td>50.0</td>
      <td>0</td>
      <td>-4.78713</td>
      <td>1.532638</td>
    </tr>
    <tr>
      <th>6</th>
      <td>[20.73, inf)</td>
      <td>27</td>
      <td>0.053360</td>
      <td>456.4</td>
      <td>3.69088</td>
      <td>16.903704</td>
      <td>7.0</td>
      <td>23.0</td>
      <td>0</td>
      <td>-5.6291</td>
      <td>0.300367</td>
    </tr>
    <tr>
      <th>7</th>
      <td>Special</td>
      <td>0</td>
      <td>0.000000</td>
      <td>0.0</td>
      <td>NaN</td>
      <td>0.000000</td>
      <td>NaN</td>
      <td>NaN</td>
      <td>0</td>
      <td>-22.5328</td>
      <td>0.000000</td>
    </tr>
    <tr>
      <th>8</th>
      <td>Missing</td>
      <td>0</td>
      <td>0.000000</td>
      <td>0.0</td>
      <td>NaN</td>
      <td>0.000000</td>
      <td>NaN</td>
      <td>NaN</td>
      <td>0</td>
      <td>-22.5328</td>
      <td>0.000000</td>
    </tr>
    <tr>
      <th>Totals</th>
      <td></td>
      <td>506</td>
      <td>1.000000</td>
      <td>11401.6</td>
      <td></td>
      <td>22.532806</td>
      <td>5.0</td>
      <td>50.0</td>
      <td>0</td>
      <td></td>
      <td>4.513036</td>
    </tr>
  </tbody>
</table>
</div>




```python
binning_table.plot()
```


```python
binning_table.analysis()
```

    -------------------------------------------------
    OptimalBinning: Continuous Binning Table Analysis
    -------------------------------------------------
    
      General metrics
    
        IV                       4.51303567
        HHI                      0.19336343
        HHI (normalized)         0.09253386
        Quality score            0.42428843
    
      Monotonic trend            descending
    
      Significance tests
    
        Bin A  Bin B  t-statistic   p-value
            0      1     2.180865  0.031181
            1      2     1.537968  0.126745
            2      3     2.976660  0.003739
            3      4     2.075258  0.044180
            4      5     3.245419  0.001354
            5      6     0.811840  0.418853
    


#### Mininum mean difference between consecutive bins

Now, we note that the mean difference between consecutive bins is not significant enough. Therefore, we decide to set ``min_mean_diff=2.0``:


```python
optb = ContinuousOptimalBinning(name=variable, dtype="numerical",
                                monotonic_trend="descending", min_mean_diff=2.0)
optb.fit(x, y)
```




    ContinuousOptimalBinning(cat_cutoff=None, dtype='numerical', max_bin_size=None,
                             max_n_bins=None, max_n_prebins=20, max_pvalue=None,
                             max_pvalue_policy='consecutive', min_bin_size=None,
                             min_mean_diff=2.0, min_n_bins=None,
                             min_prebin_size=0.05, monotonic_trend='descending',
                             name='INDUS', outlier_detector=None,
                             outlier_params=None, prebinning_method='cart',
                             special_codes=None, split_digits=None, time_limit=100,
                             user_splits=None, user_splits_fixed=None,
                             verbose=False)




```python
binning_table = optb.binning_table
binning_table.build()
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>Bin</th>
      <th>Count</th>
      <th>Count (%)</th>
      <th>Sum</th>
      <th>Std</th>
      <th>Mean</th>
      <th>Min</th>
      <th>Max</th>
      <th>Zeros count</th>
      <th>WoE</th>
      <th>IV</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>[-inf, 3.35)</td>
      <td>63</td>
      <td>0.124506</td>
      <td>1994.0</td>
      <td>8.56984</td>
      <td>31.650794</td>
      <td>16.5</td>
      <td>50.0</td>
      <td>0</td>
      <td>9.11799</td>
      <td>1.135243</td>
    </tr>
    <tr>
      <th>1</th>
      <td>[3.35, 5.04)</td>
      <td>57</td>
      <td>0.112648</td>
      <td>1615.2</td>
      <td>8.07271</td>
      <td>28.336842</td>
      <td>17.2</td>
      <td>50.0</td>
      <td>0</td>
      <td>5.80404</td>
      <td>0.653814</td>
    </tr>
    <tr>
      <th>2</th>
      <td>[5.04, 6.66)</td>
      <td>66</td>
      <td>0.130435</td>
      <td>1723.7</td>
      <td>7.87908</td>
      <td>26.116667</td>
      <td>16.0</td>
      <td>50.0</td>
      <td>0</td>
      <td>3.58386</td>
      <td>0.467460</td>
    </tr>
    <tr>
      <th>3</th>
      <td>[6.66, 16.57)</td>
      <td>131</td>
      <td>0.258893</td>
      <td>2737.5</td>
      <td>3.90562</td>
      <td>20.896947</td>
      <td>11.9</td>
      <td>35.2</td>
      <td>0</td>
      <td>-1.63586</td>
      <td>0.423513</td>
    </tr>
    <tr>
      <th>4</th>
      <td>[16.57, inf)</td>
      <td>189</td>
      <td>0.373518</td>
      <td>3331.2</td>
      <td>9.01838</td>
      <td>17.625397</td>
      <td>5.0</td>
      <td>50.0</td>
      <td>0</td>
      <td>-4.90741</td>
      <td>1.833005</td>
    </tr>
    <tr>
      <th>5</th>
      <td>Special</td>
      <td>0</td>
      <td>0.000000</td>
      <td>0.0</td>
      <td>NaN</td>
      <td>0.000000</td>
      <td>NaN</td>
      <td>NaN</td>
      <td>0</td>
      <td>-22.5328</td>
      <td>0.000000</td>
    </tr>
    <tr>
      <th>6</th>
      <td>Missing</td>
      <td>0</td>
      <td>0.000000</td>
      <td>0.0</td>
      <td>NaN</td>
      <td>0.000000</td>
      <td>NaN</td>
      <td>NaN</td>
      <td>0</td>
      <td>-22.5328</td>
      <td>0.000000</td>
    </tr>
    <tr>
      <th>Totals</th>
      <td></td>
      <td>506</td>
      <td>1.000000</td>
      <td>11401.6</td>
      <td></td>
      <td>22.532806</td>
      <td>5.0</td>
      <td>50.0</td>
      <td>0</td>
      <td></td>
      <td>4.513036</td>
    </tr>
  </tbody>
</table>
</div>




```python
binning_table.plot()
```


```python
binning_table.analysis()
```

    -------------------------------------------------
    OptimalBinning: Continuous Binning Table Analysis
    -------------------------------------------------
    
      General metrics
    
        IV                       4.51303567
        HHI                      0.25174585
        HHI (normalized)         0.12703682
        Quality score            0.73853806
    
      Monotonic trend            descending
    
      Significance tests
    
        Bin A  Bin B  t-statistic   p-value
            0      1     2.180865  0.031181
            1      2     1.537968  0.126745
            2      3     5.076919  0.000002
            3      4     4.424387  0.000014
    


24420487f4a360073137d27a15d362678e0424bb
ipynb
Jupyter Notebook
doc/source/tutorials/tutorial_continuous.ipynb
mnicstruwig/optbinning
6ce991e1ca75b4d41835f3b3bf8e0f294f6ba780
Apache-2.0
2021-02-09T02:49:32.000Z
2021-02-09T02:49:32.000Z
doc/source/tutorials/tutorial_continuous.ipynb
mnicstruwig/optbinning
6ce991e1ca75b4d41835f3b3bf8e0f294f6ba780
Apache-2.0
doc/source/tutorials/tutorial_continuous.ipynb
mnicstruwig/optbinning
6ce991e1ca75b4d41835f3b3bf8e0f294f6ba780
Apache-2.0
Qwen/Qwen-72B
1. YES
2. YES