<a href="https://colab.research.google.com/github/pedroblossbraga/MachineLearning-Pre-Processing-with-Python/blob/master/Effects_of_transformations_in_XGBoostRegressor_example.ipynb" target="_parent"></a>

## Tests with different data transformations applied to XGBoost

Samples:
- original data
- linearly transformated data
- minmax scaled data
- pseudo-random data


```python
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.impute import SimpleImputer
from xgboost import XGBRegressor
from sklearn.metrics import mean_absolute_error
```


```python
from sklearn.metrics import mean_squared_error
```


```python
import matplotlib.pyplot as plt
import seaborn as sns
import statistics
from IPython.display import display
```


```python
import warnings
warnings.filterwarnings("ignore")
```


```python
from sklearn.datasets import load_boston
X, y = load_boston(return_X_y=True)
```


```python
train_X, test_X, train_y, test_y = train_test_split(X, y, test_size=0.2)
```


```python
# lidar com os valores ausentes
df_imputer = SimpleImputer()
train_X = df_imputer.fit_transform(train_X)
test_X = df_imputer.transform(test_X)
```


```python
def test_XGBoost(train_X, train_y, test_X, test_y, plot_residuals=True):
  # instanciar o modelo XGBoost
  model = XGBRegressor()
  
  # chamar o fit para o modelo
  model.fit(train_X, train_y, verbose=False)
  
  # fazer previsões em cima do dataset de teste
  predictions = model.predict(test_X)
  
  print("Mean Absolute Error: {:.2f}".format(mean_absolute_error(predictions, test_y)))
  if plot_residuals:
    plt.figure(figsize=(15,3))
    plt.subplot(1,2,1)
    plt.title("predictions")
    sns.distplot(predictions)
    plt.axvline(statistics.mean(predictions), color='red')

    plt.subplot(1,2,2)
    plt.title(r'residuals $\epsilon = |\hat{y} - y|$')
    sns.distplot(abs(predictions-test_y))
    plt.axvline(statistics.mean(abs(predictions-test_y)), color='red')

    plt.show()

    return mean_absolute_error(y_pred=predictions, y_true=test_y), mean_squared_error(y_true=test_y, y_pred=predictions)
```

### original data


```python
mae0, mse0 = test_XGBoost(train_X, train_y, test_X, test_y)
```

### linearly transformed

\begin{equation}
X_1 = \{x_j + k\}_{j=1}^N, k \in \mathbb{N}
\end{equation}


```python
k=20
mae1, mse1 = test_XGBoost(train_X+k, train_y+k, test_X+k, test_y+k)
```

### Min-Max scaled

\begin{equation}
\hat{X} = \left\{ \frac{x_j - min \{ X \} }{max \{ X \} - min \{ X \} } \right\}_{j=1}^N
\end{equation}


```python
def minmaxscale(v):
  return (v - v.min(axis=0))/(v.max(axis=0)-v.min(axis=0))

mae2, mse2 = test_XGBoost(
    minmaxscale(train_X), 
    minmaxscale(train_y), 
    minmaxscale(test_X), 
    minmaxscale(test_y)
)
```


```python
import numpy as np

def randomize_matrix(X):
  X_ = X.copy()
  if len(X_.shape)==1: # vector
    for i in range(X_.shape[0]):
      X_[i] = np.random.randint(-20, 20)
  else: # matrix
    for lin in range(X_.shape[0]):
      for col in range(X_.shape[1]):
        X_[lin][col] = np.random.randint(-20, 20)
  return X_
```

### Pseudo-random data


```python
mae3, mse3 = test_XGBoost(
    randomize_matrix(train_X), 
    randomize_matrix(train_y), 
    randomize_matrix(test_X), 
    randomize_matrix(test_y)
)
```


```python
erros = {
    'MAE': [mae0, mae1, mae2, mae3],
    'MSE': [mse0, mse1, mse2, mse3],
    'transf': ['original', 'linear', 'minmax', 'pseudo-random']
}

display(pd.DataFrame(erros))
```


<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>MAE</th>
      <th>MSE</th>
      <th>transf</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>1.947568</td>
      <td>7.555698</td>
      <td>original</td>
    </tr>
    <tr>
      <th>1</th>
      <td>1.960215</td>
      <td>8.868490</td>
      <td>linear</td>
    </tr>
    <tr>
      <th>2</th>
      <td>0.057456</td>
      <td>0.006035</td>
      <td>minmax</td>
    </tr>
    <tr>
      <th>3</th>
      <td>11.327111</td>
      <td>170.878044</td>
      <td>pseudo-random</td>
    </tr>
  </tbody>
</table>
</div>


de408b4ce889cc61377a3c7c58c48e2a65dcd657
ipynb
Jupyter Notebook
Effects_of_transformations_in_XGBoostRegressor_example.ipynb
pedroblossbraga/MachineLearning-Pre-Processing-with-Python
55aa2897ddf75e295aa8a014049273e8636e5c5b
MIT
2020-01-13T20:09:37.000Z
2020-01-20T01:13:49.000Z
Effects_of_transformations_in_XGBoostRegressor_example.ipynb
pedroblossbraga/MachineLearning-Pre-Processing-with-Python
55aa2897ddf75e295aa8a014049273e8636e5c5b
MIT
Effects_of_transformations_in_XGBoostRegressor_example.ipynb
pedroblossbraga/MachineLearning-Pre-Processing-with-Python
55aa2897ddf75e295aa8a014049273e8636e5c5b
MIT
2021-03-16T17:53:12.000Z
2021-03-16T17:53:12.000Z
Qwen/Qwen-72B
1. YES
2. YES