# Librerias


```python
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from sklearn import datasets
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
```

# Data Set


```python
# Loading pre-defined Boston Dataset
boston_dataset = datasets.load_boston()
print(boston_dataset.DESCR)
```

    .. _boston_dataset:
    
    Boston house prices dataset
    ---------------------------
    
    **Data Set Characteristics:**  
    
        :Number of Instances: 506 
    
        :Number of Attributes: 13 numeric/categorical predictive. Median Value (attribute 14) is usually the target.
    
        :Attribute Information (in order):
            - CRIM     per capita crime rate by town
            - ZN       proportion of residential land zoned for lots over 25,000 sq.ft.
            - INDUS    proportion of non-retail business acres per town
            - CHAS     Charles River dummy variable (= 1 if tract bounds river; 0 otherwise)
            - NOX      nitric oxides concentration (parts per 10 million)
            - RM       average number of rooms per dwelling
            - AGE      proportion of owner-occupied units built prior to 1940
            - DIS      weighted distances to five Boston employment centres
            - RAD      index of accessibility to radial highways
            - TAX      full-value property-tax rate per $10,000
            - PTRATIO  pupil-teacher ratio by town
            - B        1000(Bk - 0.63)^2 where Bk is the proportion of blacks by town
            - LSTAT    % lower status of the population
            - MEDV     Median value of owner-occupied homes in $1000's
    
        :Missing Attribute Values: None
    
        :Creator: Harrison, D. and Rubinfeld, D.L.
    
    This is a copy of UCI ML housing dataset.
    https://archive.ics.uci.edu/ml/machine-learning-databases/housing/
    
    
    This dataset was taken from the StatLib library which is maintained at Carnegie Mellon University.
    
    The Boston house-price data of Harrison, D. and Rubinfeld, D.L. 'Hedonic
    prices and the demand for clean air', J. Environ. Economics & Management,
    vol.5, 81-102, 1978.   Used in Belsley, Kuh & Welsch, 'Regression diagnostics
    ...', Wiley, 1980.   N.B. Various transformations are used in the table on
    pages 244-261 of the latter.
    
    The Boston house-price data has been used in many machine learning papers that address regression
    problems.   
         
    .. topic:: References
    
       - Belsley, Kuh & Welsch, 'Regression diagnostics: Identifying Influential Data and Sources of Collinearity', Wiley, 1980. 244-261.
       - Quinlan,R. (1993). Combining Instance-Based and Model-Based Learning. In Proceedings on the Tenth International Conference of Machine Learning, 236-243, University of Massachusetts, Amherst. Morgan Kaufmann.
    



```python
df = pd.DataFrame(data = boston_dataset.data, columns=boston_dataset.feature_names)
df['House Price'] = boston_dataset.target
df.head()
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>CRIM</th>
      <th>ZN</th>
      <th>INDUS</th>
      <th>CHAS</th>
      <th>NOX</th>
      <th>RM</th>
      <th>AGE</th>
      <th>DIS</th>
      <th>RAD</th>
      <th>TAX</th>
      <th>PTRATIO</th>
      <th>B</th>
      <th>LSTAT</th>
      <th>House Price</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>0.00632</td>
      <td>18.0</td>
      <td>2.31</td>
      <td>0.0</td>
      <td>0.538</td>
      <td>6.575</td>
      <td>65.2</td>
      <td>4.0900</td>
      <td>1.0</td>
      <td>296.0</td>
      <td>15.3</td>
      <td>396.90</td>
      <td>4.98</td>
      <td>24.0</td>
    </tr>
    <tr>
      <th>1</th>
      <td>0.02731</td>
      <td>0.0</td>
      <td>7.07</td>
      <td>0.0</td>
      <td>0.469</td>
      <td>6.421</td>
      <td>78.9</td>
      <td>4.9671</td>
      <td>2.0</td>
      <td>242.0</td>
      <td>17.8</td>
      <td>396.90</td>
      <td>9.14</td>
      <td>21.6</td>
    </tr>
    <tr>
      <th>2</th>
      <td>0.02729</td>
      <td>0.0</td>
      <td>7.07</td>
      <td>0.0</td>
      <td>0.469</td>
      <td>7.185</td>
      <td>61.1</td>
      <td>4.9671</td>
      <td>2.0</td>
      <td>242.0</td>
      <td>17.8</td>
      <td>392.83</td>
      <td>4.03</td>
      <td>34.7</td>
    </tr>
    <tr>
      <th>3</th>
      <td>0.03237</td>
      <td>0.0</td>
      <td>2.18</td>
      <td>0.0</td>
      <td>0.458</td>
      <td>6.998</td>
      <td>45.8</td>
      <td>6.0622</td>
      <td>3.0</td>
      <td>222.0</td>
      <td>18.7</td>
      <td>394.63</td>
      <td>2.94</td>
      <td>33.4</td>
    </tr>
    <tr>
      <th>4</th>
      <td>0.06905</td>
      <td>0.0</td>
      <td>2.18</td>
      <td>0.0</td>
      <td>0.458</td>
      <td>7.147</td>
      <td>54.2</td>
      <td>6.0622</td>
      <td>3.0</td>
      <td>222.0</td>
      <td>18.7</td>
      <td>396.90</td>
      <td>5.33</td>
      <td>36.2</td>
    </tr>
  </tbody>
</table>
</div>




```python
df.info()
```

    <class 'pandas.core.frame.DataFrame'>
    RangeIndex: 506 entries, 0 to 505
    Data columns (total 14 columns):
     #   Column       Non-Null Count  Dtype  
    ---  ------       --------------  -----  
     0   CRIM         506 non-null    float64
     1   ZN           506 non-null    float64
     2   INDUS        506 non-null    float64
     3   CHAS         506 non-null    float64
     4   NOX          506 non-null    float64
     5   RM           506 non-null    float64
     6   AGE          506 non-null    float64
     7   DIS          506 non-null    float64
     8   RAD          506 non-null    float64
     9   TAX          506 non-null    float64
     10  PTRATIO      506 non-null    float64
     11  B            506 non-null    float64
     12  LSTAT        506 non-null    float64
     13  House Price  506 non-null    float64
    dtypes: float64(14)
    memory usage: 55.5 KB



```python
# Generate scatter plot of independent vs Dependent variable
plt.style.use('ggplot')
fig = plt.figure(figsize = (18, 18))
 
for index, feature_name in enumerate(boston_dataset.feature_names):
    ax = fig.add_subplot(4, 4, index + 1)
    ax.scatter(boston_dataset.data[:, index], boston_dataset.target)
    ax.set_ylabel('House Price', size = 12)
    ax.set_xlabel(feature_name, size = 12)
 
plt.show()
```


```python
X = df.iloc[:, :-1]
y = df.iloc[:, -1]
 
```


```python
X_train, X_test, y_train, y_test = train_test_split(
    X, y,
    test_size = 0.25)

print("Train data shape X = % s and y = % s : "%(
    X_train.shape, y_train.shape))
 
print("Test data shape X = % s and y = % s : "%(
    X_test.shape, y_test.shape))
```

    Train data shape X = (379, 13) and y = (379,) : 
    Test data shape X = (127, 13) and y = (127,) : 



```python
# StandarScale
from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
X_train_scaled = pd.DataFrame(scaler.fit_transform(X_train), columns = X_train.columns)
X_test_scaled = pd.DataFrame(scaler.transform(X_test), columns = X_test.columns)
```


```python
X_train
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>CRIM</th>
      <th>ZN</th>
      <th>INDUS</th>
      <th>CHAS</th>
      <th>NOX</th>
      <th>RM</th>
      <th>AGE</th>
      <th>DIS</th>
      <th>RAD</th>
      <th>TAX</th>
      <th>PTRATIO</th>
      <th>B</th>
      <th>LSTAT</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>211</th>
      <td>0.37578</td>
      <td>0.0</td>
      <td>10.59</td>
      <td>1.0</td>
      <td>0.489</td>
      <td>5.404</td>
      <td>88.6</td>
      <td>3.6650</td>
      <td>4.0</td>
      <td>277.0</td>
      <td>18.6</td>
      <td>395.24</td>
      <td>23.98</td>
    </tr>
    <tr>
      <th>2</th>
      <td>0.02729</td>
      <td>0.0</td>
      <td>7.07</td>
      <td>0.0</td>
      <td>0.469</td>
      <td>7.185</td>
      <td>61.1</td>
      <td>4.9671</td>
      <td>2.0</td>
      <td>242.0</td>
      <td>17.8</td>
      <td>392.83</td>
      <td>4.03</td>
    </tr>
    <tr>
      <th>195</th>
      <td>0.01381</td>
      <td>80.0</td>
      <td>0.46</td>
      <td>0.0</td>
      <td>0.422</td>
      <td>7.875</td>
      <td>32.0</td>
      <td>5.6484</td>
      <td>4.0</td>
      <td>255.0</td>
      <td>14.4</td>
      <td>394.23</td>
      <td>2.97</td>
    </tr>
    <tr>
      <th>434</th>
      <td>13.91340</td>
      <td>0.0</td>
      <td>18.10</td>
      <td>0.0</td>
      <td>0.713</td>
      <td>6.208</td>
      <td>95.0</td>
      <td>2.2222</td>
      <td>24.0</td>
      <td>666.0</td>
      <td>20.2</td>
      <td>100.63</td>
      <td>15.17</td>
    </tr>
    <tr>
      <th>90</th>
      <td>0.04684</td>
      <td>0.0</td>
      <td>3.41</td>
      <td>0.0</td>
      <td>0.489</td>
      <td>6.417</td>
      <td>66.1</td>
      <td>3.0923</td>
      <td>2.0</td>
      <td>270.0</td>
      <td>17.8</td>
      <td>392.18</td>
      <td>8.81</td>
    </tr>
    <tr>
      <th>...</th>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
    </tr>
    <tr>
      <th>128</th>
      <td>0.32543</td>
      <td>0.0</td>
      <td>21.89</td>
      <td>0.0</td>
      <td>0.624</td>
      <td>6.431</td>
      <td>98.8</td>
      <td>1.8125</td>
      <td>4.0</td>
      <td>437.0</td>
      <td>21.2</td>
      <td>396.90</td>
      <td>15.39</td>
    </tr>
    <tr>
      <th>318</th>
      <td>0.40202</td>
      <td>0.0</td>
      <td>9.90</td>
      <td>0.0</td>
      <td>0.544</td>
      <td>6.382</td>
      <td>67.2</td>
      <td>3.5325</td>
      <td>4.0</td>
      <td>304.0</td>
      <td>18.4</td>
      <td>395.21</td>
      <td>10.36</td>
    </tr>
    <tr>
      <th>372</th>
      <td>8.26725</td>
      <td>0.0</td>
      <td>18.10</td>
      <td>1.0</td>
      <td>0.668</td>
      <td>5.875</td>
      <td>89.6</td>
      <td>1.1296</td>
      <td>24.0</td>
      <td>666.0</td>
      <td>20.2</td>
      <td>347.88</td>
      <td>8.88</td>
    </tr>
    <tr>
      <th>164</th>
      <td>2.24236</td>
      <td>0.0</td>
      <td>19.58</td>
      <td>0.0</td>
      <td>0.605</td>
      <td>5.854</td>
      <td>91.8</td>
      <td>2.4220</td>
      <td>5.0</td>
      <td>403.0</td>
      <td>14.7</td>
      <td>395.11</td>
      <td>11.64</td>
    </tr>
    <tr>
      <th>73</th>
      <td>0.19539</td>
      <td>0.0</td>
      <td>10.81</td>
      <td>0.0</td>
      <td>0.413</td>
      <td>6.245</td>
      <td>6.2</td>
      <td>5.2873</td>
      <td>4.0</td>
      <td>305.0</td>
      <td>19.2</td>
      <td>377.17</td>
      <td>7.54</td>
    </tr>
  </tbody>
</table>
<p>379 rows × 13 columns</p>
</div>




```python
X_train_scaled
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>CRIM</th>
      <th>ZN</th>
      <th>INDUS</th>
      <th>CHAS</th>
      <th>NOX</th>
      <th>RM</th>
      <th>AGE</th>
      <th>DIS</th>
      <th>RAD</th>
      <th>TAX</th>
      <th>PTRATIO</th>
      <th>B</th>
      <th>LSTAT</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>-0.365345</td>
      <td>-0.496544</td>
      <td>-0.032725</td>
      <td>3.610684</td>
      <td>-0.554810</td>
      <td>-1.241634</td>
      <td>0.723924</td>
      <td>-0.089781</td>
      <td>-0.624840</td>
      <td>-0.752319</td>
      <td>0.105433</td>
      <td>0.410116</td>
      <td>1.604117</td>
    </tr>
    <tr>
      <th>1</th>
      <td>-0.403233</td>
      <td>-0.496544</td>
      <td>-0.546098</td>
      <td>-0.276956</td>
      <td>-0.724461</td>
      <td>1.198062</td>
      <td>-0.238694</td>
      <td>0.513705</td>
      <td>-0.857810</td>
      <td>-0.961052</td>
      <td>-0.258658</td>
      <td>0.383643</td>
      <td>-1.179552</td>
    </tr>
    <tr>
      <th>2</th>
      <td>-0.404699</td>
      <td>2.828737</td>
      <td>-1.510133</td>
      <td>-0.276956</td>
      <td>-1.123142</td>
      <td>2.143256</td>
      <td>-1.257319</td>
      <td>0.829468</td>
      <td>-0.624840</td>
      <td>-0.883523</td>
      <td>-1.806046</td>
      <td>0.399022</td>
      <td>-1.327457</td>
    </tr>
    <tr>
      <th>3</th>
      <td>1.106457</td>
      <td>-0.496544</td>
      <td>1.062570</td>
      <td>-0.276956</td>
      <td>1.345287</td>
      <td>-0.140277</td>
      <td>0.947951</td>
      <td>-0.758477</td>
      <td>1.704863</td>
      <td>1.567596</td>
      <td>0.833615</td>
      <td>-2.826040</td>
      <td>0.374838</td>
    </tr>
    <tr>
      <th>4</th>
      <td>-0.401108</td>
      <td>-0.496544</td>
      <td>-1.079890</td>
      <td>-0.276956</td>
      <td>-0.554810</td>
      <td>0.146021</td>
      <td>-0.063673</td>
      <td>-0.355211</td>
      <td>-0.857810</td>
      <td>-0.794066</td>
      <td>-0.258658</td>
      <td>0.376503</td>
      <td>-0.512588</td>
    </tr>
    <tr>
      <th>...</th>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
    </tr>
    <tr>
      <th>374</th>
      <td>-0.370819</td>
      <td>-0.496544</td>
      <td>1.615322</td>
      <td>-0.276956</td>
      <td>0.590338</td>
      <td>0.165198</td>
      <td>1.080967</td>
      <td>-0.948362</td>
      <td>-0.624840</td>
      <td>0.201888</td>
      <td>1.288729</td>
      <td>0.428351</td>
      <td>0.405535</td>
    </tr>
    <tr>
      <th>375</th>
      <td>-0.362493</td>
      <td>-0.496544</td>
      <td>-0.133358</td>
      <td>-0.276956</td>
      <td>-0.088268</td>
      <td>0.098076</td>
      <td>-0.025168</td>
      <td>-0.151191</td>
      <td>-0.624840</td>
      <td>-0.591297</td>
      <td>0.014410</td>
      <td>0.409787</td>
      <td>-0.296313</td>
    </tr>
    <tr>
      <th>376</th>
      <td>0.492611</td>
      <td>-0.496544</td>
      <td>1.062570</td>
      <td>3.610684</td>
      <td>0.963571</td>
      <td>-0.596436</td>
      <td>0.758928</td>
      <td>-1.264866</td>
      <td>1.704863</td>
      <td>1.567596</td>
      <td>0.833615</td>
      <td>-0.110112</td>
      <td>-0.502821</td>
    </tr>
    <tr>
      <th>377</th>
      <td>-0.162412</td>
      <td>-0.496544</td>
      <td>1.278420</td>
      <td>-0.276956</td>
      <td>0.429169</td>
      <td>-0.625203</td>
      <td>0.835937</td>
      <td>-0.665876</td>
      <td>-0.508355</td>
      <td>-0.000881</td>
      <td>-1.669512</td>
      <td>0.408688</td>
      <td>-0.117712</td>
    </tr>
    <tr>
      <th>378</th>
      <td>-0.384957</td>
      <td>-0.496544</td>
      <td>-0.000639</td>
      <td>-0.276956</td>
      <td>-1.199485</td>
      <td>-0.089593</td>
      <td>-2.160429</td>
      <td>0.662108</td>
      <td>-0.624840</td>
      <td>-0.585333</td>
      <td>0.378501</td>
      <td>0.211625</td>
      <td>-0.689794</td>
    </tr>
  </tbody>
</table>
<p>379 rows × 13 columns</p>
</div>



# Plot Function


```python
def plot_model(model_coeffs):
  # plotting the coefficient score
  fig, ax = plt.subplots(figsize =(10, 5))
  
  color =['tab:gray', 'tab:blue', 'tab:orange',
  'tab:green', 'tab:red', 'tab:purple', 'tab:brown',
  'tab:pink', 'tab:gray', 'tab:olive', 'tab:cyan',
  'tab:orange', 'tab:green', 'tab:blue', 'tab:olive']
  
  ax.bar(model_coeffs["Columns"],
  model_coeffs['Coefficient Estimate'],
  color = color)
  
  ax.spines['bottom'].set_position('zero')
  
  plt.style.use('ggplot')
  plt.show()

```

# Multiple Linear Regression 

$$y = w_0 + w_1 x_{1} + w_2 x_{2} + ... + w_n x_{n} + \epsilon$$

$$J(w) = \sum_{i=1}^m r_i^2 = \sum_{i=1}^m (y_i - \tilde{y}_i)^2 = \sum_{i=1}^m (y_i - \tilde{w}_0 - \tilde{w}_1 x_1 - ... - \tilde{w}_n x_n)^2$$




```python
# multiple Linear Regression Model
lreg = LinearRegression()
lreg.fit(X_train_scaled, y_train)
 
# Prediccion en test set
lreg_y_pred = lreg.predict(X_test_scaled)

# MSE train set
mse_train = np.mean((lreg.predict(X_train_scaled) - y_train)**2)
print("Mean squared Error on train set : ", mse_train)
 
# calculating Mean Squared Error (mse)
mean_squared_error = np.mean((lreg_y_pred - y_test)**2)
print("Mean squared Error on test set : ", mean_squared_error)
 
# Putting together the coefficient and their corresponding variable names
lreg_coefficient = pd.DataFrame()
lreg_coefficient["Columns"] = X_train.columns
lreg_coefficient['Coefficient Estimate'] = pd.Series(lreg.coef_)
print(lreg_coefficient)
```

    Mean squared Error on train set :  20.11486173313239
    Mean squared Error on test set :  37.173340973712655
        Columns  Coefficient Estimate
    0      CRIM             -0.981919
    1        ZN              1.103023
    2     INDUS              0.706474
    3      CHAS              0.181104
    4       NOX             -2.054656
    5        RM              3.302312
    6       AGE             -0.067952
    7       DIS             -2.860861
    8       RAD              2.200599
    9       TAX             -2.191210
    10  PTRATIO             -2.275399
    11        B              1.048264
    12    LSTAT             -3.420124



```python
plot_model(lreg_coefficient)
```

# Ridge Regression

\begin{equation}
  J(w)= \sum_i (w^T x_i - y_i)^2  + \lambda w^2
\end{equation}




```python
def ridge(lambd, X_train = X_train_scaled, y_train = y_train, X_test = X_test_scaled, y_test = y_test):
  from sklearn.linear_model import Ridge

  # Train the model
  ridgeR = Ridge(alpha = lambd)
  ridgeR.fit(X_train, y_train)
  y_pred = ridgeR.predict(X_test)

  # mse train set
  mse_train = np.mean((ridgeR.predict(X_train) - y_train)**2)
  #print('MSE RIDGE train set :', mse_train)

  
  # mean square error
  mean_squared_error_ridge = np.mean((y_pred - y_test)**2)
  
  #ridge coefficient
  df_ridge_coefficient = pd.DataFrame()
  df_ridge_coefficient["Columns"]= X_train.columns
  df_ridge_coefficient['Coefficient Estimate'] = pd.Series(ridgeR.coef_)
  return mean_squared_error_ridge, df_ridge_coefficient, ridgeR.coef_, mse_train
```


```python
mse_ridge, ridge_coefficient,_,mse_train = ridge(1)
print('MSE RIDGE train set :', mse_train)
print('MSE RIDGE test set :', mse_ridge)
print('-----------------')
print(ridge_coefficient)
```

    MSE RIDGE train set : 21.84825265392536
    MSE RIDGE test set : 27.298676841645808
    -----------------
        Columns  Coefficient Estimate
    0      CRIM             -1.009143
    1        ZN              0.980959
    2     INDUS             -0.194660
    3      CHAS              0.701500
    4       NOX             -1.807307
    5        RM              3.044062
    6       AGE             -0.198334
    7       DIS             -3.205758
    8       RAD              2.488452
    9       TAX             -1.959227
    10  PTRATIO             -2.060459
    11        B              0.594750
    12    LSTAT             -3.844630



```python
plot_model(ridge_coefficient)
```

### Lambda vs MSE


```python
# Veamos como varia el MSE por lambda
lambdas = np.linspace(0,1000,1000)
ridge_test_mses = [ridge(l)[0] for l in lambdas]
ridge_train_mses = [ridge(l)[3] for l in lambdas]

fig = plt.figure(figsize = (10, 5))
fig.add_subplot(1,2,1)
plt.plot(lambdas, ridge_test_mses)
plt.title('Test set')
fig.add_subplot(1,2,2)
plt.plot(lambdas, ridge_train_mses)
plt.title('Train set')
```

### Lambda vs Weights


```python
# Veamos como varia el MSE por lambda
lambdas = 10**np.linspace(10,-2,100)*0.5
ridge_coeffs = [ridge(l)[2] for l in lambdas]
ax = plt.gca()
ax.plot(lambdas, ridge_coeffs)
ax.set_xscale('log')
plt.axis('tight')
plt.xlabel('alpha')
plt.ylabel('weights')
```

# Lasso Regression

\begin{equation}
  J(w)= \sum_i (w^T x_i - y_i)^2  + \lambda ||w||_1
\end{equation}


```python
def lasso(lambd, X_train = X_train_scaled, y_train = y_train, X_test = X_test_scaled, y_test = y_test):
  from sklearn.linear_model import Lasso
 
  # Train the model
  lasso = Lasso(alpha = lambd)
  lasso.fit(X_train, y_train)
  y_pred1 = lasso.predict(X_test)

  # mse train set
  mse_train = np.mean((lasso.predict(X_train) - y_train)**2)
  #print('MSE RIDGE train set :', mse_train)
  
  # Calculate Mean Squared Error
  mean_squared_error = np.mean((y_pred1 - y_test)**2)

  df_lasso_coeff = pd.DataFrame()
  df_lasso_coeff["Columns"] = X_train.columns
  df_lasso_coeff['Coefficient Estimate'] = pd.Series(lasso.coef_)
  
  return mean_squared_error, df_lasso_coeff, lasso.coef_, mse_train



```


```python
mse_lasso, lasso_coefficient,_,mse_train = lasso(1)
print('MSE LASSO train set :', mse_train)
print('MSE LASSO :', mse_lasso)
print('-----------------')
print(lasso_coefficient)
```

    MSE LASSO train set : 28.39264682044312
    MSE LASSO : 29.095045432946005
    -----------------
        Columns  Coefficient Estimate
    0      CRIM             -0.000000
    1        ZN              0.000000
    2     INDUS             -0.000000
    3      CHAS              0.000000
    4       NOX             -0.000000
    5        RM              3.063196
    6       AGE             -0.000000
    7       DIS             -0.000000
    8       RAD             -0.000000
    9       TAX             -0.000000
    10  PTRATIO             -1.555405
    11        B              0.000000
    12    LSTAT             -3.730375



```python
plot_model(lasso_coefficient)
```

### Lambda Vs MSE


```python
# Veamos como varia el MSE por lambda
lambdas = np.linspace(0,30,1000)
lasso_test_mses = [lasso(l)[0] for l in lambdas]
lasso_train_mses = [lasso(l)[3] for l in lambdas]

fig = plt.figure(figsize = (10, 5))
fig.add_subplot(1,2,1)
plt.plot(lambdas, lasso_test_mses)
plt.title('Test set')
fig.add_subplot(1,2,2)
plt.plot(lambdas, lasso_train_mses)
plt.title('Train set')
```

### Lambda vs Weights


```python
# Veamos como varia el MSE por lambda
lambdas = 10**np.linspace(10,-2,100)*0.5
lasso_coeffs = [lasso(l)[2] for l in lambdas]
ax = plt.gca()
ax.plot(lambdas, lasso_coeffs)
ax.set_xscale('log')
plt.axis('tight')
plt.xlabel('alpha')
plt.ylabel('weights')
```

# Conclusiones:
## Ridge: 
- Incluye todas (o ninguna) las variables del modelo. Por lo tanto, la principal ventaja de la regresión ridge es el 'shrinkage' del coeficiente y la reducción de la complejidad del modelo.
- Normalmente se utiliza para reducit el sobre ajuste
- En general, funciona bien incluso en presencia de variables altamente correlacionadas, ya que las incluirá todas en el modelo, pero los coeficientes se distribuirán entre ellas según la correlación. 


## Lasso: 
- Junto con los coeficientes de reducción, Lasso también realiza la selección de variables.Algunos de los coeficientes se vuelven exactamente cero, lo que equivale a que la variable particular se excluya del modelo.

- Normalmente se utiliza para reducir las variables y obtener una'sparse solution' (Ej: cuando hay miles de variables)

- Selecciona arbitrariamente cualquier característica entre las altamente correlacionadas y reduce los coeficientes del resto a cero. Además, la variable elegida cambia aleatoriamente con el cambio en los parámetros del modelo. Por lo general, esto no funciona tan bien en comparación con la regresión ridge



# Referencias:
- https://www.cienciadedatos.net/documentos/py14-ridge-lasso-elastic-net-python.html
- https://www.analyticsvidhya.com/blog/2016/01/ridge-lasso-regression-python-complete-tutorial/
- https://www.geeksforgeeks.org/implementation-of-lasso-ridge-and-elastic-net/
- https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.Lasso.html
- https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.Ridge.html


```python

```


```python

```


```python

```

d46a0dba27ab452810c1181d37495ac5510ddafd
ipynb
Jupyter Notebook
Ayudantia 9/tEEds_Regression.ipynb
diegulio/tEEds
d2b47c0f120947b53d17c04c8ba54258988f93d3
MIT
Ayudantia 9/tEEds_Regression.ipynb
diegulio/tEEds
d2b47c0f120947b53d17c04c8ba54258988f93d3
MIT
Ayudantia 9/tEEds_Regression.ipynb
diegulio/tEEds
d2b47c0f120947b53d17c04c8ba54258988f93d3
MIT
Qwen/Qwen-72B
1. YES
2. YES