# Week 4 Lab Session (ctd..): Advertising Data 

* In this lab session we are going to look at how to answer questions involing the use of simple and multiple linear regression in Python. 

* The questions are based on the book "An introduction to Statistical Learning" by James et al. 


```python
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline
import scipy.stats as stats
import statsmodels.api as sm
import statsmodels.formula.api as smf
import seaborn as sns
```


```python
data_advert = pd.read_csv('http://www-bcf.usc.edu/~gareth/ISL/Advertising.csv', index_col=0)
data_advert.head()
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>TV</th>
      <th>radio</th>
      <th>newspaper</th>
      <th>sales</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>1</th>
      <td>230.1</td>
      <td>37.8</td>
      <td>69.2</td>
      <td>22.1</td>
    </tr>
    <tr>
      <th>2</th>
      <td>44.5</td>
      <td>39.3</td>
      <td>45.1</td>
      <td>10.4</td>
    </tr>
    <tr>
      <th>3</th>
      <td>17.2</td>
      <td>45.9</td>
      <td>69.3</td>
      <td>9.3</td>
    </tr>
    <tr>
      <th>4</th>
      <td>151.5</td>
      <td>41.3</td>
      <td>58.5</td>
      <td>18.5</td>
    </tr>
    <tr>
      <th>5</th>
      <td>180.8</td>
      <td>10.8</td>
      <td>58.4</td>
      <td>12.9</td>
    </tr>
  </tbody>
</table>
</div>




```python
data_advert.describe()
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>TV</th>
      <th>radio</th>
      <th>newspaper</th>
      <th>sales</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>count</th>
      <td>200.000000</td>
      <td>200.000000</td>
      <td>200.000000</td>
      <td>200.000000</td>
    </tr>
    <tr>
      <th>mean</th>
      <td>147.042500</td>
      <td>23.264000</td>
      <td>30.554000</td>
      <td>14.022500</td>
    </tr>
    <tr>
      <th>std</th>
      <td>85.854236</td>
      <td>14.846809</td>
      <td>21.778621</td>
      <td>5.217457</td>
    </tr>
    <tr>
      <th>min</th>
      <td>0.700000</td>
      <td>0.000000</td>
      <td>0.300000</td>
      <td>1.600000</td>
    </tr>
    <tr>
      <th>25%</th>
      <td>74.375000</td>
      <td>9.975000</td>
      <td>12.750000</td>
      <td>10.375000</td>
    </tr>
    <tr>
      <th>50%</th>
      <td>149.750000</td>
      <td>22.900000</td>
      <td>25.750000</td>
      <td>12.900000</td>
    </tr>
    <tr>
      <th>75%</th>
      <td>218.825000</td>
      <td>36.525000</td>
      <td>45.100000</td>
      <td>17.400000</td>
    </tr>
    <tr>
      <th>max</th>
      <td>296.400000</td>
      <td>49.600000</td>
      <td>114.000000</td>
      <td>27.000000</td>
    </tr>
  </tbody>
</table>
</div>




```python
plt.subplot(131)
plt.scatter(data_advert.TV,data_advert.sales)
plt.xlabel('TV')
plt.ylabel('Sales')

plt.subplot(132)
plt.scatter(data_advert.radio,data_advert.sales)
plt.xlabel('radio')

plt.subplot(133)
plt.scatter(data_advert.newspaper,data_advert.sales)
plt.xlabel('newspaper')
plt.subplots_adjust(top=0.8, bottom=0.08, left=0.0, right=1.3, hspace=5, wspace=0.5)
```

## QUESTION 1: Is there a relationship between advertising sales and budget?

* Test the null hypothesis 
\begin{equation}
H_0: \beta_1=\ldots=\beta_p=0
\end{equation}
versus the alternative
\begin{equation}
H_a: \text{at least one $\beta_j$ is nonzero}
\end{equation}
* For that compute the F-statistic in Multiple Linear Regression 'sales ~ TV+radio+newspaper' using $\texttt{ols}$ from ${\bf Statsmodels}$
* If there is no relationship between the response and predictors, the F-statistic takes values close to 1. If $H_a$ is true, than F-statistic is expected to be significantly greater than 1. Check the associated p-values. 


```python
results = smf.ols('sales ~ TV+radio+newspaper', data=data_advert).fit()
print(results.summary())
```

                                OLS Regression Results                            
    ==============================================================================
    Dep. Variable:                  sales   R-squared:                       0.897
    Model:                            OLS   Adj. R-squared:                  0.896
    Method:                 Least Squares   F-statistic:                     570.3
    Date:                Sun, 28 Oct 2018   Prob (F-statistic):           1.58e-96
    Time:                        23:46:29   Log-Likelihood:                -386.18
    No. Observations:                 200   AIC:                             780.4
    Df Residuals:                     196   BIC:                             793.6
    Df Model:                           3                                         
    Covariance Type:            nonrobust                                         
    ==============================================================================
                     coef    std err          t      P>|t|      [0.025      0.975]
    ------------------------------------------------------------------------------
    Intercept      2.9389      0.312      9.422      0.000       2.324       3.554
    TV             0.0458      0.001     32.809      0.000       0.043       0.049
    radio          0.1885      0.009     21.893      0.000       0.172       0.206
    newspaper     -0.0010      0.006     -0.177      0.860      -0.013       0.011
    ==============================================================================
    Omnibus:                       60.414   Durbin-Watson:                   2.084
    Prob(Omnibus):                  0.000   Jarque-Bera (JB):              151.241
    Skew:                          -1.327   Prob(JB):                     1.44e-33
    Kurtosis:                       6.332   Cond. No.                         454.
    ==============================================================================
    
    Warnings:
    [1] Standard Errors assume that the covariance matrix of the errors is correctly specified.


## QUESTION 2: How strong is the relationship?

You should base your discussion on the following quantities:
* $RSE$ - computed using $\texttt{scale}$ atribute: $\texttt{np.sqrt(results.scale)}$
    * Compute the percentage error, i.e. $RSE/(mean sale)$
* $R^2$ - computed using $\texttt{rsquared}$ atribute: $\texttt{results.rsquared}$

## QUESTION 3: Which media contribute to sales?

* Examine the p-values associated with each predictor’s t-statistic

## QUESTION 4: How large is the effect of each medium on sales?

* Examine 95% confidence intervals associated with each predictor
* Compare your results with three separate simple lineare regression

## QUESTION 5: Is the relationship linear?
* You can use residual versus fitted value plot to investigate this

## QUESTION 6: Is there interaction among the advertising media?
* Consider model sales ~ TV + radio + TV:radio
* How much more variability are we able to explain with this model?


```python
results = smf.ols('sales ~ TV + radio + TV:radio', data=data_advert).fit()
print(results.summary())
```

                                OLS Regression Results                            
    ==============================================================================
    Dep. Variable:                  sales   R-squared:                       0.968
    Model:                            OLS   Adj. R-squared:                  0.967
    Method:                 Least Squares   F-statistic:                     1466.
    Date:                Mon, 29 Oct 2018   Prob (F-statistic):          2.92e-144
    Time:                        00:09:17   Log-Likelihood:                -270.04
    No. Observations:                 200   AIC:                             550.1
    Df Residuals:                     195   BIC:                             566.6
    Df Model:                           4                                         
    Covariance Type:            nonrobust                                         
    ==============================================================================
                     coef    std err          t      P>|t|      [0.025      0.975]
    ------------------------------------------------------------------------------
    Intercept      6.7284      0.253     26.561      0.000       6.229       7.228
    TV             0.0191      0.002     12.633      0.000       0.016       0.022
    radio          0.0280      0.009      3.062      0.003       0.010       0.046
    newspaper      0.0014      0.003      0.438      0.662      -0.005       0.008
    TV:radio       0.0011   5.26e-05     20.686      0.000       0.001       0.001
    ==============================================================================
    Omnibus:                      126.161   Durbin-Watson:                   2.216
    Prob(Omnibus):                  0.000   Jarque-Bera (JB):             1123.463
    Skew:                          -2.291   Prob(JB):                    1.10e-244
    Kurtosis:                      13.669   Cond. No.                     1.84e+04
    ==============================================================================
    
    Warnings:
    [1] Standard Errors assume that the covariance matrix of the errors is correctly specified.
    [2] The condition number is large, 1.84e+04. This might indicate that there are
    strong multicollinearity or other numerical problems.


532f6847d07209663f127e044efdf0f02a6d9db3
ipynb
Jupyter Notebook
Week_4_Advertising_Lab_AA.ipynb
abdulra4/Analytical-Statistics
43e2679388a7aab5c64434e5222b213a9a70fa2e
MIT
Week_4_Advertising_Lab_AA.ipynb
abdulra4/Analytical-Statistics
43e2679388a7aab5c64434e5222b213a9a70fa2e
MIT
Week_4_Advertising_Lab_AA.ipynb
abdulra4/Analytical-Statistics
43e2679388a7aab5c64434e5222b213a9a70fa2e
MIT
Qwen/Qwen-72B
1. YES
2. YES