## Multiple Linear Regression
We will see how multiple input variables together influence the output variable, while also learning how the calculations differ from that of Simple LR model. We will also build a regression model using Python.
At last, we will go deeper into Linear Regression and will learn things like Multicollinearity, Hypothesis Testing, Feature Selection, and much more.


```python
# Importing required libraries
import pandas as pd
import numpy as np
%matplotlib inline
import matplotlib.pyplot as plt
from mpl_toolkits import mplot3d
from mpl_toolkits.mplot3d import Axes3D
```

### Advertising Data
We are going to use Advertising data which is available on the site of USC Marshall School of Business. 
The advertising data set consists of the sales of a product in 200 different markets, along with advertising budgets for three different media: TV, radio, and newspaper.

Loading and plotting the Data


```python
data = pd.read_csv("Advertising.csv")
data.drop("Unnamed: 0", axis =1,inplace=True)
data
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>TV</th>
      <th>Radio</th>
      <th>Newspaper</th>
      <th>Sales</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>230.1</td>
      <td>37.8</td>
      <td>69.2</td>
      <td>22.1</td>
    </tr>
    <tr>
      <th>1</th>
      <td>44.5</td>
      <td>39.3</td>
      <td>45.1</td>
      <td>10.4</td>
    </tr>
    <tr>
      <th>2</th>
      <td>17.2</td>
      <td>45.9</td>
      <td>69.3</td>
      <td>9.3</td>
    </tr>
    <tr>
      <th>3</th>
      <td>151.5</td>
      <td>41.3</td>
      <td>58.5</td>
      <td>18.5</td>
    </tr>
    <tr>
      <th>4</th>
      <td>180.8</td>
      <td>10.8</td>
      <td>58.4</td>
      <td>12.9</td>
    </tr>
    <tr>
      <th>...</th>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
    </tr>
    <tr>
      <th>195</th>
      <td>38.2</td>
      <td>3.7</td>
      <td>13.8</td>
      <td>7.6</td>
    </tr>
    <tr>
      <th>196</th>
      <td>94.2</td>
      <td>4.9</td>
      <td>8.1</td>
      <td>9.7</td>
    </tr>
    <tr>
      <th>197</th>
      <td>177.0</td>
      <td>9.3</td>
      <td>6.4</td>
      <td>12.8</td>
    </tr>
    <tr>
      <th>198</th>
      <td>283.6</td>
      <td>42.0</td>
      <td>66.2</td>
      <td>25.5</td>
    </tr>
    <tr>
      <th>199</th>
      <td>232.1</td>
      <td>8.6</td>
      <td>8.7</td>
      <td>13.4</td>
    </tr>
  </tbody>
</table>
<p>200 rows × 4 columns</p>
</div>




```python
plt.scatter(data.TV, data.Sales, color='blue', label='TV', alpha=0.5)
plt.scatter(data.Radio, data.Sales, color='red', label='radio', alpha=0.5)
plt.scatter(data.Newspaper, data.Sales, color='green', label='newspaper', alpha=0.5)

plt.legend(loc="lower right")
plt.title("Sales vs. Advertising")
plt.xlabel("Advertising [1000 $]")
plt.ylabel("Sales [Thousands of units]")
plt.grid()
plt.show()
```

----

#### Mathematically, general Linear Regression model can be expressed as:
\begin{align}
Y= β_0+ β_1 X_1+ β_2 X_2+⋯+ β_p X_p
\end{align}

Here, Y is the output variable, and X terms are the corresponding input variables.The first β term (βo) is the intercept constant and is the value of Y in absence of all predictors (i.e when all X terms are 0). 

Finding the values of these constants(β) is what regression model does by minimizing the error function and fitting the best line or hyperplane (depending on the number of input variables).
This is done by minimizing the Residual Sum of Squares (RSS), which is obtained by squaring the differences between actual and predicted outcomes.

\begin{align}
RSS=∑_i^n(y_i-y ̂_i )^2 
\end{align}

Because this method finds the least sum of squares, it is also known as the **Ordinary Least Squares** (OLS) method.

There are two primary ways to implement the OLS algorithm: **Scikit Learn** and **Statsmodels**

#### SciKit Learn: 
Just import the Linear Regression module from the Sklearn package and fit the model on the data. This method is pretty straightforward and you can see how to use it below.


```python
from sklearn.linear_model import LinearRegression
sk_model = LinearRegression()
sk_model.fit(data.drop('Sales', axis=1), data.Sales)
```




    LinearRegression(copy_X=True, fit_intercept=True, n_jobs=None, normalize=False)




```python
print("Intercept: ", sk_model.intercept_)
print("Coefficients: ", sk_model.coef_)
```

    Intercept:  2.938889369459412
    Coefficients:  [ 0.04576465  0.18853002 -0.00103749]


#### Statsmodels
Another way is to use the Statsmodels package to implement OLS. Statsmodels is a Python package that allows performing various statistical tests on the data. We will use it here because it will be helpful for us later as well.


```python
# Importing statsmodels
import statsmodels.formula.api as sm

# Fitting the OLS on data
model = sm.ols('Sales ~ TV + Radio + Newspaper', data).fit()
print(model.params)
```

    Intercept    2.938889
    TV           0.045765
    Radio        0.188530
    Newspaper   -0.001037
    dtype: float64


    /usr/local/lib/python3.7/dist-packages/statsmodels/tools/_testing.py:19: FutureWarning: pandas.util.testing is deprecated. Use the functions in the public API at pandas.testing instead.
      import pandas.util.testing as tm


These results can be interpreted as follows:

* If we fix the budget for TV & newspaper, then increasing the radio budget by 1000 USD  will lead to an increase in sales by around **189 units**(0.189*1000).

* Similarly, by fixing the radio & newspaper, we infer an approximate rise of **46 units** of products per 1000 USD increase in the TV budget.

* However, for the newspaper budget, since the coefficient is quite negligible (close to zero), it’s evident that the newspaper is not affecting the sales. In fact, it’s on the negative side of zero(-0.001) which, if the magnitude was big enough, could have meant that this agent is rather causing the sales to fall. But we cannot make that kind of inference with such negligible value.

If sales is regressed solely on newspaper (as shown below), the slope coefficient will come out to be 0.055, which is significanly large as compared to what we saw above. 


```python
# Simple Linear regression for sales vs newspaper
model_npaper = sm.ols('Sales ~ Newspaper', data).fit()
print(model_npaper.params)
```

    Intercept    12.351407
    Newspaper     0.054693
    dtype: float64


#### This is explained by **Multicollinearity**

Let's plot and observe the correlations among the variables 


```python
data.corr()
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>TV</th>
      <th>Radio</th>
      <th>Newspaper</th>
      <th>Sales</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>TV</th>
      <td>1.000000</td>
      <td>0.054809</td>
      <td>0.056648</td>
      <td>0.782224</td>
    </tr>
    <tr>
      <th>Radio</th>
      <td>0.054809</td>
      <td>1.000000</td>
      <td>0.354104</td>
      <td>0.576223</td>
    </tr>
    <tr>
      <th>Newspaper</th>
      <td>0.056648</td>
      <td>0.354104</td>
      <td>1.000000</td>
      <td>0.228299</td>
    </tr>
    <tr>
      <th>Sales</th>
      <td>0.782224</td>
      <td>0.576223</td>
      <td>0.228299</td>
      <td>1.000000</td>
    </tr>
  </tbody>
</table>
</div>



#### Plotting the heatmap to visualize the Correlation


```python
# Plotting correlation heatmap
plt.ylim(-.5,3.5)
plt.imshow(data.corr(), cmap=plt.cm.GnBu, interpolation='nearest',data=True)
plt.colorbar()
tick_marks = [i for i in range(len(data.columns))]
plt.xticks(tick_marks, data.columns, rotation=45)
plt.yticks(tick_marks, data.columns, rotation=45)

# Putting annotations
for i in range(len(data.columns)):
    for j in range(len(data.columns)):
        text = '%.2f'%(data.corr().iloc[i,j])
        plt.text(i-0.2,j-0.1,text)
```

The correlation between newspaper and radio is 0.35. This indicates a fair relationship between newspaper and radio budgets. 

Hence, it can be inferred that → when the radio budget is increased for a product, there’s a tendency to spend more on newspapers as well.

This is called Multicollinearity and is referred to as a situation in which two or more input variables are linearly related.
Hence, even though the Multiple Regression model shows no impact on sales by the newspaper, the Simple Regression model still does due to this multicollinearity and the absence of other input variables.

$Sales & Radio → probable causation
Newspaper & Radio → multicollinearity
Sales & Newspaper → transitive correlation$

---

### Hypothesis Testing

One of the fundamental questions that should be answered while running Multiple Linear Regression is, whether or not, at least one of the predictors is useful in predicting the output.

We saw that the three predictors TV, radio and newspaper had a different degree of linear relationship with the sales. But what if the relationship is just by chance and there is no actual impact on sales due to any of the predictors?

The model can only give us numbers to establish a close enough linear relationship between the response variable and the predictors. However, it cannot prove the credibility of these relationships.

To have some confidence, we take help from statistics and do something known as a Hypothesis Test. We start by forming a Null Hypothesis and a corresponding Alternative Hypothesis.

<center> NULL HYPOTHESIS </center>
\begin{align}
H_0 : \beta_1 = \beta_2 = ... = \beta_p = 0
\end{align}
\begin{align}
H_0 : \beta_TV = \beta_radio = \beta_newspaper = 0
\end{align}

<center> ALTERNATIVE HYPOTHESIS </center>
\begin{align}
H_a : At \: least \: one \: \beta_i \: is \: non-zero
\end{align}

The hypothesis test is performed by using F-Statistic. The formula for this statistic contains Residual Sum of Squares (RSS) and the Total Sum of Squares (TSS), which we can calculate using the Statsmodels. 


```python
print(model.summary2())
```

                     Results: Ordinary least squares
    =================================================================
    Model:              OLS              Adj. R-squared:     0.896   
    Dependent Variable: Sales            AIC:                780.3622
    Date:               2021-03-09 18:29 BIC:                793.5555
    No. Observations:   200              Log-Likelihood:     -386.18 
    Df Model:           3                F-statistic:        570.3   
    Df Residuals:       196              Prob (F-statistic): 1.58e-96
    R-squared:          0.897            Scale:              2.8409  
    ------------------------------------------------------------------
                    Coef.   Std.Err.     t     P>|t|    [0.025  0.975]
    ------------------------------------------------------------------
    Intercept       2.9389    0.3119   9.4223  0.0000   2.3238  3.5540
    TV              0.0458    0.0014  32.8086  0.0000   0.0430  0.0485
    Radio           0.1885    0.0086  21.8935  0.0000   0.1715  0.2055
    Newspaper      -0.0010    0.0059  -0.1767  0.8599  -0.0126  0.0105
    -----------------------------------------------------------------
    Omnibus:             60.414       Durbin-Watson:          2.084  
    Prob(Omnibus):       0.000        Jarque-Bera (JB):       151.241
    Skew:                -1.327       Prob(JB):               0.000  
    Kurtosis:            6.332        Condition No.:          454    
    =================================================================
    


If the value of F-statistic is equal to or very close to 1, then the results are in favor of the Null Hypothesis and we fail to reject it.

But as we can see that the F-statistic is many folds larger than 1, thus providing strong evidence against the Null Hypothesis (that all coefficients are zero). Hence, we **reject the Null Hypothesis** and are confident that at least one predictor is useful in predicting the output.

*Note that F-statistic is not suitable when the number of predictors(p) is large, or if p is greater than the number of data samples (n).*

Hence, we can say that at least one of the three advertising agents is useful in predicting sales.

But to find out, which predictor or predictors are useful and which one are not, we do **Feature Selection**

Two of the ways of doing Feature Selection are **Forward Selecton** & **Backward Selection**

Let's proceed with forward selection. We start with a model without any predictor and just the intercept term. We then perform simple linear regression for each predictor to find the best performer(lowest RSS). We then add another variable to it and check for the best 2-variable combination again by calculating the lowest RSS(Residual Sum of Squares). After that the best 3-variable combination is checked, and so on. The approach is stopped when some stopping rule is satisfied.


```python
# Defining the function to evaluate amodel
def evaluateModel (model):
    print("RSS = ", ((data.sales - model.predict())**2).sum())
    print("R2 = ", model.rsquared)
```

Single Predictor Models


```python
# For TV
model_TV = sm.ols('Sales ~ TV', data=data).fit()
print("model_TV")
print(model_TV.summary())
print("------------")


# For radio
model_radio = sm.ols('Sales ~ Radio', data=data).fit()
print("model_radio")
print(model_radio.summary())
print("------------")

# For newspaper
model_newspaper = sm.ols('Sales ~ Newspaper', data=data).fit()
print("model_newspaper")
print(model_newspaper.summary())
print("------------")
```

    model_TV
                                OLS Regression Results                            
    ==============================================================================
    Dep. Variable:                  Sales   R-squared:                       0.612
    Model:                            OLS   Adj. R-squared:                  0.610
    Method:                 Least Squares   F-statistic:                     312.1
    Date:                Tue, 09 Mar 2021   Prob (F-statistic):           1.47e-42
    Time:                        18:29:01   Log-Likelihood:                -519.05
    No. Observations:                 200   AIC:                             1042.
    Df Residuals:                     198   BIC:                             1049.
    Df Model:                           1                                         
    Covariance Type:            nonrobust                                         
    ==============================================================================
                     coef    std err          t      P>|t|      [0.025      0.975]
    ------------------------------------------------------------------------------
    Intercept      7.0326      0.458     15.360      0.000       6.130       7.935
    TV             0.0475      0.003     17.668      0.000       0.042       0.053
    ==============================================================================
    Omnibus:                        0.531   Durbin-Watson:                   1.935
    Prob(Omnibus):                  0.767   Jarque-Bera (JB):                0.669
    Skew:                          -0.089   Prob(JB):                        0.716
    Kurtosis:                       2.779   Cond. No.                         338.
    ==============================================================================
    
    Warnings:
    [1] Standard Errors assume that the covariance matrix of the errors is correctly specified.
    ------------
    model_radio
                                OLS Regression Results                            
    ==============================================================================
    Dep. Variable:                  Sales   R-squared:                       0.332
    Model:                            OLS   Adj. R-squared:                  0.329
    Method:                 Least Squares   F-statistic:                     98.42
    Date:                Tue, 09 Mar 2021   Prob (F-statistic):           4.35e-19
    Time:                        18:29:01   Log-Likelihood:                -573.34
    No. Observations:                 200   AIC:                             1151.
    Df Residuals:                     198   BIC:                             1157.
    Df Model:                           1                                         
    Covariance Type:            nonrobust                                         
    ==============================================================================
                     coef    std err          t      P>|t|      [0.025      0.975]
    ------------------------------------------------------------------------------
    Intercept      9.3116      0.563     16.542      0.000       8.202      10.422
    Radio          0.2025      0.020      9.921      0.000       0.162       0.243
    ==============================================================================
    Omnibus:                       19.358   Durbin-Watson:                   1.946
    Prob(Omnibus):                  0.000   Jarque-Bera (JB):               21.910
    Skew:                          -0.764   Prob(JB):                     1.75e-05
    Kurtosis:                       3.544   Cond. No.                         51.4
    ==============================================================================
    
    Warnings:
    [1] Standard Errors assume that the covariance matrix of the errors is correctly specified.
    ------------
    model_newspaper
                                OLS Regression Results                            
    ==============================================================================
    Dep. Variable:                  Sales   R-squared:                       0.052
    Model:                            OLS   Adj. R-squared:                  0.047
    Method:                 Least Squares   F-statistic:                     10.89
    Date:                Tue, 09 Mar 2021   Prob (F-statistic):            0.00115
    Time:                        18:29:01   Log-Likelihood:                -608.34
    No. Observations:                 200   AIC:                             1221.
    Df Residuals:                     198   BIC:                             1227.
    Df Model:                           1                                         
    Covariance Type:            nonrobust                                         
    ==============================================================================
                     coef    std err          t      P>|t|      [0.025      0.975]
    ------------------------------------------------------------------------------
    Intercept     12.3514      0.621     19.876      0.000      11.126      13.577
    Newspaper      0.0547      0.017      3.300      0.001       0.022       0.087
    ==============================================================================
    Omnibus:                        6.231   Durbin-Watson:                   1.983
    Prob(Omnibus):                  0.044   Jarque-Bera (JB):                5.483
    Skew:                           0.330   Prob(JB):                       0.0645
    Kurtosis:                       2.527   Cond. No.                         64.7
    ==============================================================================
    
    Warnings:
    [1] Standard Errors assume that the covariance matrix of the errors is correctly specified.
    ------------


We observe that for model_TV, the RSS is least and R² value is the most among all the models. Hence we select model_TV as our base model to move forward.

Now, we will add the radio and newspaper one by one and check the new values.


```python
# For TV & radio
model_TV_radio = sm.ols('Sales ~ TV + Radio', data=data).fit()
print("model_TV_radio")
print(model_TV_radio.summary())
print("------------")

# For TV & newspaper
model_TV_newspaper = sm.ols('Sales ~ TV + Newspaper', data=data).fit()
print("model_TV_newspaper")
print(model_TV_newspaper.summary())
print("------------")
```

    model_TV_radio
                                OLS Regression Results                            
    ==============================================================================
    Dep. Variable:                  Sales   R-squared:                       0.897
    Model:                            OLS   Adj. R-squared:                  0.896
    Method:                 Least Squares   F-statistic:                     859.6
    Date:                Tue, 09 Mar 2021   Prob (F-statistic):           4.83e-98
    Time:                        18:29:01   Log-Likelihood:                -386.20
    No. Observations:                 200   AIC:                             778.4
    Df Residuals:                     197   BIC:                             788.3
    Df Model:                           2                                         
    Covariance Type:            nonrobust                                         
    ==============================================================================
                     coef    std err          t      P>|t|      [0.025      0.975]
    ------------------------------------------------------------------------------
    Intercept      2.9211      0.294      9.919      0.000       2.340       3.502
    TV             0.0458      0.001     32.909      0.000       0.043       0.048
    Radio          0.1880      0.008     23.382      0.000       0.172       0.204
    ==============================================================================
    Omnibus:                       60.022   Durbin-Watson:                   2.081
    Prob(Omnibus):                  0.000   Jarque-Bera (JB):              148.679
    Skew:                          -1.323   Prob(JB):                     5.19e-33
    Kurtosis:                       6.292   Cond. No.                         425.
    ==============================================================================
    
    Warnings:
    [1] Standard Errors assume that the covariance matrix of the errors is correctly specified.
    ------------
    model_TV_newspaper
                                OLS Regression Results                            
    ==============================================================================
    Dep. Variable:                  Sales   R-squared:                       0.646
    Model:                            OLS   Adj. R-squared:                  0.642
    Method:                 Least Squares   F-statistic:                     179.6
    Date:                Tue, 09 Mar 2021   Prob (F-statistic):           3.95e-45
    Time:                        18:29:01   Log-Likelihood:                -509.89
    No. Observations:                 200   AIC:                             1026.
    Df Residuals:                     197   BIC:                             1036.
    Df Model:                           2                                         
    Covariance Type:            nonrobust                                         
    ==============================================================================
                     coef    std err          t      P>|t|      [0.025      0.975]
    ------------------------------------------------------------------------------
    Intercept      5.7749      0.525     10.993      0.000       4.739       6.811
    TV             0.0469      0.003     18.173      0.000       0.042       0.052
    Newspaper      0.0442      0.010      4.346      0.000       0.024       0.064
    ==============================================================================
    Omnibus:                        0.658   Durbin-Watson:                   1.969
    Prob(Omnibus):                  0.720   Jarque-Bera (JB):                0.415
    Skew:                          -0.093   Prob(JB):                        0.813
    Kurtosis:                       3.122   Cond. No.                         410.
    ==============================================================================
    
    Warnings:
    [1] Standard Errors assume that the covariance matrix of the errors is correctly specified.
    ------------


As we can see that our values have improved tremendously in TV & radio model. RSS has increased and R² has decreased further, as compared to model_TV. It’s a good sign. 

On the other hand, it's not the same for TV and newspaper. The values have improved slightly by adding newspaper too, but not as significantly as with the radio. Hence, at this step, we will proceed with the TV & radio model and will observe the difference when we add newspaper to this model.


```python
# For TV, radio & newspaper
model_all = sm.ols('Sales ~ TV + Radio + Newspaper', data=data).fit()
print("model_all")
print(model_all.summary())
print("------------")
```

    model_all
                                OLS Regression Results                            
    ==============================================================================
    Dep. Variable:                  Sales   R-squared:                       0.897
    Model:                            OLS   Adj. R-squared:                  0.896
    Method:                 Least Squares   F-statistic:                     570.3
    Date:                Tue, 09 Mar 2021   Prob (F-statistic):           1.58e-96
    Time:                        18:29:02   Log-Likelihood:                -386.18
    No. Observations:                 200   AIC:                             780.4
    Df Residuals:                     196   BIC:                             793.6
    Df Model:                           3                                         
    Covariance Type:            nonrobust                                         
    ==============================================================================
                     coef    std err          t      P>|t|      [0.025      0.975]
    ------------------------------------------------------------------------------
    Intercept      2.9389      0.312      9.422      0.000       2.324       3.554
    TV             0.0458      0.001     32.809      0.000       0.043       0.049
    Radio          0.1885      0.009     21.893      0.000       0.172       0.206
    Newspaper     -0.0010      0.006     -0.177      0.860      -0.013       0.011
    ==============================================================================
    Omnibus:                       60.414   Durbin-Watson:                   2.084
    Prob(Omnibus):                  0.000   Jarque-Bera (JB):              151.241
    Skew:                          -1.327   Prob(JB):                     1.44e-33
    Kurtosis:                       6.332   Cond. No.                         454.
    ==============================================================================
    
    Warnings:
    [1] Standard Errors assume that the covariance matrix of the errors is correctly specified.
    ------------


The values have not improved with any significance. Hence, it’s imperative to not add newspaper and finalize the model with TV and radio as selected features.
So our final model can be expressed as below:
\begin{align}
sales = 2.92 + 0.046*TV + 0.188*radio
\end{align}

Plotting the variables TV, radio, and sales in the 3D graph, we can visualize how our model has fit a regression plane to the data.


```python
fig = plt.figure(figsize=(12,10))
ax = fig.add_subplot(211, projection='3d')

fig.suptitle('Regression: Sales ~ TV & radio Advertising')


# Defining z function (or sales in terms of TV and radio)
def z_function(x,y):
    return (2.938889 + (0.045765*y) + (0.188530*x))

X, Y = np.meshgrid(range(0,50,2),range(0,300,10))
Z = z_function(X, Y)


## Creating Wireframe
ax.plot_wireframe(X, Y, Z, color='black')
ax.set_xlabel('x')
ax.set_ylabel('y')
ax.set_zlabel('z')

## Creating Surface plot
ax.plot_surface(X, Y, Z, rstride=1, cstride=1, cmap='winter', edgecolor='black', alpha=0.5)

## Adding Scatter Plot
ax.scatter(data.Radio, data.TV, data.Sales, c='red', s=25)

## Adding labels
ax.set_xlabel('Radio')
ax.set_ylabel('TV')
ax.set_zlabel('Sales')
ax.text(0,150,1, '@DataScienceWithSan')

## Rotating for better view
ax.view_init(10,30)
```

6da3c734da62ed5271f59e32d2cf68280e7323f2
ipynb
Jupyter Notebook
05_Multi Linear Regression/03_Advertisment_Solution.ipynb
kunalk3/eR_task
4d717680576bc62793c42840e772a28fb3f6c223
MIT
2021-02-25T13:58:57.000Z
2021-02-25T13:58:57.000Z
05_Multi Linear Regression/03_Advertisment_Solution.ipynb
kunalk3/eR_task
4d717680576bc62793c42840e772a28fb3f6c223
MIT
05_Multi Linear Regression/03_Advertisment_Solution.ipynb
kunalk3/eR_task
4d717680576bc62793c42840e772a28fb3f6c223
MIT
Qwen/Qwen-72B
1. YES
2. YES