<jupyter_start><jupyter_text>Advertising Sales Channel Prediction Project  Problem Statement:Sales Channel Prediction Case StudyWhen a company enters a market, the distribution strategy and channel it uses are keys to its success in the market, as well as market know-how and customer knowledge and understanding. Because an effective distribution strategy under efficient supply-chain management opens doors for attaining competitive advantage and strong brand equity in the market, it is a component of the marketing mix that cannot be ignored.The distribution strategy and the channel design have to be right the first time. The case study of Sales channel includes the detailed study of TV, radio and newspaper channel. The predict the total sales generated from all the sales channel.<jupyter_code>import warnings
warnings.simplefilter("ignore")
import joblib

import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt
%matplotlib inline
from math import sqrt

from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.svm import SVR
from sklearn.tree import DecisionTreeRegressor
from sklearn.ensemble import RandomForestRegressor
from sklearn.neighbors import KNeighborsRegressor
from sklearn.ensemble import AdaBoostRegressor
from sklearn.ensemble import ExtraTreesRegressor
from sklearn.ensemble import GradientBoostingRegressor

from sklearn.metrics import r2_score
from sklearn.metrics import mean_squared_error
from sklearn.model_selection import cross_val_score
from sklearn.model_selection import GridSearchCV<jupyter_output><empty_output><jupyter_text>Importing all the necessary libraries here.<jupyter_code>df = pd.read_csv("https://raw.githubusercontent.com/dsrscientist/DSData/master/Advertising.csv")<jupyter_output><empty_output><jupyter_text>Instead of downloading the entire dataset on my local computer I am simply loading the file directly from the GitHub repository link using the raw option.<jupyter_code>df<jupyter_output><empty_output><jupyter_text>In the above cell we can see the first five and last five rows of our data set and as per the problem statement the column "sales" is our target label for prediction making the other columns our features. Also looking at the unnamed column one can say that it is useless in terms of contributing to any data and removing it will be better for our model.Considering how we have continous data present in our target column this becomes a Regression problem!  Exploratory Data Analysis (EDA)<jupyter_code>df = df.drop("Unnamed: 0", axis=1)<jupyter_output><empty_output><jupyter_text>I have dropped the unnamed column as it was not giving any insight over the sales details and was just a numbered data that was basically of no use to us since we have indexing present for our data frame.<jupyter_code>df.isnull().sum()<jupyter_output><empty_output><jupyter_text>Checked for missing values and luckily we hav no data missing therefore we won't have to worry about handling missing data in our data set.<jupyter_code>df.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 200 entries, 0 to 199
Data columns (total 4 columns):
 #   Column     Non-Null Count  Dtype  
---  ------     --------------  -----  
 0   TV         200 non-null    float64
 1   radio      200 non-null    float64
 2   newspaper  200 non-null    float64
 3   sales      200 non-null    float64
dtypes: float64(4)
memory usage: 6.4 KB<jupyter_text>Using the info method we can see that the 3 feature column along with the target column are all of the same data type float making it easier for us to go through further data pre processing steps.<jupyter_code>df.describe()<jupyter_output><empty_output><jupyter_text>Using the describe method I can see the count, mean, standard deviation, minimum, maximum and inter quantile values of our dataset.I see that only the newspaper column shows a huge gap between it's 75% quantile to it's maximum value while the other two feature columns show reasonable increase between each of their quantiles.<jupyter_code>df.skew()<jupyter_output><empty_output><jupyter_text>While checking for skewness information we see that none of our feature columns are in the acceptable range of +/-0.5 and this indicates towards the presence of outliers which might need to be handled if they are not justifiable.  Visualization  Box Plot<jupyter_code>fig, ax = plt.subplots(ncols=4, nrows=1, figsize=(12,7))
index = 0
ax = ax.flatten()
for col, value in df.items():
    sns.boxplot(y=col, data=df, ax=ax[index], palette="plasma")
    index += 1
plt.tight_layout(pad=0.2, w_pad=0.9, h_pad=5.0)
plt.show()<jupyter_output><empty_output><jupyter_text>The above boxplot shows that only the newspaper column has 2 outliers which may be genuine or not that will need to be checked and treated if required.  Pair Plot<jupyter_code>plt.figure(figsize=(15,10))
sns.pairplot(df)
plt.show()<jupyter_output><empty_output><jupyter_text>With the help of above pairplot we can see that TV and Radio columns have a linear relationship with Sales. However the Newspaper column is too scattered and we can see an increase with respect to sales but their is no definite linear pattern into it.  Distribution Plot<jupyter_code>fig, ax = plt.subplots(ncols=4, nrows=1, figsize=(12,7))
index = 0
ax = ax.flatten()
for col, value in df.items():
    sns.distplot(value, ax=ax[index], hist=False, color="y", kde_kws={"shade": True})
    index += 1
plt.tight_layout(pad=0.5, w_pad=0.7, h_pad=5.0)
plt.show()<jupyter_output><empty_output><jupyter_text>Looking at the above distribution plot we can see that almost all the columns are maintaining the bell shape curve except for the newspaper column where we can see a visible skewness.  Correlation using a Heatmap - Positive correlation - A correlation of +1 indicates a perfect positive correlation, meaning that both variables move in the same direction together.- Negative correlation - A correlation of –1 indicates a perfect negative correlation, meaning that as one variable goes up, the other goes down.<jupyter_code>lower_triangle = np.tril(df.corr())
plt.figure(figsize=(12,7))
sns.heatmap(df.corr(), vmin=-1, vmax=1, annot=True, square=True, fmt='0.5f', 
            annot_kws={'size':10}, cmap="Accent", mask=lower_triangle)
plt.xticks(fontsize=12)
plt.yticks(fontsize=12)
plt.show()<jupyter_output><empty_output><jupyter_text>In the above heatmap we can see that there are no negative correlation present in between our features and label or between the feature columns either. Noticing the sales part we see that it has a high correlation with TV and a low correlation with Newspaper.  Line Plots<jupyter_code>sns.relplot(x="TV", y="sales", kind="line", data=df)<jupyter_output><empty_output><jupyter_text>The line plot for TV vs Sales shows a proper increasing pattern wherein the TV advertisements are playing a major role in the marketing field thus inreasing the sales.<jupyter_code>sns.relplot(x="radio", y="sales", kind="line", data=df)<jupyter_output><empty_output><jupyter_text>The line plot for Radio vs Sales do not show a dominant relationship when it comes to sales like the TV option but it is definitely contributing towards the sales parameter.<jupyter_code>sns.relplot(x="newspaper", y="sales", kind="line", data=df)<jupyter_output><empty_output><jupyter_text>Lastly, looking at the line plot for Newspaper vs Sales we see a very random trend where in the begin the contribution is way better than towards the end showing that the outliers reflecting in our data are mere flukes where in it contributed towards the sales from that mode of marketing.  Splitting the dataset into 2 variables namely 'X' and 'Y' for feature and label<jupyter_code>X = df.drop('sales', axis=1)
Y = df['sales']<jupyter_output><empty_output><jupyter_text>I have bifurcated the dataset into features and labels where X represents all the feature columns and Y represents the target label column.  Feature Scaling<jupyter_code>scaler = StandardScaler()
X = pd.DataFrame(scaler.fit_transform(X), columns=X.columns)
X # Displaying all the features after applying scaling technique to avoid bias output<jupyter_output><empty_output><jupyter_text>Even though all our feature columns were of float data type I was unhappy with the decimal place differences and was worried that it might make my model biased. Therefore I am using the Standard Scaler method to ensure all my feature columns have been standardized.Note: Trying to retain all the original data therefore not treating the outliers to ensure I do not have any data loss for my model creation.  Creating the training and testing data sets<jupyter_code>X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size=0.2, random_state=21)<jupyter_output><empty_output><jupyter_text>I am taking 20 percent of the complete dataset for training purpose and the remaing 80 percent with be used to train the machine learning models  Machine Learning Model for Regression and Evaluation Metrics<jupyter_code># Regression Model Function

def reg(model, X, Y):
    X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size=0.2, random_state=21)
    
    # Training the model
    model.fit(X_train, Y_train)
    
    # Predicting Y_test
    pred = model.predict(X_test)
    
    # RMSE - a lower RMSE score is better than a higher one
    rmse = mean_squared_error(Y_test, pred, squared=False)
    print("RMSE Score is:", rmse)
    
    # R2 score
    r2 = r2_score(Y_test, pred, multioutput='variance_weighted')*100
    print("R2 Score is:", r2)
    
    # Cross Validation Score
    cv_score = (cross_val_score(model, X, Y, cv=5).mean())*100
    print("Cross Validation Score:", cv_score)
    
    # Result of r2 score minus cv score
    result = r2 - cv_score
    print("R2 Score - Cross Validation Score is", result)<jupyter_output><empty_output><jupyter_text>I have built a function that splits the training and testing features and labels, then trains the model, predicts the label, calculates the RMSE score, generates the R2 score, calculates the Cross Validation score and finally finds the difference between the R2 score and Cross Validation score.<jupyter_code># Linear Regression Model

model=LinearRegression()
reg(model, X, Y)<jupyter_output>RMSE Score is: 1.6611334844530616
R2 Score is: 92.20737408493859
Cross Validation Score: 88.71063495438435
R2 Score - Cross Validation Score is 3.4967391305542463<jupyter_text>Created the Linear Regression model and checked it's evaluation metrics.<jupyter_code># Support Vector Regression

model=SVR(C=1.0, epsilon=0.2, kernel='poly', gamma='auto')
reg(model, X, Y)<jupyter_output>RMSE Score is: 3.604800199598648
R2 Score is: 63.30244246991651
Cross Validation Score: 72.20715307684588
R2 Score - Cross Validation Score is -8.904710606929378<jupyter_text>Created the Support Vector Regression model and checked it's evaluation metrics.<jupyter_code># Decision Tree Regressor

model=DecisionTreeRegressor(criterion="poisson", random_state=111)
reg(model, X, Y)<jupyter_output>RMSE Score is: 3.456407094079052
R2 Score is: 66.26159565701089
Cross Validation Score: 62.652074809515156
R2 Score - Cross Validation Score is 3.6095208474957303<jupyter_text>Created the Decision Tree Regressor model and checked it's evaluation metrics.<jupyter_code># Random Forest Regressor

model=RandomForestRegressor(max_depth=2, max_features="sqrt")
reg(model, X, Y)<jupyter_output>RMSE Score is: 3.413424337047948
R2 Score is: 67.0954978437343
Cross Validation Score: 59.38690016224486
R2 Score - Cross Validation Score is 7.708597681489437<jupyter_text>Created the Random Forest Regressor model and checked it's evaluation metrics.<jupyter_code># K Neighbors Regressor

KNeighborsRegressor(n_neighbors=2, algorithm='kd_tree')
reg(model, X, Y)<jupyter_output>RMSE Score is: 3.35722732875929
R2 Score is: 68.17002755908574
Cross Validation Score: 63.459372931602296
R2 Score - Cross Validation Score is 4.710654627483443<jupyter_text>Created the K Neighbors Regressor model and checked it's evaluation metrics.<jupyter_code># Gradient Boosting Regressor

model=GradientBoostingRegressor(loss='quantile', n_estimators=200, max_depth=5)
reg(model, X, Y)<jupyter_output>RMSE Score is: 1.182377293321138
R2 Score is: 96.05191108567796
Cross Validation Score: 89.46823152252635
R2 Score - Cross Validation Score is 6.583679563151605<jupyter_text>Created the Gradient Boosting Regressor model and checked it's evaluation metrics.<jupyter_code># Ada Boost Regressor

model=AdaBoostRegressor(n_estimators=300, learning_rate=1.05, random_state=42)
reg(model, X, Y)<jupyter_output>RMSE Score is: 1.163231503781359
R2 Score is: 96.17873572196949
Cross Validation Score: 95.86007140956852
R2 Score - Cross Validation Score is 0.31866431240096915<jupyter_text>Created the Ada Boost Regressor model and checked it's evaluation metrics.<jupyter_code># Extra Trees Regressor

model=ExtraTreesRegressor(n_estimators=200, max_features='sqrt', n_jobs=6)
reg(model, X, Y)<jupyter_output>RMSE Score is: 1.3992250676775948
R2 Score is: 94.47095763879216
Cross Validation Score: 93.2133957012398
R2 Score - Cross Validation Score is 1.257561937552353<jupyter_text>Created the Extra Trees Regressor model and checked it's evaluation metrics.  Hyper parameter tuning<jupyter_code># Choosing Ada Boost Regressor

fmod_param = {'n_estimators' : [50, 100, 150, 200],
              'learning_rate' : [0.001, 0.01, 0.1, 1],
              'loss' : ["linear", "square", "exponential"],
              'random_state' : [21, 42, 104, 111]
             }<jupyter_output><empty_output><jupyter_text>After comparing all the regression models I have selected Ada Boost Regressor as my best model and have listed down it's parameters above referring the sklearn webpage<jupyter_code>GSCV = GridSearchCV(AdaBoostRegressor(), fmod_param, cv=5)<jupyter_output><empty_output><jupyter_text>I am using the Grid Search CV method for hyper parameter tuning my best model.<jupyter_code>GSCV.fit(X_train,Y_train)<jupyter_output><empty_output><jupyter_text>I have trained the Grid Search CV with the list of parameters I feel it should check for best possible outcomes<jupyter_code>GSCV.best_params_<jupyter_output><empty_output><jupyter_text>Here the Grid Search CV has provided me with the best parameters list out of all the combinations it used to train the model<jupyter_code>Final_Model = AdaBoostRegressor(learning_rate=1, loss="linear", n_estimators=150, random_state=111)
Classifier = Final_Model.fit(X_train, Y_train)
fmod_pred = Final_Model.predict(X_test)
fmod_r2 = r2_score(Y_test, fmod_pred, multioutput='variance_weighted')*100
print("R2 score for the Best Model is:", fmod_r2)<jupyter_output>R2 score for the Best Model is: 96.18348660788863<jupyter_text>I have successfully incorporated the Hyper Parameter Tuning on my Final Model and received the R2 score for it  Saving the best model<jupyter_code>filename = "FinalModel_6.pkl"
joblib.dump(Final_Model, filename)<jupyter_output><empty_output>