<jupyter_start><jupyter_text>Data Cleaning   Data Reading & Data Types<jupyter_code>#Read the data in pandas
inp0= pd.read_csv("Dress_Attribute_DataSet.csv")
inp1= pd.read_csv("Dress_Sales_Dataset.csv")<jupyter_output><empty_output><jupyter_text>You have “Attribute DataSet” which contains a column named “Price”. Choose the correct statement from the following about its data type and variable type.- Integer type and numerical variable- Object type and categorical ordinal variable- Object type and categorical nominal variable- Float type and categorical variable. There is another column in “Attribute DataSet” named as “Recommendation”, choose the correct statement about its data type and variable type.- Integer type and categorical- Object type and categorical- Integer type and continuous numerical- Object type only. Which of the following column do you think are of no use in “Attribute DataSet”.- Dress_ID- Price- Size and material- NeckLine- None of the above<jupyter_code># Print the information about the attributes of inp0 and inp1.
print("Attribute DataSet")
print(inp0.info())
print("\nSales Dataset")
print(inp1.info())
print("Attribute DataSet - Price")
inp0.Price.value_counts()
print("Attribute DataSet - Recommendation")
inp0.Recommendation.value_counts()
inp0.head()<jupyter_output><empty_output><jupyter_text>Fixing the Rows and Columns  As you can see, there is a column in “Attribute Dataset” named as ‘Size’. This column contains the values in abbreviation format. Write a code in Python to convert the followings:- M into  “Medium”- L into  “Large”- XL into “Extra large”- free into “Free”- S, s & small into “Small”.Now once you are done with changes in the dataset, what is the value of the lowest percentage, the highest percentage and the percentage of Small size categories in the column named “Size”?<jupyter_code># Column fixing, correcting size abbreviation. count the percentage of each size category in "Size" column.
inp0.Size.value_counts(normalize=True)
# Function to standardize size
def standardize_size(size):
    if size == "M":
        return "Medium"
    elif size == "L":
        return "Large"
    elif size == "XL":
        return "Extra Large"
    elif size == "free":
        return "Free"
    elif size == "S" or size == "s" or size == "small":
        return "Small"
    else:
        return size    
# Print the value counts of each category in "Size" column.
inp0.Size = inp0.Size.apply(standardize_size)
inp0.Size.value_counts(normalize=True).mul(100)<jupyter_output><empty_output><jupyter_text>Impute/Remove Missing values<jupyter_code># Print the null count of each variables of inp0 and inp1.
print("Attribute Dataset - Null value counts")
print(inp0.isnull().sum())
print("\nSales Dataset - Null value counts")
print(inp1.isnull().sum())<jupyter_output>Attribute Dataset - Null value counts
Dress_ID            0
Style               0
Price               2
Rating              0
Size                0
Season              2
NeckLine            3
SleeveLength        2
Material          119
FabricType        256
Decoration        224
Pattern Type      102
Recommendation      0
dtype: int64

Sales Dataset - Null value counts
Dress_ID        0
29-08-2013      0
31-08-2013      0
09-02-2013      0
09-04-2013      0
09-06-2013      0
09-08-2013      0
09-10-2013      0
09-12-2013      0
14-09-2013      0
16-09-2013      0
18-09-2013      0
20-09-2013      0
22-09-2013      0
24-09-2013      0
26-09-2013    222
28-09-2013      0
30-09-2013    257
10-02-2013    259
10-04-2013    258
10-06-2013      0
10-08-2013    255
10-10-2013    255
10-12-2013      0
dtype: int64<jupyter_text>You are given another dataset named “Dress Sales”. Now if you observe the datatypes of the columns using ‘inp1.info()’ command, you can identify that there are certain columns defined as object data type though they primarily consist of numeric data.Now if you try and convert these object data type columns into numeric data type(float), you will come across an error message. Try to correct this error.<jupyter_code># Print the data types information of inp1 i.e. "Dress Sales" data.
inp1.info()
# Try to convert the object type into float type of data. YOU GET ERROR MESSAGE.
inp1.astype(float)
# Do the required changes in the "Dress Sales" data set to get null values on string values.
inp1.loc[inp1['09-12-2013']== 'Removed',"09-12-2013"] = np.NaN

inp1.loc[inp1['14-09-2013']== 'removed',"14-09-2013"] = np.NaN

inp1.loc[inp1['16-09-2013']== 'removed',"16-09-2013"] = np.NaN

inp1.loc[inp1['18-09-2013']== 'removed',"18-09-2013"] = np.NaN

inp1.loc[inp1['20-09-2013']== 'removed',"20-09-2013"] = np.NaN

inp1.loc[inp1['22-09-2013']== 'Orders',"22-09-2013"] = np.NaN
# Convert the object type columns in "Dress Sales" into float type of data type.
inp1.astype(float)<jupyter_output><empty_output><jupyter_text>When you see the null counts in “Dress Sales” dataset after performing all the operations that have been mentioned in jupyter notebook, you will find that there are some columns in “Dress Sales” data where there are more than 40% of missing values. Based on your understanding of dealing with missing values do the following steps.<jupyter_code># Print the null percetange of each column of inp1.
null_values_count = inp1.isnull().sum(axis=0)
total_count = inp1.sum(axis=0)
(null_values_count/total_count)*100
# Drop the columns in "Dress Sales" which have more than 40% of missing values.
max_missing_value_percentage = 0.4
inp1.drop(inp1.loc[:, ((null_values_count/total_count)*100) > max_missing_value_percentage], axis=1, inplace=True)
inp1.head()<jupyter_output><empty_output><jupyter_text>You should categorise the dates into seasons in “Dress Sales” data to simplify the analysis according to the following criteria:- June, July and August: Summer.- September, October and November: Autumn.- December, January and February: WInter.- March, April and May: Spring.<jupyter_code># Create the four seasons columns in inp1, according to the above criteria.
def get_season(date):
    month = int(date.split("-")[1])
    if  month >= 6 and month <= 8:
        return "Spring"
    elif month >= 9 and month <= 11:
        return "Autumn"
    elif month == 12 or month <= 2:
        return "Winter"
    elif month >= 3 and month <= 5:
        return "Summer"
inp1[['09-04-2013']].astype(int)
inp1['Spring'] = inp1.apply(lambda x: float(x['09-04-2013']), axis=1)

inp1['Summer'] = inp1.apply(lambda x: float(x['29-08-2013']) + float(x['31-08-2013']) + float(x['09-06-2013']) + float(x['09-08-2013']) + float(x['10-06-2013']), axis=1)

inp1['Winter'] = inp1.apply(lambda x: float(x['09-02-2013']) + float(x['09-12-2013']) + float(x['10-12-2013']), axis=1)

inp1['Autumn'] = inp1.apply(lambda x: float(x['09-10-2013']) + float(x['14-09-2013']) + float(x['16-09-2013']) + float(x['18-09-2013']) + float(x['20-09-2013']) + float(x['22-09-2013']) + float(x['24-09-2013']) + float(x['28-09-2013']), axis=1)
    
# calculate the sum of sales in each seasons in inp1 i.e. "Dress Sales".
inp1.sum()<jupyter_output><empty_output><jupyter_text>Now let's merge inp1 with inp0 with left join manner, so that the information of inp0 should remain intact.<jupyter_code># Merge inp0 with inp1 into inp0. this is also called left merge.
inp0 = pd.merge(left=inp0,right=inp1, how='left', left_on='Dress_ID', right_on='Dress_ID')
inp0.head()
# Now Drop the Date columns from inp0 as it is already combined into four seasons.
inp0.drop(inp0.loc[:,'29-08-2013':'10-12-2013'].columns, axis= 1, inplace= True)<jupyter_output><empty_output><jupyter_text>Print the null count of inp0 to get the idea about the missing values in data set.<jupyter_code># Print the null count of each columns in inp0 dataframe i.e. combined data frame of inp0 and inp1 without date columns.
inp0.isnull().sum()<jupyter_output><empty_output><jupyter_text>You can see that there are two types of variables one with a large number of missing values and another is very less number of missing values. These two columns can be categorized as:Type-1: Missing values are very less (around 2 or 3 missing values): Price, Season, NeckLine, SleeveLength, Winter and Autumn. Type-2: Missing values are large in numbers (more than 15%): Material, FabricType, Decoration and Pattern Type.<jupyter_code># Deal with the missing values of Type-1 columns: Price, Season, NeckLine, SleeveLength, Winter and Autumn.
inp0 = inp0[~inp0.Price.isnull()]

inp0 = inp0[~inp0.Season.isnull()]

inp0 = inp0[~inp0.NeckLine.isnull()]

inp0 = inp0[~inp0.SleeveLength.isnull()]

inp0 = inp0[~inp0.Winter.isnull()]

inp0 = inp0[~inp0.Autumn.isnull()]
# Deal with the missing values for Type-2 columns: Material, FabricType, Decoration and Pattern Type.
inp0.Material= inp0.Material.replace(np.nan, "Missing")

inp0.FabricType= inp0.FabricType.replace(np.nan, "Missing")

inp0.Decoration= inp0.Decoration.replace(np.nan, "Missing")

inp0['Pattern Type']= inp0['Pattern Type'].replace(np.nan, "Missing")
inp0.head()
inp0.Season.value_counts()
inp0.SleeveLength.value_counts()<jupyter_output><empty_output><jupyter_text>Standardise value  In the given dataset, there are certain discrepancies with the categorical names such as irregular spellings. Choose the correct option of columns with irregular categories and update them. - Season, NeckLine- Price, Material- fabricType, Decoration- Season, SleeveLength<jupyter_code>#correcting the spellings.
inp0.Season = inp0.Season.replace('Automn', "Autumn")
inp0.Season = inp0.Season.replace('spring', "Spring")
inp0.Season = inp0.Season.replace('winter', "Winter")
#correcting the Spellings.
inp0.SleeveLength= inp0.SleeveLength.replace(['cap-sleeves', 'capsleeves'], "cap sleeves")
inp0.SleeveLength= inp0.SleeveLength.replace('full', "full sleeves")
inp0.SleeveLength= inp0.SleeveLength.replace(['half','halfsleeve'], "half sleeves")
inp0.SleeveLength= inp0.SleeveLength.replace(['sleevless', 'sleeevless', 'sleeveless', 'sleveless'], "sleeve less")
inp0.SleeveLength= inp0.SleeveLength.replace(['threequarter','threequater', 'thressqatar'], "three quater")
inp0.SleeveLength= inp0.SleeveLength.replace(['turndowncollor','urndowncollor'], "turn down collar")
display(inp0.Season.value_counts())
display(inp0.SleeveLength.value_counts())<jupyter_output><empty_output><jupyter_text>Univariate Analysis   Categorical Unordered Univariate Analysis  There is a column named ‘Style’ in ‘Attribute Dataset’ which consists of the different style categories of the women apparels. Certain categories whose total sale is less than 50000 across all the seasons is considered under one single category as ‘Others’. Which of the following categories in ‘Style’ column can be grouped into ‘Others’ category? and perform the grouping operation in the notebook for further analysis.- Flare, fashion- Novelty, bohemian- OL, fashion, work- Novelty, fashion, Flare<jupyter_code># Group "Style" categories into "Others" which have less than 50000 sales across all the seasons.
inp0_style_grouped =inp0.groupby(['Style'])['Spring','Winter','Autumn', 'Summer'].sum()
inp0_style_grouped.sum(axis=1)
inp0_style_grouped[inp0_style_grouped.sum(axis= 1)<50000]
inp0.replace(inp0_style_grouped[inp0_style_grouped.sum(axis=1)<50000].index, 'Others', inplace=True)
inp0.Style.value_counts()<jupyter_output><empty_output><jupyter_text>What is the percentage of “cute” and “Others” category in “Style” column in “Attribute DataSet” respectively?- 46%, 5%- 9%, 2.1%- 2.1%, 5%- 13.8%, 9%<jupyter_code># Calculate the percentage of each categories in the "Style" variable.
print(inp0.Style.value_counts(normalize=True))
inp0.Style.value_counts(normalize=True).plot.barh()
plt.show()
inp0.head()<jupyter_output><empty_output><jupyter_text>Similarly Club Neckline, SLeeve length categories into "Others" which have less than 50000 sales across all the seasons.<jupyter_code># Group "Neckline" categories into "Others" which have less than 50000 sales across all the seasons.
inp0_neckline_grouped =inp0.groupby(['NeckLine'])['Spring','Winter','Autumn', 'Summer'].sum()
inp0.replace(inp0_neckline_grouped[inp0_neckline_grouped.sum(axis=1)<50000].index, 'Others', inplace=True)
inp0.NeckLine.value_counts()
# Group "Sleeve length" categories into "Others" which have less than 50000 sales across all the seasons.
inp0_sleevelength_grouped =inp0.groupby(['SleeveLength'])['Spring','Winter','Autumn', 'Summer'].sum()
inp0.replace(inp0_sleevelength_grouped[inp0_sleevelength_grouped.sum(axis=1)<50000].index, 'Others', inplace=True)
inp0.SleeveLength.value_counts()<jupyter_output><empty_output><jupyter_text>Club material, fabrictype, patterntype and decoration categories into "Others" which have less than 25000 sales across all the seasons<jupyter_code># Group "material" categories into "Others" which have less than 25000 sales across all the seasons.
inp0_material_grouped =inp0.groupby(['Material'])['Spring','Winter','Autumn', 'Summer'].sum()
inp0.replace(inp0_material_grouped[inp0_material_grouped.sum(axis=1)<25000].index, 'Others', inplace=True)
inp0.Material.value_counts()
# Group "fabric type" categories into "Others" which have less than 25000 sales across all the seasons.
inp0_fabrictype_grouped =inp0.groupby(['FabricType'])['Spring','Winter','Autumn', 'Summer'].sum()
inp0.replace(inp0_fabrictype_grouped[inp0_fabrictype_grouped.sum(axis=1)<25000].index, 'Others', inplace=True)
inp0.FabricType.value_counts()
# Group "patern type" categories into "Others" which have less than 25000 sales across all the seasons.
inp0_patterntype_grouped =inp0.groupby(['Pattern Type'])['Spring','Winter','Autumn', 'Summer'].sum()
inp0.replace(inp0_patterntype_grouped[inp0_patterntype_grouped.sum(axis=1)<25000].index, 'Others', inplace=True)
inp0["Pattern Type"].value_counts()
# Group "decoration" categories into "Others" which have less than 25000 sales across all the seasons.
inp0_decoration_grouped =inp0.groupby(['Decoration'])['Spring','Winter','Autumn', 'Summer'].sum()
inp0.replace(inp0_decoration_grouped[inp0_decoration_grouped.sum(axis=1)<25000].index, 'Others', inplace=True)
inp0.Decoration.value_counts()<jupyter_output><empty_output><jupyter_text>Caregorical Ordered Univariate Analysis Which of the following is an unordered variable in “Attribute DataSet”.- Style- Price- Season- Size  Numerical variable Univariate analysis: What is the approximate difference between the maximum value and 75th percentile in “Autumn” column.- Approx 54000- Approx 55000- Approx 52000- Approx 50000<jupyter_code># Describe the numerical variale: "Autumn".
inp0.Autumn.describe()
# plot the boxplot of "Autumn" column.
sns.boxplot(inp0.Autumn)
plt.show()<jupyter_output><empty_output><jupyter_text>Which of the following season has the highest difference between the maximum value and 99th quantile of sales?- Winter- Summer- Spring- Autumn<jupyter_code># Find the maximum and 99th percentile of Winter season.
inp0.Winter.quantile([0.9, 1])
# Find the maximum and 99th percentile of Summer season.
inp0.Summer.quantile([0.9, 1])
# Find the maximum and 99th percentile of Spring season.
inp0.Spring.quantile([0.9, 1])
# Find the maximum and 99th percentile of Autumn season.
inp0.Autumn.quantile([0.9, 1])<jupyter_output><empty_output><jupyter_text>Bivariate Analysis   Numerical- Categorical analysis Which of the following “Price” category has the lowest average value of rating?- very-high- Medium- Low- High<jupyter_code># Find the Mean of Ratings for each Price category.
inp0.groupby("Price")["Rating"].mean()<jupyter_output><empty_output><jupyter_text>What is the median of the rating of “vintage” category in Style column?- 4.6- 4.7- 4.55- 0.00<jupyter_code># Find the median of Ratings for each Style category.
inp0.groupby("Style")["Rating"].median()<jupyter_output><empty_output><jupyter_text>Which of the following season has the highest average value of sale for “Recommendation” value equals to 1.- Summer- Spring- Autumn- Winter<jupyter_code># Summer sale vs Recommendation.
inp0.groupby("Recommendation")["Summer"].mean()
# Spring sale vs Recommendation.
inp0.groupby("Recommendation")["Spring"].mean()
# Autumn sale vs Recommendation.
inp0.groupby("Recommendation")["Autumn"].mean()
# Winter sale vs Recommendation.
inp0.groupby("Recommendation")["Winter"].mean()
inp0.groupby("Recommendation")["Summer", "Spring", "Autumn", "Winter"].mean()
plt.figure(figsize=[15, 10])
plt.subplot(2, 2, 1)
sns.boxplot(data=inp0, x="Recommendation", y="Summer").set_title('Recomendation  vs Summer')
plt.subplot(2, 2, 2)
sns.boxplot(data=inp0, x="Recommendation", y="Spring").set_title('Recomendation  vs Spring')
plt.subplot(2, 2, 3)
sns.boxplot(data=inp0, x="Recommendation", y="Autumn").set_title('Recomendation  vs Autumn')
plt.subplot(2, 2, 4)
sns.boxplot(data=inp0, x="Recommendation", y="Winter").set_title('Recomendation  vs Winter')
plt.show()<jupyter_output><empty_output><jupyter_text>Categorical categorical bivariate analysis Which of the following size categories has the highest positive recommendations?- Medium and extra large- Extra large and small- Free and small- Free and medium<jupyter_code># Size vs Recommendation.
inp0.groupby("Size")["Recommendation"].mean().plot.barh()
plt.show()
# Recommendation are 0s and 1s, which means taking the sum produces the count of all positive recommendations.
inp0.groupby("Size")["Recommendation"].sum()<jupyter_output><empty_output><jupyter_text>Multivariate analysis  Which of the following pair of “Style” and “Price” category has the highest average of positive recommendations?- Price: medium and style: vintage- Price: medium and style: cute- Price: very high and style: party- Price: low and style: sexy<jupyter_code>style_recommendation_mean = inp0.groupby("Style")["Recommendation"].mean().mean()
style_recommendation_mean
# plot the heat map of Style, price and Recommendation.
res = pd.pivot_table(data=inp0, index="Style", columns="Price", values="Recommendation")
sns.heatmap(res, annot=True, cmap="RdYlGn", center=style_recommendation_mean)
plt.show()<jupyter_output><empty_output><jupyter_text>Which of the following material type has no recommendation in summer and winter seasons?- Mix and Milksilk- Nylon and Rayon- Microfiber and Silk- Milksilk and Microfiber<jupyter_code>season_recommendation_mean = inp0.groupby("Season")["Recommendation"].mean().mean()
season_recommendation_mean
# plot the heat map of Season, material and Recommendation.
res = pd.pivot_table(data=inp0, index="Season", columns="Material", values="Recommendation")
sns.heatmap(res, annot=True, cmap="RdYlGn", center=season_recommendation_mean)
plt.show()<jupyter_output><empty_output>