<jupyter_start><jupyter_text>Data Information and Exploration<jupyter_code>#Reading the Data
data = pd.read_csv('student-mat.csv',sep = ';')
data.shape
#Information about the columns
data.info()
#How data looks like
data.head()
data.describe()
columns = data.columns
columns
#Get the value counts for every column
for column in columns:
    print(data[column].value_counts())<jupyter_output>GP    349
MS     46
Name: school, dtype: int64
F    208
M    187
Name: sex, dtype: int64
16    104
17     98
18     82
15     82
19     24
20      3
22      1
21      1
Name: age, dtype: int64
U    307
R     88
Name: address, dtype: int64
GT3    281
LE3    114
Name: famsize, dtype: int64
T    354
A     41
Name: Pstatus, dtype: int64
4    131
2    103
3     99
1     59
0      3
Name: Medu, dtype: int64
2    115
3    100
4     96
1     82
0      2
Name: Fedu, dtype: int64
other       141
services    103
at_home      59
teacher      58
health       34
Name: Mjob, dtype: int64
other       217
services    111
teacher      29
at_home      20
health       18
Name: Fjob, dtype: int64
course        145
home          109
reputation    105
other          36
Name: reason, dtype: int64
mother    273
father     90
other      32
Name: guardian, dtype: int64
1    257
2    107
3     23
4      8
Name: traveltime, dtype: int64
2    198
1    105
3     65
4     27
Name: studytime, dtype: int64
0    312
1  [...]<jupyter_text>Data Distribution<jupyter_code>#Data Distribution for every column
data.hist(bins=50, figsize=(20,10), color='r')
plt.show()<jupyter_output><empty_output><jupyter_text>Final Year Score Distribution<jupyter_code>#Grade Distribution
x1 = data["G1"]
x2 = data["G2"]
x3 = data["G3"]


bins = np.linspace(0, 20, 20)

plt.hist(x1, bins, label='First Grade (G1)',color="red")
plt.hist(x2, bins, label='Second Grade (G2)',color="green")
plt.hist(x3, bins, label='Third Grade (G3)',color = "blue")
plt.legend(loc='upper right')
plt.show()<jupyter_output><empty_output><jupyter_text>Correlation between columns<jupyter_code>correlation = data.corr()
correlation["G3"].sort_values(ascending=False)<jupyter_output><empty_output><jupyter_text>G1 and G2 are very correlated to G3, so we are dropping it to get which other factors are helping in deciding G3<jupyter_code>data.drop(['G1', 'G2'], axis=1, inplace=True)
data.columns<jupyter_output><empty_output><jupyter_text>Converting binary colums to numeric<jupyter_code>data['school'].value_counts()
#Info: Majority of Students are in '<NAME>' School
#GP = 0
#MS = 1
data['school'] = data['school'].apply(lambda x: 0 if x == 'GP' else 1)
data['school'].value_counts()
#F = 0 (Female)
#M = 1 (Male)
data['sex'] = data['sex'].apply(lambda x: 0 if x =='F' else 1)
data['sex'].value_counts()
#Info: Dataset contains almost equal percentage of Female and Male records
#U = 0 (Urban)
#R = 1 (Rural)
data['address'] = data['address'].apply(lambda x: 0 if x == 'U' else 1)
data['address'].value_counts()
#Info: More then 75% students are from Urban Area
data.head()
#Family Size
#GT3 = 0 (Greater than 3)
#LT3 = 1 (Less than 3)
data['famsize'].value_counts()
data['famsize'] = data['famsize'].apply(lambda x: 0 if x == 'GT3' else 1)
data['famsize'].value_counts()
#Info: More than 2/3 of the family are greater than 3
data['Pstatus'].value_counts()
#Info: Majority of the parents are living together
#Parent's Cohabitation status
# T = 0 (Living together)
# A = 1 (Living Apart)
data['Pstatus'] = data['Pstatus'].apply(lambda x:0 if x == 'T' else 1)
data['schoolsup'].value_counts()
#Info: Majority of the students have extra educational support
#extra educational support
# no = 0
# yes = 1
data['schoolsup'] = data['schoolsup'].apply(lambda x: 0 if x == 'no' else 1)
data['famsup'].value_counts()
#Info: More than half of the students had Family Educationl Support
#family educational support 
# no = 0
# yes = 1
data['famsup'] = data['famsup'].apply(lambda x: 0 if x == 'no' else 1)
data['paid'].value_counts()
#Info: Almost 50% of the students had attended extra paid classes 
#extra paid classes within the course subject
# no = 0
# yes = 1

data['paid'] = data['paid'].apply(lambda x: 0 if x == 'no' else 1)
data['activities'].value_counts()
#extra-curricular activities
# yes = 1
# no = 0
data['activities'] = data['activities'].apply(lambda x: 0 if x == 'no' else 1)
data['higher'].value_counts()
#Info: Almost all the students wants to take higher education
#wants to take higher education
# yes = 1
# no = 0
data['higher'] = data['higher'].apply(lambda x: 0 if x == 'no' else 1)
data.columns
data['nursery'].value_counts()
#attended nursery school
# yes = 1
# no = 0
data['nursery'] = data['nursery'].apply(lambda x: 0 if x == 'no' else 1)
data['internet'].value_counts()
#Info: More than 80% students had access to Internet Connection
#Internet access at home
#yes = 1
#no = 0
data['internet'] = data['internet'].apply(lambda x: 0 if x == 'no' else 1)
data['romantic'].value_counts()
#Info: More than 30% of the students were in some kind of relationship
#with a romantic relationship
#no = 0
#yes = 1
data['romantic'] = data['romantic'].apply(lambda x: 0 if x == 'no' else 1)
data.head()
correlation = data.corr()
correlation['G3'].sort_values(ascending = False)<jupyter_output><empty_output><jupyter_text>Dropping Columns that wont have much impact on G3<jupyter_code>data.hist(bins=50, figsize=(20,15), color='r')
plt.show()<jupyter_output><empty_output><jupyter_text>Drop ColumnsBased on Correlation Matrix and Data Distribution columns like Dalc, Walc, Pstatus, traveltime, school, higher, nursery, failures, age, schoolsup won't impact that much, so dropping all these columns<jupyter_code>data.shape
data = data.drop(['Dalc', 'Walc' ,'Pstatus', 'traveltime', 'school', 'higher', 'nursery','failures', 'age','schoolsup'], axis= 1)
data.shape
data.columns
data.shape<jupyter_output><empty_output><jupyter_text>Adding Result Column based on G3's value<jupyter_code>data['result'] = data['G3'].apply(lambda x: 1 if x >= 10 else 0)
data['result'].value_counts()
#data = data.drop(['G3'], axis = 1)
data.head()
data.columns
data.shape<jupyter_output><empty_output><jupyter_text>Some Plots<jupyter_code>labels = ['Fail','Pass']
Fail = data['result'].value_counts()[0]
Pass = data['result'].value_counts()[1]
status = [Fail, Pass]
colors = ['red','green']

plt.pie(x = status, labels = labels, colors=colors, autopct='%1.1f%%', startangle=90)
plt.axis('equal')


cross_gender = pd.crosstab(data['sex'],data['result']).apply(lambda x: x/x.sum() * 100, axis=1)
cross_gender.iloc[1][0]


Female_Failed = cross_gender.iloc[0][0]
Female_Passed = cross_gender.iloc[0][1]
Male_Failed = cross_gender.iloc[1][0]
Male_Passed = cross_gender.iloc[1][1]

plt.title('Female Results')
plt.pie(x=[F_Failed, F_Passed], labels = ['Failed', 'Passed'], autopct="%1.1f%%", startangle=90)
plt.axis('equal')
plt.show()

plt.title('Male Results')
plt.axis('equal')
plt.pie(x=[M_Failed, M_Passed], labels = ['Failed', 'Passed'], autopct="%1.1f%%", startangle=90)

correlation = data.corr()
import seaborn as sns
fig, ax = plt.subplots(figsize=(12,6))
sns.heatmap(correlation,
           xticklabels=correlation.columns.values,
           yticklabels = correlation.columns.values,
            ax = ax,
           ).set_title("Correlation Between Columns")


plt.show()<jupyter_output><empty_output><jupyter_text>Finding impact of Mothers education on result<jupyter_code>cross_medu = pd.crosstab(data['Medu'], data['sex']).apply(lambda x: x/x.sum() * 100)
cross_medu

ct = pd.crosstab(data.Medu, data.result).apply(lambda x: x/x.sum() * 100)

ax = ct.plot.bar(stacked=True,  figsize=(12,6))
ax.set(xlabel="Mom's Education (from lowest to highest)", ylabel='% of Passes and Fails', title="How Mom's Education influenced Performance")
plt.show()
cross_fedu = pd.crosstab(data['Fedu'], data['result']).apply(lambda x: x/x.sum() * 100)
cross_fedu

cf = pd.crosstab(data.Fedu, data.result).apply(lambda x: x/x.sum() * 100)

ax = cf.plot.bar(stacked=True,  figsize=(12,6))
ax.set(xlabel="Dad's Education (from lowest to highest)", ylabel='% of Passes and Fails', title="How Dad's Education influenced Performance")
plt.show()
cross_address = pd.crosstab(data['address'], data['result']).apply(lambda x: x/x.sum() * 100)
cross_address
cross_paids = pd.crosstab(data['sex'], data['paid']).apply(lambda x: x/x.sum() * 100)
cross_paids
data.head()

plt.scatter(x=data['guardian'], y = data['result'])<jupyter_output><empty_output>