## Logistic Regression

- Models the probability an object belongs to a class
- Values ranges from 0 to 1
- Can use threshold to classify into which classes a class belongs
- An S-shaped curve

$
\begin{align}
\sigma(t) = \frac{1}{1 + e^{-t}}
\end{align}
$



```python
import pandas as pd
import numpy as np
from matplotlib import pyplot as plt
import math
```

### How does the Logistic Regression function looks like


```python
from matplotlib import pyplot as plt
import math

x = [i for i in range(-10, 10)]
y = [1.0/(1+math.exp(-1*i)) for i in x]

ax = plt.figure(figsize=(8, 6))
plt.plot(x, y,  color='black')
plt.axhline(0.5)
plt.show()
```

### Read the data and plot


```python
import pandas as pd

# Read the data
df_data = pd.read_csv('../data/2d_classification.csv')

# Plot the data
plt.rcParams['figure.figsize'] = [10, 7] # Size of the plots

colors = {0:'b', 1:'g', 2:'r', 3:'c', 4:'m', 5:'y', 6:'k'}
fig, ax = plt.subplots()
grouped = df_data.groupby('label')
for key, group in grouped:
    group.plot(ax=ax, kind='scatter', x='x', y='y', label=key, color=colors[key])
plt.show()
```

### Another way to plot the data


```python
import pandas as pd
import numpy as np

# Read the data
df_data = pd.read_csv('../data/2d_classification.csv')

colors = {0:'b', 1:'g', 2:'r', 3:'c', 4:'m', 5:'y', 6:'k'}

plt.figure()
unq_labels = np.unique(df_data['label'])
for i in unq_labels:
    df = df_data.loc[df_data['label'] == i][['x','y']]
    x = df['x']
    y = df['y']
    plt.scatter(x, y, c=colors[i], alpha=1)
```

### Just using some friendly variable names


```python
data = df_data[['x','y']].values
label = df_data['label'].values
```

### Our first Logistic Regression model


```python
from sklearn.linear_model import LogisticRegression
from matplotlib import pyplot as plt

log_regr = LogisticRegression()
log_regr.fit(data, label)
predictions = log_regr.predict(data)

df_data['pred_label'] = predictions
```


```python
colors = {0:'b', 1:'g', 2:'r', 3:'c', 4:'m', 5:'y', 6:'k'}

plt.figure(figsize=(8,6))
unq_labels = np.unique(predictions)
for i in unq_labels:
    df = df_data.loc[df_data['pred_label'] == i][['x','y']]
    x = df['x']
    y = df['y']
    plt.scatter(x, y, c=colors[i], alpha=1)
```

#### Problem with the previous approach
- When we develop model, using the complete dataset will not allow us to know the actual performance metrics
- Instead always have a test data for evaluation

<font color='red'> <h1> Go to Notebook '03a - Data Split' to know about data split and return from this point </h1> </font>

## Train-Test Splits


```python
from sklearn.model_selection import train_test_split
data_train, data_test, label_train, label_test = train_test_split(data, label, test_size=0.20, 
                                                                  random_state=0, stratify=None)
```


```python
df_before_pred = pd.DataFrame(data_train, columns=['x','y'])
df_before_pred['label'] = label_train

# df_before_pred['label'] = df_before_pred['labels'].apply(lambda x:x+2)
df_before_pred_test = pd.DataFrame(data_test, columns=['x','y'])
df_before_pred_test['label'] = 4

df_before_pred = pd.concat([df_before_pred, df_before_pred_test], ignore_index=True)
```

__Train-test ratio of the split data__


```python
print('Complete data:', data.shape)
print('Train Data:', data_train.shape, 'Test Data:', data_test.shape)
```

    Complete data: (100, 2)
    Train Data: (80, 2) Test Data: (20, 2)


__How the data is split__


```python
from collections import Counter
print('Training Data split', Counter(label_train))
print('Testing Data split', Counter(label_test))
```

    Training Data split Counter({0: 44, 1: 36})
    Testing Data split Counter({1: 12, 0: 8})


#### Stratified split


```python
from sklearn.model_selection import train_test_split
data_train, data_test, label_train, label_test = train_test_split(data, label, test_size=0.20, 
                                                                  random_state=0, stratify=label)

print('Complete data:', data.shape)
print('Labels distribution', Counter(label))
print('Train Data:', data_train.shape, 'Test Data:', data_test.shape)

from collections import Counter
print('Training Data split', Counter(label_train))
print('Testing Data split', Counter(label_test))
```

    Complete data: (100, 2)
    Labels distribution Counter({0: 52, 1: 48})
    Train Data: (80, 2) Test Data: (20, 2)
    Training Data split Counter({0: 42, 1: 38})
    Testing Data split Counter({1: 10, 0: 10})


### Logistic regression on the train-test stratified split


```python
from sklearn.linear_model import LogisticRegression
from matplotlib import pyplot as plt

# Run Logistic Regression
log_regr = LogisticRegression()
log_regr.fit(data_train, label_train)
predictions = log_regr.predict(data_test)

data = np.concatenate((data_train, data_test))
label = np.concatenate((label_train, predictions))

df_train_test_pred = pd.DataFrame(data, columns=['x','y'])
df_train_test_pred['label'] = label
```


```python
colors = {0:'b', 1:'g', 2:'r', 3:'c', 4:'m', 5:'y', 6:'k'}

plt.figure(figsize= (16, 6))

plt.subplot(1,2,1)
unq_labels = np.unique(df_before_pred['label'])
for i in unq_labels:
    df = df_before_pred.loc[df_before_pred['label'] == i][['x','y']]
    x = df['x']
    y = df['y']
    plt.scatter(x, y, c=colors[i], alpha=1)

plt.subplot(1,2,2)
unq_labels = np.unique(df_train_test_pred['label'])
for i in unq_labels:
    df = df_train_test_pred.loc[df_train_test_pred['label'] == i][['x','y']]
    x = df['x']
    y = df['y']
    plt.scatter(x, y, c=colors[i], alpha=1)
```

## Find the accuracy


```python
log_regr.score(data_test, label_test)
```




    0.95



## Find the coefficients


```python
log_regr.coef_
```




    array([[-2.87341187, -1.37330706]])



## Find the intercepts


```python
log_regr.intercept_
```




    array([-0.41734966])




```python

```

7a5d186ee962466d2fc730a24745360ecb5bfe61
ipynb
Jupyter Notebook
classification/notebooks/.ipynb_checkpoints/03 - Logistic Regression-checkpoint.ipynb
pshn111/Machine-Learning-Package
fbbaa44daf5f0701ea77e5b62eb57ef822e40ab2
MIT
classification/notebooks/.ipynb_checkpoints/03 - Logistic Regression-checkpoint.ipynb
pshn111/Machine-Learning-Package
fbbaa44daf5f0701ea77e5b62eb57ef822e40ab2
MIT
classification/notebooks/.ipynb_checkpoints/03 - Logistic Regression-checkpoint.ipynb
pshn111/Machine-Learning-Package
fbbaa44daf5f0701ea77e5b62eb57ef822e40ab2
MIT
Qwen/Qwen-72B
1. YES
2. YES