## Linear Models for Regression & Classification

<hr>

### Agenda
1. Simple Linear Regression using Ordinary Least Squares
2. Gradient Descent Algorithm
3. Regularized Regression Methods - Ridge, Lasso, ElasticNet
4. Logistic Regression for Classification
5. OnLine Learning Methods - Stochastic Gradient Descent & Passive Aggrasive
6. Robust Regression - Dealing with outliers & Model errors
7. Polynomial Regression
8. Bias-Variance Tradeoff



<hr>


```python
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline
```

## 1. Simple Linear Regression using Ordinary Least Squares

* Feature consist of p independent variables (p-dim)
* Target/dependent variable is represented by y
* Relation between feature & target is represented by the following equation
* w's represent weights or coef's for each feature, w0 is intercept 




```python
from sklearn.linear_model import LinearRegression
```

* Residual Squared Sum (RSS) of Error - Sum of square of difference between actual value & predicted value



## 2. Gradient Descent

* LinearRegression tries to minimize RSS using <a href="https://www.kdnuggets.com/2017/04/simple-understand-gradient-descent-algorithm.html">Gradient Descent</a>. 
* The objective of Gradient Descent is the obtain best weights such that RSS is minimal.



# Feature Transformation


### Understanding Math behind gradient descent with simplified notation
* Prediction, $y_p = Ax+B$
* Actual, y
* Simplified Loss for caclulation, Loss = $1/2 *\sum(y_p - y)^2$
* Algorithm 
  - Randomly initialize weights A & B
  - Calculate gradient .i.e change in Loss when A & B are changed.
  - Change weights by gradients calculated & reduce the loss
  - Repeat the whole process till weights don't significantly reduce any further
  



```python
import matplotlib.pyplot as plt
%matplotlib inline
```

#### Genrating Regression Dataset
* n_features - number of features to be considered
* noise - deviation from straight line
* n_samples - number of samples


```python
from sklearn.datasets import make_regression
X,Y = make_regression(n_features=1, noise=10, n_samples=1000)
```


```python
plt.xlabel('Feature - X')
plt.ylabel('Target - Y')
plt.scatter(X,Y,s=5)
```

* Hyper-parameters are initial configuration of Models
* Initialize LinearRegression model with default hyper-parameters


```python
lr = LinearRegression()
```

#### Common Hyperparameters
* fit_interceprt - Whether to calculate intercept for the model, not required if data is centered
* normalize - X will be normalized by subtracting mean & dividing by standard deviation

* <b>PS: By stanrdadizing data before subjecting to model, coef's tells the importance of features</b>

#### Common Attributes
* coef - weights for each independent variables
* intercept - bias of independent term of linear models

#### Common Functions
* fit - trains the model. Takes X & Y
* predict - Once model is trained, for given X using predict function Y can be predicted

#### Multiple Target
* Y can be of more than 1 dimension
* Advantages of multiple target are 
  - computationally fast
  - model is optimized for multiple targets
  - model do not use relationship between targets
  - model is more interpretable
  
<hr/>

### Training model
* X should be in rows of data format, X.ndim == 2
* Y should be 1D for simgle target & 2D for more than one target
* fit function for training the model


```python
lr.fit(X,Y)
```




    LinearRegression(copy_X=True, fit_intercept=True, n_jobs=1, normalize=False)




```python
lr.coef_
```




    array([31.36482038])




```python
lr.intercept_
```




    -0.16344907235574668



### Predicting using trained model


```python
pred = lr.predict(X)
```

* Blue dots represent maps to actual target data
* Orange dots represent predicted data 


```python
plt.scatter(X,Y,s=5, label='training')
plt.scatter(X,pred,s=5, label='prediction')
plt.xlabel('Feature - X')
plt.ylabel('Target - Y')
plt.legend()
plt.show()
```

#### Limitation of Ordinary Least Square Technique
* Impacted by Outliers
* Non-linearities 
* Too many independent variables
* Multicollinearity 
* Heteroskedasticity
* Noise in the Independent Variables
* <a href="http://www.clockbackward.com/2009/06/18/ordinary-least-squares-linear-regression-flaws-problems-and-pitfalls/">References</a>

## 3. Regularized Regression Methods 


### Ridge Regression
* Ridge Regression imposes penalty on size of coef.
* Less impacted by outliers.

#### Adding outliers to data


```python
ridge = Ridge(alpha=.1)
lr = LinearRegression()
ridge.fit([[0, 0], [0, 0], [1, 1]],  [0, .1, 1])
lr.fit([[0, 0], [0, 0], [1, 1]],  [0, .1, 1])
```




    LinearRegression(copy_X=True, fit_intercept=True, n_jobs=1, normalize=False)




```python
ridge.coef_
```




    array([0.44186047, 0.44186047])




```python
lr.coef_
```




    array([-1.47086948e+14,  1.47086948e+14])




```python
outliers = Y[950:] - 600
```


```python
import numpy as np
Y_Out = np.append(Y[:950],outliers)
```


```python
plt.scatter(X,Y_Out,s=5)
```


```python
lr = LinearRegression()
```


```python
lr.fit(X,Y_Out)
```




    LinearRegression(copy_X=True, fit_intercept=True, n_jobs=1, normalize=False)




```python
pred_Out = lr.predict(X)
```


```python
plt.scatter(X,Y_Out,s=5,label='actual')
plt.scatter(X,pred_Out,s=5,label='prediction with outliers')
plt.scatter(X,pred,s=5,c='k', label='prediction without outlier')
plt.legend()
plt.title('Linear Regression')
```


```python
lr.coef_
```




    array([24.43755395])




```python
from sklearn.linear_model import Ridge
```


```python
ridge = Ridge(alpha=1000)
```


```python
ridge.fit(X,Y_Out)
```




    Ridge(alpha=1000, copy_X=True, fit_intercept=True, max_iter=None,
       normalize=False, random_state=None, solver='auto', tol=0.001)




```python
pred_ridge = ridge.predict(X)
```


```python
plt.scatter(X,Y_Out,s=5,label='actual')
plt.scatter(X,pred_Out,s=5, c='r' ,label='LinearRegression with outliers')
plt.scatter(X,pred_ridge,s=5,c='k', label='RidgeRegression with outlier')
plt.legend()
plt.title('Linear Regression')
```


```python
ridge.coef_
```




    array([12.48729023])



### Effects of alpha using Ridge on Coeficients 
* Data generation


```python
X, y, w = make_regression(n_samples=10, n_features=10, coef=True,
                          random_state=1, bias=3.5)
```


```python
w
```




    array([80.71051956, 10.74941291, 38.78606441, 13.64552257,  5.99176895,
           86.35418546, 12.13434557,  4.45518785, 74.71216427, 55.6240234 ])



### Training Ridge for different values of alpha
* Coefs calculated are appended to a list
* Generate 20 alphas from 10^-6 to 10^6


```python
alphas = np.logspace(-6, 6, 200)
alphas[:20]
```




    array([1.00000000e-06, 1.14895100e-06, 1.32008840e-06, 1.51671689e-06,
           1.74263339e-06, 2.00220037e-06, 2.30043012e-06, 2.64308149e-06,
           3.03677112e-06, 3.48910121e-06, 4.00880633e-06, 4.60592204e-06,
           5.29197874e-06, 6.08022426e-06, 6.98587975e-06, 8.02643352e-06,
           9.22197882e-06, 1.05956018e-05, 1.21738273e-05, 1.39871310e-05])




```python
coefs = []
for a in alphas:
    ridge = Ridge(alpha=a, fit_intercept=False)
    ridge.fit(X, y)
    coefs.append(ridge.coef_)
```

#### Plotting alphas & coefs
* Different colors represents different coefs

#### Conclusion
* As alpha tends toward zero the coefficients found by Ridge regression stabilize towards the randomly sampled vector w (similar to LinearRegression).
* For big alpha (strong regularisation) the coefficients are smaller (eventually converging at 0) leading to a simpler and biased solution.


```python
ax = plt.gca()
ax.plot(alphas, coefs)
ax.set_xscale('log')
plt.xlabel('alpha')
plt.ylabel('weights')
plt.title('Ridge coefficients as a function of the regularization')
plt.show()
```

### Lasso
* Linear model that predict's sparse coefs
* Reduces the regressors predicting target


```python
lasso = Lasso(alpha=.1)
lasso.fit([[0, 0], [0, 0], [1, 1]],  [0, .1, 1])
```




    Lasso(alpha=0.1, copy_X=True, fit_intercept=True, max_iter=1000,
       normalize=False, positive=False, precompute=False, random_state=None,
       selection='cyclic', tol=0.0001, warm_start=False)




```python
lasso.coef_
```




    array([0.5, 0. ])



### Elastic Net   

* Elastic-net is useful when there are multiple features which are correlated with one another. Lasso is likely to pick one of these at random, while elastic-net is likely to pick both.


```python
en = ElasticNet(alpha=.1)
```


```python
en.fit([[0, 0], [0, 0], [1, 1]],  [0, .1, 1])
```




    ElasticNet(alpha=0.1, copy_X=True, fit_intercept=True, l1_ratio=0.5,
          max_iter=1000, normalize=False, positive=False, precompute=False,
          random_state=None, selection='cyclic', tol=0.0001, warm_start=False)




```python
en.coef_
```




    array([0.32589556, 0.32579954])



## 4. Logistic Regression
* Linear Model of classification, assumes linear relationship between feature & target
* y = e^(b0 + b1*x) / (1 + e^(b0 + b1*x))
* Returns class probabilities
* Hyperparameter : C - regularization coef
* Fundamentally suited for bi-class classification




```python
from sklearn.datasets import make_blobs
```


```python
X,y = make_blobs(n_features=2, n_samples=1000, cluster_std=2,centers=2)
```


```python
plt.scatter(X[:,0],X[:,1],c=y,s=10)
```


```python
h = .02
x_min, x_max = X[:, 0].min() - .5, X[:, 0].max() + .5
y_min, y_max = X[:, 1].min() - .5, X[:, 1].max() + .5
xx, yy = np.meshgrid(np.arange(x_min, x_max, h), np.arange(y_min, y_max, h))
```


```python
from sklearn.linear_model import LogisticRegression
lr = LogisticRegression()
lr.fit(X,y)
```




    LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,
              intercept_scaling=1, max_iter=100, multi_class='ovr', n_jobs=1,
              penalty='l2', random_state=None, solver='liblinear', tol=0.0001,
              verbose=0, warm_start=False)




```python
Z = lr.predict(np.c_[xx.ravel(), yy.ravel()])
```


```python
Z = Z.reshape(xx.shape)
plt.pcolormesh(xx, yy, Z, cmap=plt.cm.Paired)
plt.scatter(X[:,0],X[:,1],c=y,s=10)
```

## 5. Online Learning Models
* Stochastic Gradient Descent & Passive Aggrasive Algorithms
* Simple & Efficient to fit linear models
* Useful where number of samples is very large ( Scale of 10^5 ) 
* Supports partial_fit for out-of-core learning
* Both the algorithms support regression & classification


```python
from sklearn.datasets import make_classification, make_regression
```


```python
X,y = make_classification(n_classes=2,n_features=10,n_samples=10000)
```


```python
from sklearn.model_selection import train_test_split
```


```python
trainX,testX, trainY,testY = train_test_split(X,y)
```


```python
from sklearn.linear_model import SGDClassifier
```


```python
sgd = SGDClassifier(n_iter=10)
```


```python
sgd.partial_fit(trainX[:1500],trainY[:1500], classes=[0,1])
sgd.score(testX,testY)
```




    0.7904




```python
sgd.partial_fit(trainX[1500:5000],trainY[1500:5000])
sgd.score(testX,testY)
```




    0.8116



### 6. Robust Regression
* Robust regression is interested in fitting a regression model in the presence of corrupt data: either outliers, or error in the model.
* Three techniques supported by scikit - RANSAC, Theil Sen and HuberRegressor

#### Comparisions RANSAC, Theil Sen, HuberRegressor
* HuberRegressor should be faster than RANSAC 
* Theil Sen and RANSAC are unlikely to be as robust as HuberRegressor for the default parameters.
* RANSAC will deal better with large outliers in the y direction
* RANSAC is faster than Theil Sen and scales much better with the number of samples
* RANSAC is a good default option


```python
n_samples = 1000
n_outliers = 50
X, y, coef = make_regression(n_samples=n_samples, n_features=1,
                                      n_informative=1, noise=10,
                                      coef=True, random_state=0)
# Add outlier data
np.random.seed(0)
X[:n_outliers] = 3 + 0.5 * np.random.normal(size=(n_outliers, 1))
y[:n_outliers] = -3 + 10 * np.random.normal(size=n_outliers)
```


```python
from sklearn.linear_model import LinearRegression,RANSACRegressor
```


```python
lr = LinearRegression()
lr.fit(X, y)
ransac = RANSACRegressor()
ransac.fit(X, y)
```




    RANSACRegressor(base_estimator=None, is_data_valid=None, is_model_valid=None,
            loss='absolute_loss', max_skips=inf, max_trials=100,
            min_samples=None, random_state=None, residual_metric=None,
            residual_threshold=None, stop_n_inliers=inf, stop_probability=0.99,
            stop_score=inf)




```python
plt.scatter(X,y,s=5)
```


```python
ransac_pred = ransac.predict(X)
```


```python
lr_pred = lr.predict(X)
```


```python
plt.scatter(X,y,s=5, label='data')
plt.scatter(X,ransac_pred,s=5,label='ransac')
plt.scatter(X,lr_pred,s=5, label='linear-regression')
plt.legend()
```

## 7. Polynomial Regression
* Sometimes relationship between variables & target is of higher polynomial degree
* Transformer can be used to convert data to higher degree
* Linear models can predict coef of these higher degree polynomials


```python
from sklearn.datasets import make_circles
```


```python
X,y = make_circles(n_samples=1000, noise=.04)
```


```python
plt.scatter(X[:,0],X[:,1],c=y,s=10)
```


```python
from sklearn.preprocessing import PolynomialFeatures
```


```python
pol = PolynomialFeatures(degree=2)
```


```python

```


```python
X_tf = pol.fit_transform(X)
```


```python
lr = LogisticRegression()
```


```python
trainX,testX,trainY,testY = train_test_split(X_tf,y)
```


```python
lr.fit(trainX,trainY)
```




    LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,
              intercept_scaling=1, max_iter=100, multi_class='ovr', n_jobs=1,
              penalty='l2', random_state=None, solver='liblinear', tol=0.0001,
              verbose=0, warm_start=False)




```python
lr.score(testX,testY)
```




    0.984




```python
lr.coef_
```




    array([[ 3.56153614,  0.08571121,  0.02482483, -8.77406605, -0.03769976,
            -8.78295209]])



### 8. Bias Variance 

#### Bias

* Fitting training data poorly, but produce similar result outside training data
* we are building simple models that predicts terribly far from the reality but they don't change much from dataset to dataset.
* Situation of underfitting.
*  a linear regression model would have high bias when trying to model a non-linear relationship.

#### Variance
* Building complex model that fits the training data well but many not work similar way of other dataset.
* Model is not generalized & is overfitting.

#### Bias Variance TradeOff
* Increasing the accuracy of the model will lead to less generalization of pattern outside training data. 
* Increasing the bias will decrease the variance. 
* Increasing the variance will decrease the bias.
* We have to get perfect balance of bias & variance



# 1. Linear Regression

### 1.1 Conceptual Questions
1. What is the difference between target and prediction? $
\cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot $ ($\star$)
2. What is the loss used in standard Linear Regression Problem?  $\cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot$ ($\star$)
3. What is objective function in Linear regression? $\cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot$  ($\star \star \star$)
4. What are parameters being optimized in Linear Regression? $\cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot$  ($\star$)
5. What is the feature vector in Linear Regression $\cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot$  ($\star$)
6. What are linear models? Are they only about lines? $\cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot$ ($\star$)
### 1.2 Programming Questions
1. Write a code to  $\hspace{2 cm}\cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot $ ($\star \star \star \star \star$)  
  a. generate a 1000 points following $y=2x+3 + 0.1*U(0,1)$ where **'N'** is uniform noise between 0 and 1 and x is lying between -1 and 1.  
  b. split the data into 8:2 ratio  
  c. Fit the Linear Regression model over 80% training data and find train loss and validation loss.  
  d. Plot the prediction and actual label for the given data in the same plot.
  e. Plot the predicted line and actual line in a same plot.  
  f. Use subplot to display **d.** and **e.** sidebyside.  
  g. How do you explain extra 0.05 in the bias coeffecient?  

# 2. Gradient Descent
### 2.1. Conceptual Questions
1. What is the gradient of a multivariate function $f(x,y)$? $
\cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot $ ($\star$)  
2. Find the partial derivate of the bellow functions with respect to x and y $
\cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot $ ($\star$)  
  $ \begin{align}
 a)\hspace{5pt}x^5 &&
 b)\hspace{5pt}(2x+y^2)^2  &&
 c)\hspace{5pt}ylog(x) &&
 d)\hspace{5pt}(y-\frac{1}{1+e^{-x}})^2
  \end{align}$  
  
3. Find the gradient of the bellow functions with respect to $ \vec{x}=[x_1,x_2]$ for $x=(0,0), (1,1)\text{ and }(0,1)$.   
***Note : $\vec{w}=[2,3]$ for the problem ***  
  
  $ \begin{align}
 a)\hspace{5pt} w^\top x &&
 b)\hspace{5pt}log(1+w^\top x)  &&
 c)\hspace{5pt} \frac{1}{1+e^{-w^\top x}} &&
  \end{align}$
  
4.  What is learning rate, step size and momentum in gradient descent. What do they signify in theory and in practice? $
\cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot $ ($\star \star \star \star \star$)  
5. What is difference between stochastic gradient descent, batch descent and the standard gradient descent. $
\cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot $ ($\star \star \star$)  
6. What is a Jacobian Matrix and Hessian Matrix?  $
\cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot $ ($\star $)  
7. Write the pseudo code for gradient descent for Linear Regression.

# 3. Polynomial Regression Introduction
### 3.1 Conceptual Questions
1. Is Polynomial regression special case of linear regression? If yes, what is the feature transformation used?  
2. What does each of the coeffecient in polynomial regression represent? Slope, Slope of slope, etc.  
3. What is the order and degree of polynomial?   

# 4. LASSO AND Ridge Regression
### 4.1 Conceptual Question
1. Find the Lp Norm with p = $\frac{1}{2},1,2$ for the bellow vectors  $ \hspace{2 cm}\cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot $ ($\star \star \star $) 
  $ \begin{align}
 a)\hspace{5pt} [1,2,3] &&
 b)\hspace{5pt} [3,4]  &&
 c)\hspace{5pt} [4,3] &&
 d)\hspace{5pt} [6,8,10,30,100] &&
 e)\hspace{5pt} [1,10,100]
  \end{align}$  

2. When do we choose Manhattan distance instead of euclidean. List atleast 3 reasons.  
3. Why do we regularize weights? What is its impact on objective function?  
4. What is a sparse solution? Why do we desire it?  
5. What is the impact of strong regularization? How to choose a regularization constant?  
6. Why does L1 regularization of weights give a sparse solution ?  
7. How to theoratically find if the regularization is dominant or is it the rms loss function?

### 4.2 Programming Questions
1.  Write a code to  $ \hspace{2 cm}\cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot $ ($\star \star \star \star \star$)  
  a. generate a 1000 points following $y=2x^2+5 + N(0,1)$ where **'N'** is standard normal gaussian noise with 0 mean, 1 standard deviation and where x is between -2 and 2.  
  b.  Fit the linear equation, quadratic equation and polynomial of order 5.  
  c.  Find out the number of iteration it takes to get a solution within $\pm 0.01$ error margin for all the three cases.  
  d.  Try to use L1 Regularization and find out the feature indices which had 0 weight coefficient associated with it in the solution. i.e in $ax^4+bx^3+cx^2+dx+e$, if a,e are 0, then $x^4$,constant are the redundant features.
  e.  Select only 25 points out of 1000 points and fit the models mentioned in **'b.'** again.  
  f. Use L2 regularization and observe if it gives a better solution after 200 iterations than its counterpart unregularized versions.

# 5 Robust Regression
### 5.1 Conceptual Questions
1. Write the pseudo-code for RANSAC.  
2. How is RANSAC different from Theil Sen? Which one gives faster convergence?  

### 5.2 Programming Questions
1. Write a code to  $ \hspace{2 cm}\cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot \cdot $ ($\star \star \star \star \star$)  
  a. Generate a 1000 points following $y=2x+3 + 0.1*U(0,1)$ where **'N'** is uniform noise between 0 and 1 and x is lying between -1.   
  b. Add 50 more points with equation   $y=2x+200 + 0.1*U(0,1)$ with x lying between -1 and 1. This results in a total of 1050 points.   
  c. Compare the performance of Linear Regression and RANSAC for the same.  
  d. Plot the points considered as inliers by the RANSAC with color1 and the outliers with color2.  
  e. Plot the left and right margin for RANSAC line fit along with the points.  

# 5 Logistic Regression
### 5.1 Conceptual Questions
1. What is the decision boundary in Logistic Regression? What is the order of the equation to represent decision boundary for a feature vector of dimension n?  
2. How is classification different from regression. What are the different ways to convert binary classifier into a multiclass classifier?  
3. Why did we use sigmoid curve instead of heaviside/step function?  
4. What is entropy of a probability distribution?  
5. Write down the binary cross entropy loss for logistic regression.  
6. Why do we use cross entropy loss instead of root mean square loss?  
7. What is catagorical cross entropy loss? How is it different from binary cross entropy loss?  
8. What is difference between self entropy and cross entropy. Which one are we using for loss?  
9. Why is the output of logistic regression called logit?  
10. What is the target/label/actual output  and behavior/logit/predicted output ?  
11. What is it that we are trying to learn ? Why do we call classification/regression as a function approximation problem?  

### 5.2 Programming Questions
1. Write the code for the following.  
  a. Generate a set of 1000 points in 2D with centers as (-2,0) and (2,0) and standard deviation as 0.5.  
  b. Shuffle the data and select 80% of data for training and 20% for validation  
  c. Plot the training data  
  d. Find the accuracy for Training and Validation  
  e. Do the steps from a. to d. with 3 centers instead of two.i.e. (-2,0),(-6,0) and (0,2). Here points via center (-6,0) and (0,2) belong to the same class. Check the performance of the Logistic Regression. Explain the change in accuracy for validation data.  




99c9db8ffc960b56a3733149c537cd120efde0bc
ipynb
Jupyter Notebook
Code/4. Linear Models for Classification _ Regression.ipynb
Abhishekjha111/AI-codes
41836e8326496a0ceef1148ea35cbf4c949870c2
Apache-2.0
Code/4. Linear Models for Classification _ Regression.ipynb
Abhishekjha111/AI-codes
41836e8326496a0ceef1148ea35cbf4c949870c2
Apache-2.0
Code/4. Linear Models for Classification _ Regression.ipynb
Abhishekjha111/AI-codes
41836e8326496a0ceef1148ea35cbf4c949870c2
Apache-2.0
Qwen/Qwen-72B
1. YES
2. YES