<jupyter_start><jupyter_text>Homework 6 Due:  Tuesday, October 10 at 11:59 PM  Problem 1: Bank Account RevisitedWe are going to rewrite the bank account closure problem we had a few assignments ago, only this time developing a formal class for a Bank User and Bank Account to use in our closure (recall previously we just had a nonlocal variable amount that we changed).   Some Preliminaries:First we are going to define two types of bank accounts.  Use the code below to do this:<jupyter_code>from enum import Enum
class AccountType(Enum):
    SAVINGS = 1
    CHECKING = 2<jupyter_output><empty_output><jupyter_text>An Enum stands for an enumeration, it's a convenient way for you to define lists of things. Typing:<jupyter_code>AccountType.SAVINGS<jupyter_output><empty_output><jupyter_text>returns a Python representation of an enumeration. You can compare these account types:<jupyter_code>AccountType.SAVINGS == AccountType.SAVINGS
AccountType.SAVINGS == AccountType.CHECKING<jupyter_output><empty_output><jupyter_text>To get a string representation of an Enum, you can use:<jupyter_code>AccountType.SAVINGS.name
raise ValueError("ba")<jupyter_output><empty_output><jupyter_text>Part 1: Create a BankAccount class with the following specification:Constructor is `BankAccount(self, owner, accountType)` where `owner` is a string representing the name of the account owner and `accountType` is one of the AccountType enumsMethods `withdraw(self, amount)` and `deposit(self, amount)` to modify the account balance of the accountOverride methods `__str__` to write an informative string of the account owner and the type of account, and `__len__` to return the balance of the account<jupyter_code>class BankAccount():
    "Class Bank Account specifying owner and Account type"
    def __init__(self, owner, accountType):
        self.owner = owner
        self.balance = 0
        self.AccountType = accountType
        
    def withdraw(self,amount):
        if amount <= 0:
            raise ValueError("Cannot withdraw a negative value")
        if amount > self.balance:
            raise ValueError("Insufficient funds")
        
        self.balance = self.balance - amount
        print("Withdrawal of ",amount," from account ",self.AccountType.name,". Remaining balance ", self.balance)
    
    def deposit(self,deposit):
        if deposit <=0:
            raise ValueError
        else:
            self.balance +=deposit
            print("Deposit of ", deposit, " and new balance is ", self.balance)
        
    def __str__(self):
        
        return "Owner of account "+ self.owner + " and account type is " + self.AccountType.name
    
    def __len__(self):
        #return the balance of acc
        return self.balance
    
Bankacc1 = BankAccount("<NAME>", AccountType.SAVINGS) 
print(Bankacc1)

Bankacc1.deposit(100)
print(len(Bankacc1))<jupyter_output>Owner of account <NAME> and account type is SAVINGS
Deposit of  100  and new balance is  100
100<jupyter_text>Part 2:  Write a class BankUser with the following specification:Constructor `BankUser(self, owner)` where `owner` is the name of the account.Method `addAccount(self, accountType)` - to start, a user will have no accounts when the BankUser object is created. `addAccount` will add a new account to the user of the `accountType` specified. **Only one savings/checking account per user, return appropriate error otherwise**Methods `getBalance(self, accountType)`, `deposit(self, accountType, amount)`, and `withdraw(self, accountType, amount)` for a specific AccountType.Override `__str__` to have an informative summary of user's accounts.<jupyter_code>class BankUser():
    
    def __init__(self,owner):
        self.owner = owner
        self.savings = None
        self.checking = None
        self.myCaccount = None
        self.mySaccount = None
        
    def addAccount(self, accountType):
        #Check what account type and then whether the account is there
        if accountType == AccountType.SAVINGS:
            if self.savings == None:
                self.savings = 1
                self.mySaccount = BankAccount(self.owner,accountType)
            
            else:
                raise ValueError("You already have a savings account")
                
        elif accountType == AccountType.CHECKING:
            if self.checking == None:
                self.savings = 1
                self.myCaccount = BankAccount(self.owner,accountType)
            
            else:
                raise ValueError("You already have a checking account")
        else:
            raise ValueError("Account type is not valid")
    
    def getBalance(self,accountType):
        if accountType == AccountType.SAVINGS:
            if self.mySaccount == None:
                raise ValueError("No savings account")
            else:
                return self.mySaccount.balance
            
        elif accountType == AccountType.CHECKING:
            if self.myCaccount == None:
                raise ValueError("No checking account")
            else:
                return self.myCaccount.balance
        else:
            raise ValueError("Invalid account type inserted")
    
    def deposit(self,accountType,amount):
        if accountType == AccountType.SAVINGS:
            if self.mySaccount == None:
                raise ValueError("No savings account")
            else:
                return self.mySaccount.deposit(amount)
            
        elif accountType == AccountType.CHECKING:
            if self.myCaccount == None:
                raise ValueError("No checking account")
            else:
                return self.myCaccount.deposit(amount)
        
        else:
            raise ValueError("Invalid account type inserted")
            
    def withdraw(self,accountType,amount):
        #analogous to deposit
        if accountType == AccountType.SAVINGS:
            if self.mySaccount == None:
                raise ValueError("No savings account")
            else:
                return self.mySaccount.withdraw(amount)
            
        elif accountType == AccountType.CHECKING:
            if self.myCaccount == None:
                raise ValueError("No checking account")
            else:
                return self.myCaccount.withdraw(amount)
        
        else:
            raise ValueError("Invalid account type inserted")
        
user1 = BankUser("Filip")
user1.addAccount(AccountType.CHECKING)
user1.getBalance(AccountType.CHECKING)
user1.deposit(AccountType.CHECKING,7500)
user1.withdraw(AccountType.CHECKING,-2500)<jupyter_output>Deposit of  7500  and new balance is  7500<jupyter_text>Write some simple tests to make sure this is working. Think of edge scenarios a user might try to do.<jupyter_code>def test_withdraw_negative():
    try:
        user1 = BankUser("Filip")
        user1.addAccount(AccountType.CHECKING)
        user1.withdraw(AccountType.CHECKING,-2500)
    except ValueError as Valer:
        assert(type(Valer)==ValueError)
test_withdraw_negative()<jupyter_output><empty_output><jupyter_text>Part 3:  ATM ClosureFinally, we are going to rewrite a closure to use our bank account. We will make use of the [input function](http://anh.cs.luc.edu/python/hands-on/3.1/handsonHtml/io.html) which takes user input to decide what actions to take.Write a closure called ATMSession(bankUser) which takes in a BankUser object. Return a method called Interface that when called, would provide the following interface:First screen for user will look like:**Enter Option:****1)Exit****2)Create Account****3)Check Balance****4)Deposit****5)Withdraw**Pressing 1 will exit, any other option will show the options:**Enter Option:****1)Checking****2)Savings**If a deposit or withdraw was chosen, then there must be a third screen:**Enter Integer Amount, Cannot Be Negative:**This is to keep the code relatively simple, if you'd like you can also curate the options depending on the BankUser object (for example, if user has no accounts then only show the Create Account option), but this is up to you. In any case, you must handle any input from the user in a reasonable way that an actual bank would be okay with, and give the user a proper response to the action specified.Upon finishing a transaction or viewing balance, it should go back to the original screen<jupyter_code>def ATMSession(bankUser):
    def interface():
        while True:<jupyter_output><empty_output><jupyter_text>Part 4:  Put everything in a module Bank.pyWe will be grading this problem with a test suite. Put the enum, classes, and closure in a single file named Bank.py. It is very important that the class and method specifications we provided are used (with the same capitalization), otherwise you will receive no credit. ---  Problem 2:  Linear Regression ClassLet's say you want to create Python classes for three related types of linear regression: Ordinary Least Squares Linear Regression, Ridge Regression, and Lasso Regression.  Consider the multivariate linear model:$$y = X\beta + \epsilon$$where $y$ is a length $n$ vector, $X$ is an $m \times p$ matrix, and $\beta$is a $p$ length vector of coefficients. Ordinary Least Squares Linear Regression[OLS Regression](https://en.wikipedia.org/wiki/Ordinary_least_squares) seeks to minimize the following cost function:$$\|y - \beta\mathbf {X}\|^{2}$$The best fit coefficients can be obtained by:$$\hat{\beta} = (X^T X)^{-1}X^Ty$$where $X^T$ is the transpose of the matrix $X$ and $X^{-1}$ is the inverse of the matrix $X$. Ridge Regression[Ridge Regression](https://en.wikipedia.org/wiki/Tikhonov_regularization) introduces an L2 regularization term to the cost function:$$\|y - \beta\mathbf {X}\|^{2}+\|\Gamma \mathbf {x} \|^{2}$$Where $\Gamma = \alpha I$ for some constant $\alpha$ and the identity matrix $I$.The best fit coefficients can be obtained by:$$\hat{\beta} = (X^T X+\Gamma^T\Gamma)^{-1}X^Ty$$ Lasso Regression[Lasso Regression](https://en.wikipedia.org/wiki/Lasso_%28statistics%29) introduces an L1 regularization term and restricts the total number of predictor variables in the model.The following cost function:$${\displaystyle \min _{\beta _{0},\beta }\left\{{\frac {1}{m}}\left\|y-\beta _{0}-X\beta \right\|_{2}^{2}\right\}{\text{ subject to }}\|\beta \|_{1}\leq \alpha.}$$does not have a nice closed form solution. For the sake of this exercise, you may use the [sklearn.linear_model.Lasso](http://scikit-learn.org/stable/modules/generated/sklearn.linear_model.Lasso.html) class, which uses a coordinate descent algorithm to find the best fit. You should only use the class in the fit() method of this exercise (ie. do not re-use the sklearn for other methods in your class). $R^2$ scoreThe $R^2$ score is defined as:$${R^{2} = {1-{SS_E \over SS_T}}}$$Where:$$SS_T=\sum_i (y_i-\bar{y})^2, SS_R=\sum_i (\hat{y_i}-\bar{y})^2, SS_E=\sum_i (y_i - \hat{y_i})^2$$where  ${y_i}$ are the original data values, $\hat{y_i}$ are the predicted values, and $\bar{y_i}$ is the mean of the original data values.  Part 1:  Base ClassWrite a class called `Regression` with the following methods:$fit(X, y)$: Fits linear model to $X$ and $y$.$get\_params()$: Returns $\hat{\beta}$ for the fitted model. The parameters should be stored in a dictionary.$predict(X)$: Predict new values with the fitted model given $X$.$score(X, y)$: Returns $R^2$ value of the fitted model.$set\_params()$: Manually set the parameters of the linear model.This parent class should throw a `NotImplementedError` for methods that are intended to be implemented by subclasses.<jupyter_code>class Regression:
    
    def __init__(self):
        self.X = 0
        self.y = 0
        self.betas = 0
        self.alpha=0.1
        
    def fit(self,X,y):
        #reshape x and y arrays to get the right matrices for regre

        raise NotImplementedError("Subclasses should implement this!")
        
    def get_params(self):
        self_betas=self.betas.reshape(-1,1)
        print(self.betas.shape)
        betas_dict={}
        for index,value in enumerate(self.betas):
            betas_dict[index]=value
        return betas_dict
    
    def predict(self,X):
        #print(self.betas.shape)
        #print(sm.add_constant(X).T.shape)
        #print(self.betas)
        #print(sm.add_constant(X).T)
        ypred = np.dot(self.betas,(sm.add_constant(X).T))
        
        #raise NotImplementedError("Subclasses should implement this!")
        return ypred
        
    def score(self,X,y):
        ypred = self.predict(X)
        ypred = ypred.reshape(-1,1)
        #print(ypred)
        y_mean=np.mean(y)
        y=y.reshape(-1,1)
        #print("y_mean")
        #print(y_mean)
        #print("y-y_pred")
        #print((y - ypred)*(y - ypred))
        #print(((y - ypred)).shape)
        #print((y - y_mean)*(y-y_mean))
        #print(np.sum((y - y_mean**2)))
        score = 1- (np.sum((y - ypred)*(y - ypred))/np.sum((y - y_mean)*(y - y_mean)))
        #raise NotImplementedError("Subclasses should implement this!")
        return score
        
    def set_params(self):
        beta0 = float(input("Please input zero coefficient"))
        beta1 = float(input("Please input first coefficient"))
        self.betas = [beta0,beta1]
        return None
        
#import statsmodels.api as sm

x_train = np.array([1, 2, 3, 4])
y_train = np.array([2, 2, 4, 5])
#print(y_train.reshape(-1,1))
test_regr = Regression(x_train,y_train)

#test_regr.fit(x_train,y_train)
#test_regr.set_params()
#test_regr.get_params()<jupyter_output><empty_output><jupyter_text>Part 2: OLS Linear RegressionWrite a class called `OLSRegression` that implements the OLS Regression model described above and inherits the `Regression` class.<jupyter_code>class OLSRegression(Regression):
    
    
    def fit(self,X,y):
        #reshape x and y arrays to get the right matrices for regre
        self.X = X
        self.y= y
        #x_train = self.X.reshape(len(self.X),1)
        #y_train = self.y.reshape(len(self.y),1)

        #build matrix X by concatenating predictors and a column of ones
        n = x_train.shape[0]
        ones_col = np.ones((n, 1))
        X = np.concatenate((ones_col, x_train), axis=1)

        #matrix X^T X
        LHS = np.dot(np.transpose(X), X)
        #print(LHS)
        #matrix X^T Y
        RHS = np.dot(np.transpose(X), y_train)

        #solution beta to normal equations, since LHS is invertible by construction
        self.betas = np.dot(np.linalg.inv(LHS), RHS)
        print(self.betas)
        print(self.betas.shape)
        #return None

        
x_train = np.array([1, 2, 3, 4])
y_train = np.array([2, 2, 4, 5])
#print(y_train.reshape(-1,1))

#gamma = 3*np.eye(x_train.shape[0])
#print(gamma)
test_regr1 = OLSRegression()
test_regr1.fit(x_train,y_train)
#test_regr1.score(x_train,y_train)

test_regr1.predict(x_train)
test_regr1.score(x_train,y_train)<jupyter_output><empty_output><jupyter_text>Part 3: Ridge RegressionWrite a class called `RidgeRegression` that implements Ridge Regression and inherits the `OLSRegression` class. $$\hat{\beta} = (X^T X+\Gamma^T\Gamma)^{-1}X^Ty$$<jupyter_code>class RidgeRegression(OLSRegression):
    
    def fit(self,X,y,alpha):
        self.X = X
        self.y= y
        #x_train = self.X.reshape(len(self.X),1)
        #y_train = self.y.reshape(len(self.y),1)
        
        
        #build matrix X by concatenating predictors and a column of ones
        n = x_train.shape[0]
        ones_col = np.ones((n, 1))
        X = np.concatenate((ones_col, x_train), axis=1)
        
        
        #matrix X^T X
        LHS = np.dot(np.transpose(X), X)
        #print(LHS)
        gamma = alpha*np.eye(LHS.shape[0])
        #print(gamma)
        gammas = np.dot(np.transpose(gamma), gamma)
        
        
        #matrix X^T Y
        RHS = np.dot(np.transpose(X), y_train)

        #solution beta to normal equations, since LHS is invertible by construction
        self.betas = np.dot(np.linalg.inv(LHS+gammas), RHS)
        #print(self.betas)
        print(self.betas)
        print(self.betas.shape)
        return None
test_ridge = RidgeRegression()
test_ridge.fit(x_train,y_train,0.5)
test_ridge.predict(x_train)
test_ridge.get_params()
test_ridge.score(x_train,y_train)
test_ridge.get_params()<jupyter_output>(14,)<jupyter_text>Part 3: Lasso RegressionWrite a class called `LassoRegression` that implements Lasso Regression and inherits the `OLSRegression` class. You should only use Lasso(), Lasso.fit(), Lasso.coef_, and Lasso._intercept from the `sklearn.linear_model.Lasso` class.<jupyter_code>from sklearn.linear_model import Lasso
class LassoRegression(OLSRegression):
    
    def fit(self,X,y):
        #X=X.reshape(len(X),1)
        #y=y.reshape(len(y),1)
        
        model = Lasso(alpha=self.alpha,fit_intercept=True)
        fitted_model= model.fit(X,y)
        intercept = fitted_model.intercept_
        coeffs = np.array(fitted_model.coef_)
        self.betas=np.insert(coeffs,0,intercept)
        #self.betas = self.betas.reshape(-1,1)
        #=np.array([fitted_model.intercept_[0],fitted_model.coef_[i]]).reshape(-1,1)
        print(self.betas)
        
print(x_train.shape)
print(y_train.shape)
lasso_test= LassoRegression()
lasso_test.fit(x_train,y_train)
#lasso_test.get_params()
lasso_test.score(x_train,y_train)<jupyter_output>(14,)
(14, 354)<jupyter_text>Part 4: Model ScoringYou will use the [Boston dataset](http://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_boston.html) for this part.Instantiate each of the three models above.  Using a for loop, fit (on the training data) and score (on the testing data) each model on the Boston dataset.  Print out the $R^2$ value for each model and the parameters for the best model using the `get_params()` method. Use an $\alpha$ value of 0.1.**Hint:**  You can consider using the `sklearn.model_selection.train_test_split` method to create the training and test datasets.<jupyter_code>from sklearn.datasets import load_boston 
from sklearn.model_selection import train_test_split

boston = load_boston()
#print(boston.target)
#print(boston.data)

x_train, x_test, y_train,y_test = train_test_split(boston.data,boston.target, test_size=0.3)


print(x_train.shape)
"""
modelOLS = OLSRegression()
modelRidge = RidgeRegression()
modelLasso = LassoRegression()

models = [modelOLS, modelRidge, modelLasso]

for model in models:
    model.fit(x_train,y_train)
    model.score
"""<jupyter_output>(354, 13)