## Math Tutorial Part 1


```python
import numpy as np
import sympy as sym
```

### 1) Stats


```python
# 1
x = [2,1,3,2,-1,0,1]

mean = np.mean(x)
var = np.var(x)
std = np.std(x)

print(mean, var, std)
```

    1.1428571428571428 1.5510204081632655 1.2453996981544782


2)

Bayes Theorem
P(H|D) = P(D|H) * P(H) / P(D)

H = prob that the cab was Blue
D = Witness correctly identified the cab was Blue

P(H) = 0.15
P(D|H) = 0.8

P(D) = P(D|H) * P(H) + P(D|H_bar) * P(H_bar)
     = 0.8 * 0.2 + 0.2 * 0.85
     = 0.29
     
P(H|D) = 0.8 * 0.2 / 0.29
       = 0.41
       
What is the probability that the cab involved in the accident was Blue rather than
Green?

Answer: 41%


3)

Assuming that the xi's are independent
Bernoulli random variables with unknown parameter p, and the maximum
likelihood estimator of p: the proportion of students who own a sports car.

proportion of students who own a sports car = p = (sum x_i) / n

### 2) Regression


```python
x_1 = np.asarray([2, 1, 3, 2, 1, 3])
x_2 = np.asarray([3, 2, -4, 1, 3, 2])
x_3 = np.asarray([-2, 4, 2, 1, 2, 3])

w = np.asarray([3, 1, 2,-1, 4, 3])

y_1 = 28
y_2 = 24
y_3 = 23
```


```python
print(np.dot(x_1,w))
print(np.dot(x_2,w))
print(np.dot(x_3,w))
```

    24
    20
    18



```python
bias = 2
w = np.append(w,bias)
x_1 = np.append(x_1,1)
np.dot(w,x_1)
```




    26




```python
x_1 = np.asarray([2, 1, 3, 2, 1, 3])
x_2 = np.asarray([3, 2, -4, 1, 3, 2])
x_3 = np.asarray([-2, 4, 2, 1, 2, 3])

w = np.asarray([3, 1, 2,-1, 4, 3])

y_1 = 28
y_2 = 24
y_3 = 23

l_1 = (y_1 - np.dot(x_1,w))**2
l_2 = (y_2 - np.dot(x_2,w))**2
l_3 = (y_3 - np.dot(x_3,w))**2
print(l_1)
print(l_2)
print(l_3)
print(l_1+l_2+l_3)
```

    16
    16
    25
    57


2.4a

L = sum_n ((yn-f(x,w))^2)

f(x,w) = vp(x,w) + b

derivative w.r.t. w = dL / dw

                    = sum_n d/dw (yn-f(xn,w))^2
                    
                    = sum_n 2 * (yn-f(xn,w)) * -xn
                    
                    = sum_n 2 * (yn-vp(w,xn) + b) * -xn

2.4b
dL / dx = sum_n 2 * (yn-f(xn,w)) * -w


```python
# 2.4c

dL_1 = np.sum(2 * (y_1 - np.dot(w,x_1)) * -x_1)
dL_2 = np.sum(2 * (y_2 - np.dot(w,x_2)) * -x_2)
dL_3 = np.sum(2 * (y_3 - np.dot(w,x_3)) * -x_3)
print(dL_1)
print(dL_2)
print(dL_3)
```

    -96
    -56
    -100


2.5a

R(w) = 1/2 * vp(w,w)

d(p,q) = sum_i (q_i - p_i)^2

d(w,0) = sum_i (w_i - 0)^2 = w_i^2 = vp(w,w) = R(w)


```python
R_w = np.dot(w,w)*0.5
print(R_w)
```

    20.0


2.5c

dR(w)/dw = w

vp(w,w) is essential w * w or W^2 

2.6a

When lambda minimized, set R to 0 

2.6b

d/dw (L + lambda * R)

= deriv(L) + lambda * deriv(R)
= (sum_n 2 * (yn-vp(w,xn) + b) * -xn) + lambda * w

2.6c

Parameters: w,b
Hyperparameters: lambda, eta

Rationale: model optimizes w,b against the loss function
HPs allow for manual tuning based on trade offs


```python
# 2.6d

# L + lambda * R
57 + 1 * 20
```




    77



### Bonus: Calc w Sympy


```python
x = sym.Symbol('x')
```

##### Power Rule


```python
sym.diff(x**5)
```




$\displaystyle 5 x^{4}$



##### Product Rule


```python
sym.diff(x**2+1) * sym.diff(x**3+4)
```




$\displaystyle 6 x^{3}$



##### Chain Rule


```python
sym.diff((x**2-3*x+5)**3)
```




$\displaystyle \left(6 x - 9\right) \left(x^{2} - 3 x + 5\right)^{2}$



#### Partial Derivatives


```python
x,y = sym.symbols('x y')
```


```python
sym.diff(x**2*y,x)
```




$\displaystyle 2 x y$




```python
sym.diff(x**2*y,y)
```




$\displaystyle x^{2}$




```python

```

7f7d6050b8882070e54ac3f41b7fb1eb9511d6af
ipynb
Jupyter Notebook
math_tutorial/math_tutorial_part1.ipynb
mswaringen/uva_aml
baa584444f04fdc99732751a2ab0ceab13f6828b
MIT
math_tutorial/math_tutorial_part1.ipynb
mswaringen/uva_aml
baa584444f04fdc99732751a2ab0ceab13f6828b
MIT
math_tutorial/math_tutorial_part1.ipynb
mswaringen/uva_aml
baa584444f04fdc99732751a2ab0ceab13f6828b
MIT
Qwen/Qwen-72B
1. YES
2. YES