# STA208 Machine Learning Memo

(Contents are originally provided by James Sharpnack, annotated and revised by Gento Kato)

***THIS MEMO IS NOT COMPLETE***

## Lecture 8 (04/27/2017)

### Margin Based Methods

#### Logistic Regression Loss

 * **Model** <br>
 DV: $y \in \{-1, 1\}$ <br>
 Prediction: $\hat{y} = 1\{x^\top \hat{\beta} + \hat{\beta}_0 \geq 0\}$
 
 * **Loss** <br>
  * 0-1 loss (what we want to minimize, but nonconvex): 
  
  $$\mathit{l}_{0/1}(y_i, x_i \beta) = 1\{y_i \cdot x_i^\top \beta \leq 0 \}$$ <br>
  
  * Logistic loss (actually minimized in logit): 
  
  \begin{align}
  \mathit{l}_{logit} (y_i, x_i, \beta) &=
  \begin{cases}
  log(1 + e^{x_i^\top \beta}), & y_i = -1 \\
  log(1 + e^{-x_i^\top \beta}), & y_i = 1  
  \end{cases}
  \\ &= log(1+e^{-y_i x_i^T \beta})
  \end{align} <br>
  
  * Square error loss (we can also use but...):
  
  $$\mathit{l}_2(y_i, x_i, \beta) = (y_i - x_i^\top \beta)^2 = (1- y_i x_i^\top \beta)^2$$ <br>
  
  * Logistic loss approximates 0-1 loss more than square error loss, but **computationally inefficient**.

### Support Vector Machines (SVM)



5885ef5977833b9eb29db63df378fed39f4af7ea
ipynb
Jupyter Notebook
notebooks/STA208_MachineLearning_Memo.ipynb
gentok/Method_Notes
a7b60e50132fdda764efcfb1e163d1b31b2f99f7
MIT
notebooks/STA208_MachineLearning_Memo.ipynb
gentok/Method_Notes
a7b60e50132fdda764efcfb1e163d1b31b2f99f7
MIT
notebooks/STA208_MachineLearning_Memo.ipynb
gentok/Method_Notes
a7b60e50132fdda764efcfb1e163d1b31b2f99f7
MIT
Qwen/Qwen-72B
1. YES
2. YES