# Problem Set 1
**Due: April 26, 2021** (in class; subject to change if COVID restrictions apply)

## Problem 1

Suppose you have n i.i.d. observations from the nonlinear regression
$$y_i=(x_i+\beta)^2 + \epsilon_i \quad \epsilon_i \vert x_i \sim \mathcal{N}(0,1)$$
1. Find the asymptotic distribution of the nonlinear least squares estimator of $\beta$.
2. Find the asymptotic distribution of the maximum likelihood estimator of $\beta$. Compare the two asymptotic distributions.

**Hint**: For NLS $m(x_i;\beta)=-[y_i-(x_i+\beta)^2]^2$. For MLE $m(x_i;\beta)=\log f(y\vert x;\beta)=\log(1)-\log(\sqrt{2\pi})-\frac{1}{2}[y-(x+\beta)^2]^2$

## Problem 2
**Hint**: 
- Read Hayashi Chapter 7.3, Example 7,10
- Read Hayashi Chapter 1


Consider the linear regression model with normal errors, whose conditional density for observation $i$ is
$$\log f(y_i \vert \bf{x}_i ; \beta, \sigma^2) = - \frac{1}{2}\log (2\pi) - \frac{1}{2}\log (\sigma^2) - \frac{\big(y_i - \bf{x}_i'\beta\big)^2}{2\sigma^2}$$

Let $\big(\hat{\beta},\hat{\sigma}^2\big)$ be the unrestricted ML estimate of $\theta = \big(\beta,\sigma^2\big)$ and let $\big(\tilde{\beta},\tilde{\sigma}^2\big)$ be the restricted ML estimate subject to the constraint $\bf{R}\beta = c$ where $\bf{R}$ is an $r \times X$ matrix of known constraints. Assume that $\Theta = \mathbb{R}^K \times \mathbb{R}_++$ and that $\mathbb{E}[x_ix_i']$ is nonsingular. Also, let 
$$\hat{\Sigma} = \begin{bmatrix}
\frac{1}{\hat{\sigma^2}}\frac{1}{n}\sum_{1}^{n}x_ix_i'&0\\
0&\frac{1}{2(\hat{\sigma}^2)^2}\\
\end{bmatrix}$$
$$\quad$$
$$\tilde{\Sigma} = \begin{bmatrix}
\frac{1}{\tilde{\sigma^2}}\frac{1}{n}\sum_{1}^{n}x_ix_i'&0\\
0&\frac{1}{2(\tilde{\sigma}^2)^2}\\
\end{bmatrix}$$
- (a) Verify that $\hat{\beta}$ minimizes the sum of squared residuals. So it is the OLS estimator. Verify that $\tilde{\beta}$ minimizes the sum of squared residuals subject to the constraint $\bf{R}\beta = c$. so it is the restricted least squares estimator. **Hint**: to set up a problem, please refer to Restrictions section from EE.pdf notes. You can replace the term $\sum (y_i - \bf{x}'_i \beta)^2$ with the sum of squaredresidual: i.e.,  $SSR(\beta) \equiv \sum (y_i - \bf{x}'_i \beta)^2$.

- (b) Let $Q_n(\theta) = \log f(y_i \vert \bf{x}_i ; \beta, \sigma^2)$. Show that 
$$Q_n(\hat{\theta}) = -\frac{1}{2}\log (2\pi) -\frac{1}{2}-\frac{1}{2}\log \big(\frac{SSR_U}{n}\big)$$
$$Q_n(\tilde{\theta}) = -\frac{1}{2}\log (2\pi) -\frac{1}{2}-\frac{1}{2}\log \big(\frac{SSR_R}{n}\big)$$
where 
$SSR_U \equiv \sum (y_i - \bf{x}'_i \hat{\beta})^2$ is the unrestricted sum of squared residuals and 
$SSR_R \equiv \sum (y_i - \bf{x}'_i \tilde{\beta})^2$ is the restricted sum of squared residuals.


- (c) Verify that the $\hat{\Sigma}$ given here, although not the same as $-\frac{1}{n}\sum_{1}^{n}\bf{H}(\bf{w}_i;\hat{\theta})$, is consistent for $\mathbb{E}\big[\bf{w}_i;\theta_0\big]$. Verify that the $\tilde{\Sigma}$ given here, although not the same as $-\frac{1}{n}\sum_{1}^{n}\bf{H}(\bf{w}_i;\tilde{\theta})$, is consistent for $\mathbb{E}\big[\bf{w}_i;\theta_0\big]$. 


- (d) Show that the Wald, LM, and LR statistics using $\hat{\Sigma}$ and $\tilde{\Sigma}$ given here can be written as 
$$\begin{align}
W & = n \cdot \frac{(\bf{R}\hat{\beta}-c)'[\bf{R(X'X)^{-1}R'}]^{-1}(\bf{R}\hat{\beta}-c)}{SSR_U}\\
LM & =n \cdot \frac{(y-\bf{X}\tilde{\beta})'P(y-\bf{X}\tilde{\beta})}{SSR_R} \\
LR & =n \cdot \bigg[ \log \bigg( \frac{SSR_R}{n} \bigg)- \log \bigg(\frac{SSR_U}{n}\bigg) \bigg]
\end{align}$$
where $y$ is a vector with dimension $(n \times 1)$ and $\bf{X}$ is a matrix with dimension $(n \times K)$.


- (e) Show that the three statistics can also be written as
$$\begin{align}
W & = n \cdot \frac{SSR_R - SSR_U}{SSR_U}\\
LM & =n \cdot \frac{SSR_R - SSR_U}{SSR_R} \\
LR & =n \cdot \log \bigg( \frac{SSR_R}{SSR_U} \bigg)
\end{align}$$

**Hint**: Refer to Analytical Exercise 1,  Chapter 1 of Hayashi.
$$\begin{align}
SSR_R - SSR_U 
& = (\hat{\beta}-\tilde{\beta})'(X'X)(\hat{\beta}-\tilde{\beta})\\
& = (R\hat{\beta}-c)'[R(X'X)^{-1}R'](R\hat{\beta}-c)\\
& = (y-X\tilde{\beta})'P()(y-X\tilde{\beta})
\end{align}$$
where $P = X(X'X)^{-1}X'$

- (f) Show that $W \geq LR \geq LM$. These inequality do not always hold in nonlinear regression models.

f470844673ef7f06d3a77d49852f84fa92a792a9
ipynb
Jupyter Notebook
ps/ps1.ipynb
minyoungrho/Econometrics2
96142dd9b1db14ddf61f57408ed7505a951135f9
MIT
ps/ps1.ipynb
minyoungrho/Econometrics2
96142dd9b1db14ddf61f57408ed7505a951135f9
MIT
ps/ps1.ipynb
minyoungrho/Econometrics2
96142dd9b1db14ddf61f57408ed7505a951135f9
MIT
2021-04-21T09:25:38.000Z
2021-04-26T08:05:15.000Z
Qwen/Qwen-72B
1. YES
2. YES