<s>[INST] MSE of the Jackknife Estimator for the Uniform distribution The Jackknife is a resampling method, a predecessor of the Bootstrap, which is useful for estimating the bias and variance of a statistic. This can also be used to apply a "bias correction" to an existing estimator.
Given the estimand $\theta$ and an estimator $\hat\theta \equiv \hat\theta(X_1, X_2, \cdots X_n)$, the Jackknife estimator (with respect to $\hat\theta$) is defined as
$$\hat\theta_J = \hat\theta + (n-1)\left(\hat\theta - \frac{1}{n}\sum_{i=1}^n\hat\theta_{-i}\right),$$
where the $\hat\theta_{-i}$ terms denote the estimated value ($\hat\theta$) after "holding out" the $i^{th}$ observation.

Let $X_1, X_2, \cdots X_n \stackrel{\text{iid}}{\sim} \text{Unif}(0, \theta)$ and consider the estimator $\hat\theta = X_{(n)}$ (i.e. the maximum value, also the MLE). Note that
$$\hat\theta_{-i} = \begin{cases}
X_{(n-1)}, & X_i = X_{(n)} \\[1.2ex]
X_{(n)}, & X_i \neq X_{(n)}
\end{cases}.$$
Thus the Jackknife estimator here can be written as a linear combination of the two largest values 
\begin{align*}
\hat\theta_J &= X_{(n)} + \frac{n-1}{n}\left(X_{(n)} - X_{(n-1)}\right) \\[1.3ex]
&= \frac{2n-1}{n}X_{(n)} - \frac{n-1}{n} X_{(n-1)}.
\end{align*}
What is the bias, variance and mean square error ?
 [/INST] It is well known that the order statistics, sampled from a uniform distribution, are Beta-distributed random variables (when properly scaled).
$$\frac{X_{(j)}}{\theta} \sim \text{Beta}(j, n+1-j)$$
Using standard properties of the Beta distribution we can obtain the mean and variance of $X_{(n)}$ and $X_{(n-1)}$.
Bias
\begin{align*}
E\left(\hat\theta_J\right) &= \frac{2n-1}{n}E(X_{(n)}) - \frac{n-1}{n}E(X_{(n-1)}) \\[1.3ex]
&= \frac{2n-1}{n}\frac{n}{n+1}\theta - \frac{n-1}{n}\frac{n-1}{n+1}\theta \\[1.3ex]
&= \frac{n(n+1) - 1}{n(n+1)} \theta
\end{align*}
Therefore the bias of $\hat\theta_J$ is given by
$$\text{Bias}_\theta(\hat\theta_J) = \frac{-\theta}{n(n+1)}$$
Variance
Note: Covariance is derived here.
\begin{align*}
\text{Var}\left(\hat\theta_J\right) &= \frac{(2n-1)^2}{n^2}\text{Var}(X_{(n)}) + \frac{(n-1)^2}{n^2}\text{Var}(X_{(n-1)}) - 2 \frac{2n-1}{n}\frac{n-1}{n}\text{Cov}(X_{(n)}, X_{(n-1)}) \\[1.3ex]
&= \frac{(2n-1)^2}{n^2}\frac{n\theta^2}{(n+1)^2(n+2)} + \frac{(n-1)^2}{n^2}\frac{2(n-1)\theta^2}{(n+1)^2(n+2)} -  \frac{2(2n-1)(n-1)}{n^2}\frac{(n-1)\theta^2}{(n+1)^2(n+2)} \\[1.5ex]
\end{align*}
$$= \frac{(2n^2-1)\theta^2}{n(n+1)^2(n+2)}$$
MSE
Using the decomposition $\text{MSE}_\theta(\hat\theta) = \text{Bias}^2_\theta(\hat\theta) + \text{Var}(\hat\theta)$, we have
\begin{align*}
\text{MSE}_\theta(\hat\theta_J) &= \left(\frac{-\theta}{n(n+1)}\right)^2  + \frac{(2n^2-1)\theta^2}{n(n+1)^2(n+2)} \\[1.3ex]
&= \frac{2(n-1+1/n)\theta^2}{n(n+1)(n+2)}\\[1.3ex]
 &= \mathcal O(n^{-2})
\end{align*}
</s>