&= A^\t \Lambda^\t w + \ep^\t \Psi^\t w +U_Y =\ep^\t \Psi^\t w +U_Y.
\end{align*}

Furthermore, $U_Y=  \alpha_{0,-*}^\t Z_{-*}+\eta^\t_0  H + \ep_Y$ can be written as a linear function of $A$ plus a linear function of $\ep$. To realize this, simply express $Z_{-*}$ and $H$ by their marginal reduced form structural equations. Hence, the assumptions that $A \independent U_Y$ must entail that $A$ vanishes from the expression of $U_Y$. As a consequence we have that $R(\alpha)$ is a linear function only of $\ep$, from which the assumption that $A\independent \ep$ yields that $A \independent R(\alpha)$. That is, the null hypothesis of zero correlation implies independence in the linear structural equation model,
under the given assumptions. Thus, $E\|AR(\alpha)\|_2^2 = E\|A\|_2^2 E\|R(\alpha)\|_2^2<\i$, so the covariance matrix of $AR(\alpha)$ is well-defined.

By the established independence and <Ref>, the covariance matrix of $AR(\alpha)$ has the following representation

\Cov( AR(\alpha) )=
E(AA^\t)E(R(\alpha)^2) \succ 0.$$

The positive definiteness follows from the facts that $E(AA^\t)\succ 0$ and $E(R(\alpha)^2)>0$ for any $\alpha\in \R^{d_1+q_1}$. $E(AA^\t)\succ 0$ follows by Assumption <Ref> and $E(R(\alpha)^2)>0$ for any $\alpha\in \R^{d_1+q_1}$ follows by  Assumption <Ref>, Assumption <Ref> and <Ref>;  non-degeneracy and mutual independence of the marginal noise variables in $\ep$. To see this, expand $R(\alpha)$ in terms of the marginal reduced form structural equations of $Y$ and $Z$ and use that $(I-B^\t)$ is invertible to see that $\ep$ does not vanish in the expression $R(\alpha)$. The multi-dimensional Central Limit Theorem yields that

\frac{1}{\sqrt{n}}\fA^\t \fR(\alpha) = \sqrt{n} \lp \frac{1}{n}\sum_{i=1}^n \begin{pmatrix}	A_{i,1}R(\alpha)_{i} \\ 	\vdots \\	A_{i,q} R(\alpha)_{i}	\end{pmatrix}\rp
\convd \cN(0,\Cov(AR(\alpha))).$$

Furthermore, note that regardless of whether or not the null-hypothesis is true, we have that
\sqrt{n}(\fA^\t \fA)^{-1/2} %
\convp E(AA^\t)^{-1/2},

\frac{1}{\sqrt{n}}\|\fR(\alpha)\|_2 = \sqrt{\frac{1}{n}\sum_{i=1}^n R(\alpha)_i^2}
\convp \sqrt{E(R(\alpha)^2)}>0,

by the law of large numbers and the continuity of the matrix square root operation on the cone of symmetric positive-definite matrices.
We can represent the test-statistic as $
T_n(\alpha):= \| \sqrt{n}W_n(\alpha)\|_2^2$ with
$$W_n(\alpha):= (\fA^\t \fA)^{-1/2}\fA^\t \fR(\alpha)/\|\fR(\alpha)\|_2,$$

and have that $
\sqrt{n}W_n(\alpha) %
\convd W \sim  \cN\lp 0, I \rp$,
by Slutsky's theorem and linear transformation rules of multivariate normal distributions.	Hence,
the continuous mapping theorem yields that
T_n(\alpha) = \left\|\sqrt{n}W_n(\alpha) \right\|_2^2 \convd \|W\|_2^2 = \sum_{i=1}^qW_i^2 \sim  \chi^2_{q},$$

where $\chi^2_{q}$ is the Chi-squared distribution with $q$ degrees of freedom, since $W_1\independent \cdots \independent W_q$. This proves that the test-statistic $T_n$ has the correct asymptotic distribution under the null-hypothesis.

Now fix a distribution $P$, for which the null hypothesis of simultaneous zero correlation between the components of $A$ and the residuals $R(\alpha)$ does not hold. That is, there exists an $j\in\{1,...,q\}$ such that $E(A_{j}R(\alpha)) \not = E(A_j)E(R(\alpha))=0$.  Note that

\begin{align*}
\left\| n^{-1/2}(\fA^\t \fA)^{1/2} \right \|_{\text{op}}^2 T_n(\alpha) &= \left\| n^{-1/2}(\fA^\t \fA)^{1/2} \right \|_{\text{op}}^2 \left\| \frac{\sqrt{n}(\fA^\t \fA)^{-1/2}\frac{1}{\sqrt{n}}\fA^\t \fR(\alpha)}{\frac{1}{\sqrt{n}}\|\fR(\alpha)\|_2} \right\|_2^2 \\
&\geq \left\| \frac{\frac{1}{\sqrt{n}}\fA^\t \fR(\alpha)}{\frac{1}{\sqrt{n}}\|\fR(\alpha)\|_2} \right\|_2^2 \geq \left| \frac{\frac{1}{\sqrt{n}}\fA_j^\t \fR(\alpha)}{\frac{1}{\sqrt{n}}\|\fR(\alpha)\|_2} \right|^2= n \left| \frac{ \frac{1}{n} \fA_j^\t \fR(\alpha)}{\frac{1}{\sqrt{n}}\|\fR(\alpha)\|_2} \right|^2, %
\end{align*}

where $\fA_j^\t := (\fA_j)^\t$ and $\fA_j$ is the j'th column
of $\fA$ corresponding to the i.i.d. vector consisting of $n$ copies of the j'th exogenous variable $A_j$ and $\|\cdot\|_{\text{op}}$ is the operator norm.
The lower bound diverges to infinity in probability
as the latter factor tends to $|E(A_jR(\alpha)) / \sqrt{E(R(\alpha)^2)}|^2 > 0$ in probability by the law of large numbers and Slutsky's theorem. Hence, it holds that $T_n(\alpha) \convp \i,$
$$\left\| n^{-1/2}(\fA^\t \fA)^{1/2} \right \|_{\text{op}} \to \left\| E(A A^\t )^{1/2} \right\|_{\text{op}} \in(0,\i).$$

This concludes the proof.

Let <Ref> hold, i.e., that $\fZ^\t \fZ$ and $\fA^\t \fZ$ are of full rank.
That  $\alpha \mapsto l_{\mathrm{IV}}^n(\alpha;\fY,\fZ,\fA)$ is a convex function and $\alpha \mapsto l_{\text{OLS}}^n(\alpha)$ is a strictly convex function can be seen from the quadratic forms of their second derivatives, i.e.,
y^\t D^2l_{\text{IV}}^n(\alpha)y = 2n^{-1} y^\t \fZ^\t \fA (\fA^\t \fA)^{-1} \fA^\t \fZ y =
2n^{-1}\|(\fA^\t \fA)^{-1/2} \fA^\t \fZ y\|_2^2\geq 0,$$
$$	y^\t D^2l_{\text{OLS}}^n(\alpha) y = 2n^{-1} y^\t \fZ^\t \fZ y =2
n^{-1}\|\fZ y\|_2^2 >0,$$

for any $y\in \R^{d_1+q_1}\setminus \{0\}$.  Here, we also used that $\fA^\t \fA$ is of full rank by Assumption <Ref> and that $\fZ\in \R^{n\times(d_1+q_1)}$ is an injective linear transformation as $d_1+q_1=\text{rank}(\fZ^\t \fZ)=\text{rank}(\fZ)$.

Suppose that there exists two optimal solutions $\alpha_1,\alpha_2$ to the (Primal$.t.n$) problem. By the convexity of the feasibility set any convex combination is also feasible. However,
$$l_{\mathrm{OLS}}^n \lp \alpha_1/2 + \alpha_2/2\rp < l_{\mathrm{OLS}}^n(\alpha_1)/2 + l_{\mathrm{OLS}}^n(\alpha_2)/2 = l_{\mathrm{OLS}}^n(\alpha_1),$$
since $l_{\mathrm{OLS}}^n(\alpha_1)=l_{\mathrm{OLS}}^n(\alpha_2)$.	This means that $\alpha_1/2+\alpha_2/2$ has a strictly better objective value than the optimal point $\alpha_1$, which is a contradiction.
Hence, there cannot exist multiple solutions to the optimization problem (Primal$.t.n$).

Regarding the claim of solvability, note that $\fZ^\t \fZ$ is positive definite and as a consequence the smallest eigenvalue $\lambda_{\min}(\fZ^\t \fZ)$ is strictly positive. Thus, using the lower bound of the symmetric quadratic form $\alpha^\t \fZ^\t \fZ \alpha \geq\lambda_{\min}(\fZ^\t \fZ) \|\alpha\|_2^2$, we get that

\begin{align}
l_{\text{OLS}}^n(\alpha) &= \fY^\t \fY + \alpha^\t \fZ^\t  \fZ \alpha - 2\fY^\t \fZ \alpha \notag \geq \fY^\t \fY + \lambda_{\min}(\fZ^\t \fZ)\| \alpha\|_2^2 - 2|\fY^\t \fZ \alpha|  \notag\\
& \geq  \fY^\t \fY + \lambda_{\min}(\fZ^\t \fZ)\| \alpha\|_2^2 - 2\|\fY^\t \fZ\|_{\text{op}} \|\alpha\|_2  \to \i, \label{eq:lOLSTendsToInfinityAsAlphaGrowsLarge}
\end{align}

as $\|\alpha\|_2\to\i$, where we used that for the linear operator $\fY^\t \fZ:\R^{d_1+q_1}\to \R$ the operator
norm is given by $\|\fY^\t \fZ\|_{\text{op}} := \inf\{c\geq 0 : |\fY \fZ v|\leq c\|v\|_2, \forall v\in \R^{d_1+q_1}\}$, obviously satisfying $|\fY^\t \fZ v|\leq  \|\fY^\t \fZ\|_{\text{op}}\|v\|_2$ for any $v\in \R^{d_1+q_1}$.

Now assume that $t> \inf_\alpha l_{\text{IV}}^n(\alpha)$.
This implies that there exists at least one point $\tilde{\alpha}\in\R^{d_1+q_1}$ such that $l_{\text{IV}}^n (\tilde{\alpha}) \leq t$, hence
we only need to consider points $\alpha$ such that $l_{\text{OLS}}^n(\alpha) \leq l_{\text{OLS}}^n(\tilde{\alpha})$
as possible solutions of the optimization problem.
By the considerations
in <Ref>
above, there exists
$c\geq 0$ such that is suffices to search over the closed ball $\overline{B(0,c)}$.
for a sufficiently large $c\geq 0$ we know that $\alpha \not \in \overline{B(0,c)}$
implies that $l_{\text{OLS}}^n(\alpha)> l_{\text{OLS}}^n(\tilde{\alpha})$ by <Ref>. Furthermore, as the inequality constraint function $\alpha \mapsto l_{\text{IV}}^n(\alpha)$ is continuous, the set of feasible points $(l_{\text{IV}}^n)^{-1}((-\infty,t])$ is closed. Hence, our minimization problem is equivalent with the minimization of the continuous function $\alpha\mapsto l_{\text{OLS}}^n(\alpha)$ over the convex and compact set $\overline{B(0,c)} \cap (l_{\text{IV}}^n)^{-1}((-\infty,t])$. By the extreme value theorem, the minimum exist and is attainable. We conclude that the primal problem is solvable if $t > \inf_\alpha l_{\text{IV}}^n(\alpha)$.

By definition, Slater's condition is satisfied if there exists a point in the relative interior of the problem domain where the constraint inequality is strict <cit.>. Since the problem domain is
$\R^{d_1+q_1}$, we need the existence of $\alpha\in \R^{d_1+q_1}$ such that $l^n_{\text{IV}}(\alpha)<t$. This is clearly satisfied if $t > \inf_{\alpha} l_{\text{IV}}^n(\alpha)$. Let us now specify the exact lower bound for the constraint bound as a function of the over-identifying restrictions. 	Under- and just-identified case: $q_2 \leq d_1$ ($q\leq d_1+q_1$).
Assumption <Ref> yields that $\fA^\t \fZ\in \R^{q\times (d_1+q_1)}$  satisfies $\text{rank}(\fA^\t \fZ) = q$. That is, $\fA^\t \fZ $ is of full row rank, hence surjective. Thus, we are guaranteed the existence of a $\tilde{\alpha}\in \R^{d_1+q_1}$ such that $	\fA^\t \fZ \tilde{\alpha} = \fA ^\t \fY$, implying that  $l_{\mathrm{IV}}^n(\tilde{\alpha}) =0$. Over-identified case: $d_1 < q_2$ ($d_1+q_1< q$). Note that the constraint function $l_{\mathrm{IV}}^n(\alpha):\R^{d_1+q_1} \to \R$  is strictly  convex as the  second derivative $D^2l_{\mathrm{IV}}^n(\alpha;\fY,\fZ,\fA) \propto \fZ^\t \fA(\fA^\t \fA)^{-1}\fA^\t \fZ$ is
positive definite by the assumption that $\fA^\t \fZ\in \R^{q\times (d_1+q_1)}$ has full (column)
rank. 	The global minimum of $l_{\text{IV}}$ is
attained in the unique stationary point. Furthermore,  the stationary point is found by solving the normal equation $
Dl_{\mathrm{IV}}^n(\alpha;\fY,\fZ,\fA) =0$. The solution to the normal equation is given by $\hat{\alpha}_{\text{TSLS}}^n = (\fZ^\t P_\fA \fZ)^\t \fZ^\t P_\fA\fY,
which is the standard TSLS estimator.

Let $p_{\min}\in(0,1)$ and let  <Ref> and <Ref> hold. That is, $\fA^\t\fZ$ and $\fZ^\t \fZ$ are of full rank and $[
\fZ \, \, \fY]$ is of full column rank. Furthermore, assume that $t_n^\star(p_{\min})>-\i$ and  $T_n(\hat{\alpha}_{\text{Pr}}^n(t_n^\star(p_{\min})))\leq Q_{\chi^2_q}(1-p_{\min})$.
First assume that $\hat{\alpha}_\text{Pr}^n(t_n^\star(p_{\min})) = \hat{\alpha}_{\text{OLS}}^n$. We note that

T_n(\hat{\alpha}_{\text{OLS}}^n) = T_n(\hat{\alpha}_\text{Pr}^n(t_n^\star(p_{\min})) ) %
\leq Q_{\chi^2_q}(1-p_{\min}),

hence the global minimizer $\hat{\alpha}_{\text{OLS}}^n$ of $\alpha \mapsto l_{\text{OLS}}^n(\alpha)$ is unique, feasible and necessarily optimal in the PULSE problem, so $\hat{\alpha}_\text{Pr}^n(t_n^\star(p_{\min}))  = \hat{\alpha}_{\text{OLS}}^n = \hat{\alpha}^n_{\text{PULSE}}$ and we are done.

Now assume that $\hat{\alpha}_\text{Pr}^n(t_n^\star(p)) \not = \hat{\alpha}_{\text{OLS}}^n$. Consider the PULSE problem of interest

\begin{align} \tag{PULSE}
\begin{array}{ll}
\mathrm{min}_\alpha & l_{\mathrm{OLS}}^n(\alpha)  \\
\mathrm{subject \, to} & T_n(\alpha) \leq Q_{\chi^2_{q}}(1-p_{\min}),
\end{array} %
\end{align}

which is, in general, a non-convex quadratically constrained quadratic program. First we argue that the problem is solvable, i.e., the optimum is attainable.
To see this, let $p=p_{\min}$, $Q=Q_{\chi^2_{q}}(1-p_{\min})$ and note that by the assumption  $t_n^\star(p_{\min})>-\i$ we have that the feasible set of the PULSE problem is non-empty. By the assumptions that $[ \fZ \, \, \fY]$ is of full column rank we have that $T_n(\alpha)$ is well-defined for any $\alpha \in \R^{d_1+q_1}$, as the denominator is never zero. By continuity of $\R^{d_1+q_1} \ni \alpha \mapsto T_n(\alpha)$ we have that the feasible set $
\cF := T_n^{-1} \lp (-\i,Q] \rp,$
is closed and non-empty, since it is the continuous preimage of a closed set. Applying the same arguments as seen earlier in the proof of <Ref>, we know that $l_{\text{OLS}}^n(\alpha)\to \i$ when $\|\alpha\| \to \i$. Hence, for a sufficiently large $c>0$ we know that if $\alpha \not \in \overline{B(0,c)}$, where $\overline{B(0,c)}\subset \R^{d_1+q_1}$ is the closed ball with centre 0 and radius $c$, then we only get suboptimal objective values $l_{\text{OLS}}^n(\alpha) > l_{\text{OLS}}^n(\hat{\alpha}_{\text{Pr}}^n (t_n^\star(p)))$. That is, we can without loss of optimality or loss of solutions restrict the feasible set to $\cF'= T_n^{-1} ( (-\i, Q] ) \cap \overline{B(0,c)}$ a closed and bounded set in $\R^{d_1+q_1}$. Hence, by the extreme value theorem the minimum over $\cF'$ is guaranteed to be attained. That is, the PULSE problem is solvable.

However, by the non-convexity of $T_n$, the preimage $T_n^{-1} ( (-\i, Q] )$ is in general not convex, so the minimum is not yet guaranteed to be attained in a unique point.
We will show that the minimum of the PULSE problem is attained in a unique point, that exactly coincides with the primal PULSE solution. Fix any solution $\hat{\alpha}$ to
PULSE problem and realize that the PULSE constraint is active in  $\hat{\alpha}$,
\begin{align}
T_n(\hat{\alpha})= Q. \label{Eq:PULSEsolActiveInConstraint}
\end{align}

This is seen by noting that $\hat{\alpha}_\text{Pr}^n(t_n^\star(p)) \not = \hat{\alpha}_{\text{OLS}}^n$ by assumption, so $\hat{\alpha}_{\text{OLS}}^n$ is not feasible in the PULSE problem, that is,  $
\hat{\alpha}_{\text{OLS}}^n\not \in \cF$.
If $\hat{\alpha}_{\text{OLS}}^n$ was feasible, then $t_n^\star(p) =  \sup \{ t \in D_{\text{Pr}} : T_n(\hat{\alpha}_{\mathrm{Pr}}^n(t))\leq Q_{\chi^2_{q}}(1-p_{\min})\} = l_{\text{IV}}^n(\hat{\alpha}^n_{\text{OLS}}),$	since $T_n(\hat{\alpha}_{\text{Pr}}^n(l_{\text{IV}}^n(\hat{\alpha}^n_{\text{OLS}})) = T_n(\hat{\alpha}_{\text{OLS}}^n)\leq Q$,  hence
\hat{\alpha}_\text{Pr}^n(t_n^\star(p))  =   \argmin_{\alpha: l_{\text{IV}}^n(\alpha) \leq l_{\text{IV}}^n(\hat{\alpha}_{\text{OLS}}^n)}
l_{\text{OLS}}^n(\alpha)   =  \hat{\alpha}_{\text{OLS}}^n,

which is a contradiction. That the optimum must be attained in a point, where the PULSE inequality constraint is active then
follows from <Ref> of <Ref>
and the conclusion above that the only stationary point of $l_{\text{OLS}}^n$, $\hat{\alpha}_{\text{OLS}}^n$, is not feasible.

\begin{align} \label{eq:PULSEsolutionTIGHTconstraint}
T_n(\hat{\alpha}) = n \frac{l_{\text{IV}}^n(\hat{\alpha})}{l_{\text{OLS}}^n(\hat{\alpha})} = Q \iff   l_{\text{IV}}^n(\hat{\alpha}) = \frac{Q}{n} l_{\text{OLS}}^n(\hat{\alpha}).
\end{align}

Furthermore, the assumption that $T_n(\hat{\alpha}_{\text{Pr}}^n(t_n^\star(p)))\leq Q$ means that the solution to the primal PULSE, $\hat{\alpha}_{\text{Pr}}^n(t_n^\star(p))$, is feasible in the PULSE problem. That is, $\hat{\alpha}_\text{Pr}^n(t_n^\star(p)) \in \cF$. As a consequence of this we have that

\begin{align} \label{eq:TheoremPULSEequalpPULSE_pPULSEfeasibleInPULSE}
l_{\text{OLS}}^n(\hat{\alpha}_\text{Pr}^n(t_n^\star(p)))  \geq \min_{\alpha\in \cF} l_{\text{OLS}}^n(\alpha) = l_{\text{OLS}}^n(\hat{\alpha}).
\end{align}

Now we show that the PULSE solution $\hat{\alpha}$ is feasible in the primal PULSE problem (Primal$.t_n^*(p).n$).
To see this, Note that the feasibility set of the PULSE problem can be shrunk in the following manner

\begin{align*}
\cF &= \lb \alpha \in \R^{d_1+q_1} :  l_{\text{IV}}^n(\alpha )\leq \frac{Q}{n}l_{\text{OLS}}^n(\alpha) \rb \\
&=\lb \alpha \in \R^{d_1+q_1} :  l_{\text{IV}}^n(\alpha )\leq \frac{Q}{n}l_{\text{OLS}}^n(\alpha) , l_{\text{OLS}}^n(\alpha) \geq  l_{\text{OLS}}^n(\hat{\alpha})\rb \\
&\supseteq \lb \alpha \in \R^{d_1+q_1} :  l_{\text{IV}}^n(\alpha )\leq \frac{Q}{n}l_{\text{OLS}}^n(\hat{\alpha}) , l_{\text{OLS}}^n(\alpha) \geq  l_{\text{OLS}}^n(\hat{\alpha})\rb \\
&= \lb \alpha \in \R^{d_1+q_1} :  l_{\text{IV}}^n(\alpha )\leq  l_{\text{IV}}^n(\hat{\alpha} ) , l_{\text{OLS}}^n(\alpha) \geq  l_{\text{OLS}}^n(\hat{\alpha})\rb  \\
&= \lb \alpha \in \R^{d_1+q_1} :  l_{\text{IV}}^n(\alpha )\leq  l_{\text{IV}}^n(\hat{\alpha} ) \rb  =: \hat{\cF}(\hat{\alpha}),
\end{align*}

where the third equality follows from <Ref>. 	The only claim above that needs justification is that:

\begin{equation} \label{eq:toprove}
l_{\text{IV}}^n(\alpha )\leq  l_{\text{IV}}^n(\hat{\alpha} ) \implies l_{\text{OLS}}^n(\alpha) \geq  l_{\text{OLS}}^n(\hat{\alpha}).
\end{equation}

For now we assume that this claim holds and provide a proof later. Thus, we have that $\hat{\cF}(\hat{\alpha}) \subseteq \cF$ and we note that $\hat{\alpha}\in \hat{\cF}(\hat{\alpha})$.
An important consequence of this is that the PULSE solution $\hat{\alpha}$ is also the unique solution to the primal problem (Primal$.l_{\text{IV}}^{n}(\hat{\alpha}).n$). That is,

\begin{align*}
\hat{\alpha}  = \hat{\alpha}_{\text{Pr}}^n(l_{\text{IV}}^n(\hat{\alpha})) = \begin{array}{ll}
\mathrm{argmin}_\alpha & l_{\mathrm{OLS}}^n(\alpha)  \\
\mathrm{subject \, to} & l_{\text{IV}}^n(\alpha) \leq l_{\text{IV}}^n(\hat{\alpha}).
\end{array}
\end{align*}

We will now prove that
l_\text{IV}^n(\hat{\alpha}) \in \cE:=
\{ t\in [ \min_{\alpha}l_{\mathrm{IV}}^n(\alpha),l_{\text{IV}}^n(\hat{\alpha}_{\text{OLS}}^n) ] : T_n(\hat{\alpha}_{\text{Pr}}^n(t)) \leq  Q_{\chi^2_{q}}(1-p)\}.
This follows from the following two observations:
$\min_{\alpha}l_{\mathrm{IV}}^n(\alpha) \leq l_{\text{IV}}^n(\hat{\alpha}) <
and (2) $T_n(\hat{\alpha}_{\text{Pr}}^n(l_\text{IV}^n(\hat{\alpha}))) \leq  Q_{\chi^2_{q}}(1-p)$.
(1) follows
$\hat{\alpha}_{\text{OLS}}^n\not \in \cF$,
which implies, by the above inclusion, that $\hat{\alpha}_{\text{OLS}}^n\not \in \hat{\cF}(\hat{\alpha})$.
(2)	follows because $\hat \alpha$ solves
(Primal$.l_{\text{IV}}^{n}(\hat{\alpha}).n$) and thus
$\hat{\alpha}_{\text{Pr}}^n(l_\text{IV}^n(\hat{\alpha})) = \hat{\alpha}$;
$T_n(\hat{\alpha}) \leq Q_{\chi^2_q}(1-p)$
holds because $\hat{\alpha}$ is feasible for the PULSE problem.

Now, since $t_n^\star(p)=\sup(\cE\setminus \{\min_{\alpha}l_{\mathrm{IV}}^n(\alpha)\})\in \R$ implies $t_n^\star(p)= \sup(\cE)$, it follows that	 $l^n_{\text{IV}}(\hat{\alpha}) \leq t_n^\star(p)$. In other words, any solution $\hat{\alpha}$ to the PULSE problem is feasible in the primal PULSE problem (Primal$.t_n^*(p).n$).

\begin{align} \label{eq:TheoremPULSEequalpPULSE_PULSEfeasibleInpPULSE}
l_{\text{OLS}}^n (\hat{\alpha}) \geq l_{\text{OLS}}(\hat{\alpha}_{\text{Pr}}^n(t_n^\star(p))).
\end{align}

<Ref> and <Ref>  now yield that $l_{\text{OLS}}^n(\hat{\alpha}_{\text{Pr}}^n(t_n^\star(p))) =  l_{\text{OLS}}^n (\hat{\alpha})$ for any PULSE solution $\hat{\alpha}$.
Thus, any solution $\hat{\alpha}$ to the PULSE problem  is feasible in the primal PULSE problem (Primal$.t_n^*(p).n$) and it attains the optimal primal PULSE objective value. We conclude that $\hat{\alpha}$ solves the primal PULSE problem. Furthermore, it must hold that $
\hat{\alpha} = \hat{\alpha}_{\text{Pr}}^n(t_n^\star(p))$,
by uniqueness of solutions to the primal PULSE problem(see <Ref>). This implies two things: solutions to the PULSE problem are unique and the PULSE solution coincides with the primal PULSE solution.

It only remains to prove the claim of <Ref>, which ensures
$\hat{\cF}(\hat{\alpha}) \subseteq \cF$. Assume for contradiction that there exists an
$\alpha$ such that $l_{\text{IV}}^n(\alpha )\leq  l_{\text{IV}}^n(\hat{\alpha} )$ and $l_{\text{OLS}}^n(\alpha) <  l_{\text{OLS}}^n(\hat{\alpha})$, that is, we assume that

\begin{align*}
\cA := \underset{=:\cB}{\underbrace{\{\alpha \in \R^{d_1+q_1} : l_{\text{OLS}}^n(\alpha) < l_{\text{OLS}}^n(\hat{\alpha})\}}} \cap \underset{=:\cC}{\underbrace{\{\alpha \in \R^{d_1+q_1}: l_{\text{IV}}^n(\alpha) \leq l_{\text{IV}}^n(\hat{\alpha})\}}}   \not = \emptyset.
\end{align*}

Define $\cM_{\text{IV}} := \{\alpha : l_{\text{IV}}^n(\alpha) = \min_{\alpha'} l_{\text{IV}}^n(\alpha')\}$ as the solution space to the generalized method of moments formulation of the instrumental variable minimization problem.
We now prove that  $
\cM_{\text{IV}} \cap \cA = \emptyset$.
That is, we claim that
in the just- and over-identified setup  $\hat{\alpha}_{\text{TSLS}}^n\not \in \cA$ and in the under-identified setup none of the infinitely many solutions in the solution space of the instrumental variable minimization problem  lies in $\cA$. These statements follow by first noting that $\cM_{\text{IV}} \subset \cF$ in any identification setting. In the under- and -just identified setup this is seen by noting that $l_{\text{IV}}^n(\alpha)=0$ for any $\alpha \in \cM_{\text{IV}}$, which implies $T_n(\alpha)=0\leq  Q$, hence $\cM_{\text{IV}}\subset \cF$.
In the over-identified setup, where $\cM_{\mathrm{IV}}=\{\hat{\alpha}_{\text{TSLS}}^n\}$, we will now  argue that $\cM_{\text{IV}} \subset \cF$
follows from the assumption that $t_n^\star(p) <\infty$.
We first prove
that $D_{\text{Pr}}\ni t \mapsto T_n(\hat{\alpha}^n_{\text{Pr}}(t))$ is weakly increasing. If $t_1<t_2$ are two constraint bounds for which the primal problem is solvable, then $l_{\text{OLS}}^n(\hat{\alpha}^n_{\text{Pr}}(t_1)) \geq l_{\text{OLS}}^n(\hat{\alpha}^n_{\text{Pr}}(t_2))$ as the feasibility set
for $t_2$ is larger than the
one for $t_1$.
Furthermore, the solution $\hat{\alpha}^n_{\text{Pr}}(t_2)$ either equals
$\hat{\alpha}^n_{\text{Pr}}(t_1)$ or is contained in the set $\{\alpha\in \R^{d_1+q_1}: t_1< l_{\text{IV}}^n(\alpha) \leq t_2 \}$; in the latter case we have
$l_{\text{IV}}^n (\hat{\alpha}^n_{\text
{Pr}}(t_1)) \leq t_1 < l_{\text{IV}}^n (\hat{\alpha}^n_{\text
{Pr}}(t_2))  \leq t_2$. Thus, we have in both cases that
$l_{\text{IV}}^n (\hat{\alpha}^n_{\text
{Pr}}(t_1)) \leq  l_{\text{IV}}^n (\hat{\alpha}^n_{\text
{Pr}}(t_2))$. Combining
two observations above we have that
T_n(\hat{\alpha}^n_{\text{Pr}}(t_1)) = n \frac{l_{\text{IV}}^n(\hat{\alpha}^n_{\text{Pr}}(t_1))}{l_{\text{OLS}}^n(\hat{\alpha}^n_{\text{Pr}}(t_1))} \leq n \frac{l_{\text{IV}}^n(\hat{\alpha}^n_{\text{Pr}}(t_2))}{l_{\text{OLS}}^n(\hat{\alpha}^n_{\text{Pr}}(t_2))} = T_n(\hat{\alpha}^n_{\text{Pr}}(t_2)).$$

Hence, as $-\infty<\min_{\alpha}l_{\text{IV}}^n(\alpha) = l_{\text{IV}}^n(\hat{\alpha}_{\text{TSLS}}^n) < t_n^\star(p) < \i$ are two points for which the primal problem is solvable we get that
T_n(\hat{\alpha}_{\text{TSLS}}^n) = T_n(\hat{\alpha}^n_{\text{Pr}}(l_{\text{IV}}^n(\hat{\alpha}_{\text{TSLS}}^n))) \leq T_n(\hat{\alpha}^n_{\text{Pr}}(t_n^\star(p))) \leq Q.$$

This proves that $\cM_{\text{IV}} \subset \cF$ in the over-identified setup.
Now, if $\cM_{\text{IV}}\cap \cA \not = \emptyset$, there exists an $\alpha\in \cM_{\text{IV}}\cap \cA\subseteq \cF\cap \cA$ such that $\alpha$ is feasible in the PULSE problem $(\alpha\in \cF)$ and $\alpha$ is super-optimal compared to $\hat{\alpha}$, $l_{\text{OLS}}^n(\alpha) < l_{\text{IV}}^n(\hat{\alpha})$ ($\alpha\in \cA$), contradicting that $\hat{\alpha}$ is an solution to the PULSE problem. We can thus conclude that
$\cM_{\mathrm{IV}} \cap \cA = \emptyset$.

This allows us to fix two distinct points $\bar{\alpha}\not = \alpha'$ such that  $\bar{\alpha} \in \cA $ and $\alpha'\in \cM_{\text{IV}}$. Consider the proper line segment function between $\bar{\alpha}$ and  $\alpha'$, $f(t):[0,1]\to \R^{d_1+q_1}$ given by
f(t) := t\alpha' + (1-t) \bar{\alpha}.
A multivariate convex function is convex in any direction from any given starting point in its domain, so both $l_{\text{IV}}^n\circ f:[0,1]\to \R_+$ and $l_{\text{OLS}}^n\circ f:[0,1]\to \R_+$ are convex. Since $\cM_{\mathrm{IV}} \cap \cA = \emptyset$ it is obvious that the function $f$ will for sufficiently large $t$ 'leave' the set $\cA$.
We will now prove that $f$ actually leaves the superset $ \cB\supset \cA$.
More precisely, we will prove that
there exists a $t_1 \in (0,1]$ such that
for all $t'\in[0,t_1)$ it holds that $f(t') \in \cB$
for all $t'\in[t_1,1]$ it holds that
$f(t') \notin \cB$
(which implies
$f(t') \notin \cA$).
Because $l_{\text{IV}}^n(\alpha')=\min_{\alpha}l_{\text{IV}}^n(\alpha)$ we have that $\alpha'\in \cC=\{\alpha : l_{\text{IV}}^n(\alpha) \leq l_{\text{IV}}^n(\hat{\alpha})\}$.
By convexity of $l_{\text{IV}}^n$ 	(Lemma <ref>) the sublevel set $\cC$ is convex and thus contains the entire line segment between $\bar{\alpha}$ and $\alpha'$. As a consequence $a'\not\in \cB$.
It therefore suffices to
construct a $t_1 \in (0,1]$ such that
for all $t'\in[0,t_1)$ it holds that $f(t') \in \cB$
for all $t'\in[t_1,1]$ it holds that
$f(t') \notin \cB$.
We now consider the set
\{t \in [0,1] \,:\,
l_{\text{OLS}}^n(f(t)) < l_{\text{OLS}}^n(\hat{\alpha})
\} = f^{-1}(\cB).
This set
contains $0$
because $\bar{\alpha} \in \cA\subset \cB$;
it does not contain $1$ because
$\alpha'\not \in \cB$;
is convex, as it is a sublevel set of a convex function ($l_{\text{OLS}}^n \circ f$);
it is relatively open in $[0,1]$ because
it is a pre-image of an open set under a continuous function ($l_{\text{OLS}}^n \circ f$).
Thus, the set must be of the form
$[0,t_1)$ for some $t_1 \in (0,1]$. This $t_1$ satisfies the desired criteria.

We constructed $t_1$ above such that for all $t'\in[0,t_1)$ it holds that
$l_{\text{OLS}}^n(f(t'))< l_{\text{OLS}}^n(\hat{\alpha})$
and for all $t'\in[t_1,1]$ it holds that
$l_{\text{OLS}}^n(f(t'))\geq l_{\text{OLS}}^n(\hat{\alpha})$. By continuity of  $l_{\text{OLS}}^n \circ f$ we must therefore have that $l_{\text{OLS}}^n(f(t_1))= l_{\text{OLS}}^n(\hat{\alpha})$.
Since $f(1)= \alpha'$ is a global minimum for $l_{\text{IV}}^n$, we have that 1 must also be a global minimum for $l_{\text{IV}}^n\circ f$, implying that the convex the function $l_{\text{IV}}^n\circ f:[0,1]\to \R_+$ is monotonically decreasing.
It must therefore hold that
l_{\text{IV}}^n(f(t_1)) <  l_{\text{IV}}^n(f(0)) =  l_{\text{IV}}^n(\bar{a}) \leq l_{\text{IV}}^n(\hat{\alpha})

The first inequality is strict because if $l_{\text{IV}}^n(f(t_1)) = l_{\text{IV}}^n(f(0)) =l_{\text{IV}}^n(\bar{\alpha})$, then convexity of $l_{\text{IV}}^n$ implies that

l_{\text{IV}}^n(f(t_1)) = l_{\text{IV}}^n(t_1 \alpha' + (1-t_1) \bar\alpha)  \leq t_1 l_{\text{IV}}^n(\alpha') + (1-t_1)l_{\text{IV}}^n(\bar{\alpha}),$$
which happens if and only if $
l_{\text{IV}}^n(\bar{\alpha})\leq l_{\text{IV}}^n(\alpha')
contradicting the already established fact that $l_{\text{IV}}^n(\bar{\alpha})> l_{\text{IV}}^n(\alpha')$, which holds since $\alpha'\in \cM_{\text{IV}}$ but $\bar{\alpha} \not \in \cM_{\mathrm{IV}}$. We conclude that $l_{\text{IV}}^n(f(t_1))< l_{\text{IV}}^n(\hat{\alpha})$.

Thus, we have argued that $\cM_{\mathrm{IV}} \cap \cA = \emptyset$ implies the existence of an $\tilde{\alpha} := f(t_1) = t_1 \alpha ' + (1-t_1) \bar{\alpha}$ such that $l_{\text{IV}}^n(\tilde{\alpha}) < l_{\text{IV}}^n(\hat{\alpha})$ and $l_{\text{OLS}}^n( \tilde{\alpha}) = l_{\text{OLS}}^n(\hat{\alpha})$. We have illustrated the above considerations  in <Ref>.

Illustration of the described procedure in the
just- or over-identified setup with $d_1+q_1=2$, where we show that  $\cA \not = \emptyset$ leads to a contradiction. Here, $L_g(c) := \{\alpha: g(\alpha) \leq c\}$
is the $c$ sublevel set of the function $g$ and $A^\circ$ denotes the interior of a set $A$.
The illustration is simplified, e.g., because
the sublevel sets are convex but not necessarily Euclidean balls.
Note that the position of $\hat{\alpha}_{\text{OLS}}^n$ is not specified, as it can possibly be in either $\cA$ or $L_{l_{\text{OLS}}^n}(l_{\text{OLS}}^n(\hat{\alpha}))\setminus \cA$. In the under-indentified setup $\alpha'$ would lie in the $d-q_2=1$ dimensional subspace $\cM_{\text{IV}}$ and the level sets would be slabs around this line.

It follows that
T_n(\tilde{\alpha}) = n \frac{l_{\text{IV}}^n(\tilde{\alpha})}{l_{\text{OLS}}^n(\tilde{\alpha})} =  n \frac{l_{\text{IV}}^n(\tilde{\alpha})}{l_{\text{OLS}}^n(\hat{\alpha})} < n \frac{l_{\text{IV}}^n(\hat{\alpha})}{l_{\text{OLS}}^n(\hat{\alpha})}  = Q,$$

implying that $\tilde{\alpha}$ is
strictly feasible in the PULSE problem and, in fact, a solution
as the objective value is optimal.
We argued earlier in <Ref> that any solution to the PULSE problem must be tight in the inequality constraint, hence we have arrived at a contradiction.  We conclude that $\cA =\emptyset$, which implies
that <Ref> must hold.

Assume that we are in the just- or over-identified setup and that <Ref> are satisfied. That is, $\fZ^\t\fZ$, $\fA^\t \fZ$ and $\fA^\t \fA$ are almost surely of full rank. In particular we have that
$\fZ$, $\fA^\t \fZ$ and $P_\fA \fZ$ are almost surely of full column rank (injective linear maps).  Furthermore, let $\ep$ have density with respect to the Lebesgue measure and let $B$ be lower triangular.
Fix $\lambda \geq 0$ and $\omega\in W_\lambda$, where

\begin{align*}
W_\lambda:=& (\hat{\alpha}_{\mathrm{K}}^n(\lambda)= \hat{\alpha}_{\text{TSLS}}^n) \cap (\text{rank}(\fZ^\t \fZ)=d_1+q_1) \\&\quad \cap  (\text{rank}(\fA^\t \fZ)=d_1+q_1) \cap (\text{rank}(\fA^\t \fA)=q),
\end{align*}

$P(\hat{\alpha}_{\mathrm{K}}^n(\lambda)= \hat{\alpha}_{\text{TSLS}}^n) = P(W_\lambda)$. 	By <Ref> we have that

\begin{align*}
\hat{\alpha}_{\mathrm{K}}^n(\lambda) &= \argmin_{\alpha} \{l_{\mathrm{OLS}}^n(\alpha) + \lambda l_{\mathrm{IV}}^n(\alpha)\}
\\
&=\argmin_{\alpha} \{(\fY-\fZ\alpha)^\t (\fY-\fZ\alpha) + \lambda (\fY-\fZ\alpha)^\t P_\fA (\fY-\fZ\alpha)\} \\
&=\argmin_{\alpha}  (\fY-\fZ\alpha)^\t(\fI+  \lambda P_\fA) (\fY-\fZ\alpha) \\
&= \argmin_{\alpha} \| (\fI + \lambda P_\fA)^{1/2} (\fY - \fZ \alpha ) \|_2^2 \\
&= \argmin_{\alpha} \| \fY - \fZ \alpha \|_{(\fI+\lambda P_\fA)}^2 ,
\end{align*}

where $\| \cdot \|_{(\fI+\lambda P_\fA)}$ is the norm induced by the inner product $
\la x , y \ra_{(\fI+\lambda P_\fA)} = x^\t (\fI+\lambda P_\fA) y$.
The solution $\fZ \hat{\alpha}_{\mathrm{K}}^n(\lambda)$ is well-known to coincide with the orthogonal projection of $\fY$ onto $\cR(\fZ)$, the range of $\fZ$, with respect to the inner product $\la \cdot , \cdot \ra_{(\fI+\lambda P_\fA)}$. Hence, $\fZ \hat{\alpha}_{\mathrm{K}}^n(\lambda)$ is the unique element in this closed linear subspace such that for all $z\in \cR(\fZ)$ it holds that

\la \fY - \fZ \hat{\alpha}_{\mathrm{K}}^n(\lambda),  z \ra_{(\fI+\lambda P_\fA)} = \la \fY - \fZ \hat{\alpha}_{\mathrm{K}}^n(\lambda), (\fI+\lambda P_\fA) z \ra = 0  ,$$
or equivalently,

\begin{align} \label{eq:LemmaKclassDifferentFromIVSolutionsKclassPredictionUniqueEquality}
\la \fY - \fZ \hat{\alpha}_{\mathrm{K}}^n(\lambda),  z \ra = - \lambda \la \fY - \fZ \hat{\alpha}_{\mathrm{K}}^n(\lambda),  P_\fA z \ra, \quad \forall z\in \cR(\fZ).
\end{align}

We note that if $\lambda = 0$ then $\hat{\alpha}_{\mathrm{K}}^n(\lambda) = \hat{\alpha}_{\text{OSL}}^n$, seen either by directly inspecting the closed form solution of $\hat{\alpha}_{\mathrm{K}}^n(\lambda)$ or concluding the same from <Ref>.

Furthermore, when $\lambda > 0$
we have that  $\hat{\alpha}_{\mathrm{K}}^n(\lambda)=\hat{\alpha}_{\text{TSLS}}^n$ implies that, again, 	$\hat{\alpha}_{\mathrm{K}}^n(\lambda) = \hat{\alpha}_{\text{OSL}}^n$.
To see this, we note that
\hat{\alpha}_{\text{TSLS}}^n =\argmin_\alpha l_{\text{IV}}^n(\alpha) =  \argmin_\alpha \| P_\fA \fY - P_\fA \fZ \alpha \|_2^2,$$

so $P_\fA \fZ\hat{\alpha}_{\text{TSLS}}^n$ is the orthogonal projection of $P_\fA \fY$ onto $\cR(P_\fA\fZ)$. That is, $P_\fA \fZ\hat{\alpha}_{\text{TSLS}}^n$ is the unique element in $\cR(P_\fA \fZ )$ such that $\la P_\fA \fY - P_\fA \fZ \hat{\alpha}_{\text{TSLS}}^n , s \ra = 0$ for all $s\in \cR(P_\fA \fZ )$, i.e.,

$$\la P_\fA \fY - P_\fA \fZ \hat{\alpha}_{\text{TSLS}}^n, P_\fA z \ra  = 0,$$
for all  $z\in \cR(\fZ)$. Thus, if $\hat{\alpha}_{\mathrm{K}}^n(\lambda)= \hat{\alpha}_{\text{TSLS}}^n$ for some $\lambda >0$ we have that

\begin{align*}
0 &= 	\la P_\fA \fY - P_\fA \fZ \hat{\alpha}_{\text{TSLS}}^n , P_\fA z \ra  \\
&= 	\la P_\fA \fY - P_\fA \fZ \hat{\alpha}_{\mathrm{K}}^n(\lambda) , P_\fA z \ra
\\
&= \la \fY -  \fZ \hat{\alpha}_{\mathrm{K}}^n(\lambda), P_\fA z \ra \\
&= -\lambda^{-1}\la \fY - \fZ \hat{\alpha}_{\mathrm{K}}^n(\lambda),  z \ra,  %
\end{align*}

hence $\la \fY - \fZ \hat{\alpha}_{\mathrm{K}}^n(\lambda),  z \ra = 0 $ for all $z\in \cR(\fZ)$, where we used <Ref> and in the third equality we used
that $P_\fA$ is idempotent and symmetric. This implies that $\hat{\alpha}_{\mathrm{K}}^n(\lambda) = \hat{\alpha}_{\text{OLS}}^n$, as it satisfies the uniquely determining condition for the ordinary least square estimator.

Hence, for any $\lambda \geq 0$, whenever $\hat{\alpha}_{\mathrm{K}}^n(\lambda)= \hat{\alpha}_{\text{TSLS}}^n$ we know that $
\hat{\alpha}_{\text{TSLS}}^n =\hat{\alpha}_{\mathrm{K}}^n(\lambda) = \hat{\alpha}_{\text{OLS}}^n$.
Thus, for any $\lambda \geq 0$ it holds that
P(\hat{\alpha}_{\mathrm{K}}^n(\lambda)= \hat{\alpha}_{\text{TSLS}}^n) \leq P(\hat{\alpha}_{\text{TSLS}}^n = \hat{\alpha}_{\text{OLS}}^n).$$

Recall that the reduced form equations of our system are given by
$	[\fY  \, \, \fX \, \,  \fH ]
= \fA \Pi + \bm{\ep}\Gamma^{-1}$ where $\Gamma := I-B$. When $B$ is lower triangular, so is $\Gamma$ and $\Gamma^{-1}$.
By selecting the relevant columns of $\Pi$ and $\Gamma^{-1}$ we may express the marginal reduced form structural equations of $\fS$  that consist of any collection of columns from $[
\fY \, \,  \fX \, \, \fH
]$ by $\fS = \fA \Pi_{S} + \bm{\ep} \Gamma^{-1}_S$ for conformable matrices $\Pi_{S}$ and $\Gamma^{-1}_S$. In particular, we have that the marginal reduced form structural equations for $\fY$ and $\fX_{*}$ are given by $
\fY = \fA \Pi_Y + \bm{\ep}\Gamma^{-1}_{Y}$ and $
\fX_{*} = \fA \Pi_{X_*} + \bm{\ep}\Gamma^{-1}_{X_*}$,
where $\Pi_Y, \Pi_{X_*},\Gamma^{-1}_{Y}$ and $\Gamma^{-1}_{X_{*}}$ are matrices conformable with the following block representation

\begin{align*}
\Pi &= [ \,
\underbrace{\Pi_{Y_{}}}_{q\times 1} \,\,  \underbrace{\Pi_{X_*}}_{q\times d_1} \, \, \underbrace{\Pi_{X_{-*}}}_{d\times q_2} \, \, \underbrace{\Pi_{H_{}}}_{q\times r} \,
] \in \R^{q \times l},
\quad \text{and} \quad
\Gamma^{-1} =
[ \,\underbrace{\Gamma^{-1}_{Y_{}}}_{l\times 1}  \,\,
\underbrace{\Gamma^{-1}_{X_*}}_{l\times d_1} \,\,
\underbrace{\Gamma^{-1}_{X_{-*}}}_{l \times d_2} \,\,
\underbrace{\Gamma^{-1}_{H_{}}}_{l \times r}  \,]  \in \R^{l\times l},
\end{align*}

where $l:=1+d+r$. 	Note that by the lower triangular structure of $\Gamma^{-1}$ we have that the only matrix among $ \Gamma^{-1}_{Y}$, $\Gamma^{-1}_{X_*}$, $\Gamma^{-1}_{X_{-*}}$ and $\Gamma^{-1}_{H}$ that has a non-zero first row is $\Gamma^{-1}_Y$.

Now assume without loss of generality that the first row of $\Gamma^{-1}$ is given by the first canonical Euclidean basis vector $(1,0,...,0)\in \R^{1\times l}$ such that we have the following partitionings

\begin{align*}
\bm{\ep} &= [ \,
\underbrace{\bm{\ep}_Y}_{n\times 1}  \, \,  \underbrace{\bm{\ep}_{-Y}}_{n\times(d+r)}
\, ]\in \R^{n\times l}, &
\Gamma_{Y}^{-\t} &= [ \, 1  \, \,
\underbrace{\Gamma_{-Y,Y}^{-\t}}_{1\times (d+r)}
\,] \in \R^{1 \times l},\\
\Gamma_{X_{*}}^{-\t} &= [ \,
\bm{0}_{d_1\times 1}  \, \,
\underbrace{\Gamma_{-Y,X_{*}}^{-\t}}_{d_1\times(d+r)}
\, ] \in \R^{d_1 \times l}, &
\Gamma_{X_{1}}^{-\t} &= [ \,
0  \, \,
\underbrace{\Gamma_{-Y,X_{1}}^{-\t}}_{1\times (d+r)}
\,] \in \R^{1 \times l},
\end{align*}

where $\fX_1$ is the first column of $\fX$.
Hence, we note that $\bm{\ep}\Gamma^{-1}_{Y} =\bm{\ep}_Y + \bm{\ep}_{-Y}\Gamma_{-Y,Y}^{-1}$, such that the marginal reduced form structural equation for $\fY$ has the following representation
\fY = \fA \Pi_Y + \bm{\ep}\Gamma^{-1}_{Y}
= \fA \Pi_Y + \bm{\ep}_{-Y}\Gamma_{-Y,Y}^{-1}+ \bm{\ep}_Y  =: f_y(\fA,\bm{\ep}_{-Y}) + \bm{\ep}_{Y}.$$

We can also represent $\fZ$ in terms of these structural coefficient block matrices by

\begin{align*}
\fZ &=  \begin{bmatrix}
\fX_{*} & \fA_{*}
\end{bmatrix} =  \begin{bmatrix}
\fA \Pi_{X_{*}} + \bm{\ep}\Gamma^{-1}_{X_{*}} & \fA_{*}
\end{bmatrix}  = \begin{bmatrix}
\fA \Pi_{X_{*}}  & \fA_{*}
\end{bmatrix} + \bm{\ep} \begin{bmatrix}
\Gamma^{-1}_{X_{*}} & \bm{0}_{l \times q_1}
\end{bmatrix}  \\
&= \fA \begin{bmatrix}
\Pi_{X_{*}}  & \begin{bmatrix}
\fI_{q_1\times q_1} \\ \bm{0}_{q_2\times q_1}
\end{bmatrix}
\end{bmatrix} +  \begin{bmatrix}
\bm{\ep}_{-Y} \Gamma_{-Y,X_{*}}^{-1} & \bm{0}_{l \times q_1}
\end{bmatrix} =:f_z(\fA,\bm{\ep}_{-Y}).
\end{align*}

Assumption <Ref> and Assumption <Ref> together  with the assumption that the data matrices consist of row-wise i.i.d. copies of the system variables, yield that $\fA \independent \bm{\ep}_{Y}$ and $\bm{\ep}_{-Y}\independent \bm{\ep}_{Y} $. This implies that the conditional distribution of $\bm{\ep}_{Y}$ given $\fA$ and $\bm{\ep}_{-Y}$ satisfies $P_{\bm{\ep}_{Y} |\fA = A,\bm{\ep}_{-Y}=e} = P_{\bm{\ep}_{Y}}$ for $P_{\fA,\bm{\ep}_{-Y}}$-almost all $(A,e)\in \R^{n\times q}\times \R^{n\times (d+r)}$. Hence, conditional on $\fA = A$ and $\bm{\ep}_{-Y} =e$ we have that
\fY|(\fA = A,\bm{\ep}_{-Y} =e) \stackrel{a.s.}{=} f_y(A,e) + \bm{\ep}_Y,$
$ \fZ |(\fA = A,\bm{\ep}_{-Y} =e) \stackrel{a.s.}{=} f_z(A,e).$
Now let $(P_\fA\fZ)^+ = (\fZ^\t P_\fA \fZ)^{-1}\fZ^\t P_\fA$ and $
\fZ^+ =(\fZ^\t \fZ)^{-1}\fZ^\t$ denote the pseudo-inverse matrices of the almost surely full column rank matrices $P_\fA \fZ$ and $\fZ$. Furthermore, note that the pseudo-inverses are unique for all matrices,
i.e., if $P_\fA \fZ \not = \fZ$, then $(P_\fA\fZ)^+ \not = \fZ^+$.   We realize that $
\hat{\alpha}^n_{\text{TSLS}} =(\fZ^\t P_\fA \fZ)^{-1}\fZ^\t P_\fA \fY = (P_\fA \fZ)^+\fY $ and $ \hat{\alpha}^n_{\text{OLS}} = (\fZ^\t\fZ)^{-1} \fZ^\t \fY =\fZ^+\fY$.
Thus, with slight abuse of notation we let $Z:=f_z(A,e)$ for any $A,e$, and note that

\begin{align} \notag
&P(\hat{\alpha}^n_{\text{TSLS}}= \hat{\alpha}^n_{\text{OLS}})\\ \notag
&= P((P_\fA\fZ)^+ \fY =\fZ^+\fY) \\\notag
&= \int P\lp [(P_\fA \fZ)^+- \fZ^+] \fY =0 | \fA = A,\bm{\ep}_{-Y} =e \rp \, \mathrm{d} P_{\fA,\bm{\ep}_{-Y}} (A,e) \\\notag
&= \int P\lp [(P_A Z)^+- Z^+](f_y(A,e) + \bm{\ep}_Y)  =0 \rp \, \mathrm{d} P_{\fA,\bm{\ep}_{-Y}} (A,e) \\ \label{Eq:KclassNotEqTSLSTempEq1}
&= \int \mathbbm{1}_{(P_A Z \not = Z)}P\lp [(P_A Z)^+- Z^+](f_y(A,e) + \bm{\ep}_Y)  =0 \rp \, \mathrm{d} P_{\fA,\bm{\ep}_{-Y}} (A,e),
\end{align}

where $P_A = A(A^\t A)^{-1}A^\t\in \R^{n\times n}$. The last equality is due to the claim that $\mathbbm{1}_{(P_A Z \not = Z)}=1$ for $P_{\fA,\bm{\ep}_{-Y}}$ almost all $(A,e)$, or equivalently
\int \mathbbm{1}_{(P_A Z  = Z)} \, \mathrm{d} P_{\fA,\bm{\ep}_{-Y}} (A,e)
= \int \mathbbm{1}_{(P_\fA \fZ  = \fZ)} \, \mathrm{d} P
=  P(P_\fA \fZ =\fZ)=0.$$

We prove this claim now.
We now prove that $P(P_\fA \fZ =\fZ)=0$. First we note that $P_\fA \fZ =\fZ$ implies that $\cR(\fZ) \subseteq \cR(\fA)$.
Since $ \fZ = [ \fX_* \, \, \fA_* ]$ with $\fA = [\fA_*\,\,  \fA_{-*} ]$ it must hold that $\cR(\fX_*) \subseteq \cR(\fA)$.
Assume without loss of generality that $\fX_1$, the first column of $\fX$, is also a column of $\fX_*$.
Note that $\cR(\fX_*)\subseteq \cR(\fA)$ implies that $\fX_1$ can be written as a linear combination of the columns in $\fA$, i.e., there exists a $b=(b_1,...,b_q)\in \R^{q}$ such that $
\fX_1 = b_1 \fA_1 + \cdots + b_{q} \fA_q = \fA b$,
namely $b=(\fA^\t \fA)^{-1}\fA \fX_1$. The marginal reduced form structural equation for $\fX_1$ is given by $		\fX_1 = \fA \Pi_{X_1} + \bm{\ep}\Gamma^{-1}_{X_1} = \fA \Pi_{X_1} + \bm{\ep}_{-Y} \Gamma_{-Y,X_{1}}^{-1}=\fA \Pi_{X_1} + \tilde{\bm{\ep}}$,
where $\tilde{\bm{\ep}}:= \bm{\ep}_{-Y}\Gamma_{-Y,X_{1}}^{-1}$. These two equalities are only possible if $\tilde{\bm{\ep}}\in \cR(\fA)$.
Note that $\tilde{\bm{\ep}}$ has jointly independent marginals (i.i.d. observations). Each coordinate is an independent copy of a linear combination of $1+d+r$ independent random variables $\ep_1,...,\ep_{1+d+r}$ all with density with respect to Lebesgue measure.
We conclude that $\tilde{\bm{\ep}}$ has density with respect to the $n$-dimensional Lebesgue measure as the linear
combination is non-vanishing. This holds because
$\Gamma_{-Y,X_{1}}^{-1}\not=0$ by virtue of being a column of the invertible matrix $
\Gamma^{-1}$, where we have removed the first entry (which was
a zero element). Furthermore, since $\fA \independent \bm{\ep}$, we also have that $\fA
\independent \tilde{\bm{\ep}}$. Hence, the conditional distribution of $\tilde{\bm{\ep}}$ given $\fA$ satisfies
$P_{\tilde{\bm{\ep}} |\fA = A} = P_{\tilde{\bm{\ep}}}$ for $P_{\fA}$-almost all $A\in \R^{n\times q}$. We conclude that

\begin{align*}
P(P_\fA \fZ =\fZ)  &\leq P(\tilde{\bm{\ep}} \in \cR(\fA)) \\
&= \int P(\tilde{\bm{\ep}} \in \cR(\fA)|\fA=A) \, \mathrm{d} P_{\fA}(A)
\\
&= \int P(\tilde{\bm{\ep}} \in \cR(A)) \, \mathrm{d} P_{\fA}(A) \\
\end{align*}

The last equality follows from the fact that $q= \text{rank}(\fA^\t\fA) = \text{rank}(\fA)< n$ implies that $\cR(\fA)$ is a $q$-dimensional subspace of $\R^n$. Hence, for $P_\fA$-almost all $A\in \R^{n\times q}$ it
holds that $\cR(A)$ is a $q$-dimensional subspace of $\R^n$. The probability that $\tilde{\bm{\ep}}$ lies in a $q$-dimensional subspace of $\R^n$ is zero, since it has density with respect to the $n$-dimensional Lebesgue measure.

Thus, it suffices to show that
$$ \label{Eq:EpYInaffineSubspaceWithProbZero}
P\lp [(P_A Z)^+- Z^+](f_y(A,e) + \bm{\ep}_Y)  =0 \rp =0,$$

for any $A\in \R^{n\times q}$
and $Z=f_z(A,e)\in \R^{n\times (d_1+q_1)}$ with $P_AZ \not = Z$.
Therefore, let $A\in \R^{n\times q}$
and $Z=f_z(A,e)\in \R^{n\times (d_1+q_1)}$ with $P_AZ \not = Z$. It holds that $(P_AZ)^+\not = Z^+$, which implies that $(P_AZ)^+-Z^+\not = 0$. Furthermore, we have that
[(P_A Z)^+- Z^+](f_y(A,e) + \bm{\ep}_Y)  =0 ,$$
if and only if
$$\bm{\ep}_\fY \in \text{ker}((P_A Z)^+- Z^+)-[(P_A Z)^+- Z^+]f_y(A,e),$$

so it suffices to show that $\bm{\ep}_Y$ has zero probability to be in the affine (translated) subspace
$$\text{ker}((P_A Z)^+- Z^+)-[(P_A Z)^+- Z^+]f_y(A,e)\subset \R^n.$$
This affine subspace has dimension $n$ if and only if $(P_AZ)^+-Z^+ = 0$, which we know is false. Hence, the dimension of the affine subspace is strictly less than $n$. As $\bm{\ep}_Y$ has density with respect to the $n$-dimensional Lebesgue measure, we know that the probability of being in a $N<n$ dimensional affine subspace is zero.

Hence, we have shown that $
P(\hat{\alpha}^n_{\text{TSLS}}= \hat{\alpha}^n_{\text{OLS}}) = 0$.
Combining all of our observations we get that $
P(\hat{\alpha}_{\mathrm{K}}^n(\lambda)= \hat{\alpha}_{\text{TSLS}}^n) \leq P(\hat{\alpha}_{\text{TSLS}}^n = \hat{\alpha}_{\text{OLS}}^n) = 0.$
We conclude that
P \lp   \hat{\alpha}_{\text{TSLS}}^n \not = \hat{\alpha}_{\mathrm{K}}^n(\lambda) \rp = 1,\quad \text{for all }\lambda \geq 0.$$
However, we can easily strengthen this to
$P ( \cap_{\lambda \geq 0}   (\hat{\alpha}_{\text{TSLS}}^n \not = \hat{\alpha}_{\mathrm{K}}^n(\lambda) ) ) = 1.
To this end, let $\omega$ be a realization in the almost sure set $\cap_{\lambda \in \bQ_+}	W_\lambda.
$\omega \in \bigcap_{\lambda \geq 0 } \lp \hat{\alpha}_{\text{TSLS}}^n \not = \hat{\alpha}_{\mathrm{K}}^n(\lambda) \rp$.
there exists an $\tilde{\lambda} \in \R_+ \setminus \bQ_+$ such that $\hat{\alpha}_{\text{TSLS}}^n = \hat{\alpha}_{\mathrm{K}}^n(\tilde{\lambda})$. By <Ref> we have that $\lambda \mapsto l_{\text{IV}}^n( \hat{\alpha}_{\text{IV}}^n (\lambda))$ is monotonically decreasing, but since  $\hat{\alpha}_{\mathrm{K}}^n(\tilde{\lambda})$ already minimizes the $l_{\text{IV}}^n$ function, so will all $\hat{\alpha}_{\mathrm{K}}^n(\lambda )$ for all $\lambda \geq \tilde{\lambda}$. As $\hat{\alpha}_{\text{TSLS}}^n$ is the unique point that minimizes $l_{\text{IV}}^n$ we conclude that $\hat{\alpha}_{\text{TSLS}}^n = \hat{\alpha}_{\mathrm{K}}^n(\lambda )$ for all $\lambda \geq \tilde{\lambda}$, which yields a contradiction.
We conclude that $
P \lp \cap_{\lambda \geq 0 } \lp \hat{\alpha}_{\text{TSLS}}^n \not = \hat{\alpha}_{\mathrm{K}}^n(\lambda) \rp \rp =1$.

Let <Ref> and <Ref> hold, i.e., that $\fZ^\t \fZ$ and $\fA^\t \fZ$ are  of full rank and  $[ \fZ \,\, \fY]$ is of full column rank. Furthermore, let <Ref> hold, i.e., that $\hat{\alpha}_{\mathrm{K}}^n(\lambda)\not \in \cM_{\mathrm{IV}}$ for all $\lambda\geq 0$. It holds that (Primal$.t.n$) has a unique solution and satisfies Slater's condition for all  $t>  \min_{\alpha}l_{\mathrm{IV}}^n(\alpha)$ (<Ref>). Furthermore,  (Dual$.\lambda.n$) has a unique solution for all $\lambda \geq 0$ (<Ref>).

First consider an arbitrary $t\in D_{\text{Pr}}$ and note that the dual problem of (Primal$.t.n$), 	not to be confused with the problem (Dual$.\lambda.n$), is given by

\begin{align} \label{Eq:DualOfPrimal}
\begin{array}{ll}
\text{maximize}_\lambda & g_t(\lambda)  \\
\text{subject to} & \lambda \geq 0.
\end{array}
\end{align}

However, (Dual$.\lambda.n$) is equivalent with the infimum problem in the definition of  $g_t:\R_+ \to \R$ given by
g_t(\lambda) := \inf_\alpha \{l_{\text{OLS}}^n(\alpha)+ \lambda (l_{\text{IV}}^n(\alpha) - t)\}.$$

Now consider $\hat{\alpha}_{\text{Pr}}^n(t)$ solving the primal (Primal$.t.n$).
Slater's condition is satisfied, so there exists a $\lambda(t)\geq 0$ solving the dual problem and strong duality holds, $l_{\text{OLS}}^n(\hat{\alpha}_{\text{Pr}}^n(t)) = g_t(\lambda(t))$.
We will now show that $\hat{\alpha}_{\text{Pr}}^n(t)$ also solves to the K-class penalized regression problem (Dual.$\lambda(t).n$). That is, we will show that $
\hat{\alpha}_{\text{Pr}}^n(t) = \begin{array}{ll}
\underset{\alpha}{\text{argmin}} & l_{\text{OLS}}^n(\alpha) + \lambda(t) l_{\text{IV}}^n(\alpha).
\end{array}
To that end, note that

\begin{align*}
g_t(\lambda(t)) &=  \inf_\alpha \{ {l_{\text{OLS}}^n(\alpha)+ \lambda(t) (l_{\text{IV}}^n(\alpha) - t)} \} = \inf_\alpha \{ {l_{\text{OLS}}^n(\alpha)+ \lambda(t)l_{\text{IV}}^n(\alpha) } \}- \lambda(t) t\\
&\leq l_{\text{OLS}}^n(\hat{\alpha}_{\text{Pr}}^n(t))+ \lambda(t) (l_{\text{IV}}^n(\hat{\alpha}_{\text{Pr}}^n(t)) - t)
=  l_{\text{OLS}}^n(\hat{\alpha}_{\text{Pr}}^n(t))  =g_t(\lambda(t)),
\end{align*}

where in the last equality
we used strong duality
and  the second last equality we used that for any constraint bound $t\in D_{\text{Pr}}$ the inequality constraint of (Primal$.t.n$) is active in the solution $\hat{\alpha}_{\text{Pr}}^n(t)$, i.e., $l_{\text{IV}}^n(\hat{\alpha}_{\text{Pr}}^n(t))=t$; see <Ref> of <Ref>.
Thus, it holds that

\begin{align*}
& \inf_\alpha \{ {l_{\text{OLS}}^n(\alpha)+ \lambda(t) (l_{\text{IV}}^n(\alpha) - t)} \} = l_{\text{OLS}}^n(\hat{\alpha}_{\text{Pr}}^n(t))+ \lambda(t) (l_{\text{IV}}^n(\hat{\alpha}_{\text{Pr}}^n(t)) - t) \\
\iff &
\inf_\alpha \{ l_{\text{OLS}}^n(\alpha)+ \lambda(t) l_{\text{IV}}^n(\alpha) \} = l_{\text{OLS}}^n(\hat{\alpha}_{\text{Pr}}^n(t))+ \lambda(t) l_{\text{IV}}^n(\hat{\alpha}_{\text{Pr}}^n(t))

proving that $\hat{\alpha}_{\text{Pr}}^n(t)$ coincides with the unique solution $\hat{\alpha}_{\text{K}}^n(\lambda(t))$ to the K-class problem (Dual.$\lambda(t).n$) as it attains the same objective. Furthermore, there can only be one $\lambda(t)$ solving the dual problem in <Ref>.
If there are two distinct solutions $\lambda',\lambda''\geq 0$ with $\lambda'\not = \lambda''$, then by the above observations we get that $
\hat{\alpha}_{\text{Pr}}^n(t) = \hat{\alpha}_{\text{K}}^n(\lambda') = \hat{\alpha}_{\text{K}}^n(\lambda''),$
in contradiction to <Ref>.

Conversely, fix $\lambda \geq 0$ and recall that $\hat{\alpha}_{\mathrm{K}}^n(\lambda)$ solves the
penalized K-class regression problem (Dual.$\lambda.n$), that is, $
\hat{\alpha}_{\mathrm{K}}^n(\lambda) = \argmin_{\alpha} \{ l_{\text{OLS}}^n(\alpha) + \lambda l_{\text{IV}}^n(\alpha) \}$.
Now consider a primal constraint bound $t(\lambda) :=  l_{\text{IV}}^n(\hat{\alpha}_{\text{K}}^n(\lambda)) $ and consider the corresponding primal optimization problem (Primal.$t(\lambda).n$) and its dual form given by

\begin{align} \label{Eq:PrimalKclassEquivalence_PrimalDualSpecificToTheorem}
\textit{Primal}:\begin{array}{lr}
\begin{array}{ll}
\text{minimize} & l_{\text{OLS}}^n(\alpha)  \\
\text{subject to} &  l_{\text{IV}}^n(\alpha) \leq  t(\lambda)
\end{array}&  \quad \quad \textit{Dual}:	\begin{array}{ll}
\text{maximize} & g_{t(\lambda)}(\gamma)  \\
\text{subject to} & \gamma \geq 0,
\end{array}
\end{array}
\end{align}

where  $g_{t(\lambda)}:[0,\i) \to \R$ is given by
g_{t(\lambda)}(\gamma) = \inf_\alpha \{ l_{\text{OLS}}^n(\alpha) + \gamma [l_{\text{IV}}^n(\alpha)-t(\lambda)] \} .
Here we note that the proposed primal problem satisfies Slater's condition. To see this note that $\hat{\alpha}_{\text{K}}^n(\lambda)\not \in \cM_{\text{IV}}$, by <Ref>,
hence $\inf_{\alpha}l_{\text{IV}}^n(\alpha)=\min_{\alpha}l_{\text{IV}}^n(\alpha) < t(\lambda)= l_{\text{IV}}^n( \hat{\alpha}_{\text{K}}^n(\lambda))$.
Furthermore, we conclude that $t(\lambda) \in (\min_{\alpha}l_{\text{IV}}^n(\alpha),l_{\text{IV}}(\hat{\alpha}_{\text{OLS}}^n)]=D_{\text{Pr}}$ as $\lambda \mapsto l_{\text{IV}}(\hat{\alpha}_{\text{K}}^n(\lambda))$
is monotonically decreasing and $\hat{\alpha}_{\text{K}}^n(0) = \hat{\alpha}_{\text{OLS}}^n$; see <Ref>.

Let $p^\star$ and $d^\star$ denote the optimal objective values for the above primal and dual problem in <Ref>, respectively. It
holds that $\hat{\alpha}_{\text{K}}^n(\lambda)$  is primal feasible since it satisfies the inequality constraint of the primal problem in	 <Ref>. This implies that $p^\star \leq l_{\text{OLS}}^n(\hat{\alpha}_{\text{K}}^n(\lambda))$
since $p^\star$ is the infimum of all attainable objective values. By the non-negative duality gap we also have that

\begin{align*}
p^\star & \geq d^\star = \sup_{\gamma\geq 0} g_{t(\lambda)}(\gamma)  \geq  g_{t(\lambda)}(\lambda) =\inf_\alpha \{ l_{\text{OLS}}^n(\alpha) + \lambda [l_{\text{IV}}^n(\alpha)-t(\lambda)] \} \\
&=\inf_\alpha \{ l_{\text{OLS}}^n(\alpha) + \lambda l_{\text{IV}}^n(\alpha) \} - \lambda t(\lambda)=   l_{\text{OLS}}^n(\hat{\alpha}_{\text{K}}^n(\lambda)) + \lambda [l_{\text{IV}}^n(\hat{\alpha}_{\text{K}}^n(\lambda))-l_{\text{IV}}^n(\hat{\alpha}_{\text{K}}^n(\lambda))] \\
&= l_{\text{OLS}}^n(\hat{\alpha}_{\text{K}}^n(\lambda)),
\end{align*}

implying that $l_{\text{OLS}}^n(\hat{\alpha}_{\text{K}}^n(\lambda))= p^\star$. This proves that strong duality holds and that the solution
to the K-class regression problem $\alpha_{\text{K}}^n(\lambda)$ solves the primal optimization problem (Primal.$t(\lambda).n$), since it attains the unique optimal objective value while also satisfying the inequality constraint.

Fix any $p_{\min}\in(0,1)$ and let <Ref> hold,
i.e., that $\fZ^\t \fZ$ and $\fA^\t \fZ$ are  of full rank and  $[ \fZ \,\, \fY]$ is of  full column rank. Furthermore, let <Ref> hold, i.e., that $\hat{\alpha}_{\mathrm{K}}^n(\lambda)\not \in \cM_{\mathrm{IV}}$ for all $\lambda \geq 0$. It holds that (Primal$.t.n$) has a unique solution and satisfies Slater's condition for all  $t>  \min_{\alpha}l_{\mathrm{IV}}^n(\alpha)$ (<Ref>), that (Dual$.\lambda.n$) has a unique solution for all $\lambda \geq 0$ (<Ref>) and that $\{\hat{\alpha}_{\text{Pr}}^n(t):t\in D_{\text{Pr}}\}=\{\hat{\alpha}_{\text{K}}^n(\lambda):\lambda \geq 0\}$ (<Ref>).
Finally, we assume that $\lambda_n^\star(p_{\min})<\i$.
To simplify notation, we write $Q=Q_{\chi^2_{q}}(1-p_{\min})$.

We claim that the PULSE estimator can be represented in the dual form of the primal PULSE problem. That is, as a K-class estimator $\hat{\alpha}_{\text{PULSE}}^n(p_{\min}) = \hat{\alpha}^n_{\text{K}}(\lambda_n^\star(p_{\min}))$ with stochastic penalty parameter given by $
\lambda_n^\star(p_{\min}) := \inf\{\lambda \geq 0 : T_n(\hat{\alpha}_{\text{K}}^n (\lambda))\leq Q_{\chi^2_{q}}(1-p_{\min}) \}.
We show this by proving that $\hat{\alpha}_{\text{K}}^n(\lambda_n^\star(p_{\min}))= \hat{\alpha}_{\text{Pr}}^n(t_n^\star(p_{\min})))$, which by <Ref> implies that the claim is true, if the conditions $t_n^\star(p_{\min})>-\i$ and
$T_n(\hat{\alpha}_{\text{Pr}}^n(t_n^\star(p_{\min})))\leq Q$ can be verified from the assumption that $\lambda_n^\star(p_{\min})<\i$. First, we note that if $\lambda_n^\star(p_{\min})<\i$, then $t_n^\star(p_{\min})>-\i$.
This follows by noting that, with $t(\lambda) := l_{\text{IV}}^n(\hat{\alpha}_{\text{K}}^n(\lambda))$,  proof of <Ref> ii) yields that $\hat{\alpha}_{\text{K}}^n(\lambda) = \hat{\alpha}_{\text{Pr}}^n(t(\lambda))$ for any $\lambda \geq 0$
which yields $
\lambda_n^\star(p_{\min}) = \inf \left\{  \lambda  \geq 0:   T_n ( \hat{\alpha}_{\text{Pr}}^n \circ t(\lambda) )\leq Q \right\}.
Hence, if $\lambda_n^\star(p_{\min}) <\i$ we know there exists a $\lambda' \geq 0$ such that $T_n ( \hat{\alpha}_{\text{Pr}}^n \circ t(\lambda') )\leq Q$, i.e., there exists a $t'=t(\lambda')\in(\min_{\alpha'} l_{\text{IV}}^n(\alpha'),\i)$ such that $T_n ( \hat{\alpha}_{\text{Pr}}^n (t') )\leq Q$.
We have excluded that $t'=\min_{\alpha'} l_{\text{IV}}^n(\alpha')$ as $t'=l_{\text{IV}}^n(\hat{\alpha}_{\text{K}}^n(\lambda')) > \min_{\alpha'} l_{\text{IV}}^n(\alpha')$ since $\hat{\alpha}_{\text{K}}(\lambda') \not \in \cM_{\text{IV}}$.
Furthermore, we can without loss of generality assume that $t' \in (\min_{\alpha'} l_{\text{IV}}^n(\alpha'),l_{\text{IV}}^n(\hat{\alpha}^n_{\text{OLS}})]$ because if $t' > l_{\text{IV}}^n(\hat{\alpha}^n_{\text{OLS}})$, then it holds that $T_n ( \hat{\alpha}_{\text{Pr}}^n (l_{\text{IV}}^n(\hat{\alpha}^n_{\text{OLS}})) )\leq Q$ as $\hat{\alpha}_{\text{Pr}}^n (l_{\text{IV}}^n(\hat{\alpha}^n_{\text{OLS}})) = \hat{\alpha}_{\text{Pr}}^n (t')$ since the ordinary least square solution solves
all (Primal$.t.n$) with constraints bounds larger than
$l_{\text{IV}}^n(\hat{\alpha}^n_{\text{OLS}})$. As a consequence, the set for which we take the supremum over in the definition of $t_n^\star(p_{\min})$ is non-empty, such that $t_n^\star(p_{\min}) >-\i$.

Next we show that $\hat{\alpha}_{\text{K}}^n(\lambda_n^\star(p_{\min}))= \hat{\alpha}_{\text{Pr}}^n(t_n^\star(p_{\min})))$.
When this equality is shown, then the remaining condition that
$T_n(\hat{\alpha}_{\text{Pr}}^n(t_n^\star(p_{\min})))\leq Q$
follows by <Ref>
and we are done. For any constraint bound $t \in D_{\text{Pr}}= ( \min_{\alpha}l_{\mathrm{IV}}^n(\alpha),l_{\text{IV}}^n(\hat{\alpha}_{\text{OLS}}^n) ]$, consider the primal and corresponding dual optimization problems

\begin{align} \label{eq:dPULSEequalpPULSEPrimalDualProblems}
\textit{Primal}:\begin{array}{lr}
\begin{array}{ll}
\text{minimize} & l_{\mathrm{OLS}}^n(\alpha)  \\
\text{subject to} & l_{\mathrm{IV}}^n(\alpha)\leq t
\end{array} &  \quad \quad \textit{Dual}:	\begin{array}{ll}
\text{maximize} & g_t(\lambda)  \\
\text{subject to} & \lambda \geq 0,
\end{array}
\end{array}
\end{align}

with dual function $g_t:\R_{\geq 0} \to \R$ given by $
g_t(\lambda) := \inf_\alpha\{l_{\mathrm{OLS}}^n(\alpha)+ \lambda (l_{\mathrm{IV}}^n(\alpha) - t)\}.$
The proof of <Ref> yields that there exists a unique $\tilde{\lambda}(t)\geq 0$ solving the dual problem of <Ref> such that   $ \label{eq:TheoremPrimalDualConnectionPvalConstraintEqThatPrimalEqualKclass}
\hat{\alpha}_{\text{Pr}}^n(t) = \hat{\alpha}_{\text{K}}^n(\tilde{\lambda}(t)).
We now prove that $D_{\text{Pr}}\ni t \mapsto \tilde{\lambda}(t)$ is strictly decreasing.
Note that by the definition of $g_t$ and <Ref> (or  equivalently the discussion in the beginning of <Ref>) we have that

\begin{align}
g_t(\lambda) &= \inf_\alpha\{l_{\mathrm{OLS}}^n(\alpha)+ \lambda l_{\mathrm{IV}}^n(\alpha) \} -\lambda t  \label{Eq:RepresentationOfGtInTermsOfKclass}
= l_{\mathrm{OLS}}^n(\hat{\alpha}_{\text{K}}^n (\lambda))+ \lambda l_{\mathrm{IV}}^n(\hat{\alpha}_{\text{K}}^n (\lambda)) -\lambda t.
\end{align}

For any $t_1$, $t_2$ with
$0\leq  \min_{\alpha}l_{\mathrm{IV}}^n(\alpha)<t_1<t_2 \leq l_{\text{IV}}^n(\hat{\alpha}_{\text{OLS}}^n) $ we have that $g_{t_1}(\tilde{\lambda}(t_1)) \geq g_{t_1}(\tilde{\lambda}(t_2))$ and $g_{t_2}(\tilde{\lambda}(t_2)) \geq g_{t_2}(\tilde{\lambda}(t_1))$ as $\tilde{\lambda}(t)$ maximizes $g_t$.
Hence, by bounding the first term we get that

\begin{align} \label{eq:TheoremLambdaStarFiniteTStarFinite_Ineq1}
g_{t_1}(\tilde{\lambda}(t_1)) - g_{t_2}(\tilde{\lambda}(t_2)) & \geq g_{t_1}(\tilde{\lambda}(t_2)) - g_{t_2}(\tilde{\lambda}(t_2))
= \tilde{\lambda}(t_2)(t_2-t_1),
\end{align}

where the last equality follows from the representation in <Ref>.
Similarly, by bounding the other term we get that

\begin{align} \label{eq:TheoremLambdaStarFiniteTStarFinite_Ineq2}
g_{t_1}(\tilde{\lambda}(t_1)) - g_{t_2}(\tilde{\lambda}(t_2)) & \leq  g_{t_1}(\tilde{\lambda}(t_1)) - g_{t_2}(\tilde{\lambda}(t_1))
= \tilde{\lambda}(t_1)(t_2-t_1).
\end{align}

Combining the inequalities from <Ref> we conclude that $
\tilde{\lambda}(t_2) (t_2-t_1) \leq \tilde{\lambda}(t_1) (t_2-t_1)$ which implies $\tilde{\lambda}(t_2) \leq \tilde{\lambda}(t_1),
proving that $D_{\text{Pr}} \ni t \mapsto \tilde{\lambda}(t)$, the dual solution as a function of the primal problem constraint bound, is weakly decreasing. We now  strengthen this statement to strictly decreasing.
For any constraint bound $t\in D_{\text{Pr}}= ( \min_{\alpha}l_{\mathrm{IV}}^n(\alpha),l_{\text{IV}}^n(\hat{\alpha}_{\text{OLS}}^n)]$ we have that the solution $\hat{\alpha}^n_{\text{Pr}}(t)$ yields an active inequality constraint in the (Primal.$t.n$) problem, i.e., $l_{\text{IV}}^n(\hat{\alpha}_{\text{Pr}}(t))=t$; see <Ref> of <Ref>.
Therefore, for any $\min_{\alpha}l_{\mathrm{IV}}^n(\alpha) < t_1 < t_2 \leq  l_{\text{IV}}^n(\hat{\alpha}_{\text{OLS}}^n)$ we get that
l_{\text{IV}}^n(\hat{\alpha}_{\text{K}}^n(\tilde{\lambda}(t_1))) = l_{\text{IV}}^n(\hat{\alpha}_{\text{Pr}}^n(t_1)) = t_1 < t_2 = l_{\text{IV}}^n(\hat{\alpha}_{\text{Pr}}^n(t_2)) = l_{\text{IV}}^n(\hat{\alpha}_{\text{K}}^n(\tilde{\lambda}(t_2))),
proving that $\tilde{\lambda}(t_1) \not = \tilde{\lambda}(t_2) $, which implies that $D_{\text{Pr}} \ni t\mapsto \tilde{\lambda}(t)$ is strictly increasing.

Recall, by <Ref> that the K-class estimators for $\kappa\in[0,1)$ coincides with the collection of solutions to every primal problem satisfying Slater's condition.
That is,

\begin{align} \label{Eq:EqualityOfPrimalSoltuionsAndKclassSolutions}
\{\hat{\alpha}_{\text{K}}^n(\lambda):\lambda \geq 0\}
= \{\hat{\alpha}_{\text{Pr}}^n(t): t \in D_{\text{Pr}} \}
= \{\hat{\alpha}_{\text{K}}^n(\tilde{\lambda}(t)):t \in D_{\text{Pr}}\},
\end{align}

where $\tilde{\lambda}$ is as introduced above.

It now only remains to show
that $\tilde{\lambda}( t_n^\star(p_{\min}))= \lambda_n^\star(p_{\min})$, which implies the
conclusion as $\hat{\alpha}_{\text{Pr}}^n(t_n^\star(p_{\min})) = \hat{\alpha}_{\text{K}}^n(\tilde{\lambda}(t_n^\star(p_{\min}))) = \hat{\alpha}_{\text{K}}^n(\lambda_n^\star(p_{\min}))$.
We know that $\hat{\alpha}_{\text{K}}^n \circ \tilde{\lambda} (t) = \hat{\alpha}_{\text{Pr}}^n (t)$, hence for all $t\in D_{\text{Pr}}$, $
(T_n \circ \hat{\alpha}_{\text{K}}^n \circ \tilde{\lambda}) (t) = (T_n \circ \hat{\alpha}_{\text{Pr}}^n) (t)$,
and that for any $A\subset [0,\infty)$ it holds that  $\tilde{\lambda} ( \tilde{\lambda}^{-1}(A)) = A \cap \cR (\tilde{\lambda})$,
where $\cR (\tilde{\lambda}) = \{\tilde{\lambda}(t): t\in D_{\text{Pr}}\}\subseteq [0,\i)$ is the range of the reparametrization function $\tilde{\lambda}:D_{\text{Pr}} \to [0,\i)$. In fact,
$\tilde{\lambda}$ is surjective. To see this, note that $[0,\i) \ni \lambda \mapsto \hat{\alpha}_{\text{K}}^n(\lambda)$ is injective by <Ref>. Thus, $\cR(\tilde{\lambda})=[0,\i)$ must hold, for otherwise 	<Ref> would not hold. Hence, by surjectivity of $\tilde{\lambda}$ we get that for $A\subset [0,\infty)$ it holds that  $\tilde{\lambda} ( \tilde{\lambda}^{-1}(A)) = A$.

Now consider $\hat{\alpha}_{\text{Pr}}^n: D_{\text{Pr}} \to \R^{d_1+q_1}$, $\hat{\alpha}_{\text{K}}^n: [0,\infty) \to \R^{d_1+q_1}$ and $\tilde{\lambda}:D_{\text{Pr}} \to [0,\infty)$  as measurable (which follows by continuity and monotonicity) mappings  such that
t_n^\star(p_{\min}) = \sup \{ (T_n \circ \hat{\alpha}_{\text{Pr}}^n)^{-1} (-\i,  Q] \}
= \sup \{  (T_n \circ \hat{\alpha}_{\text{K}}^n \circ \tilde{\lambda})^{-1} (-\i,  Q] \}.$$

Since $t\mapsto \tilde{\lambda}(t)$ is strictly decreasing, we get that

\begin{align*}
\tilde{\lambda}(t_n^\star(p_{\min})) &= \tilde{\lambda} (   \sup \{   (T_n \circ \hat{\alpha}_{\text{K}}^n \circ \tilde{\lambda})^{-1} (-\i,  Q] \} ) \\
&= \inf \left\{ \tilde{\lambda}\lp  (T_n \circ \hat{\alpha}_{\text{K}}^n \circ \tilde{\lambda})^{-1} (-\i,  Q]  \rp \right\} \\
&= \inf \{ \tilde{\lambda} ( \tilde{\lambda}^{-1} (  (T_n \circ \hat{\alpha}_{\text{K}}^n )^{-1} (-\i,  Q] )) \} \\
&= \inf \left\{   (T_n \circ \hat{\alpha}_{\text{K}}^n )^{-1} (-\i,  Q]  \right\} \\
&=  \inf \left\{  \lambda \geq 0  :   T_n ( \hat{\alpha}_{\text{K}}^n (\lambda) )\leq Q \right\}
\\&= \lambda_n^\star(p_{\min}),
\end{align*}

Let $p_{\min}\in(0,1)$ and let <Ref> hold. We have that
l_{\mathrm{OLS}}^n(\hat{\alpha}_{\mathrm{K}}^n (\lambda)) \geq l_{\mathrm{OLS}}^n(\hat{\alpha}_{\text{OLS}}^n)
= n^{-1}\|\fY-\fZ\hat{\alpha}_{\text{OLS}}^n\|_2^2
= n^{-1}\| \fY - P_\fZ \fY \|_2^2

as $P_\fZ \fY \not = \fY$ (by <Ref> we have that $\fY \not \in \text{span}(\fZ)$, such that the projection of $\fY$ onto the column space of $\fZ$ does not coincide with $\fY$ itself). Hence,
$T_n:\R^{d_1+q_1}\to \R$ is well-defined, and the following upper bound

T_n (\hat{\alpha}_{\text{K}}^n (\lambda) ) = n \frac{l_{\text{IV}}^n(\hat{\alpha}_{\text{K}}^n (\lambda))}{l_{\mathrm{OLS}}^n(\hat{\alpha}_{\text{K}}^n (\lambda))}  \leq n \frac{l_{\text{IV}}^n(\hat{\alpha}_{\text{K}}^n (\lambda)) }{l_{\mathrm{OLS}}^n(\hat{\alpha}_{\text{OLS}}^n)}

is valid for every $\lambda \geq 0$. In the under- and just-identified setup we know that there exists an $\tilde{\alpha}\in \cM_{\text{IV}} \subset \R^{d_1+q_1}$ such that $
Now let $\Lambda>0$ be given by

\begin{align} \label{eq:LambdaEquality}
\Lambda
n \frac{  l_{\mathrm{OLS}}^n(\tilde{\alpha}) }{l_{\mathrm{OLS}}^n(\hat{\alpha}_{\text{OLS}}^n) Q_{\chi^2_{q}}(1-p_{\min})}.
\end{align}

For any $\lambda > \Lambda$ we have by the non-negativity of $l_{\text{OLS}}^n(\alpha)/\lambda$ that

\begin{align*}
l_{\text{IV}}^n(\hat{\alpha}_{\text{K}}^n (\lambda))  &\leq  \lambda^{-1}l_{\mathrm{OLS}}^n(\hat{\alpha}_{\text{K}}^n (\lambda)) + l_{\text{IV}}^n(\hat{\alpha}_{\text{K}}^n (\lambda)) = \min_\alpha \{ \lambda^{-1}l_{\mathrm{OLS}}^n(\alpha) + l_{\text{IV}}^n(\alpha)\} \\
&\leq  \lambda^{-1}l_{\mathrm{OLS}}^n(\tilde{\alpha}) + l_{\text{IV}}^n(\tilde{\alpha})  <  \frac{l_{\text{OLS}}^n(\tilde{\alpha})}{\Lambda} = \frac{l_{\mathrm{OLS}}^n(\hat{\alpha}_{\text{OLS}}^n)Q_{\chi^2_{q}}(1-p_{\min})}{n},
\end{align*}

This implies
T_n (\hat{\alpha}_{\mathrm{K}}^n (\lambda) )  \leq n \frac{l_{\text{IV}}^n(\hat{\alpha}_{\text{K}}^n (\lambda))}{l_{\mathrm{OLS}}^n(\hat{\alpha}_{\text{OLS}}^n)}  < Q_{\chi^2_{q}}(1-p_{\min}),

$\lambda > \Lambda$, proving that $
\lambda_n^\star(p_{\min}) = \inf \{\lambda \geq 0 : T_n(\hat{\alpha}_{\text{K}}^n(\lambda)) \leq Q_{\chi^2_{q}}(1-p_{\min})\}<\i.
Now consider the over-identified setup $(q> d_1+q_1)$. We claim that $
\lambda^\star_n(p_{\text{min}}) < \i$ if and only if $T_n(\hat{\alpha}_{\text{TSLS}}^n)< Q_{\chi^2_q}(1-p_{\min})$.
If $T_n(\hat{\alpha}_{\text{TSLS}}^n)< Q_{\chi^2_q}(1-p_{\min})$, then by continuity of $\lambda \mapsto \hat{\alpha}_{\text{K}}^n(\lambda)$ and $\alpha \mapsto T_n(\alpha)$ it must hold that $\lambda^\star_n(p_{\text{min}}) < \i$. This follows by noting that
$$ %
T_n(\hat{\alpha}_{\text{K}}^n(\lambda)) \downarrow T_n(\lim_{\lambda \to \i }\hat{\alpha}_{\text{K}}^n(\lambda))= T_n(\hat{\alpha}_{\text{TSLS}}^n) < Q_{\chi^2_q}(1-p_{\min}),$$

when $\lambda \to \i$, as $\lambda \mapsto T_n(\hat{\alpha}_{\text{K}}^n(\lambda))$
is strictly decreasing (<Ref>)
Here, we also used that

\begin{align*}
\lim_{\lambda \to \i } \hat{\alpha}_\text{K}^n(\lambda) &= \lim_{\lambda \to \i } (\fZ^\t (\fI+\lambda P_\fA)\fZ)^{-1} \fZ^\t(\fI+\lambda P_\fA)\fY  \\
&=  \lim_{\lambda \to \i } (\fZ^\t (\lambda^{-1}\fI+P_\fA)\fZ)^{-1} \fZ^\t(\lambda^{-1}\fI+ P_\fA)\fY \\ \notag
&= (\fZ^\t P_\fA \fZ)^{-1} \fZ^\t  P_\fA\fY  \\
&= \hat{\alpha}_{\text{TSLS}}^n.
\end{align*}

Hence, there must exist a $\lambda \in [0,\i)$ such that $T_n(\hat{\alpha}_{\text{K}}^n(\lambda))  <Q_{\chi^2_q}(1-p_{\min})$, proving that $\lambda^\star_n(p_{\text{min}}) < \i$.  Furthermore, note that the above arguments also imply that $\label{Eq:TSLSyieldsSmallestTestAmongKclass}
T_n(\hat{\alpha}_{\text{K}}^n(\lambda)) > T_n(\hat{\alpha}_{\text{TSLS}}^n)
for any $\lambda \geq 0$, as $\lambda \mapsto T_n(\hat{\alpha}_{\text{K}}^n(\lambda))$ is strictly decreasing and $T_n(\hat{\alpha}_{\text{TSLS}}^n)$ is the limit as $\lambda \to\i$.

Conversely, assume that $\lambda^\star_n(p_{\text{min}}) < \i$, which implies that there exists a $\lambda'\in[0,\i)$ such that $T_n(\hat{\alpha}_{\text{K}}^n(\lambda'))\leq Q_{\chi^2_q}(1-p_{\min} )$. Thus,

T_n(\hat{\alpha}_{\text{TSLS}}^n) < T_n(\hat{\alpha}_{\text{K}}^n(\lambda'))\leq Q_{\chi^2_q}(1-p_{\min}),

proving that the converse implication also holds.

We furthermore note that, if the acceptance region is empty, that is
\cA_n(1-p_{\min}) := \{\alpha\in \R^{d_1+q_1}:T_n(\alpha) \leq Q_{\chi^2_q}(1-p_{\min})\} = \emptyset,$$

then it obviously holds that $\lambda^\star_n(p_{\min})=\{\lambda \geq 0 :T_n(\hat{\alpha}_{\text{K}}^n(\lambda) )\leq Q_{\chi^2_q}(1-p_{\min}) \} = \i$.
The possibility of the acceptance region being empty, follows from the fact that the Anderson-Rubin confidence region can be empty; see <Ref>. To realize that the Anderson-Rubin confidence region can be empty we refer to the discussions and Monte-Carlo simulations of 25.

Assume that $\lambda_n^\star(p_{\min})<\i$ and that Assumption  <Ref> and <Ref> hold. Consider <Ref> for any fixed $N\in \N$.
The first `while loop' guarantees that $\lambda_{\min}$ and $\lambda_{\max}$ are such that
$\lambda^{\star}\in(\lambda_{\min},\lambda_{\max}]$. This is seen by noting that $\lambda \mapsto T_n(\hat{\alpha}_\text{K}^n(\lambda))$ is monotonically decreasing
(<Ref>) and that $\lambda_n^\star(p_{\min})<\i$. Hence, $T_n(\hat{\alpha}_\text{K}^n(\lambda_{\max}))$
eventually drops below $Q_{\chi^2_{q}}(1-p)$.
The second `while loop' keeps iterating until the
interval $(\lambda_{\min},\lambda_{\max}]$,  which  is guaranteed to contain $\lambda_n^\star(p_{\min})$, has a length less than or equal to $1/N$. Let $\lambda_{\min}$ and $\lambda_{\max}$ denote the last boundaries achieved
before the procedure terminates. Then  $
0\leq \mathrm{Binary.Search}(N,p) - \lambda_n^\star(p_{\min}) =
\lambda_{\max} - \lambda_n^\star(p_{\min})  \leq
\lambda_{\max}- \lambda_{\min} 	\leq 1/N$.
Hence,  $
\mathrm{Binary.Search}(N,p) - \lambda_n^\star(p_{\min}) \to 0$, as $N\to\i$.

Consider the just- or over-identified setup $(q\geq d_1+q_1)$, let  <Ref> hold. We furthermore assume that the population rank condition, <Ref>, i.e., $E(AZ^\t)$ is of full rank, are satisfied. We furthermore work under the finite-sample conditions of  <Ref>, i.e., that $\fA^\t \fA$ and  $\fZ^\t\fA$ are of full rank and $\fY\not\in \text{span}(\fZ)$ for all sample-sizes $n\in \N$ almost surely.
The  first two
of these are not strictly necessary as the population version of these rank assumptions guarantee that $\fA^\t \fA$ and  $\fZ^\t\fA$ are of full rank with probability tending to one; see
proof of
Likewise, we can drop the last finite-sample assumption as it is almost surely guaranteed if we assume that the distribution of $\ep_Y$ has density with respect to Lebesgue measure.
The proof below is easily modified
to accommodate these more relaxed assumptions, but for notational simplicity we
prove the statement
under the stronger finite-sample assumptions. We also let <Ref> hold which in addition with the previous assumptions guarantees that the dual representation of the PULSE holds whenever $\lambda^\star_n(p_{\min})<\i$; see <Ref>.
Furthermore, many of the previous theorems and lemmas were shown for a specific realization that satisfies the finite sample assumptions. Hence, we may only invoke the conclusions of these theorems almost surely.	 Note that the assumptions guarantee that the TSLS estimator is consistent, i.e., $\hat \alpha_{\mathrm{TSLS}}^n \convp \alpha_0$.

Fix any $p_{\min}\in(0,1)$ and let an arbitrary $\ep>0$ be given. We want to prove  that  $
P(\|\hat{\alpha}_{\text{PULSE}+}^n(p_{\min})-\alpha_0\|>\ep)  \to 0.$
To that end, define the events $(A_n)_{n\in \N}$ by $
A_n := (T_n(\hat{\alpha}_{\text{TSLS}}^n)< Q_{\chi^2_{q}}(1-p_{\min})),
such that

\begin{align} \label{eq:ConsistencyFirstTerm}
P(\|\hat{\alpha}_{\text{PULSE}+}^n(p_{\min})-\alpha_0\|>\ep) =& P((\|\hat{\alpha}_{\text{PULSE}}^n(p_{\min})-\alpha_0\|>\ep)\cap A_n) \\
&+P((\|\hat{\alpha}_{\text{ALT}}^n-\alpha_0\|>\ep)\cap A_n^c), \label{eq:ConsistencySecondTerm}
\end{align}

for all $n\in\N$. The last term,  <Ref>, tends to zero as $n\to\i$,
P((\|\hat{\alpha}_{\text{ALT}}^n-\alpha_0\|>\ep)\cap A_n^c) \leq P(\|\hat{\alpha}_{\text{ALT}}^n-\alpha_0\|>\ep) \to 0,

by the assumption that $\hat{\alpha}_{\text{ALT}}^n\convp \alpha_0$ as $n\to\i$. In regards to the first term, the right-hand side of <Ref>, we note that $A_n = (\lambda^\star_n(p_{\min})<\i)$, by <Ref>. Formally, this event equality only holds when intersecting both sides with the almost sure event that the finite sample rank condition holds. However, we suppress this intersection for ease of notation. Thus, on $A_n$, it holds that $\hat{\alpha}_{\text{PULSE}}^n(p_{\min}) = \hat{\alpha}_{\text{K}}^n(\lambda^\star_n(p_{\min})),$ by <Ref>, implying that
P((\|\hat{\alpha}_{\text{PULSE}}^n(p_{\min})-\alpha_0\|>\ep)\cap A_n) = P((\|\hat{\alpha}_{\text{K}}^n(\lambda^\star_n(p_{\min}))-\alpha_0\|>\ep)\cap A_n).$$

Furthermore, <Ref> yields that on $A_n$, it holds that
T_n(\hat{\alpha}_{\text{K}}^n(\lambda^\star_n(p_{\min}))) \leq Q_{\chi^2_{q}}(1-p_{\min}),$$
or equivalently
$$l_{\text{IV}}^n(\hat{\alpha}_{\text{K}}^n(\lambda^\star_n(p_{\min})) \leq n^{-1}Q_{\chi^2_{q}}(1-p_{\min}) l_{\text{OLS}}^n(\hat{\alpha}_{\text{K}}^n(\lambda^\star_n(p_{\min})).$$

On $A_n$, the stochastic factor in the upper bound above, is further bounded from above by

\begin{align*}
l_{\mathrm{OLS}}^n(\hat{\alpha}_{\mathrm{K}}^n (\lambda_n^\star(p_{\min}))) &\leq \sup_{\lambda\geq 0 }l_{\mathrm{OLS}}^n(\hat{\alpha}_{\mathrm{K}}^n (\lambda ))  \\
&= \lim_{\lambda \to \i }l_{\mathrm{OLS}}^n(\hat{\alpha}_{\mathrm{K}}^n (\lambda )) \\
&= l_{\mathrm{OLS}}^n(\lim_{\lambda \to \i }\hat{\alpha}_{\mathrm{K}}^n (\lambda )) \\
&= l_{\mathrm{OLS}}^n(\hat{\alpha}_{\text{TSLS}}^n),
\end{align*}

where we used continuity of $\alpha \mapsto l_{\mathrm{OLS}}^n(\alpha)$, that $\lambda \mapsto l_{\mathrm{OLS}}^n(\hat{\alpha}_{\mathrm{K}}^n (\lambda ))$ is weakly increasing (<Ref>) and that	$\lim_{\lambda \to \i } \hat{\alpha}_\text{K}^n(\lambda) = \hat{\alpha}_{\text{TSLS}}^n$.
Recall that the TSLS estimator is consistent 	$\hat{\alpha}_{\text{TSLS}}^n \convp \alpha_0$, where $\alpha_0$ is the causal coefficient of $Z$ onto $Y$. Hence, Slutsky's theorem and the weak law of large numbers yield that

\begin{align*}
l_{\mathrm{OLS}}^n(\hat{\alpha}_{\text{TSLS}}^n)  &= n^{-1} (\fY - \fZ \hat{\alpha}_{\text{TSLS}}^n)^\t (\fY - \fZ \hat{\alpha}_{\text{TSLS}}^n) \\&=
n^{-1} \fY^\t \fY + (\hat{\alpha}_{\text{TSLS}}^n)^\t n^{-1} \fZ^\t \fZ\hat{\alpha}_{\text{TSLS}}^n - 2n^{-1}\fY^\t \fZ\hat{\alpha}_{\text{TSLS}}^n  \\&\convp E(Y^2) + \alpha_0^\t E(Z Z^\t ) \alpha_0 - 2 E(YZ^\t) \alpha_0 \\&= E[(Y-Z\alpha_0)^2].
\end{align*}

Thus, on the event $A_n$, we have that
0\leq l_{\text{IV}}^n(\hat{\alpha}_{\text{K}}^n(\lambda^\star_n(p_{\min})) \leq n^{-1}Q_{\chi^2_{q}}(1-p_{\min}) l_{\text{OLS}}^n(\hat{\alpha}_{\text{TSLS}}^n)=:H_n,$$

where the upper bound $H_n$ converges to zero in probability by Slutsky's theorem. Furthermore, note that

\begin{align*}
\notag
l_\text{IV}^n(\alpha_0) &= \|n^{-1/2} (\fA^\t \fA)^{-1/2} \fA^\t(\fY - \fZ\alpha_0)\|_2^2 \\ \notag
&=\| (n^{-1}\fA^\t \fA)^{-1/2} n^{-1}\fA^\t\fU_Y\|_2^2 \\
&\convp \| E(AA^\t)^{-1/2} E(AU_Y)\|_2^2 \\
&= \| E(AA^\t)^{-1/2} E(A)E(U_Y)\|_2^2 \\\notag
& =
\end{align*}

where we used that $Y=Z^\t \alpha_0 +U_Y$, Assumption <Ref>: $A\independent U_Y$, and Assumption <Ref>: $E(A)=0$ (Alternatively,  $E(U_Y|A)=0$).

Now define a sequence of (everywhere) well-defined estimators $(\tilde{\alpha}_n)_{n\in\N}$ by
\tilde{\alpha}_n :=
\mathbbm{1}_{A_n} \hat{\alpha}_{\text{K}}^n(\lambda^\star_n(p_{\min}))
+ \mathbbm{1}_{A_n^c} \alpha_0,$$

for each $n\in \N$. We claim that the loss function $l_{\text{IV}}^n$ evaluated in this estimator tends to zero in probability, i.e., as $n\to\i$ it holds that

\begin{align} \label{eq:lIVinAlphatildeConvPZero}
l_{\text{IV}}^n(\tilde{\alpha}_n) = \| (n^{-1}\fA^\t \fA)^{-1/2} n^{-1}\fA^\t(\fY - \fZ\tilde{\alpha}_n)\|_2^2 \convp 0.
\end{align}

This holds by the above observations as for any $\ep'>0$ we have that

\begin{align*}
P(|l_{\text{IV}}^n(\tilde{\alpha}_n)| > \ep')
&= P((|l_{\text{IV}}^n(\hat{\alpha}_{\text{K}}^n(\lambda^\star_n(p_{\min})) )| > \ep')\cap A_n) +P((|l_{\text{IV}}^n(\alpha_0)| > \ep')\cap A_n^c) \\
&\leq    P((|H_n| > \ep')\cap A_n) +P((|l_{\text{IV}}^n(\alpha_0)| > \ep')\cap A_n^c) \\
&\leq P(|H_n| > \ep') +P(|l_{\text{IV}}^n(\alpha_0)| > \ep') \to 0,
\end{align*}

when $n\to\i$. 	Now define the random linear maps $g_n:\Omega\times \R^{d_1+q_1}\to \R^q$ by
g_n(\omega,\alpha) := (n^{-1}\fA^\t(\omega) \fA(\omega))^{-1/2}n^{-1} \fA^\t(\omega)  \fZ(\omega) \alpha,$$

for all $n\in \N$.
The maps $(g_n)$ converge point-wise, that is, for each $\alpha$, in probability to $g:\R^{d_1+q_1}\to \R^q$, given by $
g(\alpha) := E(AA^\t)^{-1/2}E(AZ^\t)\alpha$,
as $n\to \i$. The map $g$ is injective. This follows by  Assumption <Ref> and  <Ref>, which implies and state that $E(AA^\t)\in \R^{q\times q}$ and $E(AZ^\t)\in \R^{q\times(d_1+q_1)}$ are of full rank, respectively, hence
\text{rank}(E(AA^\t)^{-1/2}E(AZ^\t))=\text{rank}(E(AZ^\t))= d_1+q_1,
since we are in the just- and over-identified setup, where $q\geq d_1+q_1$. We conclude that $g$ is injective, as its matrix representation is of full column rank.
Furthermore, by <Ref> it holds that $
g_n(\tilde{\alpha}_n) \convp E(AA^\t)^{-1/2}E(AY)$.
Hence, we have that

\begin{align*}
g_n(\tilde{\alpha}_n)-g_n(\alpha_{0}) &\convp E(AA^\t)^{-1/2}E(AY) -E(AA^\t)^{-1/2}E(AZ^\t)\alpha_{0} \\
&= E(AA^\t)^{-1/2}E(AU_Y) \\
&= 0,
\end{align*}

as $n\to \i$. <Ref> of <Ref> now yields that $\tilde{\alpha}_n \convp \alpha_0$. Finally, note that as $\hat{\alpha}_{\text{K}}^n(\lambda^\star_n(p_{\min}))= \tilde{\alpha}_n$ on $A_n$ we have that

\begin{align*}
P((\|\hat{\alpha}_{\text{K}}^n(\lambda^\star_n(p_{\min}))-\alpha_0\|>\ep)\cap A_n) &=P((\|\tilde{\alpha}_n-\alpha_0\|>\ep)\cap A_n)\\
&\leq P(\|\tilde{\alpha}_n-\alpha_0\|>\ep) \\
&\to 0,
\end{align*}

proving that  $\hat{\alpha}_{\text{PULSE}+}^n(p_{\min}) \convp \alpha_0$, as $n\to\i$.

§ AUXILIARY LEMMAS

Suppose that $g_n: \mathbb{R}^G \to \mathbb{R}^K$ are random linear maps
converging point-wise in probability to a non-random  linear map $g:\R^G\to \R^K$ that is injective. If

g_n(\hat{\beta}_n - \beta_0) \underset{n\to\i}{\convp}0,

then $\hat{\beta}_n$ is a consistent estimator of $\beta_0$. That  is, $	\hat{\beta}_n \underset{n\to\i}{\convp}\beta_0$.

As $g$ is injective, we have that $\text{rank}(g) = G$, and as such $\text{rank}(g^\t g) = \text(g) = G$ which implies that $g^\t g$ is invertible. Furthermore, by Slutsky's theorem we get that $
g_n\convp g \implies g_n^\t g_n \convp g^\t g$,
as $n\to\i$, that is, for any $\ep>0$,
P(||g_n^\t g_n-g^\t g\| \leq \ep) =P(g_n^\t g_n \in \overline{B(g^\t g,\ep)})  \underset{n\to\i}{\to }  1.$$

Here $\|\cdot\|$ is any norm on the set of $G\times G$ matrices and $\overline{B(g^\t g,\ep)}$ is the closed ball around $g^\t g$ with radius $\ep$ with respect the this norm.
Now note that the set $\text{NS}_G$ of all non-singular $G\times G$ matrices  is an open subset of all $G\times G$ matrices, which implies that there exists an $\ep>0$, such that  $\overline{B(g^\t g,\ep)} \subset \text{NS}_G.$
Hence, $g_n^\t g_n$ is invertible with probability tending towards 1, that is, $
P(g_n^\t g_n \in \text{NS}_G) \underset{n\to\i}{\to } 1$.
Let $h_n:\Omega\to\text{NS}_G$ be given by
h_n(\omega)  := \mathbbm{1}_{(g_n^\t g_n \in \text{NS}_G)} g_n^\t(\omega) g_n(\omega) + \mathbbm{1}_{(g_n^\t g_n \in \text{NS}_G)^c} I.$$

Then $h_n \underset{n\to\i}{\convp} g^\t g$, since for any $\ep >0$

\begin{align*}
P(\|h_n-g^\t g\| >\ep ) &= P((\|g_n^\t g_n-g^\t g\|> \ep) \cap (g_n^\t g_n \in \text{NS}_G)) \\
& \qquad + P((\|I-g^\t g\|> \ep )\cap (g_n^\t g_n \in \text{NS}_G)^c) \\
&\leq  P(\|g_n^\t g_n-g^\t g\|> \ep) + P(g_n^\t g_n \in \text{NS}_G)^c) \\
&\underset{n\to\i}{\to}  0,
\end{align*}

Continuity of the inverse operator and the continuous mapping theorem, yield that $\|h_n^{-1}\|_{\text{op}}\convp \|(g^\t g)^{-1}\|_{\text{op}} \in \R$ and $\|g_n^\t \|_{\text{op}} \convp \|g^\t\|_{\text{op}}\in\R$ as $n$ tends to infinity, where $\|\cdot\|_{\text{op}}$ is the operator norm induced by the Euclidean norm $\|\cdot\|_2$. Furthermore,

\begin{align*}
\|g_n^\t g_n(\hat{\beta}_n-\beta_0) \|_2 &\leq \|g_n^\t \|_{\text{op}} \| g_n(\hat{\beta}_n-\beta_0) \|_2 \\ &\underset{n\to\i}{\convp} \|g^\t \|_{\text{op}}  \cdot  0\\
&= 0,
\end{align*}

by the assumptions and Slutsky's theorem. Hence, for any  $\ep>0$

\begin{align*}
P(	\|h_n(\hat{\beta}_n-\beta_0) \|_2 > \ep ) &= P(	(\|g_n^\t g_n(\hat{\beta}_n-\beta_0) \|_2 > \ep ) \cap (g_n^\t g_n \in \text{NS}_G) ) \\
&\qquad + P(	(\|\hat{\beta}_n-\beta_0\|_2 > \ep) \cap  (g_n^\t g_n \in \text{NS}_G)^c) \\
&\leq  P(	(\|g_n^\t g_n(\hat{\beta}_n-\beta_0) \|_2 > \ep ) ) + P((g_n^\t g_n \in \text{NS}_G)^c)\\
&\underset{n\to\i}{\to}  0.
\end{align*}

\begin{align*}
\|\hat{\beta}_n-\beta_0\|_2 &= \| h_n^{-1} h_n(\hat{\beta}_n-\beta_0) \|_2 \\& \leq \|h_n^{-1}\|_{\text{op}} \| h_n(\hat{\beta}_n-\beta_0) \|_2 \\ &\underset{n\to\i}{\convp} \|(g^\t g)^{-1}\|_{\text{op}} \cdot 0\\ &=0,
\end{align*}

by Slutsky's theorem, yielding that $\hat{\beta}_n$ is an consistent estimator of $\beta_0$.

Let $\hat{\alpha}$ be a solution to a constrained optimization problem of the form

\begin{align*}
\begin{array}{ll}
\underset{\alpha \in \R^{k}}{\mathrm{minimize}} & f(\alpha) \\
\mathrm{subject \, to} & g(\alpha) \leq c,
\end{array}
\end{align*}

where $f$ is an everywhere differentiable strictly convex function on $\R^k$ for which a stationary point exists, $g$ is continuous and $c\in \R$.  If the stationary point of $f$ is not feasible, then the constraint inequality is tight (active)
in the solution $\hat{\alpha}$, that is, $
g(\hat{\alpha}) = c$.

$\hat{\alpha}$ feasible and the stationary point of $f$ is not feasible,
we know that $\hat{\alpha}$ is not a stationary point of $f$, hence $Df(\hat{\alpha}) \not = 0$. Now assume that the constraint bound is not tight (active) in the solution $\hat{\alpha}$, that is $g(\hat{\alpha})<c$. By continuity of $g$, we know that there exists an $\epsilon >0$, such that for all $\alpha \in B(\hat{\alpha}, \ep)$, it holds that $g(\alpha)<c$. Furthermore, since $Df(\hat{\alpha}) =0$, we can look at the line segment going through $\hat{\alpha}$ in the direction of the negative gradient of $f$ in $\hat{\alpha}$. That is, $l:\R \to \R^k$ defined by $l(t) = \hat{\alpha} - tDf(\hat{\alpha})$. Note that
$$D (f\circ l)(0) = Df(l(0))Dl(0) = -Df(\hat{\alpha}) Df(\hat{\alpha})^\t = -\|Df(\hat{\alpha})\|<0,$$
meaning that the derivative of $f\circ l:\R \to \R$ is negative in zero. Therefore, there exists a $\delta>0$, such that for all $t\in(0,\delta)$ it holds that $f\circ l (t) < f \circ l(0)$, i.e.,
$$ f(\hat{\alpha}-tDf(\hat{\alpha})) < f(\hat{\alpha}).$$
Thus, for sufficiently small $t'$, is it holds that $t'<\delta$ and $\hat{\alpha}-t'Df(\hat{\alpha}) \in B(\hat{\alpha},\ep)$. We conclude that $\tilde{\alpha}:= \hat{\alpha}-tDf(\hat{\alpha})$ is feasible, $g(\tilde{\alpha})<c$, and super-optimal compared to $\hat{\alpha}$, $f(\tilde{\alpha})< f(\hat{\alpha})$, which contradicts that $\hat{\alpha}$ solves the optimization problem. In words, if the solution is not tight we can take a small step in the negative gradient direction of the objective function and get a better objective value while still being feasible.

§ ADDITIONAL REMARKS

Theorem <ref> still holds under
the following three model misspecifications, which may arise from erroneous non-sample information and unobserved endogenous variables
(these violations
may break the identification of $\alpha_{0,*}$ and
generally render
the K-class estimators inconsistent even when $\plim\kappa  =1$).

(a) 	Exclude included endogenous variables. Consider the setup where no hidden variable enters the target equation given by $Y = \gamma_0^\t X + \beta_0^\t A + \ep_Y$, with $\ep_Y \independent A$.  If we
exclude an  endogenous variable that directly affects $Y$, i.e., $\gamma_{0,-*} \not =0$, this is equivalent to drawing inference from the model $	Y= \gamma_{0,*}^\t X_{*} + \beta_{0,*}^\t A_{*} +U $,
where $U= \ep_Y + \gamma_{0,-*}^\t X_{-*}$. If $E(A_{-*}U)= E(A_{-*}X_{-*}^\t)\gamma_{0,-*}\not = 0$, we have introduced dependence that renders at least some of the instruments $A_{-*}$ invalid, breaking identifiability.

(b) Exclude included exogenous variables. Consider again the setup from $(a)$ where there is no hidden variables entering the target equation. If we
exclude a exogenous variable that directly affects $Y$, i.e., $\beta_{0,-*} \not =0$, then this is equivalent with drawing inference from the model $	Y= \gamma_{0,*}^\t X_{*} + \beta_{0,*}^\t A_{*} +U $,
where $U= \ep_Y + \beta_{0,-*}^\t A_{-*}$. It holds that $E(A_{-*}U)= E(A_{-*}A_{-*}^\t)\beta_{0,-*}\not =0$, again rendering the instruments invalid.

(c) Possibility of hidden endogenous variables.
Consider the case with
included hidden variables that are directly influenced by the excluded exogenous variables, i.e., $A_{-*}\to H\to Y$.
This implies that the excluded exogenous variables $A_{-*}$ are correlated with the collapsed noise variable in the structural equation $Y =  \alpha^\t_{0,*} Z_*  + U$, where $U = \ep_Y + \eta_0^\t H$ with $\eta_0\not =0$. In the case that $E(A_{-*}U) = E(A_{-*}H^\t)\eta_0\not =0$ the instruments are invalid.

Our acceptance region $
\cA_n^c(1-p_{\min}) := \{\alpha \in \R^{d_1+q_1}: T_n^c(\alpha) \leq Q_{\chi^2_{q}}(1-p_{\min})\}$,
is closely related to the Anderson-Rubin <cit.> confidence region of the simultaneous causal parameter $\alpha_0 = (\gamma_0,\alpha_0)$ in an identified model. When the causal parameter $\alpha_0$ is identifiable, i.e., in a just- or over-identified setup $(q\geq d_1+q_2)$ and <Ref>
holds, only the causal parameter yields regression residuals $Y-\alpha_0^\t Z$ that are uncorrelated with the exogenous variables $A$. In this restricted setup, our null hypothesis is equivalent with $\tilde{H}_0(\alpha): \alpha = \alpha_0$.
The hypothesis $\tilde{H}_0(\alpha)$
can be tested by the Anderson-Rubin test and all non-rejected coefficients constitute
the Anderson-Rubin confidence region of $\alpha_0$, which is given by $
\text{CR}_{\text{AR}}^{ex,n}(1-p_{\min}) := \left\{\alpha \in \R^{d_1+q_1}:T^{\text{AR}}_n(\alpha) \leq Q_{F(q,n-q)}(1-p_{\min}) \right\}$,
where $Q_{F(q,n-q)}(1-p_{\min})$ is the $1-p_{\min}$ quantile of the $F$ distribution with $q$ and $n-q$ degrees of freedom and the
Anderson-Rubin test-statistic $T^{\text{AR}}_n(\alpha)$ is  given by

\begin{align*}
T^{\text{AR}}_n(\alpha) :=  \frac{n-q}{q}\frac{(\fY-\fZ\alpha)^\t P_\fA(\fY-\fZ\alpha)}{(\fY-\fZ\alpha)^\t P_\fA^\perp(\fY-\fZ\alpha)} = \frac{n-q}{q}\frac{l_{\text{IV}}^n(\alpha)}{l_{\text{OLS}}^n(\alpha)-l_{\text{IV}}^n(\alpha)}.
\end{align*}

The confidence region $\text{CR}_{\text{AR}}^{ex,n}$ is exact whenever several regularity conditions are satisfied, such as
deterministic exogenous variables and normal distributed errors
In a general SEM model the regularity conditions are not fulfilled, but changing the rejection threshold to $Q_{\chi^2_q/q}(1-p_{\min})$, we obtain an asymptotically valid confidence region. That is,

\begin{align*}
\text{CR}_{\text{AR}}^{as,n}(1-p_{\min}) := \left\{\alpha \in \R^{d_1+q_1}:T^{\text{AR}}_n(\alpha) \leq Q_{\chi^2_q/q}(1-p_{\min}) \right\},
\end{align*}

is an asymptotically valid approximate confidence region
This relies on the fact that $T^{\text{AR}}_n(\alpha) \convd \chi^2_q/q$ under the null and $T^{\text{AR}}_n$ diverges to infinity under the general alternative. The test-statistic $T_n^c(\alpha)$ can be seen as a scaled coefficient of determination ($R^2$-statistic) for which $T_n^{\text{AR}}(\alpha)$ is the  corresponding $F$-statistic. That is, one can realize that

\begin{align*}
T_n^{\text{AR}}(\alpha) = \frac{n-q}{q} \frac{T_n^c(\alpha)/c(n)}{1-T_n^c(\alpha)/c(n)} \leq Q_{\chi^2_q/q}(1-p_{\min}),
\end{align*}

is equivalent to

\begin{align*}
\frac{n-q+ Q_{\chi^2_q}(1-p_{\min})}{c(n)} T_n^c(\alpha) \leq Q_{\chi^2_q}(1-p_{\min}).
\end{align*}

Thus, if
$Q_{\chi^2_q}(1-p_{\min}) \geq q$, then  $\cA_n(1-p_{\min}) \supseteq \text{CR}_{\text{AR}}^{as,n}(1-p_{\min})$ and $\cA_n(1-p_{\min}) \subset \text{CR}_{\text{AR}}^{as,n}(1-p_{\min})$ otherwise, where $\cA_n(1-p_{\min})$ is the acceptance region when using the scaling scheme $c(n)=n$.
Furthermore, $\cA_n^c(1-p_{\min})$, the acceptance region under a general scaling scheme $c(n) \sim n$,  is asymptotically equivalent to the Anderson-Rubin approximate confidence region $\text{CR}_{\text{AR}}^{as,n}(1-p_{\min})$. If we choose the specific scaling to be
$c(n)=n-q+ Q_{\chi^2_q}(1-p_{\min})$, then
they coincide,
$\text{CR}_{\text{AR}}^{as,n}(1-p_{\min})=\cA_n^c(1-p_{\min})$ for each $n\in \N$. Whenever the Anderson-Rubin confidence region is exact, we could change the rejection threshold from $Q_{\chi^2_{q}}(1-p_{\min})$ to $c(n)Q_{B(q/2,(n-q)/2)}(1-p_{\min})$ and also get an exact acceptance region, where $B(q/2,(n-q)/2)$ is the Beta distribution with shape and scale parameter $q/2$ and $(n-q)/2$ respectively.

It has been suggested to use pre-test for choosing
between the TSLS and OLS estimator. When using the Hausman test for endogeneity <cit.> one considers the pre-test estimator studied by, e.g., 21 and 36. If $H$ denotes the Hausman test-statistic that rejects the hypothesis of endogeneity when $H\leq Q$, the pre-test estimator is given by $\alpha_{\mathrm{pretest}}^n=1_{(H\leq Q)}\alpha_{\mathrm{OLS}}^n+1_{(H>Q)}\alpha_{\mathrm{TSLS}}^n$.  The PULSE estimator can  be seen as a
pre-test estimator using the Anderson-Rubin test as a test for endogeneity.  However, PULSE differs from the above in the sense that when endogeneity is not rejected we do not revert to the TSLS estimate but rather to the coefficient within the Anderson-Rubin confidence region that minimizes the mean squared prediction error.

§ SIMULATION STUDY

§.§ Distributional robustness

We first illustrate the distributional robustness property of K-class estimators
discussed in Section <ref> in
a finite sample setting.
We consider the model given by

\begin{align*}
X := A + U_X, \qquad
Y := \gamma  X + U_Y,
\end{align*}

where $\gamma=1$ and $A\sim N(0,1)$ independent of $
\begin{psmallmatrix}
U_X \\U_Y
\end{psmallmatrix} \sim \mathcal{N} \left( \begin{psmallmatrix}
0 \\ 0
\end{psmallmatrix}, \begin{psmallmatrix}
1 & 0.5 \\
0.5 & 1
\end{psmallmatrix}\right)$.
We estimate $\gamma$ from $n=2000$  observations generated by the above system and estimate $\hat \gamma^n_{\text{K}}(\kappa)$ for all $\kappa \in \{0,3/4,1\}$ for which the corresponding population coefficients are given by $\gamma_{\text{K}}(0)=\gamma_{\text{OLS}}=1.25$, $\gamma_{\text{K}}(3/4)=1.1$ and $\gamma_{\text{K}}(1)=\gamma_{\text{TSLS}}=1$.
We repeat the simulation 50 times and save the estimated coefficients.
illustrates the distributional robustness property of <Ref>. For all estimated coefficients $\hat \gamma$ of $\gamma$ we have plotted the analytically computed worst case mean squared prediction error (MSPE) under all hard interventions of absolute strength up to $x$ given by
\begin{align} \label{eq:analyticDerivationOfWCMSPE}
\sup_{|v| \leq x} E^{\mathrm{do}(A:=v)} [ \lp Y-\hat \gamma X\rp^2 ] %
&=  x^2(1-\hat \gamma)^2 +\hat \gamma^2+3(1-\hat \gamma)
\end{align}

against the maximum intervention strength $x$ for the range $x\in[0,6]$. The plot also shows results for the population coefficient as seen in  <cit.>.

Distributional Robustness of K-class estimators. The plot shows
the worst case MSPE against the maximum intervention strength condsidered. Each of the 50 repetitions corresponds
to three lines (green, red, blue), corresponding to the three
estimates using $\kappa \in \{0,3/4,1\}$, respectively. The solid black line corresponds to the population coefficients. The OLS is optimal for small interventions but yields a large loss for strong interventions; the TSLS is optimal
for large interventions but yields a relatively large loss for small interventions. Choosing a $\kappa$ different from zero and allows us to trade off these two regimes.
The dashed and the dotted lines correspond
to the two samples,
for which
the interval on which the
$\kappa=3/4$ estimator
TSLS and OLS in terms of worst case MSPE
is shortest and longest, respectively.

In all 50 repetitions the K-class estimator for $\kappa=3/4$  outperforms both OLS and TSLS in terms of worst case MSPE for maximum intervention strength of $2$.
This is in line with the theory presented in Section <ref>.
In terms of population coefficients  our theoretical results predict
that $\kappa=3/4$ is worst case MSPE superior, relative to OLS and TSLS, for all maximum intervention strengths in the range $[1.37,3]$.
Among the 50 repetitions we find the outcomes for which the superiority range of $\kappa = 3/4$ has the shortest and longest superiority range length. The shortest superiority range is $[1.27, 2.15]$ and the longest is $[1.46,5.54]$. Clearly, these numbers vary with changing sample size and number of repetitions. For example, with $50$, $200$, $500$, $2000$, $5000$ and $10000$ observations and 50 repetitions, the median lengths of the MSPE superiority range for $\kappa=3/4$ equal 0.82, 1.16, 1.44, 1.74, 1.58 and 1.63, respectively (1.63 is also the length of the theoretically computed interval $[1.37, 3]$).

§.§ Estimating causal effects

In this subsection we
investigate the finite sample behaviour of the PULSE estimator by simulation experiments. We look at how the PULSE estimator fairs in comparison to other well-known single equation estimators in terms of different performance measures.  We generate $n \in \N$ realizations of the SEM in question and construct the estimators of interest based on these $n$ observations. This is repeated $N\in \N$ times, allowing us to estimate different finite sample performance measures of the estimators of interest. The characterization of weak instruments through the minimum eigenvalue of $G_n$, a multivariate analogue to the first stage F-statistic, as introduced in 85 is important for some of our experimental findings. We refer the reader to <Ref> for a brief introduction.

§.§.§ Benchmark Estimators and Performance Measures

We compare the PULSE(5)  estimator, that is PULSE with $p_{\min}=0.05$, to four specific K-class estimators that are well-known to have second moments (in sufficiently over-identified setups). This will allow us to conduct both bias and mean squared error analysis of estimators.
Most importantly, we benchmark against  Fuller estimators.
The $\kappa$-parameter of the Fuller estimators are given by
$\kappa_{\text{FUL}}^n(a) =\kappa_{\text{LIML}}^n -\frac{a}{n-q},$
where $n-q$ is the degrees of freedom in the first stage regression, $a>0$ is a hyper parameter
and $\kappa_{\text{LIML}}^n$ is the stochastic $\kappa$-parameter corresponding the to LIML estimator. One way to represent the $\kappa$-parameter of the LIML estimator is
\kappa_{\text{LIML}} = \lambda_{\min}(W_1W^{-1})  %
where $\lambda_{\min}$ denotes the smallest eigenvalue, $W_1$ and $W$ are defined as $
W = [
\fY  \, \, \fX
]^\t P_\fA^\perp [
\fY  \, \, \fX
]$ and $W_1 =[
\fY  \, \, \fX
]^\t P_{\fA_*}^\perp [
\fY  \, \, \fX
and $P_\fA^\perp = \fI-\fA (\fA^\t \fA)^{-1}\fA^\t$; see, e.g.,  4.  We choose to benchmark the PULSE estimator against the
following K-class estimators: OLS ($\kappa=0$), TSLS ($\kappa=1$), Fuller(1) ($\kappa =\kappa_{\text{FUL}}^n(1)$) and Fuller(4) ($\kappa =\kappa_{\text{FUL}}^n(4)$).

The Fuller(1) estimator is approximately unbiased in that the mean bias is zero up to $\mathcal{O}(n^{-2})$ <cit.> and  Fuller(4) exhibits approximate superiority in terms of MSE compared to all other Fuller estimators <cit.>. As we shall see below the PULSE estimator has good MSE performance when instruments are weak and therefore we especially benchmark against Fuller(4) which  has shown better MSE performance than TSLS in simulation studies when instruments are weak; see e.g.  40. In the over-identified setup we let the PULSE estimator revert to Fuller(4) whenever the dual representation is infeasible.

We compare the estimators in terms of bias and mean squared error (MSE), which for an $n$-sample estimator $\hat{\alpha}_n$ with target $\alpha\in \R^{d_1+q_1}$
are given by $
\text{Bias}(\hat \alpha_n) = E(\hat \alpha_n)-\alpha\in \R^{d_1+q_1}$, $
\text{MSE}(\hat \alpha_n) = E[(\hat \alpha_n-\alpha) (\hat \alpha_n-\alpha)^\t]\in \R^{(d_1+q_1)\times (d_1+q_1)}$.
The empirical quantities, estimated
from $N$ independent repetitions are denoted by
$\widehat{\text{MSE}}(\hat \alpha_n)$.
In the multivariate setting,
we compare biases by comparing their Euclidean norms
When comparing MSEs, we
MSE superior to
if they are ordered in the partial ordering generated by the proper cone of positive semi-definite matrices (that is, $\widehat{\text{MSE}}(\tilde{\alpha}_n) - \widehat{\text{MSE}}(\hat\alpha_n)$ is
positive semi-definite).
We also consider
the ordering of its scalarizations given by the determinant and trace (the latter satisfies
$\text{trace}(\widehat{\text{MSE}}(\hat{\alpha}_n)) = \text{trace}(\widehat{\text{Var}}(\hat{\alpha}_n))+ \|\widehat{\text{Bias}}(\hat{\alpha}_n)\|_2^2$).

We conduct the simulation experiments
even though it is not proved that the
PULSE estimator has finite second moments.
In the simulations, the empirical estimates of the mean squared error were stable, possibly
even more so than for the Fuller estimators for which we know that second moments exists in settings where the noise is Gaussian; see e.g., 32, 19.

Below we describe two multivariate simulation experiments and refer the reader to <Ref> in the main paper for a univariate simulation experiment.

§.§.§ Varying confounding multivariate experiment.

In this simulation scheme we consider just-identified two-dimensional instrumental variable models with the
SEM and causal graph illustrated in <Ref>.  Since we want to compare MSE statistics that require estimators with second moments we drop comparisons with the TSLS estimator.

\begin{align*}
A &:= N_A \in \mathbb{R}^2, \\[7pt]
H &:= N_H \in \mathbb{R}^2, \\[7pt]
X &:=  \xi^\t A +  \delta^\t H + N_X \in \mathbb{R}^2 , \\[7pt]
Y &:=  \gamma^\t X +  \mu^\t H + N_Y  \in \mathbb{R}.
\end{align*}

The SEM and graph representation used for simulating data in the experiments described in Section <ref>.

Here, $\xi, \delta\in \R^{2\times 2}$, $\mu\in \R^2$ and $(N_A,N_X,N_X,N_Y)$ are independent noise innovations. We let $\gamma=(0,0)$ and let the noise innovations for $A,H,Y$ have distribution $(N_A ,N_H, N_Y) \sim  \mathcal{N} (0, I )$.

We randomly generate 10000 models by letting $
N_X \sim \mathcal{N} \left( \begin{psmallmatrix}
0 \\ 0
\end{psmallmatrix}, \begin{psmallmatrix}
\sigma_{1}^2 & 0 \\
0 & \sigma_{2}^2
\end{psmallmatrix} \right)$,
where the standard deviations is drawn by $\sigma_{1}^2,\sigma_{2}^2\sim \text{Unif}(0.1,1)$ and all other model coefficients are drawn according to $
\xi_{11},\xi_{12},\xi_{21},\xi_{22},\delta_{11},\delta_{12},\delta_{21},\delta_{22},\mu_1,\mu_2 \sim \text{Unif}(-2,2)$.
The hidden confounding induces dependence between the collapsed noise variables $U_X = \delta^\t H + N_X$ and $U_Y = \mu^\t H + N_Y$, which we capture by a normalized cross covariance vector $\rho := \Sigma_{U_X}^{-1/2} \Sigma_{U_XU_Y} \Sigma_{U_Y}^{-1/2}\in \R^2$, where $\Sigma_{U_X} = \text{Var}(U_X)$, $\Sigma_{U_XU_Y}=\text{Cov}(U_X,U_Y)$ and $\Sigma_{U_Y}= \text{Var}(U_Y)$. As such, the degree of confounding can be explained by the norm of $\rho$ given by $
\| \rho\|_2^2 =  \Sigma_{U_YU_X} \Sigma_{U_X}^{-1}\Sigma_{U_XU_Y}/\Sigma_{U_Y} = \mu^\t \delta (\delta^\t \delta + \mathrm{diag}(\sigma_{1}^2,\sigma_{2}^2))^{-1}{\delta}^\t \mu /(\mu^\t \mu + 1)$.
For each of the 10000 generated models we simulate $n=50$ observations and compute the PULSE and benchmark estimators and repeat this $N=5000$ times to estimate the performance measures.

<Ref> shows
the relative change in the determinant and trace of the MSE matrix and the Euclidean norm of the bias vector.
Similarly to the univariate setup,
PULSE seems to perform better than Fuller(1) and Fuller(4) in terms of the determinant and trace for
settings with weak
confounding (small $\|\rho\|_2$)
and weak instruments (small $\lambda_{\min}(\hat E_NG_n)$).
Most of the MSE matrices do not allow for an ordering:
PULSE is MSE superior to Fuller(1),
Fuller(4), and
9.2%, 4.6% and $1\%$ of the cases, while
the MSE matrices are not ordered
90.8%, 95.4% and 95.8% of the cases. Note that both Fuller(1) and Fuller(4) is never MSE superior to PULSE.
In contrast to the univariate setup, there are  models with very  weak instruments for which Fuller outperforms PULSE; these models seems to be exclusively with strong confounding.
We also see models with strong confounding and moderate to strong instrument strength where the PULSE is superior and models with weak confounding where PULSE is inferior. Hence, the degree of confounding $\|\rho\|_2$  does not completely characterize whether or not PULSE is superior to the Fuller estimators in terms of MSE performance measures in the multi-dimensional setting.
In regards to the bias we see that both Fuller estimators are less biased than PULSE for all but a few models with very weak instruments. Furthermore, PULSE is for models with strong confounding less biased than OLS but has comparable bias for models with small to moderate confounding.

Illustrations of the relative change in the determinant (left) and trace (middle) of the MSE matrix and the Euclidean norm of the bias vector (right) (a positive relative change means that PULSE is better).  Each of the 10000 models corresponds to a point which is color-graded according the the value of $\|\rho\|_2$ (which indicates the strength of confounding), see <Ref>. PULSE tends to outperform the Fuller estimators for weak instruments and weak confounding.
The vertical dotted line at $\log(15.5)$ corresponds to a rejection threshold for weak instruments based on relative change in bias for Fuller estimators <cit.>. Note that the lowest possible negative relative change is $-1$.

We also conducted the above simulation experiment for $\gamma=(1,1)$ and $\gamma=(-1,1)$. The results (not shown but available in the folder 'Plots' in the code repository) are similar to the case $\beta = (0,0)$ and the above observations still apply. <Ref> shows the results of additional experiments,
where we consider, e.g., PULSE with $p_{\min} = 0.1$.

§.§.§ Fixed confounding multivariate experiment.

In the varying confounding experiment, we saw that when  $\|\rho\|_2$ is small then the majority of the simulated models had PULSE superior to Fuller(1) and Fuller(4) in terms of the determinant and trace of MSE. However, we also saw models with large $\|\rho\|_2$ where PULSE was still superior and models with small $\|\rho\|_2$ where PULSE was inferior.
In this experiment, we will investigate this further by fixing the confounding strength $\|\rho\|_2$ and investigating other model aspects that affect which estimator is superior. That is, we consider models with structural assignments given by

\begin{align*}
A := N_A \in \mathbb{R}^2, \quad
X :=  \xi^\t A +U_X\in \mathbb{R}^2 ,  \quad
Y :=  \gamma^\t X + U_Y \in \mathbb{R},
\end{align*}

for some $\xi = \in \R^{2\times 2}$ and independent noise innovations $(N_A,(U_X,U_Y))$. We let $\gamma=(0,0)$ and fix the noise innovations for $A$ with distribution $N_A \sim  \mathcal{N} (0, I )$. We let

\begin{align*}
\begin{pmatrix}
U_X \\ U_Y
\end{pmatrix} \sim  \cN \lp \begin{pmatrix}
0 \\ 0 \\ 0
\end{pmatrix}, \begin{pmatrix}
1 & \eta & \phi_1 \\
\eta & 1 & \phi_2\\
\phi_1 & \phi_2 & 1
\end{pmatrix}\rp,
\end{align*}

for some $\eta,\phi_1,\phi_2\in[0,1)$. With this noise structure we have that
\|\rho \|_2^2 = (\phi_1^2 +\phi_2^2-2\eta \phi_1\phi_2)/(1-\eta^2),
and when $\phi=\phi_1=\phi_2$ it holds that $\|\rho \|_2^2  =  2\phi^2/(1+\eta)$.
We randomly generate 5000 copies of $\xi$ with each entry drawn by $\text{Unif}(-2,2)$ distribution. For each model, that is, each combination of selected noise-parameter values and $\xi$, we simulate $n=50$ observations and compute the estimators. This is repeated $N=5000$ times to estimate the performance measures.

In <Ref> we have illustrated the relative change in the performance measures when comparing
PULSE to Fuller(4). For setups with weak confounding ($\|\rho\|_2=0.2$), it is seen that if instruments are sufficiently weak ($\lambda_{\min}(\hat{E}_N(G_n))\leq 15.5$), then PULSE is superior to Fuller(4) in terms of both the determinant and trace performance measures. For setups with larger confounding there are still models where PULSE is superior but the characterization of superiority by weakness of instruments is no longer valid.

In <Ref> the percentage of models for which PULSE is superior to Fuller(4) in terms of the MSE partial ordering, determinant and trace performance measures is presented. It is seen that setups with identical $\|\rho\|_2$ does
not yield
similar comparisons between PULSE and Fuller(4).

MSE superiority

to  X[c] X[c] X[c] X[c] | X[r] X[r] X[c]
4cModel Parameters      3cPULSE Superiority (%)

$\|\rho\|_2$     	$\eta$     $\phi_1$     $\phi_2$     MSE     determinant     trace
0.20     0.80     0.19     0.19     48.46     85.52     86.74

0.20     0.20     0.15     0.15     32.34     98.66     92.16

0.50     0.80     0.47     0.47     1.60     13.04     19.80

0.50     0.20     0.39     0.39     0.76     19.86     27.86

0.80     0.80     0.76     0.76     0.14     7.48     12.80

0.80     0.20     0.62     0.62     0.06     7.64     15.50

Note: The rows show different  noise-parameter values for the different experimental setups. The last three columns describe the percentage of models (out of the 5000 randomly generated models) for which PULSE (with $p_{\min}=0.05$) is superior to Fuller(4) in terms of the MSE partial ordering, determinant and trace performance measures. Whenever PULSE is not superior to Fuller(4) in terms of the MSE partial ordering the MSE matrices are not comparable.

For any two setups with identical confounding strength $\|\rho\|_2$ we see that decreasing $\eta$ yields a larger percentage of models for which PULSE is superior in terms of the determinant and trace. Furthermore, we see that decreasing $\|\rho\|_2$ (for fixed $\eta$) has a similar effect. Thus, it seems that both $\rho$ and $\eta$ negatively influences the size of the parameter space of $\xi$ for which PULSE is superior to Fuller(4) in terms of both the determinant and trace performance measures. However, superiority with respect to the partial ordering of the MSE matrices does not exhibit similar behaviour. Decreasing $\|\rho\|_2$ (for fixed $\eta$) still leads to a percentage increase but decreasing $\eta$ (for fixed $\|\rho\|_2$) leads to a percentage decrease, of models for which PULSE is superior to Fuller(4).

Illustrations of the relative change from PULSE to Fuller(4) in the determinant and trace of the MSE matrix and the Euclidean norm of the bias vector. 	 The vertical dotted line at $\log(15.5)$ corresponds to a rejection threshold for weak instruments based on relative change in bias for Fuller estimators <cit.>.

§.§ Under-identified setup

In an under-identified setup the causal parameter is not identified by instrumental variable methods. Instead the usual two stage least square procedure, $\argmin_\alpha l_\mathrm{IV}(\alpha)$, yields an entire linear solution space of coefficients that renders the regression residuals uncorrelated with the instruments. The causal coefficient lies within this solution space but we are unable to identify it. In the under-identified setup, the population PULSE coefficient is the point in the solution space which provides the best mean squared prediction error. That is, the population PULSE coefficient is given by

\begin{align*}
\alpha^*= \argmin_{\alpha: E[A(Y-Z\alpha)]=0} E[(Y-Z\alpha)^2] = \argmin_{\alpha:l_{\mathrm{IV}}(\alpha)=0}l_{\mathrm{OLS}}(\alpha).
\end{align*}

The PULSE estimator in the under-identified setup remains unchanged from the exposition in the main paper.
Here, the function $l_{\mathrm{IV}}^n$ does not have a unique solution but we can define a modified TSLS estimator

\begin{align*}
\hat \alpha_{\mathrm{TSLS.mod}}^n := \lim_{\kappa\uparrow 1}\alpha_{\mathrm{K}}^n(\kappa) = \argmin_{\alpha: l_{\mathrm{IV}}^n(\alpha)=0} l_{\mathrm{OLS}}^n (\alpha).
\end{align*}

The modified TSLS estimator is the minimum of a quadratic
subject to a feasible linear constraint, and can be computed efficiently using QP solvers.

§.§.§ Underindentified example
Consider an under-identified setup with structural assignments given by

\begin{align*}
A &:= \ep_A, \quad \quad H := \ep_H, \quad \quad X_1 := \eta A + \delta_1H + \ep_1,\\
Y &:=	\beta X_1 + \delta_2 H + \ep_Y,\quad \quad X_2 := \gamma Y + \ep_2,
\end{align*}

with $(\ep_A,\ep_H,\ep_Y,\ep_1,\ep_2)\sim \cN(0,I_5)$. The causal graph of this structural equation model is illustrated in <Ref>.

Causal graph of the under-indentified setup in <Ref> Here, $H$ is hidden and the causal parameter $\beta$ is, in general, not identifiable from the distribution over $(A, X_1, X_2, Y)$.
Existing methods in machine learning try to find invariant sets of covariates (i.e., sets $S$ that, after regressing $Y$ on $X_S$, yield residuals which are uncorrelated with $A$). In this example, no such set exists. PULSE finds a solution and outputs a vector with non-zero coefficients for $X_1$ and $X_2$.

In general, the causal parameter $\beta$ is not identifiable.
Existing methods <cit.> propose to look for invariant sets that yield residuals which are uncorrelated with $A$ after regressing $Y$ on that set. In general, because of the hidden variable $H$, no such sets exist either.
The best predictive model under all invariant models, however, is still well-defined.
To see this, let us derive the population PULSE coefficient

\begin{align*}
\alpha^* = \argmin_{\alpha: l_{\mathrm{IV}}(\alpha)=0} E[(Y-\alpha_1X_1-\alpha_2X_2)^2] .
\end{align*}

We know that a necessary and sufficient condition for $l_{\mathrm{IV}}(\alpha)=0$ is that $\Corr(Y-\alpha_1X_1 -\alpha_2X_2,A)=0$. We have

\begin{align*}
Y-\alpha_1X_1-\alpha_2X_2 &=  Y-\alpha_1X_1-\alpha_2(\gamma Y + \ep_2) \\
&= (1-\alpha_2 \gamma) (\beta X_1 + \delta_2 H + \ep_Y) -\alpha_1X_1-\alpha_2 \ep_2 \\
&= (\beta-\alpha_1-\alpha_2\gamma \beta) X_1 + (1-\alpha_2\gamma)\delta_2 H + (1-\alpha_2\gamma)\ep_Y - \alpha_2 \ep_2.
\end{align*}

As $\eta\not = 0$, the regression residuals are uncorreleted with $A$ if and only if $  \alpha_1 = (1-\alpha_2\gamma) \beta$. Hence,

\begin{align*}
\alpha^* &= \argmin_{\alpha: \alpha_1 = (1-\alpha_2\gamma)\beta} E[((1-\alpha_2\gamma)\delta_2 H + (1-\alpha_2\gamma)\ep_Y - \alpha_2 \ep_2)^2] \\
&= \argmin_{\alpha: \alpha_1 = (1-\alpha_2\gamma)\beta} (1-\alpha_2\gamma)^2\delta_2^2 \mathrm{Var}(H) + (1-\alpha_2\gamma)^2 \mathrm{Var}(\ep_Y) + \alpha_2^2 \mathrm{Var}(\ep_2).
\end{align*}

The latter function is convex in $\alpha_2$, so the minimum is attained in a stationary point. We have that

\begin{align*}
&\frac{\partial}{\partial \alpha_2}   (1-\alpha_2\gamma)^2\delta_2^2 \mathrm{Var}(H) + (1-\alpha_2\gamma)^2 \mathrm{Var}(\ep_Y) +\alpha_2^2 \mathrm{Var}(\ep_2) \\
&= 2\left[ \alpha_2(\mathrm{Var}(\ep_2)+\gamma^2\delta_2^2 \mathrm{Var}(H) + \gamma^2 \mathrm{Var}(\ep_Y)) - \gamma \delta_2^2 \mathrm{Var}(H) - \gamma\mathrm{Var}(\ep_Y)    \right]=0,
\end{align*}

if and only if

\begin{align*}
\alpha_2(\mathrm{Var}(\ep_2)+\gamma^2\delta_2^2 \mathrm{Var}(H) + \gamma^2 \mathrm{Var}(\ep_Y)=   \gamma \delta_2^2 \mathrm{Var}(H) + \gamma\mathrm{Var}(\ep_Y).
\end{align*}

\begin{align} \label{eq:alst}
\alpha_2^* &= \frac{(\mathrm{Var}(\ep_Y)+\delta_2^2\mathrm{Var}(H))\gamma }{\mathrm{Var}(\ep_2)+(\mathrm{Var}(\ep_Y)+\delta_2^2\mathrm{Var}(H))\gamma^2} = \frac{(1+\delta_2^2)\gamma}{1+(1+\delta_2^2)\gamma^2 };
\quad  \alpha_1^* = (1-\alpha_2^*\gamma)\beta.
\end{align}

We now generate
models by randomly drawing the model coefficients using $\alpha \sim \mathrm{Unif}(1,2)$, $\delta_1 \sim \mathrm{Unif}(1,2), \delta_2 \sim \mathrm{Unif}(1,2), \gamma \sim \mathrm{Unif}(1,2)$ and $\eta \sim \mathrm{Unif}(0.1,1)$
and compute the corresponding population quantities according to Equation (<ref>).

For different sample sizes, we then simulate data sets from such models and compute the PULSE estimator.
<Ref> shows the trace of the estimated MSE of the PULSE estimator (with $p_{\min}=0.05$)
when comparing to the population quantity derived above.
For each model and sample size, the MSE is estimated based on 100 repetitions. As sample size increases, the MSE indeed approaches the population quantity.

Illustration of the trace of the estimated MSE matrix of the PULSE estimator in the under-identified setup based on 100 repetitions. PULSE converges towards the population quantities computed in Equation (<ref>).

As a comparison, we also implemented the TSLS modification from Equation (<ref>).
Similarly to the identified setups, the TSLS modification may come with poor finite sample properties, in particular for weak instruments and small sample size. Indeed, in this example
we observe that PULSE has superior MSE properties for small sample sizes. For example, the trace MSE for the PULSE estimator is on average (over 1000 random models) 50%  lower than the trace MSE of the modified TSLS estimator for a sample size of 50.

§ EMPIRICAL APPLICATIONS

We now consider three classical
instrumental variable applications (see 2 and 16 for discussions on the underlying assumptions).

<ref> “Does compulsory school attendance affect schooling and earnings?” by [10]. This paper investigates the effects of education on wages. The endogenous effect of education on wages are remedied by instrumenting education on quarter of birth indicators.

<ref> “Using geographic variation in college proximity to estimate the return to schooling” by [17]. This paper also investigates the effects of education on wages. In this paper education is instrumented by proximity to college indicator.

<ref> “The colonial origins of comparative development: An empirical investigation” by [1]. This paper investigates the effects of extractive institutions (proxied by protection against expropriation) on the gross domestic product (GDP) per capita. The endogeneity of the explanatory variables are remedied by instrumenting protection against expropriation on early European settler mortality rates.

For each study, we replicate the OLS and TSLS estimates of these studies and provide in addition the corresponding Fuller(4) (see
and PULSE estimates.
Since we do not have access to interventional data,
we cannot directly test the distributional
robustness properties discussed in Section <ref>.
For the third study, however, the exogenous variable is continuous, which allows us to investigate
distributional robustness empirically
by holding out data points with extreme values of the exogenous variable and predict on these held-out data.

For the remainder of this section we use the PULSE estimator with $p_{\min}=0.05$ and the test scaling-scheme that renders the test equivalent to the asymptotic version of the Anderson-Rubin test (see <Ref>).
Code replicating this analysis is available on GitHub.[<https://github.com/MartinEmilJakobsen/PULSE/tree/master/Empirical_Applications>]

§.§ [10]

The dataset of [10] consists, in part, of 1980 US census data of 329,509 men born between 1930–1939. The endogenous target of interest is log weakly wages and the main endogenous regressor is years of education is instrumented on year and quarter of birth indicators. We consider four models M1–M4 corresponding to the models presented in column (1)–(8) in Table 5 of [10]. Model M1 is given by the structural reduced form equations

\begin{align*}
\log \mathrm{weakly \, wage} &= \mathrm{educ}\cdot \gamma + \sum_{i} \mathrm{YR}_i \cdot \beta_i + U_1, \\
\mathrm{educ} &= \sum_{i} \mathrm{YR}_i \cdot  \delta_i + \sum_{i,j} \mathrm{YR}_i\cdot \mathrm{QOB}_j \cdot \delta_{i,j} + U_2,
\end{align*}

where $\mathrm{educ}$ is years of education, $(\mathrm{YR}_i)$ is year of birth indicators and $(\mathrm{QOB}_j)$ is quarter of birth indicators. Model M2 is given by M1 with the additional included exogenous regressors of age and age-squared. Models M3 and M4 are given by model M1 and M2, respectively, with additional included exogenous indicators describing race, marital status, metropolitan area and eight regional indicators. All models are over-identified, instrumenting education on a total of 30 binary instruments.

<Ref> shows the OLS and TSLS estimates, as well as the Fuller(4) and PULSE estimates for the linear effect of education on log weakly wages.  In all models the PULSE estimates coincide with the OLS estimates.

The estimated return of education on log weakly wage.

to r c c c c c c c
Model     OLS     TSLS     FUL     PULSE     Message     Test     Threshold

M1    0.0711     0.0891     0.0926     0.0711     OLS Accepted     26.9231     55.7585

M2    0.0711     0.0760     0.0739     0.0711     OLS Accepted     23.1512     55.7585

M3    0.0632     0.0806     0.0835     0.0632     OLS Accepted     23.7903     68.6693

M4     0.0632     0.0600     0.0555     0.0632     OLS Accepted     19.5856     68.6693

Point estimates for the return of education on log weakly wage. The OLS and TSLS values coincide with the ones in
Table V of 10.
The right columns show the values of the test statistic (evaluated in the PULSE estimates) and the test rejection thresholds.
For all models, the OLS is accepted and the PULSE coincides with the OLS.

§.§ [17]

The dataset of [17] consists of a US National Longitudinal Survey of Young Men spanning from 1966 to 1981. The subset of interest consists of 3010 observations
for which there is recorded a valid wage and education level in a 1976 interview.
The endogenous target of interest is log hourly wages and the main endogenous regressor is years of education.
Proximity to a four year college, recorded in 1966, is used as an instrument.
We consider two models, M1 and M2, corresponding to models in Panel B, column (5) and (6) of Table 3 <cit.>
, respectively.
Model M1 is given by regressing the target, log hourly wages, on included exogenous  indicators of race, metropolitan area and region; the included endogenous regressors are years of education, work-experience and work-experience-squared. The endogenous regressors are instrumented by the excluded exogenous variables age, age-squared and indicator of proximity to college.	In model M2, we have model M1 with the addition of several exogenous indicators of parents education level.

<Ref> shows the OLS and TSLS estimates, as well as the Fuller(4) and PULSE estimates for the linear effect of education on log hourly wages.
Again, in all models the OLS estimates are not rejected by the Anderson-Rubin test. Hence, all PULSE estimates coincide with the OLS estimates.

The estimated return of education on log hourly wages.

to r c c c c c c c
Model     OLS     TSLS     FUL     PULSE     Message     Test     Threshold

M1    0.0747     0.1224     0.1156       0.0747     OLS Accepted     1.2218     26.2962

M2    0.0726     0.1324     0.1283      0.0726     OLS Accepted     1.7095     43.7730

Point estimates for the return of education on log hourly wage. The OLS and TSLS values coincide with the ones shown in
Table 3 of 17.
The right columns show the values of the test statistic (evaluated in the PULSE estimates) and the test rejection thresholds.
For all models, the OLS is accepted and the PULSE coincides with the OLS.

§.§ [1]

In <Ref> of the main paper we describe the data and models of [1]. Furthermore, we replicate the OLS and TSLS estimates and presented the corresponding Fuller(4) and PULSE estimates.

To investigate
distributional robustness,
we conduct an out-of-sample
mean squared prediction error (MSPE) analysis on a mean-centered dataset of the just-identified identified model M1.
This is the simplest model proposed in [1] but the MSPE robustness property of <Ref>
is robust to model misspecifications; see <Ref>.
We do not have access to interventional data.
for different values of $n_{\mathrm{test}} \in \mathbb{N}$, that is,
for each $n_{\mathrm{test}}\in\{4,8,...,32\}$,
we remove the data points with the  $n_{\mathrm{test}}/2$ lowest and $n_{\mathrm{test}}/2$ highest settler mortality rates.
We then fit the OLS, TSLS, PULSE and Fuller(4) on the remaining $64-n_{\mathrm{test}}$ observations and compute the out-of-sample MSPE on the $n_{\mathrm{test}}$ held-out observations,
measuring the model's ability to generalize.

The instrument has a larger variance on the held-out data and the population robustness property of K-class estimators (see <Ref>)
that PULSE and Fuller(4)
might generalize slightly better than OLS or TSLS.[Here, we consider a just-identified model, so the Fuller(4) K-class parameter $\kappa$ lies in $(0,1)$.]
The results of this analysis is summarised in  <Ref>.
Indeed, we see that the
OLS is optimal for a small number of held-out data points (when little generalization is required)
and that for an
number of held-out data points,
PULSE and FULLER(4) outperform the other estimators in terms of MSPE.

For comparison, we also consider
random sample splits, i.e., taking out a random subset of the dataset rather.
Here, no generalization is required and as expected,
OLS performs better than the other estimates, see <Ref>.
The MSPE is minimized by
OLS, PULSE, Fuller(4), and TSLS
65.9%, 21.8%, 6.1%, and  6.2% of the cases, respectively.

log GPD MSPE orderings on random sample splits.

to r | r r r r
MSPE   4cOutperforms

OLS     PULSE     FUL      TSLS

OLS        65.9%      79.7%     85.3%

PULSE     34.1%        87.7%     90.5%

FUL     20.3%     12.3%       93.8%

TSLS     14.7%     9.5%     6.2%

The table shows generalization performance for different estimators on model M1 of [1].
The data set is split randomly into a subset of
90% of the data (that is, 58 observations) and the MSPE for the OLS, PULSE, Fuller(4), and TSLS
are calculated on the remaining 10% of the data.
This procedure is repeated 1000 times.
The table shows how often the estimators outperform each other. E.g.,
OLS has lower MSPE than TSLS in 85.3% of the cases.
Here, no generalziation is needed
and, as expected, the OLS performs best.

log GPD MSPE on extreme out-of-sample instrument observations.

to r | c c c c| c c  | c c c c
4c|Estimated coefficient     2c|K-class $\kappa$     4cMSPE

$n_{\mathrm{test}}$   OLS   TSLS   PULSE   FUL   PULSE   FUL   OLS   TSLS   PULSE   FUL

4     0.5015     1.1592     0.7852     0.9509     0.8286     0.9322      0.2072     2.0358     0.3211     0.8613

6     0.5113     0.9441     0.6590     0.8313     0.7075     0.9298      0.8282     1.5889     0.8692     1.2034

8     0.5017     0.9433     0.6287     0.8150     0.6781     0.9273      0.7800     1.5331     0.7796     1.0961

10     0.4978     0.8795     0.5810     0.7717     0.5733     0.9245     0.7018     1.0850     0.6769     0.8479

12     0.4901     0.8693     0.5390     0.7512     0.4407     0.9216     0.6605     1.0346     0.6357     0.7788

14     0.4748     0.8439     0.4748     0.7091     0.0000     0.9184     0.6562     0.8910     0.6562     0.6722

16     0.4581     0.7655     0.4581     0.6359     0.0000     0.9149     0.7290     0.7581     0.7290     0.6573

18     0.4247     0.6861     0.4247     0.5451     0.0000     0.9111     0.7476     0.6263     0.7476     0.6263

20     0.3883     0.8604     0.3883     0.6096     0.0000     0.9070     0.8886     0.8354     0.8886     0.6632

22     0.3789     0.8867     0.3789     0.6046     0.0000     0.9024     0.8285     0.8315     0.8285     0.6072

24     0.3784     0.7016     0.3784     0.5450     0.0000     0.8974     0.9152     0.7251     0.9152     0.7334

26     0.4156     0.8753     0.5240     0.6723     0.6682     0.8919     0.8794     1.0333     0.7957     0.8012

28     0.4155     0.7867     0.4676     0.6306     0.4789     0.8857     0.8340     0.8530     0.7880     0.7468

30     0.4016     0.8725     0.4710     0.6278     0.5754     0.8788     0.7989     0.9223     0.7370     0.6991

32     0.4087     0.9103     0.4893     0.6228     0.6344     0.8710     0.7823     0.9880     0.7225     0.7016

The table shows generalization performance for different estimators on model M1 of [1].
We remove the $n_{\mathrm{test}}$
observations with the most extreme values of settler mortality,
OLS, TSLS, PULSE and Fuller(4)
on the $64-n_{\mathrm{test}}$ samples, and compute the MSPE on the $n_{\mathrm{test}}$ held-out samples (four right-most columns).
Indeed, in particular for larger values of $n_{\mathrm{test}}$, where more generalization is needed, PULSE and Fuller(4) outperform OLS and TSLS in the majority of cases.
The columns “Estimated coefficient” show the estimates for the linear effect of average expropriation risk on log GPD of each estimation method.
The column “K-class $\kappa$” shows K-class $\kappa$ parameters for both the PULSE and Fuller(4) estimates; EQ is computed  according to  $\mathrm{Var}(\mathrm{\text{out-of-sample}})=\mathrm{Var}(\mathrm{\text{in-sample}})/(1-\kappa_{\mathrm{EQ}})$.

§ WEAK INSTRUMENTS

There is a wide variety of attempts to quantify weakness of instruments, see e.g. 9  and  84 for an overview.  Heuristically, the presence of weak instruments in a instrumental variable setup refers to the notion that the causal effects of the instruments onto regressors are weak relative to the noise variance of the regressors. This strength of the instruments has direct effects on the finite sample behavior of instrumental variable estimators. For simplicity consider a mean zero collapsed causal structural model with no included exogenous variables entering the equation of interest, that is,

\begin{align} \label{eq:weakInstrumentSetup}
\begin{split}
Y= \gamma^\t X + U_Y, \qquad X= \xi^\t A + U_X,
\end{split}
\end{align}

where $A\in \R^q$ are the collection of exogenous variables and the noise variables $U_X$ and $U_Y$ are possibly correlated. Let $\fA,\fX,\fY$ be a  $n$-sample data matrices of i.i.d. realizations of the system in <Ref>. A key statistic used to quantify weakness of instruments is the concentration matrix  given by
$ \mu_n = \Sigma_{U_X}^{-1/2} \xi^\t \fA^\t \fA \xi \Sigma_{U_X}^{-1/2}$,
where $\Sigma_{U_X}$ is the variance matrix of $U_X$. This statistic turns up in numerous different aspect of the finite sample properties of the two-stage least square estimator.
that the one-dimensional analogue of $ \mu_n$ under deterministic instruments and normal distributed noise variables directly influences the goodness of approximating a finite sample standardized two-stage least square estimator by its Gaussian asymptotic distribution.
He argues that for large concentration parameters the Gaussian approximation is good.
The concentration parameter can also be connected to approximate bias of the two-stage least squares estimator. Under assumptions similar to the above,   showed that an approximate (to the order of $\mathcal{O}(n^{-1})$) finite sample bias of the two-stage least square estimator is inversely proportional to $ \mu_n$. Note that the concentration matrix $\mu_n$ is not observable, but may be approximated by
$\hat \mu_n = \hat{\Sigma}_{U_X}^{-1/2}\fX^\t P_{\fA} \fX \hat{\Sigma}_{U_X}^{-1/2}$,
where  $\hat{\Sigma}_{U_X} =  \frac{1}{n-q} \fX^\t P_{\fA}^\perp \fX$ is an estimator of the variance matrix of $U_X$ and $P_\fA \fX$ is the ordinary least square prediction of $\fA \xi$.   Now define

\begin{align*}
G_n := \frac{\hat \mu_n}{q}=\frac{ \hat{\Sigma}_{U_X}^{-1/2}\fX^\t P_{\fA} \fX \hat{\Sigma}_{U_X}^{-1/2}}{q},
\end{align*}

which can be seen
as a multivariate
first-stage $F$-statistic for testing the hypothesis $H_0:\xi=0$. That is, when $X\in \R$, then $G_n = \frac{n-q}{q} \frac{\fX^\t P_\fA \fX }{\fX^\t \fX - \fX^\t P_\fA \fX }$
is recognized as the F-test for testing $H_0$. 85 propose to reject the
hypothesis of a presence of weak instruments if the test-statistic $\lambda_{\min}(G_n)$, the smallest eigenvalue of $G_n$, is larger than a critical value that, for example, depends on
how much bias you allow your estimator to have.  Prior to this $G_n$ had been  used to test under-identifiability in the sense that the concentration matrix is singular <cit.>, while the former uses a small minimum eigenvalue of $G_n$  as a proxy for the presence of weak instruments in identified models. From the work of 83 a frequently appearing rule of thumb for instruments being non-weak is that the F-statistic $G_n$ ($\lambda_{\min}(G_n)$ in higher dimensions) is larger than 10.  A more formal justification of this rule is due to  85 who showed (under weak-instrument asymptotics) that it approximately (in several models) corresponds to a 5% significance test that the bias of TSLS is at most 10% of the bias of OLS.

We can, under further model simplification, strengthen the intuition on how the concentration matrix $G_n$ and especially the minimum eigenvalue $\lambda_{\min}(G_n)$ governs the weakness of instruments. To this end assume that $\text{Var}(U_X)= \Sigma_{
U_X}=I$  and note that $\hat \mu_n$ is approximately proportional to the Hessian of the two-stage least squares objective function. That is, $\hat \mu_n  \approx \Sigma_{U_X}^{-1/2} \fX^\t P_{\fA} \fX \Sigma_{U_X}^{-1/2} = \fX^\t \fA ( \fA^\t \fA)^{-1} \fA^\t \fX \propto H(l_{\text{IV}}^n)$. Hence, we have that $\lambda_{\min}(G_n)$ is approximately proportional to the curvature of two-stage least squares objective function in the direction of least curvature. Thus, if $\lambda_{\min}(G_n)$ is small,
then, heuristically,
the objective function $l_{\text{IV}}^n$ has weak identification in the direction of the corresponding eigenvector. That is, changes to the point estimate of $\beta$ away from the two-stage least square solution in this direction does not have a strong effect on the objective value.
Finally, the weak instrument problem is a small sample problem. To this end note that
n^{-1} G_n= n^{-1}\hat{\Sigma}_{U_X}^{-1/2} \fX^\t P_{\fA} \fX \hat{\Sigma}_{U_X}^{-1/2} \convp \text{Var}(U_X)^{-1/2} \xi^\t \text{Var}(A)^{-1} \xi  \text{Var}(U_X)^{-1/2}$,
by the continuity of the minimum eigenvalue operator, we have that $\lambda_{\min}(G_n) \convp \i$.

§ ADDITIONAL SIMULATION EXPERIMENTS

§.§ Additional illustrations for the univariate experiment

Illustrations of the relative change in the absolute value of the mean bias (a positive relative change means that PULSE is better). The vertical dotted line corresponds to the rule of thumb for classifying instruments as weak, i.e., an  F-test rejection threshold of 10. The first stage F-test for $H_0:\bar \xi=0$, i.e.,  for the relevancy of instruments,
at a significance level of 5%,