and $\hat C_i,\hat C_j$ respectively. Similar results apply to the sine of the angle between an estimated generalized eigenvector and the true one [@demmel2000templates] Section 5.7.

Regularization
--------------

An additional concern with finite samples is that $\hat C_m$ may not be full rank as we have assumed until now. In particular, if there are fewer than $d$ examples in class $m$, then $\hat C_m$ is guaranteed to be rank deficient. When such a matrix appears in the denominator of , estimation of the eigenvectors can be unstable and overly sensitive to the sample at hand. A common solution [@platt2010translingual] is to regularize the denominator matrix by adding a multiple of the identity to the denominator, i.e., maximizing $$R^\gamma_{ij}(v)=\frac{v^\top \hat  C_i v}{v^\top ( \hat C_j +\gamma I)v},
\label{eqn:regsignoise}$$ which is equivalent to maximizing equation with an additional upper-bound constraint on the norm of $v$. We typically set $\gamma$ to be a small multiple of the average eigenvalue of $\hat C_j$ [@friedman1989regularized] which can be easily obtained as the trace of $\hat C_j$ divided by $d$. In Section \[sec:experiments\] we find this strategy empirically effective.

An Algorithm
------------

$S=\{(x_i,y_i)\}_{i=1}^n$, $\theta \geq 0$ and $\gamma \geq 0$ $F \leftarrow \emptyset$ Solve $\hat C_i 