magnitude and the direction of the principal eigenvector
$\mathbf{v=v}_{1}+\mathbf{v}_{2}$ of the joint covariance matrix $\mathbf{Q}$
and the inverted joint covariance matrix $\mathbf{Q}^{-1}$ are both functions
of differences between joint variabilities of normal _extreme vectors_
$\mathbf{x}_{1_{i\ast}}\mathbf{\sim}p\left(\mathbf{x};\boldsymbol{\mu}_{1},\mathbf{\Sigma}_{1}\right)$
and
$\mathbf{x}_{2_{i\ast}}\mathbf{\sim}p\left(\mathbf{x};\boldsymbol{\mu}_{2},\mathbf{\Sigma}_{2}\right)$.

Moreover, given the novel principal eigen-coordinate transform method
expressed by Theorem 7.1 and Corollary 7.2, we are guaranteed the existence of
an exclusive principal eigen-coordinate system—which is the principal part of
an equivalent representation of a certain quadratic form—that is the solution
of an equivalent form of the vector algebra locus equation of the geometric
locus of a certain quadratic curve or surface, such that the exclusive
principal eigen-coordinate system is the principal eigenaxis of the geometric
locus of the quadratic curve or surface, so that the principal eigenaxis
satisfies the geometric locus of the quadratic curve or surface in terms of
its total allowed eigenenergy, and the uniform property exhibited by all of
the points that lie on the geometric locus of the quadratic curve or surface
is the total allowed eigenenergy exhibited by the principal eigenaxis of the
geometric locus of the quadratic curve or surface.

Furthermore, conditions expressed by Theorems 6.2 \- 6.4 substantiate Theorem
7.1 and Corollary 7.2—since Theorems 6.2 \- 6.4 guarantee the existence of the
general vector algebra locus equations of (6.1) - (6.3), wherein the principal
eigenaxis of the locus of any given quadratic curve or surface provides an
exclusive principal eigen-coordinate system for the locus of the quadratic
curve or surface, such that all of the points that lie on the locus of the
quadratic curve or surface explicitly and exclusively reference the principal
eigenaxis and also satisfy the total allowed eigenenergy exhibited by the
principal eigenaxis, at which point the principal eigenaxis satisfies the
locus of the quadratic curve or surface in terms of its eigenenergy, and the
uniform property exhibited by all of the points that lie on the locus of the
quadratic curve or surface is the eigenenergy exhibited by the principal
eigenaxis.

Therefore, let the pair of random quadratic forms $\mathbf{v}^{T}\mathbf{Qv}$
and $\mathbf{v}^{T}\mathbf{Q}^{-1}\mathbf{v}$ that jointly represent the pair
of signed random quadratic forms

$\mathbf{x}^{T}\mathbf{\Sigma}_{1}^{-1}\mathbf{x-x}^{T}\mathbf{\Sigma}_{2}^{-1}\mathbf{x}$

be solutions of a system of well-posed vector algebra locus equations.

By Theorem 7.1 and Corollaries 7.2 \- 7.3, it follows that a principal
eigenaxis $\widetilde{\mathbf{v}}\ \mathbf{=\
}\widetilde{\mathbf{v}}_{1}-\widetilde{\mathbf{v}}_{2}$ of a certain quadratic
curve or surface is the principal part of an equivalent representation of the
pair of random quadratic forms $\mathbf{v}^{T}\mathbf{Qv}$ and
$\mathbf{v}^{T}\mathbf{Q}^{-1}\mathbf{v}$, such that the principal eigenaxis
$\widetilde{\mathbf{v}}\ \mathbf{=\
}\widetilde{\mathbf{v}}_{1}-\widetilde{\mathbf{v}}_{2}$ of the quadratic curve
or surface is symmetrically and equivalently related to the principal
eigenvector $\mathbf{v=v}_{1}+\mathbf{v}_{2}$ of the joint covariance matrix
$\mathbf{Q}$ of the random quadratic form $\mathbf{v}^{T}\mathbf{Qv}$ _and_
the inverted joint covariance matrix $\mathbf{Q}^{-1}$ of the random quadratic
form $\mathbf{v}^{T}\mathbf{Q}^{-1}\mathbf{v}$, so that the principal
eigenaxis $\widetilde{\mathbf{v}}\ \mathbf{=\
}\widetilde{\mathbf{v}}_{1}-\widetilde{\mathbf{v}}_{2}$ satisfies the
quadratic curve or surface in terms of its total allowed eigenenergy
$\left\|\widetilde{\mathbf{v}}_{1}-\widetilde{\mathbf{v}}_{2}\right\|^{2}$—which
is regulated by the eigenvalues of the symmetric matrices $\mathbf{Q}$ and
$\mathbf{Q}^{-1}$ of the random quadratic forms $\mathbf{v}^{T}\mathbf{Qv}$
and $\mathbf{v}^{T}\mathbf{Q}^{-1}\mathbf{v}$, wherein the magnitude and the
direction of the principal eigenaxis $\widetilde{\mathbf{v}}\ \mathbf{=\
}\widetilde{\mathbf{v}}_{1}-\widetilde{\mathbf{v}}_{2}$ are both functions of
differences between joint variabilities of normal _extreme vectors_
$\mathbf{x}_{1_{i\ast}}\mathbf{\sim}p\left(\mathbf{x};\boldsymbol{\mu}_{1},\mathbf{\Sigma}_{1}\right)$
and
$\mathbf{x}_{2_{i\ast}}\mathbf{\sim}p\left(\mathbf{x};\boldsymbol{\mu}_{2},\mathbf{\Sigma}_{2}\right)$.

We have generalized these findings by letting the elements of $\mathbf{Q}$ and
$\mathbf{Q}^{-1}$ both describe differences between joint variabilities of
random vectors $\mathbf{x\sim}p\left(\mathbf{x};\omega_{1}\right)$ and
$\mathbf{x\sim}p\left(\mathbf{x};\omega_{1}\right)$ that belong to a
collection of two categories $\omega_{1}$ and $\omega_{2}$ of random vectors
$\mathbf{x\in}$ $\mathbb{R}^{d}$, where $p\left(\mathbf{x};\omega_{1}\right)$
and $p\left(\mathbf{x};\omega_{2}\right)$ are certain probability density
functions for the two classes $\omega_{1}$ and $\omega_{2}$ of random vectors
$\mathbf{x}$. Thereby, the magnitude and the direction of the principal
eigenvector $\mathbf{v=v}_{1}+\mathbf{v}_{2}$ of the joint covariance matrix
$\mathbf{Q}$ and the inverted joint covariance matrix $\mathbf{Q}^{-1}$ are
both functions of differences between joint variabilities of extreme vectors
$\mathbf{x}_{1_{i\ast}}\mathbf{\sim}p\left(\mathbf{x};\omega_{1}\right)$ and
$\mathbf{x}_{2_{i\ast}}\mathbf{\sim}p\left(\mathbf{x};\omega_{1}\right)$ that
belong to the classes $\omega_{1}$ and $\omega_{2}$ of random vectors
$\mathbf{x\in}$ $\mathbb{R}^{d}$.

Accordingly, let the pair of random quadratic forms
$\mathbf{v}^{T}\mathbf{Qv}$ and $\mathbf{v}^{T}\mathbf{Q}^{-1}\mathbf{v}$ be
solutions of a system of well-posed vector algebra locus equations, such that
$\mathbf{v=v}_{1}+\mathbf{v}_{2}$ is the principal eigenvector of both
$\mathbf{Q}$ and $\mathbf{Q}^{-1}$, so that the magnitude and the direction of
the principal eigenvector $\mathbf{v=v}_{1}+\mathbf{v}_{2}$ of $\mathbf{Q}$
and $\mathbf{Q}^{-1}$ are both functions of differences between joint
variabilities of extreme vectors
$\mathbf{x}_{1_{i\ast}}\mathbf{\sim}p\left(\mathbf{x};\omega_{1}\right)$ and
$\mathbf{x}_{2_{i\ast}}\mathbf{\sim}p\left(\mathbf{x};\omega_{1}\right)$ that
belong to the classes $\omega_{1}$ and $\omega_{2}$ of random vectors
$\mathbf{x\in}$ $\mathbb{R}^{d}$.

By Theorem 7.1 and Corollaries 7.2 \- 7.3, it follows that a principal
eigenaxis $\widetilde{\mathbf{v}}\ \mathbf{=\
}\widetilde{\mathbf{v}}_{1}-\widetilde{\mathbf{v}}_{2}$ of a certain quadratic
curve or surface is the principal part of an equivalent representation of the
pair of random quadratic forms $\mathbf{v}^{T}\mathbf{Qv}$ and
$\mathbf{v}^{T}\mathbf{Q}^{-1}\mathbf{v}$, such that the principal eigenaxis
$\widetilde{\mathbf{v}}\ \mathbf{=\
}\widetilde{\mathbf{v}}_{1}-\widetilde{\mathbf{v}}_{2}$ of the quadratic curve
or surface is symmetrically and equivalently related to the principal
eigenvector $\mathbf{v=v}_{1}+\mathbf{v}_{2}$ of the joint covariance matrix
$\mathbf{Q}$ of the random quadratic form $\mathbf{v}^{T}\mathbf{Qv}$ _and_
the inverted joint covariance matrix $\mathbf{Q}^{-1}$ of the random quadratic
form $\mathbf{v}^{T}\mathbf{Q}^{-1}\mathbf{v}$, so that the principal
eigenaxis $\widetilde{\mathbf{v}}\ \mathbf{=\
}\widetilde{\mathbf{v}}_{1}-\widetilde{\mathbf{v}}_{2}$ satisfies the
quadratic curve or surface in terms of its total allowed eigenenergy
$\left\|\widetilde{\mathbf{v}}_{1}-\widetilde{\mathbf{v}}_{2}\right\|^{2}$—which
is regulated by the eigenvalues of the symmetric matrices $\mathbf{Q}$ and
$\mathbf{Q}^{-1}$ of the random quadratic forms $\mathbf{v}^{T}\mathbf{Qv}$
and $\mathbf{v}^{T}\mathbf{Q}^{-1}\mathbf{v}$, wherein the magnitude and the
direction of the principal eigenaxis $\widetilde{\mathbf{v}}\ \mathbf{=\
}\widetilde{\mathbf{v}}_{1}-\widetilde{\mathbf{v}}_{2}$ are both functions of
differences between joint variabilities of extreme vectors
$\mathbf{x}_{1_{i\ast}}\mathbf{\sim}p\left(\mathbf{x};\omega_{1}\right)$ and
$\mathbf{x}_{2_{i\ast}}\mathbf{\sim}p\left(\mathbf{x};\omega_{1}\right)$ that
belong to the classes $\omega_{1}$ and $\omega_{2}$ of random vectors
$\mathbf{x\in}$ $\mathbb{R}^{d}$.

Even more, Theorem 8.3 promises us that a geometric locus of a novel principal
eigenaxis

$\displaystyle\boldsymbol{\rho}$
$\displaystyle=\sum\nolimits_{i=1}^{l_{1}}\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}-\sum\nolimits_{i=1}^{l_{2}}\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}$
$\displaystyle=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$

provides dual representation of the discriminant function and an exclusive
principal coordinate eigen-coordinate system of the geometric locus of the
decision boundary of _any_ given minimum risk binary classification system,
along with an eigenaxis of symmetry that spans the decision space of the given
system, such that the geometric locus of the decision boundary of the system
is the geometric locus of a certain quadratic curve or surface.

Moreover, we have conducted numerous simulation studies and corresponding
analysis that substantiate Theorem 8.3, wherein a well-posed constrained
optimization algorithm finds the geometrical and statistical components of a
geometric locus of a novel principal eigenaxis [69].

Finally, conditions expressed by Axiom 4.4, Lemma 6.1, Theorem 7.1,
Corollaries 7.2 \- 7.3 and Theorem 8.3 provide us with a collective guarantee
that we can determine an _equivalent form_ of the vector algebra locus
equation of (5.4)

$\displaystyle d\left(\mathbf{x}\right)$
$\displaystyle:\mathbf{x}^{T}\mathbf{\Sigma}_{1}^{-1}\mathbf{x}-2\mathbf{x}^{T}\mathbf{\Sigma}_{1}^{-1}\boldsymbol{\mu}_{1}+\boldsymbol{\mu}_{1}^{T}\mathbf{\Sigma}_{1}^{-1}\boldsymbol{\mu}_{1}-\ln\left(\left|\mathbf{\Sigma}_{1}\right|\right)$
$\displaystyle-\mathbf{x}^{T}\mathbf{\Sigma}_{2}^{-1}\mathbf{x}+2\mathbf{x}^{T}\mathbf{\Sigma}_{2}^{-1}\boldsymbol{\mu}_{2}\mathbf{-}\boldsymbol{\mu}_{2}^{T}\mathbf{\Sigma}_{2}^{-1}\boldsymbol{\mu}_{2}+\ln\left(\left|\mathbf{\Sigma}_{2}\right|\right)=0$

_if_ we can determine _how_ to _transform_ the basis of the intrinsic
coordinate system
$\mathbf{x}^{T}\mathbf{\Sigma}_{1}^{-1}\mathbf{x-x}^{T}\mathbf{\Sigma}_{2}^{-1}\mathbf{x}$,
such that
$\mathbf{x\sim}p\left(\mathbf{x};\boldsymbol{\mu}_{1},\mathbf{\Sigma}_{1}\right)$
and
$\mathbf{x\sim}p\left(\mathbf{x};\boldsymbol{\mu}_{2},\mathbf{\Sigma}_{2}\right)$,
_into_ a locus of signed and scaled extreme vectors
$\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}$ and
$-\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}$

$\displaystyle\boldsymbol{\rho}$
$\displaystyle=\sum\nolimits_{i=1}^{l_{1}}\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}-\sum\nolimits_{i=1}^{l_{2}}\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}$
$\displaystyle=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\text{,}$

so that likelihood values and likely locations of a collection of extreme
points
$\mathbf{x}_{1_{\ast}}\mathbf{\sim}p\left(\mathbf{x};\boldsymbol{\mu}_{1},\mathbf{\Sigma}_{1}\right)$
and
$\mathbf{x}_{2_{\ast}}\mathbf{\sim}p\left(\mathbf{x};\boldsymbol{\mu}_{2},\mathbf{\Sigma}_{2}\right)$
_determine_ the _positions_ of the _basis_ of the _transformed_ intrinsic
coordinate system
$\mathbf{x}^{T}\mathbf{\Sigma}_{1}^{-1}\mathbf{x-\mathbf{x}^{T}\mathbf{\Sigma}_{2}^{-1}\mathbf{x}}$,
such that a geometric locus of a novel principal eigenaxis $\boldsymbol{\rho}$
$=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ is the solution of the
equivalent form of (5.4), at which point the geometric locus of the novel
principal eigenaxis
$\boldsymbol{\rho}=\sum\nolimits_{i=1}^{l_{1}}\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}-\sum\nolimits_{i=1}^{l_{2}}\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}$
is the principal part of an equivalent representation of a pair of random
quadratic forms $\mathbf{v}^{T}\mathbf{Qv}$ and
$\mathbf{v}^{T}\mathbf{Q}^{-1}\mathbf{v}$ associated with a joint covariance
matrix $\mathbf{Q}$ and the inverted joint covariance matrix
$\mathbf{Q}^{-1}$, such that $\mathbf{v=v}_{1}+\mathbf{v}_{2}$ is the
principal eigenvector of the joint covariance matrix $\mathbf{Q}$ and the
inverted joint covariance matrix $\mathbf{Q}^{-1}$.

Even so, how do we implement an _algorithm_ that is subject to essential
criterion in the novel principal eigen-coordinate transform method expressed
by Theorem 7.1 and Corollaries 7.2 \- 7.3?

###  9.1 A System of Well-posed Vector Algebra Locus Equations

Given the guarantees expressed by Theorem 7.1 and Corollaries 7.2 \- 7.3, take
any given random quadratic form $\mathbf{v}^{T}\mathbf{Qv}$, such that
$\left(1\right)$ $\mathbf{v=v}_{1}+\mathbf{v}_{2}$ is the principal
eigenvector of $\mathbf{Q}$ and $\mathbf{Q}^{-1}$; and $\left(2\right)$ the
elements of $\mathbf{Q}$ and $\mathbf{Q}^{-1}$ both describe differences
between joint variabilities of random vectors
$\mathbf{x\sim}p\left(\mathbf{x};\omega_{1}\right)$ and
$\mathbf{x\sim}p\left(\mathbf{x};\omega_{1}\right)$ that belong to two classes
$\omega_{1}$ and $\omega_{2}$, where $p\left(\mathbf{x};\omega_{1}\right)$ and
$p\left(\mathbf{x};\omega_{2}\right)$ are certain probability density
functions for the two classes $\omega_{1}$ and $\omega_{2}$ of random vectors
$\mathbf{x\in}$ $\mathbb{R}^{d}$.

We have discovered a system of well-posed vector algebra locus equations that
are satisfied by the pair of random quadratic forms
$\mathbf{v}^{T}\mathbf{Qv}$ and
$\mathbf{v}^{T}\mathbf{Q}^{-1}\mathbf{v}$—wherein the principal eigenvector
$\mathbf{v=v}_{1}+\mathbf{v}_{2}$ of $\mathbf{Q}$ and $\mathbf{Q}^{-1}$ is
symmetrically and equivalently related to the principal part of an equivalent
representation of the random quadratic forms $\mathbf{v}^{T}\mathbf{Qv}$ and
$\mathbf{v}^{T}\mathbf{Q}^{-1}\mathbf{v}$, such that an exclusive principal
eigen-coordinate system—structured as a locus of signed and scaled extreme
vectors $\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}$ and
$-\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}$

$\displaystyle\boldsymbol{\rho}$
$\displaystyle=\sum\nolimits_{i=1}^{l_{1}}\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}-\sum\nolimits_{i=1}^{l_{2}}\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}$
$\displaystyle=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$

is the _solution_ of the system of well-posed vector algebra locus equations,
at which point the geometric locus of the novel principal eigenaxis
$\boldsymbol{\rho}$ $=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ is
symmetrically and equivalently related to the principal eigenvector
$\mathbf{v=v}_{1}+\mathbf{v}_{2}$ of the joint covariance matrix $\mathbf{Q}$
and the inverted joint covariance matrix $\mathbf{Q}^{-1}$, such that the
geometric locus of the novel principal eigenaxis $\boldsymbol{\rho}$
$=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ is the principal eigenaxis of
the decision boundary of a minimum risk binary classification system, so that
the novel principal eigenaxis $\boldsymbol{\rho}$
$=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ satisfies the geometric locus
of the decision boundary of the minimum risk binary classification system in
terms of a critical minimum eigenenergy
$\left\|\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right\|^{2}$, such that
the total allowed eigenenergy
$\left\|\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right\|^{2}$ exhibited by
the geometric locus of the novel principal eigenaxis $\boldsymbol{\rho}$
$=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ is regulated by the eigenvalues
of the symmetric matrices $\mathbf{Q}$ and $\mathbf{Q}^{-1}$ of the pair of
random quadratic forms $\mathbf{v}^{T}\mathbf{Qv}$ and
$\mathbf{v}^{T}\mathbf{Q}^{-1}\mathbf{v}$.

Accordingly, we have discovered that the geometric locus of the novel
principal eigenaxis $\boldsymbol{\rho}$
$=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ is subject to essential
criterion in the novel principal eigen-coordinate transform method expressed
by Theorem 7.1 and Corollaries 7.2 \- 7.3, along with conditions expressed by
Theorem 8.3, such that the geometric locus of the novel principal eigenaxis
$\boldsymbol{\rho}$ $=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ is the
principal eigenaxis of the geometric locus of the decision boundary of a
minimum risk binary classification system, so that the geometric locus of the
novel principal eigenaxis $\boldsymbol{\rho}$
$=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ satisfies the geometric locus
of the decision boundary in terms of a critical minimum eigenenergy
$\left\|\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right\|_{\min_{c}}^{2}$
and a minimum expected risk
$\mathfrak{R}_{\mathfrak{\min}}\left(\left\|\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right\|_{\min_{c}}^{2}\right)$,
at which point the geometric locus of the novel principal eigenaxis
$\boldsymbol{\rho}$ $=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ provides
dual representation of the discriminant function, an exclusive principal
eigen-coordinate system of the geometric locus of the decision boundary, and
an eigenaxis of symmetry that spans the decision space of the minimum risk
binary classification system.

Even so, we realize that we cannot expect to change the positions of the basis
of _any_ given intrinsic coordinate system
$\mathbf{x}^{T}\mathbf{\Sigma}_{1}^{-1}\mathbf{x-x}^{T}\mathbf{\Sigma}_{2}^{-1}\mathbf{x}$,
such that
$\mathbf{x\sim}p\left(\mathbf{x};\boldsymbol{\mu}_{1},\mathbf{\Sigma}_{1}\right)$
and
$\mathbf{x\sim}p\left(\mathbf{x};\boldsymbol{\mu}_{2},\mathbf{\Sigma}_{2}\right)$,
in the vector algebra locus equation of (5.4) by _simply_ _rotating_ the
coordinates _axes_ of a Cartesian coordinate system.

_So_ , how do we change the positions of the basis of _any_ given intrinsic
coordinate system
$\mathbf{x}^{T}\mathbf{\Sigma}_{1}^{-1}\mathbf{x-\mathbf{x}^{T}\mathbf{\Sigma}_{2}^{-1}\mathbf{x}}$,
such that
$\mathbf{x\sim}p\left(\mathbf{x};\boldsymbol{\mu}_{1},\mathbf{\Sigma}_{1}\right)$
and
$\mathbf{x\sim}p\left(\mathbf{x};\boldsymbol{\mu}_{2},\mathbf{\Sigma}_{2}\right)$—to
which geometric loci of decision boundaries of minimum risk binary
classification systems in (5.3) are referenced—such that a geometric locus of
a novel principal eigenaxis $\boldsymbol{\rho}$
$=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ is the principal part of an
equivalent representation of a pair of random quadratic forms
$\mathbf{v}^{T}\mathbf{Qv}$ and $\mathbf{v}^{T}\mathbf{Q}^{-1}\mathbf{v}$
associated with a joint covariance matrix $\mathbf{Q}$ and the inverted joint
covariance matrix $\mathbf{Q}^{-1}$, where $\mathbf{v=v}_{1}+\mathbf{v}_{2}$
is the principal eigenvector of $\mathbf{Q}$ and $\mathbf{Q}^{-1}$, so that
the geometric locus of the novel principal eigenaxis $\boldsymbol{\rho}$
$=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ provides dual representation of
the discriminant function, an exclusive principal eigen-coordinate system of
the geometric locus of the decision boundary, and an eigenaxis of symmetry
that spans the decision space—of _any_ given minimum risk binary
classification system?

###  9.2 A Principal Eigen-coordinate Transform Algorithm

We have discovered that changing the positions of the basis of any given
intrinsic coordinate system
$\mathbf{x}^{T}\mathbf{\Sigma}_{1}^{-1}\mathbf{x-x}^{T}\mathbf{\Sigma}_{2}^{-1}\mathbf{x}$
in the vector algebra locus equation of (5.4)—is accomplished by a well-posed
constrained optimization algorithm that executes a _novel principal eigen-
coordinate transform_ , wherein a geometric locus of a novel principal
eigenaxis

$\displaystyle\boldsymbol{\rho}$
$\displaystyle=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$
$\displaystyle=\sum\nolimits_{i=1}^{l_{1}}\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}-\sum\nolimits_{i=1}^{l_{2}}\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}$

is the principal part of an equivalent representation of a pair of correlated
random quadratic forms $\mathbf{v}^{T}\mathbf{Qv}$ and
$\mathbf{v}^{T}\mathbf{Q}^{-1}\mathbf{v}$ associated with a joint covariance
matrix $\mathbf{Q}$ and the inverted joint covariance matrix
$\mathbf{Q}^{-1}$, such that $\mathbf{v=v}_{1}+\mathbf{v}_{2}$ is the
principal eigenvector of $\mathbf{Q}$ and $\mathbf{Q}^{-1}$, at which point
the geometric locus of the novel principal eigenaxis $\boldsymbol{\rho}$
$=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ provides dual representation of
the discriminant function, an exclusive principal eigen-coordinate system of
the geometric locus of the decision boundary, and an eigenaxis of symmetry
that spans the decision space—of _any_ given minimum risk binary
classification system, so that the geometric locus of the novel principal
eigenaxis $\boldsymbol{\rho}$ $=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$
satisfies the geometric locus of the decision boundary in terms of a critical
minimum eigenenergy
$\left\|\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right\|_{\min_{c}}^{2}$
and a minimum expected risk
$\mathfrak{R}_{\mathfrak{\min}}\left(\left\|\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right\|_{\min_{c}}^{2}\right)$,
such that the total allowed eigenenergy
$\left\|\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right\|_{\min_{c}}^{2}$
and the expected risk
$\mathfrak{R}_{\mathfrak{\min}}\left(\left\|\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right\|_{\min_{c}}^{2}\right)$
exhibited by the geometric locus of the novel principal eigenaxis
$\boldsymbol{\rho}$ $=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ are
regulated by the eigenvalues of the symmetric matrices $\mathbf{Q}$ and
$\mathbf{Q}^{-1}$ associated with the pair of random quadratic forms
$\mathbf{v}^{T}\mathbf{Qv}$ and $\mathbf{v}^{T}\mathbf{Q}^{-1}\mathbf{v}$.

We have determined that the algorithm finds the solution for a well-posed
inequality constrained optimization problem, known as the primal problem, such
that the geometric locus of a novel principal eigenaxis
$\boldsymbol{\rho}=\sum\nolimits_{i=1}^{l_{1}}\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}-\sum\nolimits_{i=1}^{l_{2}}\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}$
is subject to certain constraints, by using Lagrange multipliers $\psi_{i}\geq
0$ and a Lagrangian function, wherein the objective function of the novel
principal eigenaxis $\boldsymbol{\rho}$
$=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ and its constraints are
combined with each other.

Thereby, we have determined that the algorithm introduces a geometric locus of
a Wolfe dual novel principal eigenaxis

$\displaystyle\boldsymbol{\psi}$
$\displaystyle=\sum\nolimits_{i=1}^{l_{1}}\psi_{1i\ast}\frac{k_{\mathbf{x}_{1i\ast}}}{\left\|k_{\mathbf{x}_{1i\ast}}\right\|}+\sum\nolimits_{i=1}^{l_{2}}\psi_{2i\ast}\frac{k_{\mathbf{x}_{2i\ast}}}{\left\|k_{\mathbf{x}_{2i\ast}}\right\|}$
$\displaystyle=\boldsymbol{\psi}_{1}+\boldsymbol{\psi}_{2}$

that is _symmetrically_ and _equivalently_ _related to_ the geometric locus of
the primal novel principal eigenaxis

$\displaystyle\boldsymbol{\rho}$
$\displaystyle=\sum\nolimits_{i=1}^{l_{1}}\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}-\sum\nolimits_{i=1}^{l_{2}}\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}$
$\displaystyle=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$

inside a vector space that we have named the “Wolfe-dual principal
eigenspace,” and finds extrema for the _restriction_ of the primal novel
principal eigenaxis $\boldsymbol{\rho}$
$=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ _to_ the Wolfe-dual principal
_eigenspace_ , such that the scale factors $\psi_{1i\ast}$ and $\psi_{2i\ast}$
of the vector components of the novel principal eigenaxes
$\boldsymbol{\psi}=\boldsymbol{\psi}_{1}+\boldsymbol{\psi}_{2}$ and
$\boldsymbol{\rho}$ $=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ are the
fundamental unknown parameters associated with the algorithm.

We have also determined that the algorithm finds the scale factors
$\psi_{1i\ast}$ and $\psi_{2i\ast}$ by solving the Wolfe dual problem

$\max\Xi_{\boldsymbol{\psi}}\left(\boldsymbol{\psi}\right)=\mathbf{1}^{T}\boldsymbol{\psi}-\frac{\boldsymbol{\psi}^{T}\mathbf{Q}\boldsymbol{\psi}}{2}\text{,}$

where $\mathbf{Q}$ is a joint covariance matrix of a collection of two
categories $\omega_{1}$ and $\omega_{2}$ of random vectors $\mathbf{x\sim}$
$p\left(\mathbf{x};\omega_{1}\right)$ and $\mathbf{x\sim}$
$p\left(\mathbf{x};\omega_{2}\right)$, such that the Wolfe dual novel
principal eigenaxis $\boldsymbol{\psi}$ is subject to the constraints
$\boldsymbol{\psi}^{T}\mathbf{y}=0$ and $\psi_{i}\geq 0$, wherein the
inequalities $\psi_{i}>0$ only hold for certain values of $\psi_{i}$, so that
the Wolfe dual novel principal eigenaxis $\boldsymbol{\psi}$ is symmetrically
and equivalently related to the principal eigenvector
$\boldsymbol{\psi}_{\max}$ of the joint covariance matrix $\mathbf{Q}$ and the
inverted joint covariance matrix $\mathbf{Q}^{-1}$ associated with the pair of
random quadratic forms $\boldsymbol{\psi}^{T}\mathbf{Q}\boldsymbol{\psi}$ and
$\boldsymbol{\psi}^{T}\mathbf{Q}^{-1}\boldsymbol{\psi}$, _and_ is _also_
symmetrically and equivalently related to the primal novel principal eigenaxis
$\boldsymbol{\rho}$.

In addition, we have determined that each scale factor $\psi_{1i\ast}$ and
$\psi_{2i\ast}$ determines a likelihood value and a likely location—both of
which are normalized relative to length—for a correlated extreme point
$\mathbf{x}_{1_{i\ast}}$ and $\mathbf{x}_{2_{i\ast}}$, so that each scale
factor $\psi_{1i\ast}$ and $\psi_{2i\ast}$ determines the magnitude
$\left\|\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}\right\|$ and
$\left\|\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}\right\|$ of a correlated
scaled extreme vector $\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}$ and
$\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}$ that lies on the geometric
locus of the primal novel principal eigenaxis $\boldsymbol{\rho}$
$=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$, as well as a likelihood value
and a likely location for the correlated extreme vector
$k_{\mathbf{x}_{1_{i\ast}}}$ and $k_{\mathbf{x}_{2_{i\ast}}}$.

Moreover, we have determined that the solution of the Wolfe dual problem is
_implemented_ by a Wolfe-dual _eigenenergy functional_ of a minimum risk
binary classification system

$\max\Xi_{\boldsymbol{\psi}}\left(\boldsymbol{\psi}\right)=\mathbf{1}^{T}\boldsymbol{\psi}-\boldsymbol{\psi}^{T}\mathbf{Q}\boldsymbol{\psi/}2\text{,}$

wherein $\boldsymbol{\psi}^{T}\mathbf{y}=0$ and $\psi_{i}\geq 0$, such that
the _objective_ of the Wolfe-dual eigenenergy functional is to _find_ the
principal eigenvector $\boldsymbol{\psi}_{\max}$ of the joint covariance
matrix $\mathbf{Q}$ and the inverted joint covariance matrix $\mathbf{Q}^{-1}$
associated with the pair of random quadratic forms
$\boldsymbol{\psi}^{T}\mathbf{Q}\boldsymbol{\psi}$ and
$\boldsymbol{\psi}^{T}\mathbf{Q}^{-1}\boldsymbol{\psi}$, so that the geometric
locus of the Wolfe dual novel principal eigenaxis $\boldsymbol{\psi}$ is
symmetrically and equivalently related to the principal eigenvector
$\boldsymbol{\psi}_{\max}$ of the joint covariance matrix $\mathbf{Q}$
associated with the random quadratic form
$\boldsymbol{\psi}^{T}\mathbf{Q}\boldsymbol{\psi}$ and the inverted joint
covariance matrix $\mathbf{Q}^{-1}$ associated with the random quadratic form
$\boldsymbol{\psi}^{T}\mathbf{Q}^{-1}\boldsymbol{\psi}$, and is also
symmetrically and equivalently related to the geometric locus of the primal
novel principal eigenaxis $\boldsymbol{\rho}$ in such a manner that the
geometric locus of the novel principal eigenaxis $\boldsymbol{\rho}$
$=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ is the principal part of an
equivalent representation of the pair of random quadratic forms
$\boldsymbol{\psi}^{T}\mathbf{Q}\boldsymbol{\psi}$ and
$\boldsymbol{\psi}^{T}\mathbf{Q}^{-1}\boldsymbol{\psi}$.

Thereby, the geometric locus of the novel principal eigenaxis
$\boldsymbol{\rho}$ $=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ is the
principal eigenaxis of the geometric locus of the decision boundary of a
minimum risk binary classification system, so that the geometric locus of the
novel principal eigenaxis $\boldsymbol{\rho}$
$=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ satisfies the geometric locus
of the decision boundary in terms of a critical minimum eigenenergy
$\left\|\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right\|_{\min_{c}}^{2}$
and a minimum expected risk
$\mathfrak{R}_{\mathfrak{\min}}\left(\left\|\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right\|_{\min_{c}}^{2}\right)$,
such that the total allowed eigenenergy
$\left\|\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right\|_{\min_{c}}^{2}$
and the expected risk
$\mathfrak{R}_{\mathfrak{\min}}\left(\left\|\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right\|_{\min_{c}}^{2}\right)$
exhibited by the geometric locus of the novel principal eigenaxis
$\boldsymbol{\rho}$ $=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ are
regulated by the eigenvalues of the symmetric matrices $\mathbf{Q}$ and
$\mathbf{Q}^{-1}$ associated with the pair of random quadratic forms
$\boldsymbol{\psi}^{T}\mathbf{Q}\boldsymbol{\psi}$ and
$\boldsymbol{\psi}^{T}\mathbf{Q}^{-1}\boldsymbol{\psi}$, wherein the magnitude
and the direction of the novel principal eigenaxis $\boldsymbol{\rho}$
$=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ are both functions of
differences between joint variabilities of extreme vectors
$\mathbf{x}_{1_{i\ast}}\mathbf{\sim}p\left(\mathbf{x};\omega_{1}\right)$ and
$\mathbf{x}_{2_{i\ast}}\mathbf{\sim}p\left(\mathbf{x};\omega_{1}\right)$ that
belong to two classes $\omega_{1}$ and $\omega_{2}$ of random vectors
$\mathbf{x\in}$ $\mathbb{R}^{d}$.

###  9.3 Minimization of a Vector-Valued Cost Function

We have discovered that the Wolfe-dual eigenenergy functional of a minimum
risk binary classification system

$\max\Xi_{\boldsymbol{\psi}}\left(\boldsymbol{\psi}\right)=\mathbf{1}^{T}\boldsymbol{\psi}-\boldsymbol{\psi}^{T}\mathbf{Q}\boldsymbol{\psi/}2\text{,}$

wherein $\boldsymbol{\psi}^{T}\mathbf{y}=0$ and $\psi_{i}\geq 0$, uses a
_vector-valued cost function_ to find the principal eigenvector
$\boldsymbol{\psi}_{\max}$ of the joint covariance matrix $\mathbf{Q}$ and the
inverted joint covariance matrix $\mathbf{Q}^{-1}$ associated with the pair of
random quadratic forms $\boldsymbol{\psi}^{T}\mathbf{Q}\boldsymbol{\psi}$ and
$\boldsymbol{\psi}^{T}\mathbf{Q}^{-1}\boldsymbol{\psi}$, such that the
eigenenergy exhibited by both $\boldsymbol{\psi}_{\max}$ and
$\boldsymbol{\rho}$ is minimized in accordance with the eigenenergy condition

$\lambda_{1}\left\|\boldsymbol{\psi}\right\|_{\min_{c}}^{2}=\boldsymbol{\psi}_{\max}^{T}\mathbf{Q}\boldsymbol{\psi}_{\max}\equiv\left\|\boldsymbol{\rho}\right\|_{\min_{c}}^{2}\text{,}$

at which point the random quadratic form
$\boldsymbol{\psi}_{\max}^{T}\mathbf{Q}\boldsymbol{\psi}_{\max}$ is
equivalently related to the critical minimum eigenenergy
$\left\|\boldsymbol{\rho}\right\|_{\min_{c}}^{2}$ exhibited by the geometric
locus of the primal novel principal eigenaxis $\boldsymbol{\rho}$, so that the
random quadratic form
$\boldsymbol{\psi}_{\max}^{T}\mathbf{Q}\boldsymbol{\psi}_{\max}$, plus the
total allowed eigenenergy $\left\|\boldsymbol{\rho}\right\|_{\min_{c}}^{2}$
and the expected risk
$\mathfrak{R}_{\mathfrak{\min}}\left(\left\|\boldsymbol{\rho}\right\|_{\min_{c}}^{2}\right)$
exhibited by the geometric locus of the primal novel eigenaxis
$\boldsymbol{\rho}$ jointly reach their minimum values.

We have also discovered that the Wolfe-dual eigenenergy functional

$\max\Xi_{\boldsymbol{\psi}}\left(\boldsymbol{\psi}\right)=\mathbf{1}^{T}\boldsymbol{\psi}-\boldsymbol{\psi}^{T}\mathbf{Q}\boldsymbol{\psi/}2\text{,}$

wherein $\boldsymbol{\psi}^{T}\mathbf{y}=0$ and $\psi_{i\ast}>0$, is
_maximized_ by the largest eigenvector $\boldsymbol{\psi}_{\max}$ of the joint
covariance matrix $\mathbf{Q}$

$\mathbf{Q}\boldsymbol{\psi}_{\max}=\lambda_{1}\boldsymbol{\psi}_{\max}\text{,}$

at which point the random quadratic form
$\boldsymbol{\psi}_{\max}^{T}\mathbf{Q}\boldsymbol{\psi}_{\max}$ reaches its
minimum value, so that the total allowed eigenenergy
$\left\|\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right\|_{\min_{c}}^{2}$
and the expected risk
$\mathfrak{R}_{\mathfrak{\min}}\left(\left\|\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right\|_{\min_{c}}^{2}\right)$
exhibited by the geometric locus of the primal novel principal eigenaxis
$\boldsymbol{\rho}$ $=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ are
_jointly minimized_.

Thereby, we have discovered a novel principal eigen-coordinate transform
algorithm—that minimizes a vector-valued cost function—so that the total
allowed eigenenergy
$\left\|\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right\|_{\min_{c}}^{2}$
and the correlated expected risk
$\mathfrak{R}_{\mathfrak{\min}}\left(\left\|\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right\|_{\min_{c}}^{2}\right)$
exhibited by a minimum risk binary classification system are jointly
minimized.

We have also discovered that the novel principal eigen-coordinate transform
algorithm finds scale factors $\psi_{1i\ast}$ and $\psi_{2i\ast}$ for the
components
$\psi_{1i\ast}\frac{k_{\mathbf{x}_{1i\ast}}}{\left\|k_{\mathbf{x}_{1i\ast}}\right\|}$
and
$\psi_{2i\ast}\frac{k_{\mathbf{x}_{2i\ast}}}{\left\|k_{\mathbf{x}_{2i\ast}}\right\|}$
of the principal eigenvector $\boldsymbol{\psi}_{\max}$ of a joint covariance
matrix $\mathbf{Q}$ and the inverted joint covariance matrix
$\mathbf{Q}^{-1}$, so that the principal eigenvector
$\boldsymbol{\psi}_{\max}$ is symmetrically and equivalently related to the
geometric locus of the Wolfe dual novel principal eigenaxis
$\boldsymbol{\psi}=\sum\nolimits_{i=1}^{l_{1}}\psi_{1i\ast}\frac{k_{\mathbf{x}_{1i\ast}}}{\left\|k_{\mathbf{x}_{1i\ast}}\right\|}+\sum\nolimits_{i=1}^{l_{2}}\psi_{2i\ast}\frac{k_{\mathbf{x}_{2i\ast}}}{\left\|k_{\mathbf{x}_{2i\ast}}\right\|}$
of a minimum risk binary classification system, _and_ is also symmetrically
and equivalently related to the geometric locus of the primal novel principal
eigenaxis
$\boldsymbol{\rho}=\sum\nolimits_{i=1}^{l_{1}}\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}-\sum\nolimits_{i=1}^{l_{2}}\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}$
of the system, such that the scale factors $\psi_{1i\ast}$ and $\psi_{2i\ast}$
determine critical minimum eigenenergies
$\left\|\psi_{1_{i\ast}}k_{\mathbf{x}_{1_{i\ast}}}\right\|_{\min_{c}}^{2}$ and
$\left\|\psi_{2_{i\ast}}k_{\mathbf{x}_{2_{i\ast}}}\right\|_{\min_{c}}^{2}$
exhibited by the principal eigenaxis components
$\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}$ and
$\psi_{2_{i\ast}}k_{\mathbf{x}_{2_{i\ast}}}$ that lie on the sides
$\boldsymbol{\rho}_{1}$ and $\boldsymbol{\rho}_{2}$ of the novel principal
eigenaxis $\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ of
the system

$\displaystyle\boldsymbol{\rho}$
$\displaystyle=\sum\nolimits_{i=1}^{l_{1}}\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}-\sum\nolimits_{i=1}^{l_{2}}\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}$
$\displaystyle=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\text{,}$

so that eigenenergies
$\left\|\psi_{1_{i\ast}}k_{\mathbf{x}_{1i\ast}}\right\|_{\min_{c}}^{2}$ and
$\left\|\psi_{2_{i\ast}}k_{\mathbf{x}_{2i\ast}}\right\|_{\min_{c}}^{2}$
related to likely locations of corresponding extreme points
$\mathbf{x}_{1_{i\ast}}$ and $\mathbf{x}_{2_{i\ast}}$ determine _costs_ for
expected counter risks of making _right decisions_ or _costs_ for expected
risks of making _wrong decisions_.

Accordingly, we have determined that the novel principal eigen-coordinate
transform algorithm _finds_ the extreme vectors $k_{\mathbf{x}_{1_{i\ast}}}$
and $k_{\mathbf{x}_{2_{i\ast}}}$ and the scale factors $\psi_{1i\ast}$ and
$\psi_{2i\ast}$ that determine the geometrical and statistical structure of
the geometric locus of a novel principal eigenaxis
$\boldsymbol{\rho}=\sum\nolimits_{i=1}^{l_{1}}\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}-\sum\nolimits_{i=1}^{l_{2}}\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}$,
so that the geometric locus of the novel principal eigenaxis
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ is the
principal part of an equivalent representation of a pair of random quadratic
forms $\boldsymbol{\psi}^{T}\mathbf{Q}\boldsymbol{\psi}$ and
$\boldsymbol{\psi}^{T}\mathbf{Q}^{-1}\boldsymbol{\psi}$, at which point the
random quadratic forms $\boldsymbol{\psi}^{T}\mathbf{Q}\boldsymbol{\psi}$ and
$\boldsymbol{\psi}^{T}\mathbf{Q}^{-1}\boldsymbol{\psi}$ and the total allowed
eigenenergy
$\left\|\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right\|_{\min_{c}}^{2}$
exhibited by the novel principal eigenaxis
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ each reach
their minimum value.

Correspondingly, we have determined that any given geometric locus of a novel
principal eigenaxis
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ provides dual
representation of the discriminant function, an exclusive principal eigen-
coordinate system of the geometric locus of the decision boundary, and an
eigenaxis of symmetry that spans the decision space—of a certain minimum risk
binary classification system—such that the geometric locus of the novel
principal eigenaxis
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ is the
principal eigenaxis of the geometric loci of the decision boundary and a pair
of symmetrically positioned decision borders, so that the geometric locus of
the novel principal eigenaxis
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ satisfies the
geometric locus of the decision boundary in terms of a critical minimum
eigenenergy
$\left\|\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right\|_{\min_{c}}^{2}$
and a minimum expected risk
$\mathfrak{R}_{\mathfrak{\min}}\left(\left\|\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right\|_{\min_{c}}^{2}\right)$,
such that the total allowed eigenenergy
$\left\|\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right\|_{\min_{c}}^{2}$
and the expected risk
$\mathfrak{R}_{\mathfrak{\min}}\left(\left\|\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right\|_{\min_{c}}^{2}\right)$
exhibited by the geometric locus of the novel principal eigenaxis
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ are regulated
by the eigenvalues of the symmetric matrices $\mathbf{Q}$ and
$\mathbf{Q}^{-1}$ of a pair of random quadratic forms
$\boldsymbol{\psi}^{T}\mathbf{Q}\boldsymbol{\psi}$ and
$\boldsymbol{\psi}^{T}\mathbf{Q}^{-1}\boldsymbol{\psi}$, wherein the magnitude
and the direction of the novel principal eigenaxis $\boldsymbol{\rho}$
$=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ are both functions of
differences between joint variabilities of extreme vectors
$\mathbf{x}_{1_{i\ast}}\mathbf{\sim}p\left(\mathbf{x};\omega_{1}\right)$ and
$\mathbf{x}_{2_{i\ast}}\mathbf{\sim}p\left(\mathbf{x};\omega_{1}\right)$ that
belong to two classes $\omega_{1}$ and $\omega_{2}$ of random vectors
$\mathbf{x\in}$ $\mathbb{R}^{d}$.

We are now in a position to define fundamental locus equations of binary
classification.

##  10 Locus Equations of Binary Classification

In this section, we develop vector algebra locus equations of binary
classification that express fundamental laws of binary classification—that
discriminant functions of minimum risk binary classification systems are
subject to. We begin by defining the remaining terms in the equivalent form of
the vector algebra locus equation of (5.4).

###  10.1 Locus of Average Risk

Let any given random vector that has the form

$\displaystyle\frac{1}{l}\left(\sum\nolimits_{i=1}^{l_{1}}k_{\mathbf{x}_{1_{i\ast}}}+\sum\nolimits_{i=1}^{l_{2}}k_{\mathbf{x}_{2_{i\ast}}}\right)$
$\displaystyle=\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\text{,}$

wherein $l$ reproducing kernels $k_{\mathbf{x}_{i\ast}}$ of extreme points
$\mathbf{x}_{i\ast}\mathbf{\ }$belong to class $\omega_{1}$ and class
$\omega_{2}$, provide an equivalent representation of the random vector

$2\left(\mathbf{\Sigma}_{1}^{-1}\boldsymbol{\mu}_{1}-\mathbf{\Sigma}_{2}^{-1}\boldsymbol{\mu}_{2}\right)\text{,}$

so that the locus of each random vector
$\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}$ and
$2\left(\mathbf{\Sigma}_{1}^{-1}\boldsymbol{\mu}_{1}-\mathbf{\Sigma}_{2}^{-1}\boldsymbol{\mu}_{2}\right)$
represents a _locus of average risk_ within the decision space $Z=Z_{1}\cup
Z_{2}$ of a minimum risk binary classification system, such that the locus of
average risk is located near the locus of the decision boundary of the system.

Thereby, we realize that an equivalent representation of the vector

$2\mathbf{x}^{T}\left(\mathbf{\Sigma}_{1}^{-1}\boldsymbol{\mu}_{1}-\mathbf{\Sigma}_{2}^{-1}\boldsymbol{\mu}_{2}\right)$

is represented by a vector that determines the position of a random vector
$k_{\mathbf{s}}$ relative to a locus of average risk
$\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}$

(10.1)
$k_{\mathbf{s}}-\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\text{,}$

such that the locus of average risk
$\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}$ is located on or
near the locus of a linear decision boundary or is centrally located and
bounded by quadratic loci of a quadratic decision boundary.

###  10.2 Expected Likelihood of Observing Extreme Vectors

An equivalent form of the vector algebra locus equation of (5.4) contains an
equivalent representation of the statistical expression

$\left(\boldsymbol{\mu}_{1}^{T}\mathbf{\Sigma}_{1}^{-1}\boldsymbol{\mu}_{1}-\boldsymbol{\mu}_{2}^{T}\mathbf{\Sigma}_{2}^{-1}\boldsymbol{\mu}_{2}\right)+\left(\ln\left(\left|\mathbf{\Sigma}_{2}\right|\right)-\ln\left(\left|\mathbf{\Sigma}_{1}\right|\right)\right)\text{,}$

where the expression
$\left(\boldsymbol{\mu}_{1}^{T}\mathbf{\Sigma}_{1}^{-1}\boldsymbol{\mu}_{1}-\boldsymbol{\mu}_{2}^{T}\mathbf{\Sigma}_{2}^{-1}\boldsymbol{\mu}_{2}\right)$
represents the difference between expected likelihoods of observing normal
random vectors that belong to two classes, and the expression
$\ln\left(\left|\mathbf{\Sigma}_{2}\right|\right)-\ln\left(\left|\mathbf{\Sigma}_{1}\right|\right)$
represents the difference between expected distributions of normal random
vectors that belong to the two classes—which are related to expected
likelihoods of observing the normal random vectors.

Since we are going to use a constrained optimization algorithm to find
discriminant functions of minimum risk binary classification systems, we
realize that the statistic $\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}$ is a
simple and effective way to represent the _difference_ between expected
likelihoods of observing $l$ extreme vectors
$\left\\{k_{\mathbf{x}_{i\ast}}\right\\}_{i=1}^{l}$ that belong to two classes
$\omega_{1}$ and $\omega_{2}$ of random vectors $\mathbf{x\in}$
$\mathbb{R}^{d}$ such that $\mathbf{x\sim}$
$p\left(\mathbf{x};\omega_{1}\right)$ and $\mathbf{x\sim}$
$p\left(\mathbf{x};\omega_{2}\right)$

(10.2) $\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}\text{,}$

where $l$ is the number of extreme vectors $k_{\mathbf{x}_{i\ast}}$ that
belong to the two classes $\omega_{1}$ and $\omega_{2}$, wherein $y_{i}=+1$ if
an extreme vector $k_{\mathbf{x}_{i\ast}}$ belongs to class $\omega_{1}$ and
$y_{i}=-1$ if an extreme vector $k_{\mathbf{x}_{i\ast}}$ belongs to class
$\omega_{2}$.

###  10.3 Locus Equation of a Decision Boundary

We are now in a position to define a vector algebra locus equation that
represents the geometric locus of the decision boundary of any given minimum
risk binary classification system that is subject to two categories of random
vectors $\mathbf{x\in}$ $\mathbb{R}^{d}$. Corollary 10.1 expresses an
equivalent form of the vector algebra locus equation in (5.4) that represents
the geometric locus of the decision boundary of a minimum risk binary
classification system, so that the discriminant function of the system and the
exclusive intrinsic eigen-coordinate system of the geometric locus of the
decision boundary of the system are both represented by a geometric locus of a
novel principal eigenaxis.

###### Corollary 10.1

Let

$\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\left(\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right)+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}\overset{\omega_{1}}{\underset{\omega_{2}}{\gtrless}}0$

be any given minimum risk binary classification system that is subject to
random inputs $\mathbf{x\in}$ $\mathbb{R}^{d}$ such that $\mathbf{x\sim}$
$p\left(\mathbf{x};\omega_{1}\right)$ and $\mathbf{x\sim}$
$p\left(\mathbf{x};\omega_{2}\right)$, where
$p\left(\mathbf{x};\omega_{1}\right)$ and
$p\left(\mathbf{x};\omega_{2}\right)$ are certain probability density
functions for two classes $\omega_{1}$ and $\omega_{2}$ of random vectors
$\mathbf{x\in}$ $\mathbb{R}^{d}$, at which point the discriminant function of
the system is represented by a geometric locus of a novel principal eigenaxis
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$.

The geometric locus of the decision boundary of the system is represented by
the graph of a vector algebra locus equation that has the form

$\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\left(\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right)+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}=0\text{,}$

so that the geometric locus of the novel principal eigenaxis
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ of the system
is the solution of the locus equation, at which point the geometric locus of
the novel principal eigenaxis
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ provides dual
representation of the discriminant function of the system and an exclusive
principal eigen-coordinate system of the geometric locus of the decision
boundary of the system, such that all of the points $\mathbf{s}$ that lie on
the geometric locus of the decision boundary exclusively reference the
principal eigen-coordinate system
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$, where the
statistic $\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}:y_{i}=\pm 1$ is an expected
likelihood of observing $l=l_{1}+l_{2}$ extreme vectors
$k_{\mathbf{x}_{i\ast}}$, and the vector difference of
$k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}$
determines the distance between the locus of a random vector $\mathbf{s}$ and
a locus of average risk
$\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}$ within the decision
space $Z=Z_{1}\cup Z_{2}$ of the system.

Proof. We prove Corollary 10.1 by a constructive proof that demonstrates how a
well-posed constrained optimization algorithm resolves the inverse problem of
the binary classification of random vectors.

Returning now to the conditions expressed by Corollary 3.9 and Axioms 5.4 \-
5.5, along with the conditions expressed by Corollaries 3.8 \- 3.10, recall
that likely locations of extreme points $\mathbf{x}_{1_{i\ast}}$ and
$\mathbf{x}_{2_{i\ast}}$ are spread throughout the decision space $Z=Z_{1}\cup
Z_{2}$ of any given minimum risk binary classification system in such a manner
that likely locations of the extreme points $\mathbf{x}_{1_{i\ast}}$ and
$\mathbf{x}_{2_{i\ast}}$ determine right and wrong decisions made by the
system. Properties of a minimum risk binary classification system to make
right and wrong decisions are defined by the notions of counter and risk—which
we originally defined in our working paper [69].

###  10.4 Counter Risks of Making Right Decisions

###### Definition 10.2

The probability of finding an extreme point $\mathbf{x}_{1_{i\ast}}$ inside
the decision space $Z=Z_{1}\cup Z_{2}$ of a minimum risk binary classification
system is said to determine a region of counter risk—where counter risk is the
property of the system to make right decisions—if and only if likely locations
of the extreme point $\mathbf{x}_{1_{i\ast}}$ are inside the decision region
$Z_{1}$ of the system, at which point the critical minimum eigenenergy
$\left\|\psi_{1_{i\ast}}k_{\mathbf{x}_{1_{i\ast}}}\right\|_{\min_{c}}^{2}$
exhibited by the principal eigenaxis component
$\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}$ on the side
$\boldsymbol{\rho}_{1}$ of the geometric locus of the novel principal
eigenaxis $\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ of
the system contributes to both the counter risk
$\overline{\mathfrak{R}}_{\mathfrak{\min}}\left(Z_{1}|\boldsymbol{\rho}_{1}\right)$
and the total allowed eigenenergy
$\left\|Z_{1}|\boldsymbol{\rho}_{1}\right\|_{\min_{c}}^{2}$ that is given by
the integral

(10.3)
$\overline{\mathfrak{R}}_{\mathfrak{\min}}\left(Z_{1}|\boldsymbol{\rho}_{1}\right)=\int_{Z_{1}}\boldsymbol{\rho}_{1}d\boldsymbol{\rho}_{1}\equiv\left\|Z_{1}|\boldsymbol{\rho}_{1}\right\|_{\min_{c}}^{2}\equiv
P\left(Z_{1}|\boldsymbol{\rho}_{1}\right)\text{,}$

over the decision region $Z_{1}$ of the system, where
$P\left(Z_{1}|\boldsymbol{\rho}_{1}\right)$ is the conditional probability
that extreme points $\mathbf{x}_{1_{i\ast}}$ are located inside the decision
region $Z_{1}$, and
$\left\|Z_{1}|\boldsymbol{\rho}_{1}\right\|_{\min_{c}}^{2}$ is the total
allowed eigenenergy exhibited by all of the principal eigenaxis components
$\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}$ on the side
$\boldsymbol{\rho}_{1}$ of the novel principal eigenaxis
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$.

Correspondingly, the probability of finding an extreme point
$\mathbf{x}_{2_{i\ast}}$ inside the decision space $Z=Z_{1}\cup Z_{2}$ of a
minimum risk binary classification system is said to determine a region of
counter risk if and only if likely locations of the extreme point
$\mathbf{x}_{2_{i\ast}}$ are inside the decision region $Z_{2}$ of the system,
at which point the critical minimum eigenenergy
$\left\|\psi_{2_{i\ast}}k_{\mathbf{x}_{2_{i\ast}}}\right\|_{\min_{c}}^{2}$
exhibited by the principal eigenaxis component
$\psi_{2_{i\ast}}k_{\mathbf{x}_{2_{i\ast}}}$ on the side
$\boldsymbol{\rho}_{2}$ of the geometric locus of the novel principal
eigenaxis $\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ of
the system contributes to both the counter risk
$\overline{\mathfrak{R}}_{\mathfrak{\min}}\left(Z_{2}|\boldsymbol{\rho}_{2}\right)$
and the total allowed eigenenergy
$\left\|Z_{2}|\boldsymbol{\rho}_{2}\right\|_{\min_{c}}^{2}$ that is given by
the integral

(10.4)
$\overline{\mathfrak{R}}_{\mathfrak{\min}}\left(Z_{2}|\boldsymbol{\rho}_{2}\right)=\int_{Z_{2}}\boldsymbol{\rho}_{2}d\boldsymbol{\rho}_{2}\equiv\left\|Z_{2}|\boldsymbol{\rho}_{2}\right\|_{\min_{c}}^{2}\equiv
P\left(Z_{2}|\boldsymbol{\rho}_{2}\right)\text{,}$

over the decision region $Z_{2}$ of the system, where
$P\left(Z_{2}|\boldsymbol{\rho}_{2}\right)$ is the conditional probability
that extreme points $\mathbf{x}_{2_{i\ast}}$ are located inside the decision
region $Z_{2}$, and
$\left\|Z_{2}|\boldsymbol{\rho}_{2}\right\|_{\min_{c}}^{2}$ is the total
allowed eigenenergy exhibited by all of the principal eigenaxis components
$\psi_{2_{i\ast}}k_{\mathbf{x}_{2_{i\ast}}}$ on the side
$\boldsymbol{\rho}_{2}$ of the novel principal eigenaxis
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$.

###  10.5 Risks of Making Wrong Decisions

###### Definition 10.3

The probability of finding an extreme point $\mathbf{x}_{1_{i\ast}}$ inside
the decision space $Z=Z_{1}\cup Z_{2}$ of a minimum risk binary classification
system is said to determine a region of risk—where risk is the property of the
system to make wrong decisions—if and only if likely locations of the extreme
point $\mathbf{x}_{1_{i\ast}}$ are inside the decision region $Z_{2}$ of the
system, at which point the critical minimum eigenenergy
$\left\|\psi_{1_{i\ast}}k_{\mathbf{x}_{1_{i\ast}}}\right\|_{\min_{c}}^{2}$
exhibited by the principal eigenaxis component
$\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}$ on the side
$\boldsymbol{\rho}_{1}$ of the geometric locus of the novel principal
eigenaxis $\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ of
the system contributes to both the risk
$\mathfrak{R}_{\mathfrak{\min}}\left(Z_{2}|\boldsymbol{\rho}_{1}\right)$ and
the total allowed eigenenergy
$\left\|Z_{2}|\boldsymbol{\rho}_{1}\right\|_{\min_{c}}^{2}$ that is given by
the integral

(10.5)
$\mathfrak{R}_{\mathfrak{\min}}\left(Z_{2}|\boldsymbol{\rho}_{1}\right)=\int_{Z_{2}}\boldsymbol{\rho}_{1}d\boldsymbol{\rho}_{1}\equiv\left\|Z_{2}|\boldsymbol{\rho}_{1}\right\|_{\min_{c}}^{2}\equiv
P\left(Z_{2}|\boldsymbol{\rho}_{1}\right)\text{,}$

over the decision region $Z_{2}$ of the system, where
$P\left(Z_{2}|\boldsymbol{\rho}_{1}\right)$ is the conditional probability
that extreme points $\mathbf{x}_{1_{i\ast}}$ are located inside the decision
region $Z_{2}$, and
$\left\|Z_{2}|\boldsymbol{\rho}_{1}\right\|_{\min_{c}}^{2}$ is the total
allowed eigenenergy exhibited by all of the principal eigenaxis components
$\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}$ on the side
$\boldsymbol{\rho}_{1}$ of the novel principal eigenaxis
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$.

Correspondingly, the probability of finding an extreme point
$\mathbf{x}_{2_{i\ast}}$ inside the decision space $Z=Z_{1}\cup Z_{2}$ of a
minimum risk binary classification system is said to determine a region of
risk if and only if likely locations of the extreme point
$\mathbf{x}_{2_{i\ast}}$ are inside the decision region $Z_{1}$ of the system,
at which point the critical minimum eigenenergy
$\left\|\psi_{2_{i\ast}}k_{\mathbf{x}_{2_{i\ast}}}\right\|_{\min_{c}}^{2}$
exhibited by the principal eigenaxis component
$\psi_{2_{i\ast}}k_{\mathbf{x}_{2_{i\ast}}}$ on the side
$\boldsymbol{\rho}_{2}$ of the geometric locus of the novel principal
eigenaxis $\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ of
the system contributes to both the risk
$\mathfrak{R}_{\mathfrak{\min}}\left(Z_{1}|\boldsymbol{\rho}_{2}\right)$ and
the total allowed eigenenergy
$\left\|Z_{1}|\boldsymbol{\rho}_{2}\right\|_{\min_{c}}^{2}$ that is given by
the integral

(10.6)
$\mathfrak{R}_{\mathfrak{\min}}\left(Z_{1}|\boldsymbol{\rho}_{2}\right)=\int_{Z_{1}}\boldsymbol{\rho}_{2}d\boldsymbol{\rho}_{2}\equiv\left\|Z_{1}|\boldsymbol{\rho}_{2}\right\|_{\min_{c}}^{2}\equiv
P\left(Z_{1}|\boldsymbol{\rho}_{2}\right)\text{,}$

over the decision region $Z_{1}$ of the system, where
$P\left(Z_{1}|\boldsymbol{\rho}_{2}\right)$ is the conditional probability
that extreme points $\mathbf{x}_{2_{i\ast}}$ are located inside the decision
region $Z_{1}$, and
$\left\|Z_{1}|\boldsymbol{\rho}_{2}\right\|_{\min_{c}}^{2}$ is the total
allowed eigenenergy exhibited by all of the principal eigenaxis components
$\psi_{2_{i\ast}}k_{\mathbf{x}_{2_{i\ast}}}$ on the side
$\boldsymbol{\rho}_{2}$ of the novel principal eigenaxis
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$.

###  10.6 Vector Algebra Locus Equations of Decision Spaces

Given the conditions expressed by Theorem 7.1, Corollary 7.2 and Theorem 8.3,
we realize that the geometric locus of the novel principal eigenaxis
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ of any given
minimum risk binary classification system
$\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\left(\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right)+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}\overset{\omega_{1}}{\underset{\omega_{2}}{\gtrless}}0$
completely determines the shape of the decision space $Z=Z_{1}\cup Z_{2}$ of
the system.

Thereby, given the properties of any given minimum risk binary classification
system to make right and wrong decisions—expressed by the notions of counter
and risk in Definitions 10.2 and 10.3, along with Theorem 7.1, Corollary 7.2
and Theorem 8.3, we realize that any given geometric locus of a novel
principal eigenaxis
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ is the
solution of each and every one of the vector algebra locus equations expressed
by Corollary 10.4.

###### Corollary 10.4

Let

$\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\left(\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right)+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}\overset{\omega_{1}}{\underset{\omega_{2}}{\gtrless}}0$

be any given minimum risk binary classification system that is subject to
random inputs $\mathbf{x\in}$ $\mathbb{R}^{d}$ such that $\mathbf{x\sim}$
$p\left(\mathbf{x};\omega_{1}\right)$ and $\mathbf{x\sim}$
$p\left(\mathbf{x};\omega_{2}\right)$, where
$p\left(\mathbf{x};\omega_{1}\right)$ and
$p\left(\mathbf{x};\omega_{2}\right)$ are certain probability density
functions for two classes $\omega_{1}$ and $\omega_{2}$ of random vectors
$\mathbf{x\in}$ $\mathbb{R}^{d}$, at which point the discriminant function of
the system is represented by a geometric locus of a novel principal eigenaxis
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$.

Also, let $d\left(\mathbf{s}\right)=0$ denote the geometric locus of the
decision boundary of the system, let $d\left(\mathbf{s}\right)=+1$ denote the
geometric locus of the decision border of the decision region $Z_{1}$ of the
system, and let $d\left(\mathbf{s}\right)=-1$ denote the geometric locus of
decision border of the decision region $Z_{2}$ of the system, so that the
geometric loci of the decision boundary $d\left(\mathbf{s}\right)=0$ and the
symmetrically positioned decision borders $d\left(\mathbf{s}\right)=+1$ and
$d\left(\mathbf{s}\right)=-1$ partition the decision space $Z=Z_{1}\cup Z_{2}$
of the minimum risk binary classification system in a symmetrically balanced
manner.

The geometric locus of the decision boundary of the minimum risk binary
classification system is represented by the graph of a vector algebra locus
equation that has the form

$\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\left(\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right)+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}=0\text{,}$

wherein the geometric locus of the novel principal eigenaxis
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ of the system
is the solution of the locus equation, so that the geometric locus of the
novel principal eigenaxis
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ provides dual
representation of the discriminant function of the system and an exclusive
principal eigen-coordinate system of the geometric locus of the decision
boundary of the system, at which point all of the points $\mathbf{s}$ that lie
on the geometric locus of the decision boundary exclusively reference the
novel principal eigenaxis
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$, and the
geometric locus of the novel principal eigenaxis
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ represents an
eigenaxis of symmetry that spans the decision space $Z=Z_{1}\cup Z_{2}$ of the
minimum risk binary classification system
$\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\left(\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right)+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}\overset{\omega_{1}}{\underset{\omega_{2}}{\gtrless}}0$,
so that the shape of the decision space $Z=Z_{1}\cup Z_{2}$ is completely
determined by the exclusive intrinsic eigen-coordinate system
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ in the
following manner.

The geometric locus of the decision border of the decision region $Z_{1}$ of
the minimum risk binary classification system is represented by the graph of a
vector algebra locus equation that has the form

$\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\left(\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right)+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}=+1\text{,}$

wherein the geometric locus of the novel principal eigenaxis
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ of the system
is the solution of the locus equation, so that all of the points $\mathbf{s}$
that lie on the geometric locus of the decision border
$d\left(\mathbf{s}\right)=+1$ exclusively reference the novel principal
eigenaxis $\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$, at
which point the shape of the decision region $Z_{1}$ of the minimum risk
binary classification system is completely determined by the exclusive
intrinsic eigen-coordinate system
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$, such that the
shape of the decision region $Z_{1}$ is determined by the shapes of the
geometric loci of the decision border $d\left(\mathbf{s}\right)=+1$ and the
decision boundary $d\left(\mathbf{s}\right)=0$ of the system.

The geometric locus of the decision border of the decision region $Z_{2}$ of
the minimum risk binary classification system is represented by the graph of a
vector algebra locus equation that has the form

$\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\left(\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right)+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}=-1\text{,}$

wherein the geometric locus of the novel principal eigenaxis
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ of the system
is the solution of the locus equation, so that all of the points $\mathbf{s}$
that lie on the geometric locus of the decision border
$d\left(\mathbf{s}\right)=-1$ exclusively reference the novel principal
eigenaxis $\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$, at
which point the shape of the decision region $Z_{2}$ of the minimum risk
binary classification system is completely determined by the exclusive
intrinsic eigen-coordinate system
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$, such that the
shape of the decision region $Z_{2}$ is determined by the shapes of the
geometric loci of the decision border $d\left(\mathbf{s}\right)=-1$ and the
decision boundary $d\left(\mathbf{s}\right)=0$ of the system.

Proof. Corollary 10.4 is substantiated by conditions expressed by Corollaries
3.8 \- 3.10, Axioms 5.4 \- 5.5, Theorem 7.1 and Corollary 7.2, and Theorem
8.3.

We prove Corollary 10.4 by a constructive proof that demonstrates how a well-
posed constrained optimization algorithm resolves the inverse problem of the
binary classification of random vectors.

Returning now to the integral equation in (3.4) expressed by Corollary 3.8,
along with the integral equation in (3.5) expressed by Corollary 3.10, we
realize that the total allowed eigenenergy and the expected risk exhibited by
the geometric locus of the novel principal eigenaxis
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ of any given
minimum risk binary classification system are jointly minimized within the
decision space $Z=Z_{1}\cup Z_{2}$ of the system—including the situation of
_completely overlapping distributions_ of random points—such that $100\%$ of
the training data are extreme points and the error rate of the system is
$50\%$—which is the lowest possible error rate of the system.

Figure $7$ illustrates how a discriminant function of a minimum risk binary
classification system minimizes the expected risk of the system within the
decision space of the system—for completely overlapping distributions of
random points $\mathbf{x\in}$ $\mathbb{R}^{2}$—such that the geometric locus
of the novel principal eigenaxis of the system is the solution of each and
every one of the vector algebra locus equations expressed by Corollary 10.4.

Thereby, the geometric locus of the novel principal eigenaxis represents an
eigenaxis of symmetry that spans the decision space of the minimum risk binary
classification system, so that the geometric loci of a hyperbolic decision
boundary and a pair of symmetrically positioned hyperbolic decision borders
symmetrically partition the decision space of the system. The decision
boundary is black, the decision borders are blue and red, and each extreme
point is enclosed in a black circle.

Figure 7: The constrained optimization algorithm that resolves the inverse
problem of the binary classification of random vectors finds discriminant
functions of minimum risk binary classification systems—for completely
overlapping distributions of random points—such that $100\%$ of the training
data are extreme points, and the error rate of any given system is
$50\%$—which is the lowest possible error rate of the system.

###  10.7 Joint Minimization of Eigenenergy and Risk

Recall that a geometric locus of a novel principal eigenaxis provides dual
representation of the discriminant function, an exclusive principal eigen-
coordinate system of the geometric locus of the decision boundary, and an
eigenaxis of symmetry that spans the decision space—of any given minimum risk
binary classification system.

Given the conditions expressed by Corollaries 3.8 and 3.10, along with the
conditions expressed by Theorems 6.2 \- 6.4, Theorem 7.1 and Corollaries 7.2
\- 7.3, we realize that any given geometric locus of a novel principal
eigenaxis exhibits a critical minimum eigenenergy and a minimum expected risk
in the manner that is expressed by Corollary 10.5.

###### Corollary 10.5

Let $\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ be the
geometric locus of the novel principal eigenaxis of any given minimum risk
binary classification system

$\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\left(\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right)+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}\overset{\omega_{1}}{\underset{\omega_{2}}{\gtrless}}0$

that is subject to random inputs $\mathbf{x\in}$ $\mathbb{R}^{d}$ such that
$\mathbf{x\sim}$ $p\left(\mathbf{x};\omega_{1}\right)$ and $\mathbf{x\sim}$
$p\left(\mathbf{x};\omega_{2}\right)$, where
$p\left(\mathbf{x};\omega_{1}\right)$ and
$p\left(\mathbf{x};\omega_{2}\right)$ are certain probability density
functions for two classes $\omega_{1}$ and $\omega_{2}$ of random vectors
$\mathbf{x\in}$ $\mathbb{R}^{d}$, at which point the discriminant function

$d\left(\mathbf{s}\right)=\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\left(\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right)+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}$

of the system is represented by the geometric locus of the novel principal
eigenaxis $\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$.

The geometric locus of the novel principal eigenaxis
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ exhibits a
total allowed eigenenergy and an expected risk in such a manner that the dual
locus $\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ of the
discriminant function of the minimum risk binary classification system is in
statistical equilibrium at the geometric locus of the decision boundary of the
system, at which point the geometric locus of the novel principal eigenaxis
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ is an
eigenaxis of symmetry that satisfies the law of cosines in the symmetrically
balanced manner

(10.7) $\displaystyle\frac{1}{2}\left\|\boldsymbol{\rho}\right\|_{\min}^{2}$
$\displaystyle=\left\|\boldsymbol{\rho}_{1}\right\|_{\min}^{2}-\left\|\boldsymbol{\rho}_{1}\right\|\left\|\boldsymbol{\rho}_{2}\right\|\cos\theta_{\boldsymbol{\rho}_{1}\boldsymbol{\rho}_{2}}$
$\displaystyle=\left\|\boldsymbol{\rho}_{2}\right\|_{\min}^{2}-\left\|\boldsymbol{\rho}_{2}\right\|\left\|\boldsymbol{\rho}_{1}\right\|\cos\theta_{\boldsymbol{\rho}_{2}\boldsymbol{\rho}_{1}}\text{,}$

so that the dual locus
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ of the
discriminant function satisfies the geometric locus of the decision boundary
in terms of a critical minimum eigenenergy and a minimum expected risk, such
that critical minimum eigenenergies related to likely locations of extreme
points $\mathbf{x}_{1_{i\ast}}\mathbf{\sim}$
$p\left(\mathbf{x};\omega_{1}\right)$ and
$\mathbf{x}_{2_{i\ast}}\mathbf{\sim}$ $p\left(\mathbf{x};\omega_{2}\right)$
determine conditional probabilities that the extreme points
$\mathbf{x}_{1_{i\ast}}$ and $\mathbf{x}_{2_{i\ast}}$ will be observed within
localized areas of counter risk and risk throughout the decision space
$Z=Z_{1}\cup Z_{2}$ of the system, such that the magnitude and the direction
of the novel principal eigenaxis $\boldsymbol{\rho}$
$=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ are both functions of
differences between joint variabilities of extreme vectors
$\mathbf{x}_{1_{i\ast}}\mathbf{\sim}p\left(\mathbf{x};\omega_{1}\right)$ and
$\mathbf{x}_{2_{i\ast}}\mathbf{\sim}p\left(\mathbf{x};\omega_{1}\right)$ that
belong to the two classes $\omega_{1}$ and $\omega_{2}$ of random vectors
$\mathbf{x\in}$ $\mathbb{R}^{d}$.

Proof. We prove Corollary 10.5 by a constructive proof that demonstrates how a
well-posed constrained optimization algorithm resolves the inverse problem of
the binary classification of random vectors.

Returning again to the integral equations expressed by Corollaries 3.8 and
3.10, it follows that the discriminant function of any given minimum risk
binary classification system satisfies a pair of fundamental integral
equations of binary classification.

###  10.8 Integral Equations of Binary Classification

Corollary 10.6 expresses the requirement that a discriminant function of a
minimum risk binary classification system is the solution of an integral
equation—corresponding to the integral equation in (3.4), over the decision
space $Z=Z_{1}\cup Z_{2}$ of the system, so that the total allowed eigenenergy
and the expected risk exhibited by the system are regulated by the equilibrium
requirement on the dual locus of the discriminant function at the geometric
locus of the decision boundary of the system expressed by Corollary 10.5, at
which point counter risks and risks exhibited by the system are symmetrically
balanced with each other throughout the decision regions $Z_{1}$ and $Z_{2}$
of the system.

###### Corollary 10.6

Take the discriminant function

$d\left(\mathbf{s}\right)=\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\left(\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right)+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}$

of any given minimum risk binary classification system

$\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\left(\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right)+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}\overset{\omega_{1}}{\underset{\omega_{2}}{\gtrless}}0$

that is subject to random inputs $\mathbf{x\in}$ $\mathbb{R}^{d}$ such that
$\mathbf{x\sim}$ $p\left(\mathbf{x};\omega_{1}\right)$ and $\mathbf{x\sim}$
$p\left(\mathbf{x};\omega_{2}\right)$, where
$p\left(\mathbf{x};\omega_{1}\right)$ and
$p\left(\mathbf{x};\omega_{2}\right)$ are certain probability density
functions for two classes $\omega_{1}$ and $\omega_{2}$ of random vectors
$\mathbf{x\in}$ $\mathbb{R}^{d}$, at which point the discriminant function of
the system is represented by the geometric locus of a novel principal
eigenaxis $\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$.

Given the integral equation in (3.4) expressed by Corollary 3.8 and the
conditions expressed by (10.7) in Corollary 10.5, it follows that the
discriminant function is the solution of the integral equation

(10.8) $\displaystyle f_{1}\left(d\left(\mathbf{s}\right)\right)$
$\displaystyle:\int_{Z_{1}}\boldsymbol{\rho}_{1}d\boldsymbol{\rho}_{1}+\int_{Z_{2}}\boldsymbol{\rho}_{1}d\boldsymbol{\rho}_{1}+C_{1}$
$\displaystyle=\int_{Z_{1}}\boldsymbol{\rho}_{2}d\boldsymbol{\rho}_{2}+\int_{Z_{2}}\boldsymbol{\rho}_{2}d\boldsymbol{\rho}_{2}+C_{2}\text{,}$

over the decision space $Z=Z_{1}\cup Z_{2}$ of the system, where $C_{1}$ and
$C_{2}$ are certain integration constants, so that the total allowed
eigenenergy
$\left\|\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right\|_{\min_{c}}^{2}$
and the expected risk
$\mathfrak{R}_{\mathfrak{\min}}\left(\left\|\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right\|_{\min_{c}}^{2}\right)$
exhibited by the system are jointly regulated by the equilibrium requirement
on the dual locus
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ of the
discriminant function at the geometric locus of the decision boundary of the
system

$\displaystyle d\left(\mathbf{s}\right)$
$\displaystyle:\left\|\boldsymbol{\rho}_{1}\right\|_{\min_{c}}^{2}-\left\|\boldsymbol{\rho}_{1}\right\|\left\|\boldsymbol{\rho}_{2}\right\|\cos\theta_{\boldsymbol{\rho}_{1}\boldsymbol{\rho}_{2}}$
$\displaystyle=\left\|\boldsymbol{\rho}_{2}\right\|_{\min_{c}}^{2}-\left\|\boldsymbol{\rho}_{2}\right\|\left\|\boldsymbol{\rho}_{1}\right\|\cos\theta_{\boldsymbol{\rho}_{2}\boldsymbol{\rho}_{1}}$
$\displaystyle=\frac{1}{2}\left\|\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right\|_{\min_{c}}^{2}\text{,}$

at which point the dual locus
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ of the
discriminant function satisfies the geometric locus of the decision boundary
in terms of a critical minimum eigenenergy
$\left\|\boldsymbol{\rho}\right\|_{\min_{c}}^{2}$ and a minimum expected risk
$\mathfrak{R}_{\mathfrak{\min}}\left(\left\|\boldsymbol{\rho}\right\|_{\min_{c}}^{2}\right)$
in such a manner that regions of counter risks of the system are symmetrically
balanced with regions of risks of the system.

Thereby, critical minimum eigenenergies
$\left\|\psi_{1_{i\ast}}k_{\mathbf{x}_{1_{i\ast}}}\right\|_{\min_{c}}^{2}$
exhibited by principal eigenaxis components
$\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}$ on side
$\boldsymbol{\rho}_{1}$ of the novel principal eigenaxis
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$—that determine
probabilities of finding extreme points $\mathbf{x}_{1_{i\ast}}$ located
throughout the decision space $Z=Z_{1}\cup Z_{2}$ of the system, are
symmetrically balanced with critical minimum eigenenergies
$\left\|\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}\right\|_{\min_{c}}^{2}$
exhibited by principal eigenaxis components
$\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}$ on side
$\boldsymbol{\rho}_{2}$ of the novel principal eigenaxis
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$—that determine
probabilities of finding extreme points $\mathbf{x}_{2_{i\ast}}$ located
throughout the decision space $Z=Z_{1}\cup Z_{2}$ of the system.

Proof. We prove Corollary 10.6 by a constructive proof that demonstrates how a
well-posed constrained optimization algorithm resolves the inverse problem of
the binary classification of random vectors.

Corollary 10.7 expresses the requirement that a discriminant function of a
minimum risk binary classification system minimize an integral equation over
the decision regions $Z_{1}$ and $Z_{2}$ of the system, so that the total
allowed eigenenergy and the expected risk exhibited by the system are jointly
minimized within the decision space $Z=Z_{1}\cup Z_{2}$ of the system in such
a manner that the system satisfies a state of statistical equilibrium, at
which point regions of counter risks and risks of the system—located
throughout the decision region $Z_{1}$ of the system—are symmetrically
balanced with regions of counter risks and risks of the system—located
throughout the decision region $Z_{2}$ of the system. Thereby, the minimum
risk binary classification system satisfies a state of statistical equilibrium
so that the total allowed eigenenergy and the expected risk exhibited by the
system are jointly minimized within the decision space $Z=Z_{1}\cup Z_{2}$ of
the system, at which point the system exhibits the minimum probability of
classification error.

The integral equation expressed by Corollary 10.7 is derived from the integral
equation in (10.8) expressed by Corollary 10.6.

###### Corollary 10.7

Take the discriminant function

$d\left(\mathbf{s}\right)=\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\left(\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right)+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}$

of any given minimum risk binary classification system

$\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\left(\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right)+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}\overset{\omega_{1}}{\underset{\omega_{2}}{\gtrless}}0$

that is subject to random inputs $\mathbf{x\in}$ $\mathbb{R}^{d}$ such that
$\mathbf{x\sim}$ $p\left(\mathbf{x};\omega_{1}\right)$ and $\mathbf{x\sim}$
$p\left(\mathbf{x};\omega_{2}\right)$, where
$p\left(\mathbf{x};\omega_{1}\right)$ and
$p\left(\mathbf{x};\omega_{2}\right)$ are certain probability density
functions for two classes $\omega_{1}$ and $\omega_{2}$ of random vectors
$\mathbf{x\in}$ $\mathbb{R}^{d}$, at which point the discriminant function of
the system is represented by the geometric locus of a novel principal
eigenaxis $\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$.

Given the integral equation in (3.5) expressed by Corollary 3.10 and the
integral equation in (10.8) expressed by Corollary 10.6, it follows that the
discriminant function minimizes the integral equation

(10.9) $\displaystyle f_{2}\left(d\left(\mathbf{s}\right)\right)$
$\displaystyle:\int_{Z_{1}}\boldsymbol{\rho}_{1}d\boldsymbol{\rho}_{1}-\int_{Z_{1}}\boldsymbol{\rho}_{2}d\boldsymbol{\rho}_{2}+C_{1}$
$\displaystyle=\int_{Z_{2}}\boldsymbol{\rho}_{2}d\boldsymbol{\rho}_{2}-\int_{Z_{2}}\boldsymbol{\rho}_{1}d\boldsymbol{\rho}_{1}+C_{2}\text{,}$

over the decision regions $Z_{1}$ and $Z_{2}$ of the system, where $C_{1}$ and
$C_{2}$ are certain integration constants, so that the system satisfies a
state of statistical equilibrium such that the total allowed eigenenergy
$\left\|\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right\|_{\min_{c}}^{2}$
and the expected risk
$\mathfrak{R}_{\mathfrak{\min}}\left(\left\|\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right\|_{\min_{c}}^{2}\right)$
exhibited by the system are jointly minimized within the decision space
$Z=Z_{1}\cup Z_{2}$ of the system in such a manner that critical minimum
eigenenergies
$\left\|\psi_{1_{i\ast}}k_{\mathbf{x}_{1_{i\ast}}}\right\|_{\min_{c}}^{2}$ and
$\left\|\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}\right\|_{\min_{c}}^{2}$
exhibited by corresponding principal eigenaxis components
$\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}$ and
$\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}$ that lie on side
$\boldsymbol{\rho}_{1}$ and side $\boldsymbol{\rho}_{2}$ of the novel
principal eigenaxis
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ are minimized
throughout the decision regions $Z_{1}$ and $Z_{2}$ of the system, at which
point regions of counter risks and risks of the system—located throughout the
decision region $Z_{1}$ of the system—are symmetrically balanced with regions
of counter risks and risks of the system—located throughout the decision
region $Z_{2}$ of the system.

Thereby, the minimum risk binary classification system satisfies a state of
statistical equilibrium so that the total allowed eigenenergy and the expected
risk exhibited by the system are jointly minimized within the decision space
$Z=Z_{1}\cup Z_{2}$ of the system, at which point the system exhibits the
minimum probability of classification error for any given random vectors
$\mathbf{x\in}$ $\mathbb{R}^{d}$ such that $\mathbf{x\sim}$
$p\left(\mathbf{x};\omega_{1}\right)$ and $\mathbf{x\sim}$
$p\left(\mathbf{x};\omega_{2}\right)$.

Proof. We prove Corollary 10.7 by a constructive proof that demonstrates how a
well-posed constrained optimization algorithm resolves the inverse problem of
the binary classification of random vectors.

We are now in a position to express the direct problem of the binary
classification of random vectors—according to the theoretical model that we
have developed. Theorem 11.1 is an existence theorem that expresses
fundamental laws of binary classification—that discriminant functions of
minimum risk binary classification systems are subject to—in terms of a
general locus formula.

##  11 The Direct Problem

###### Theorem 11.1

Let

(11.1)
$\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\left(\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right)+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}\overset{\omega_{1}}{\underset{\omega_{2}}{\gtrless}}0$

be any given minimum risk binary classification system that is subject to
random inputs $\mathbf{x\in}$ $\mathbb{R}^{d}$ such that $\mathbf{x\sim}$
$p\left(\mathbf{x};\omega_{1}\right)$ and $\mathbf{x\sim}$
$p\left(\mathbf{x};\omega_{2}\right)$, where
$p\left(\mathbf{x};\omega_{1}\right)$ and
$p\left(\mathbf{x};\omega_{2}\right)$ are certain probability density
functions for two classes $\omega_{1}$ and $\omega_{2}$ of random vectors
$\mathbf{x\in}$ $\mathbb{R}^{d}$, where $y_{i}=\pm 1$ and $\omega_{1}$ or
$\omega_{2}$ is the true category, satisfying the following geometrical and
statistical criteria:

$1$. The discriminant function

(11.2)
$d\left(\mathbf{s}\right)=\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\left(\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right)+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}$

is represented by a geometric locus of a novel principal eigenaxis

(11.3) $\displaystyle\boldsymbol{\rho}$
$\displaystyle=\sum\nolimits_{i=1}^{l_{1}}\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}-\sum\nolimits_{i=1}^{l_{2}}\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}$
$\displaystyle=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$

structured as a locus of signed and scaled extreme vectors
$\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}$ and
$-\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}$, so that a dual locus of
likelihood components and principal eigenaxis components
$\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}$ and
$\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}$ represents an exclusive
principal eigen-coordinate system of the geometric locus of the decision
boundary of the system, and also represents an eigenaxis of symmetry that
spans the decision space of the system, such that each scale factor
$\psi_{1_{i_{\ast}}}$ or $\psi_{2_{i_{\ast}}}$ determines a scaled extreme
vector $\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}$ or
$\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}$ that represents a principal
eigenaxis component that determines a likely location for a correlated extreme
point $\mathbf{x}_{1_{i\ast}}\mathbf{\sim}$
$p\left(\mathbf{x};\omega_{1}\right)$ or $\mathbf{x}_{2_{i\ast}}\mathbf{\sim}$
$p\left(\mathbf{x};\omega_{2}\right)$, along with a likelihood component that
determines a likelihood value for the correlated extreme point
$\mathbf{x}_{1_{i\ast}}$ or $\mathbf{x}_{2_{i\ast}}$, where the reproducing
kernel for each extreme point $k_{\mathbf{x}_{1_{i\ast}}}$ and
$k_{\mathbf{x}_{2_{i\ast}}}$ has the preferred form of either
$k_{\mathbf{x}}\left(\mathbf{s}\right)=\left(\mathbf{s}^{T}\mathbf{x}+1\right)^{2}$
or
$k_{\mathbf{x}}\left(\mathbf{s}\right)=\exp\left(-\gamma\left\|\mathbf{s}-\mathbf{x}\right\|^{2}\right)$,
wherein $0.01\leq\gamma\leq 0.1$;

$2$. The geometric locus of the novel principal eigenaxis
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ is the
solution of the vector algebra locus equation

(11.4)
$\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\left(\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right)+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}=0$

that represents the geometric locus of the decision boundary of the system,
where the expression
$\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}$ represents a locus
of average risk in the decision space $Z=Z_{1}\cup Z_{2}$ of the system, and
the statistic $\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}:y_{i}=\pm 1$ represents
an expected likelihood of observing $l$ extreme vectors
$\left\\{k_{\mathbf{x}_{i\ast}}\right\\}_{i=1}^{l}$ within the decision space
$Z=Z_{1}\cup Z_{2}$, so that all of the points $\mathbf{s}$ that lie on the
geometric locus of the decision boundary exclusively reference the novel
principal eigenaxis
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$, as well as
the vector algebra locus equations

(11.5)
$\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\left(\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right)+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}=+1$

and

(11.6)
$\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\left(\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right)+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}=-1$

that represent the geometric loci of the decision borders of the corresponding
decision regions $Z_{1}$ and $Z_{2}$ of the system, so that all of the points
$\mathbf{s}$ that lie on the geometric loci of the decision borders
exclusively reference the novel principal eigenaxis
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$.

Thereby, the geometric locus of the novel principal eigenaxis
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ represents an
eigenaxis of symmetry that spans the decision space $Z=Z_{1}\cup Z_{2}$ of the
minimum risk binary classification system

$\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\left(\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right)+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}\overset{\omega_{1}}{\underset{\omega_{2}}{\gtrless}}0\text{,}$

at which point the shape of the decision space $Z=Z_{1}\cup Z_{2}$ is
completely determined by the exclusive principal eigen-coordinate system
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$;

$3$. The discriminant function is the solution of the integral equation

(11.7) $\displaystyle f_{1}\left(d\left(\mathbf{s}\right)\right)$
$\displaystyle:\int_{Z_{1}}\boldsymbol{\rho}_{1}d\boldsymbol{\rho}_{1}+\int_{Z_{2}}\boldsymbol{\rho}_{1}d\boldsymbol{\rho}_{1}+C_{1}$
$\displaystyle=\int_{Z_{1}}\boldsymbol{\rho}_{2}d\boldsymbol{\rho}_{2}+\int_{Z_{2}}\boldsymbol{\rho}_{2}d\boldsymbol{\rho}_{2}+C_{2}\text{,}$

over the decision space $Z=Z_{1}\cup Z_{2}$ of the minimum risk binary
classification system

$\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\left(\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right)+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}\overset{\omega_{1}}{\underset{\omega_{2}}{\gtrless}}0\text{,}$

where $C_{1}$ and $C_{2}$ are certain integration constants, so that the total
allowed eigenenergy
$\left\|\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right\|_{\min_{c}}^{2}$
and the expected risk
$\mathfrak{R}_{\mathfrak{\min}}\left(\left\|\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right\|_{\min_{c}}^{2}\right)$
exhibited by the system are jointly regulated by the equilibrium requirement
on the dual locus
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ of the
discriminant function at the geometric locus of the decision boundary of the
system

$\displaystyle d\left(\mathbf{s}\right)$
$\displaystyle:\left\|\boldsymbol{\rho}_{1}\right\|_{\min_{c}}^{2}-\left\|\boldsymbol{\rho}_{1}\right\|\left\|\boldsymbol{\rho}_{2}\right\|\cos\theta_{\boldsymbol{\rho}_{1}\boldsymbol{\rho}_{2}}$
$\displaystyle=\left\|\boldsymbol{\rho}_{2}\right\|_{\min_{c}}^{2}-\left\|\boldsymbol{\rho}_{2}\right\|\left\|\boldsymbol{\rho}_{1}\right\|\cos\theta_{\boldsymbol{\rho}_{2}\boldsymbol{\rho}_{1}}$
$\displaystyle=\frac{1}{2}\left\|\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right\|_{\min_{c}}^{2}\text{,}$

at which point the dual locus
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ of the
discriminant function satisfies the geometric locus of the decision boundary
in terms of a critical minimum eigenenergy
$\left\|\boldsymbol{\rho}\right\|_{\min_{c}}^{2}$ and a minimum expected risk
$\mathfrak{R}_{\mathfrak{\min}}\left(\left\|\boldsymbol{\rho}\right\|_{\min_{c}}^{2}\right)$
in such a manner that regions of counter risks of the system are symmetrically
balanced with regions of risks of the system, so that critical minimum
eigenenergies
$\left\|\psi_{1_{i\ast}}k_{\mathbf{x}_{1_{i\ast}}}\right\|_{\min_{c}}^{2}$
exhibited by principal eigenaxis components
$\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}$ on side
$\boldsymbol{\rho}_{1}$ of the novel principal eigenaxis
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$—that determine
probabilities of finding extreme points $\mathbf{x}_{1_{i\ast}}$ located
throughout the decision space $Z=Z_{1}\cup Z_{2}$ of the system, are
symmetrically balanced with critical minimum eigenenergies
$\left\|\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}\right\|_{\min_{c}}^{2}$
exhibited by principal eigenaxis components
$\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}$ on side
$\boldsymbol{\rho}_{2}$ of the novel principal eigenaxis
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$—that determine
probabilities of finding extreme points $\mathbf{x}_{2_{i\ast}}$ located
throughout the decision space $Z=Z_{1}\cup Z_{2}$ of the system;

$4$. The discriminant function minimizes the integral equation

(11.8) $\displaystyle f_{2}\left(d\left(\mathbf{s}\right)\right)$
$\displaystyle:\int_{Z_{1}}\boldsymbol{\rho}_{1}d\boldsymbol{\rho}_{1}-\int_{Z_{1}}\boldsymbol{\rho}_{2}d\boldsymbol{\rho}_{2}+C_{1}$
$\displaystyle=\int_{Z_{2}}\boldsymbol{\rho}_{2}d\boldsymbol{\rho}_{2}-\int_{Z_{2}}\boldsymbol{\rho}_{1}d\boldsymbol{\rho}_{1}+C_{2}\text{,}$

over the decision regions $Z_{1}$ and $Z_{2}$ of the minimum risk binary
classification system
$\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\left(\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right)+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}\overset{\omega_{1}}{\underset{\omega_{2}}{\gtrless}}0$,
where $C_{1}$ and $C_{2}$ are certain integration constants, so that the
system satisfies a state of statistical equilibrium such that the total
allowed eigenenergy
$\left\|\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right\|_{\min_{c}}^{2}$
and the expected risk
$\mathfrak{R}_{\mathfrak{\min}}\left(\left\|\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right\|_{\min_{c}}^{2}\right)$
exhibited by the system are jointly minimized within the decision space
$Z=Z_{1}\cup Z_{2}$ of the system, at which point critical minimum
eigenenergies
$\left\|\psi_{1_{i\ast}}k_{\mathbf{x}_{1_{i\ast}}}\right\|_{\min_{c}}^{2}$ and
$\left\|\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}\right\|_{\min_{c}}^{2}$
exhibited by corresponding principal eigenaxis components
$\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}$ and
$\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}$ on side
$\boldsymbol{\rho}_{1}$ and side $\boldsymbol{\rho}_{2}$ of the novel
principal eigenaxis
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ are minimized
throughout the decision regions $Z_{1}$ and $Z_{2}$ of the system, so that
regions of counter risks and risks of the system—located throughout the
decision region $Z_{1}$ of the system—are symmetrically balanced with regions
of counter risks and risks of the system—located throughout the decision
region $Z_{2}$ of the system.

Thereby, the minimum risk binary classification system

$\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\left(\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right)+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}\overset{\omega_{1}}{\underset{\omega_{2}}{\gtrless}}0$

satisfies a state of statistical equilibrium so that the total allowed
eigenenergy and the expected risk exhibited by the system are jointly
minimized within the decision space $Z=Z_{1}\cup Z_{2}$ of the system, at
which point the system exhibits the minimum probability of classification
error for any given random vectors $\mathbf{x\in}$ $\mathbb{R}^{d}$ such that
$\mathbf{x\sim}$ $p\left(\mathbf{x};\omega_{1}\right)$ and $\mathbf{x\sim}$
$p\left(\mathbf{x};\omega_{2}\right)$;

$5$. The geometric locus of the novel principal eigenaxis
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ satisfies the
law of cosines in the symmetrically balanced manner

(11.9)
$\displaystyle\frac{1}{2}\left\|\boldsymbol{\rho}\right\|_{\min_{c}}^{2}$
$\displaystyle=\left\|\boldsymbol{\rho}_{1}\right\|_{\min_{c}}^{2}-\left\|\boldsymbol{\rho}_{1}\right\|\left\|\boldsymbol{\rho}_{2}\right\|\cos\theta_{\boldsymbol{\rho}_{1}\boldsymbol{\rho}_{2}}$
$\displaystyle=\left\|\boldsymbol{\rho}_{2}\right\|_{\min_{c}}^{2}-\left\|\boldsymbol{\rho}_{2}\right\|\left\|\boldsymbol{\rho}_{1}\right\|\cos\theta_{\boldsymbol{\rho}_{2}\boldsymbol{\rho}_{1}}\text{,}$

where $\theta$ is the angle between $\boldsymbol{\rho}_{1}$ and
$\boldsymbol{\rho}_{2}$, so that the geometric locus of the novel principal
eigenaxis $\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$
represents an eigenaxis of symmetry that exhibits symmetrical dimensions and
densities, such that the magnitude and the direction of the novel principal
eigenaxis $\boldsymbol{\rho}$ $=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$
are both functions of differences between joint variabilities of extreme
vectors $k_{\mathbf{x}_{1_{i\ast}}}$ and $k_{\mathbf{x}_{2_{i\ast}}}$ that
belong to the two classes $\omega_{1}$ and $\omega_{2}$ of random vectors
$\mathbf{x\in}$ $\mathbb{R}^{d}$, at which point the critical minimum
eigenenergy $\left\|\boldsymbol{\rho}_{1}\right\|_{\min_{c}}^{2}$ exhibited by
side $\boldsymbol{\rho}_{1}$ is symmetrically balanced with the critical
minimum eigenenergy $\left\|\boldsymbol{\rho}_{2}\right\|_{\min_{c}}^{2}$
exhibited by side $\boldsymbol{\rho}_{2}$

$\left\|\boldsymbol{\rho}_{1}\right\|_{\min_{c}}^{2}=\left\|\boldsymbol{\rho}_{2}\right\|_{\min_{c}}^{2}\text{,}$

the length of side $\boldsymbol{\rho}_{1}$ equals the length of side
$\boldsymbol{\rho}_{2}$

$\left\|\boldsymbol{\rho}_{1}\right\|=\left\|\boldsymbol{\rho}_{2}\right\|\text{,}$

and counteracting and opposing forces and influences of the minimum risk
binary classification system

$\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\left(\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right)+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}\overset{\omega_{1}}{\underset{\omega_{2}}{\gtrless}}0$

are symmetrically balanced with each other about the geometric center of the
locus of the novel principal eigenaxis
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$

$\displaystyle\left\|\boldsymbol{\rho}_{1}\right\|\left(\sum\nolimits_{i=1}^{l_{1}}\operatorname{comp}_{\overrightarrow{\boldsymbol{\rho}_{1}}}\left(\overrightarrow{\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}}\right)-\sum\nolimits_{i=1}^{l_{2}}\operatorname{comp}_{\overrightarrow{\boldsymbol{\rho}_{1}}}\left(\overrightarrow{\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}}\right)\right)$
$\displaystyle=\left\|\boldsymbol{\rho}_{2}\right\|\left(\sum\nolimits_{i=1}^{l_{2}}\operatorname{comp}_{\overrightarrow{\boldsymbol{\rho}_{2}}}\left(\overrightarrow{\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}}\right)-\sum\nolimits_{i=1}^{l_{1}}\operatorname{comp}_{\overrightarrow{\boldsymbol{\rho}_{2}}}\left(\overrightarrow{\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}}\right)\right)\text{,}$

whereon the statistical fulcrum of the novel principal eigenaxis
$\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ is located.

Thereby, counteracting and opposing components of critical minimum
eigenenergies related to likely locations of extreme points from class
$\omega_{1}$ and class $\omega_{2}$ that determine regions of counter risks
and risks of the system—along the dual locus of side
$\boldsymbol{\rho}_{1}$—are symmetrically balanced with counteracting and
opposing components of critical minimum eigenenergies related to likely
locations of extreme points from class $\omega_{2}$ and class $\omega_{1}$
that determine regions of counter risks and risks of the system—along the dual
locus of side $\boldsymbol{\rho}_{2}$;

$6$. The center of total allowed eigenenergy and expected risk of the minimum
risk binary classification system

$\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\left(\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right)+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}\overset{\omega_{1}}{\underset{\omega_{2}}{\gtrless}}0$

is located at the geometric center of the locus of the novel principal
eigenaxis $\boldsymbol{\rho}=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$ of
the system, whereon the statistical fulcrum of the system is located;

Then the minimum risk binary classification system acts to jointly minimize
its eigenenergy and risk by locating a point of equilibrium, at which point
critical minimum eigenenergies exhibited by the system are symmetrically
concentrated in such a manner that the geometric locus of the novel principal
eigenaxis of the system is an eigenaxis of symmetry that exhibits symmetrical
dimensions and densities, so that the dual locus of the discriminant function
of the system is in statistical equilibrium at the geometric locus of the
decision boundary of the system, such that counteracting and opposing forces
and influences of the system are symmetrically balanced with each other—about
the geometric center of the locus of the novel principal eigenaxis—whereon the
statistical fulcrum of the system is located.

Thereby, the minimum risk binary classification system satisfies a state of
statistical equilibrium so that the total allowed eigenenergy and the expected
risk exhibited by the system are jointly minimized within the decision space
of the system, at which point the system exhibits the minimum probability of
classification error.

The general locus formula that resolves the direct problem of the binary
classification of random vectors—that is expressed by Theorem 11.1—is readily
generalized to minimum risk multiclass classification systems.

###  11.1 Minimum Risk Multiclass Classification Systems

Corollary 11.2 generalizes the fundamental laws of binary classification
expressed by Theorem 11.1 to minimum risk multiclass classification systems.

###### Corollary 11.2

Any given minimum risk multiclass classification system that is subject to $M$
sources of random vectors $\mathbf{x\in}$ $\mathbb{R}^{d}$ is determined by
$M$ ensembles of $M-1$ minimum risk binary classification systems, such that
each ensemble is determined by an architecture wherein one class is compared
with all of the other $M-1$ classes, so that every one of the $M-1$ minimum
risk binary classification systems in each and every one of the $M$ ensembles
satisfies the geometrical and statistical criteria expressed by Theorem 11.1.

Thereby, the minimum risk multiclass classification system satisfies a state
of statistical equilibrium so that the total allowed eigenenergy and the
expected risk exhibited by the system are jointly minimized within the
decision space of the system, at which point the system exhibits the minimum
probability of classification error.

Proof. Corollary 11.2 is proved by Theorem 11.1 and the superposition
principle [39]—since any given minimum risk multiclass classification system
is based on a “one versus all” architecture.

###  11.2 Fundamental Laws of Binary Classification

Theorem 11.1 expresses fundamental laws of binary classification that
discriminant functions of minimum risk binary classification systems are
subject to. These laws are summarized below.

####  11.2.1 The Law of Total Allowed Eigenenergy

We have named the locus formula in (11.7) “the law of total allowed energy for
minimum risk binary classification systems.” The law of total allowed energy
demonstrates that the total allowed energy and the expected risk exhibited by
any given minimum risk binary classification system are jointly regulated by
an equilibrium requirement—on the dual locus of the discriminant function of
the system at the geometric locus of the decision boundary of the system—at
which point the dual locus of the discriminant function is an eigenaxis of
symmetry that spans the decision space of the system, so that the dual locus
of the discriminant function satisfies the geometric locus of the decision
boundary in terms of a critical minimum eigenenergy and a minimum expected
risk.

####  11.2.2 The Law of Statistical Equilibrium

We have named the locus formula in (11.8) “the law of statistical equilibrium
for minimum risk binary classification systems.” The law of statistical
equilibrium demonstrates that any given minimum risk binary classification
system acts to jointly minimize its eigenenergy and risk by locating a point
of equilibrium, at which point the geometric locus of the novel principal
eigenaxis of the system represents an eigenaxis of symmetry that exhibits
symmetrical dimensions and densities, such that critical minimum eigenenergies
are minimized throughout the decision space of the system, so that regions of
counter risks and risks of the system are symmetrically balanced with each
other, at which point the total allowed eigenenergy and the expected risk
exhibited by the system are jointly minimized within the decision space of the
system—so that the system exhibits the minimum probability of classification
error.

####  11.2.3 The Law of Symmetry

We have named the locus formula in (11.9) “the law of symmetry for minimum
risk binary classification systems.” The law of symmetry demonstrates that the
cost of finding any given minimum risk binary classification system under
uncertainty is the critical minimum eigenenergy that is necessary for the
system to achieve a state of statistical equilibrium, at which point critical
minimum eigenenergies exhibited by the system are symmetrically concentrated
in such a manner that the geometric locus of the novel principal eigenaxis of
the system represents an eigenaxis of symmetry that exhibits symmetrical
dimensions and densities, so that counteracting and opposing forces and
influences of the system are symmetrically balanced with each other—about the
geometric center of the locus of the novel principal eigenaxis—whereon the
statistical fulcrum of the system is located.

In the next part of our treatise, we will prove the fundamental laws of binary
classification expressed by Theorem 11.1 by means of a constructive proof that
demonstrates how a certain constrained optimization algorithm executes each
and every one of the fundamental laws.

We now turn our attention to the constrained optimization algorithm that
resolves the inverse problem of the binary classification of random vectors.

##  12 Finding a Novel Principal Eigenaxis

In this part of our treatise, we use a well-posed constrained optimization
algorithm and a collection of labeled feature vectors to produce an example of
a minimum risk binary classification system that satisfies the geometrical and
statistical criteria expressed by Theorem 11.1. We define a class of
discriminant functions of minimum risk binary classification systems—by an
objective function of an inequality constrained optimization problem—so that a
discriminant function of a minimum risk binary classification system is
represented by a geometric locus of a novel principal eigenaxis, such that the
constrained objective function of the geometric locus of the novel principal
eigenaxis is subject to the geometrical and statistical criteria expressed by
Theorem 11.1.

###  12.1 Objective Function of a Novel Principal Eigenaxis

Take any given collection of labeled feature vectors

$\left(\mathbf{x}_{1}\mathbf{,}y_{1}\right),\ldots,\left(\mathbf{x}_{N}\mathbf{,}y_{N}\right)\in\mathbb{R}^{d}\times
Y,Y=\left\\{\pm 1\right\\}\text{,}$

where $N$ feature vectors $\mathbf{x}\in\mathbb{R}^{d}$ are generated by
certain probability density functions $p\left(\mathbf{x};\omega_{1}\right)$
and $p\left(\mathbf{x};\omega_{2}\right)$ that determine either overlapping
distributions, such that $\bigcap\nolimits_{i=1}^{2}\omega_{i}\neq\emptyset$
and $\bigcap\nolimits_{i=1}^{2}\mathbf{x}\omega_{i}\neq\emptyset$, or non-
overlapping distributions, such that
$\bigcap\nolimits_{i=1}^{2}\omega_{i}=\emptyset$ and
$\bigcap\nolimits_{i=1}^{2}\mathbf{x}\omega_{i}=0$, of $d$-dimensional
numerical features $\mathbf{x\in}$ $\mathbb{R}^{d}$.

We produce an example of a minimum risk binary classification system

$\left(k_{\mathbf{s}}\boldsymbol{-}\frac{1}{l}\sum\nolimits_{i=1}^{l}k_{\mathbf{x}_{i\ast}}\right)\left(\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}\right)+\frac{1}{l}\sum\nolimits_{i=1}^{l}y_{i}\overset{\omega_{1}}{\underset{\omega_{2}}{\gtrless}}0$

that satisfies the geometrical and statistical criteria expressed by Theorem
11.1, so that the discriminant function of the system is represented by a
geometric locus of a novel principal eigenaxis

$\displaystyle\boldsymbol{\rho}$
$\displaystyle=\sum\nolimits_{i=1}^{l_{1}}\psi_{1_{i_{\ast}}}k_{\mathbf{x}_{1_{i\ast}}}-\sum\nolimits_{i=1}^{l_{2}}\psi_{2_{i_{\ast}}}k_{\mathbf{x}_{2_{i\ast}}}$
$\displaystyle=\boldsymbol{\rho}_{1}-\boldsymbol{\rho}_{2}$

that satisfies the conditions of Theorem 11.1, by using the collection of
labeled feature vectors and a well-posed constrained optimization algorithm to
determine the solution for the inequality constrained optimization
problem—known as the primal problem

(12.1) $\displaystyle\min\Psi\left(\boldsymbol{\kappa}\right)$
$\displaystyle=\left\|\boldsymbol{\kappa}\right\|^{2}/2+C/2\sum\nolimits_{i=1}^{N}\xi_{i}^{2}\text{,}$
$\displaystyle\text{s.t.
}y_{i}\left(k_{\mathbf{x}_{i}}\boldsymbol{\kappa}+\boldsymbol{\kappa}_{0}\right)$
$\displaystyle\geq 1-\xi_{i},\ \ i=1,...,N\text{,}$

where
$\boldsymbol{\kappa}\triangleq\boldsymbol{\kappa}_{1}{\large-}\boldsymbol{\kappa}_{2}$
is a geometric locus of a novel principal eigenaxis, $k_{\mathbf{x}_{i}}$ is a
reproducing kernel for the feature vector $\mathbf{x}_{i}$, where the
reproducing kernel $k_{\mathbf{x}}\left(\mathbf{s}\right)$ is either a
Gaussian reproducing kernel
$k_{\mathbf{x}}\left(\mathbf{s}\right)=\exp\left(-\gamma\left\|\mathbf{s}-\mathbf{x}\right\|^{2}\right):0.01\leq\gamma\leq
0.1$, or a second-order polynomial reproducing kernel
$k_{\mathbf{x}}\left(\mathbf{s}\right)=\left(\mathbf{s}^{T}\mathbf{x}+1\right)^{2}$,
$\left\|\boldsymbol{\kappa}\right\|^{2}$ is the eigenenergy exhibited by the
geometric locus of the novel principal eigenaxis $\boldsymbol{\kappa}$,
$\boldsymbol{\kappa}_{0}$ is a functional of $\boldsymbol{\kappa}$, $C$ and
$\xi_{i}$ are regularization parameters for a joint covariance matrix, and
$y_{i}$ are class membership statistics, where $y_{i}=+1$ if
$\mathbf{x}_{i}\in\omega_{1}$, and $y_{i}=-1$ if
$\mathbf{x}_{i}\in\omega_{2}$.

###  12.2 Objective of the Constrained Optimization Algorithm

The objective of the constrained optimization algorithm that solves the primal
optimization problem in (12.1) is to _find_ the geometric locus of the novel
principal eigenaxis $\boldsymbol{\kappa}$ that minimizes the total allowed
eigenenergy $\left\|\boldsymbol{\kappa}\right\|_{\min_{c}}^{2}$ and the
expected risk
$\mathfrak{R}_{\mathfrak{\min}}\left(\left\|\boldsymbol{\kappa}\right\|_{\min_{c}}^{2}\right)$
exhibited by the minimum risk binary classification system
$k_{\mathbf{s}}\boldsymbol{\kappa}+$
$\boldsymbol{\kappa}_{0}\overset{\omega_{1}}{\underset{\omega_{2}}{\gtrless}}0$—within
the decision space $Z=Z_{1}\cup Z_{2}$ of the system—at which point the
geometric locus of the novel principal eigenaxis $\boldsymbol{\kappa}$ is
subject to a critical minimum eigenenergy constraint

$\gamma\left(\boldsymbol{\kappa}\right)=\left\|\boldsymbol{\kappa}\right\|_{\min_{c}}^{2}\text{,}$

so that the system of $N$ inequalities

$y_{i}\left(k_{\mathbf{x}_{i}}\boldsymbol{\kappa}+\boldsymbol{\kappa}_{0}\right)\geq
1-\xi_{i},\ \ i=1,...,N$

is satisfied in the most suitable manner.

###  12.3 The Primal Eigenenergy Functional

The solution for the primal optimization problem in (12.1) is found by using
Lagrange multipliers $\psi_{i}\geq 0$ and the _primal eigenenergy functional_

(12.2)
$\displaystyle\Xi_{\boldsymbol{\kappa}}\left(\boldsymbol{\kappa}\mathbf{,}\boldsymbol{\kappa}_{0},\psi_{i}\right)$
$\displaystyle=\left\|\boldsymbol{\kappa}\right\|^{2}/2+C/2\sum\nolimits_{i=1}^{N}\xi_{i}^{2}$
$\displaystyle-\sum\nolimits_{i=1}^{N}\psi_{i}\left\\{y_{i}\left(k_{\mathbf{x}_{i}}\boldsymbol{\kappa}+\boldsymbol{\kappa}_{0}\right)-1+\xi_{i}\right\\}$

of a minimum risk binary classification system
$k_{\mathbf{s}}\boldsymbol{\kappa}+$
$\boldsymbol{\kappa}_{0}\overset{\omega_{1}}{\underset{\omega_{2}}{\gtrless}}0$,
so that the objective function and its constraints in (12.1) are combined with
each other, at which point the primal eigenenergy functional in (12.2) is
minimized with respect to the primal variables $\boldsymbol{\kappa}$ and
$\boldsymbol{\kappa}_{0}$ and is maximized with respect to the dual variables
$\psi_{i}$.

###  12.4 The Wolfe-dual Principal Eigenspace

The constrained optimization algorithm that resolves the inverse problem—of
the binary classification of random vectors—introduces a _dual eigenenergy
functional_ of a minimum risk binary classification system
$k_{\mathbf{s}}\boldsymbol{\kappa}+$
$\boldsymbol{\kappa}_{0}\overset{\omega_{1}}{\underset{\omega_{2}}{\gtrless}}0$
inside a vector space that we have named the “Wolfe-dual principal
eigenspace,” so that the Wolfe-dual novel principal eigenaxis
$\boldsymbol{\psi}$ of the system $k_{\mathbf{s}}\boldsymbol{\kappa}+$
$\boldsymbol{\kappa}_{0}\overset{\omega_{1}}{\underset{\omega_{2}}{\gtrless}}0$
is symmetrically and equivalently __ related to the primal novel principal
eigenaxis $\boldsymbol{\kappa}$ of the system
$k_{\mathbf{s}}\boldsymbol{\kappa}+$
$\boldsymbol{\kappa}_{0}\overset{\omega_{1}}{\underset{\omega_{2}}{\gtrless}}0$,
and finds _extrema_ for the _restriction_ of the geometric locus of the novel
principal eigenaxis $\boldsymbol{\kappa}$ to the Wolfe-dual principal
_eigenspace_.

We determine the dual eigenenergy functional and the extrema by evaluating the
Karush-Kuhn-Tucker (KKT) conditions on the primal eigenenergy functional in
(12.2).

###  12.5 The Karush-Kuhn-Tucker Conditions

We use the Karush-Kuhn-Tucker theorem [80] and the KKT conditions [17, 73] on
the primal eigenenergy functional in (12.2) to determine geometrical and
statistical conditions that the constrained objective function of the
geometric locus of the novel principal eigenaxis $\boldsymbol{\kappa}$ is
subject to—at which point the minimum risk binary classification system
$k_{\mathbf{s}}\boldsymbol{\kappa}+$
$\boldsymbol{\kappa}_{0}\overset{\omega_{1}}{\underset{\omega_{2}}{\gtrless}}0$
satisfies the geometrical and statistical criteria expressed by Theorem 11.1.

Accordingly, we use the KKT conditions

(12.3)
$\boldsymbol{\kappa}-\sum\nolimits_{i=1}^{N}\psi_{i}y_{i}k_{\mathbf{x}_{i}}=0,\text{\
}i=1,...N\text{,}$ (12.4) $\sum\nolimits_{i=1}^{N}\psi_{i}y_{i}=0,\text{ \
}i=1,...,N\text{,}$ (12.5)
$C\sum\nolimits_{i=1}^{N}\xi_{i}-\sum\nolimits_{i=1}^{N}\psi_{i}=0\text{,}$
(12.6) $\psi_{i}\geq 0,\text{ \ }i=1,...,N\text{,}$ (12.7)
$\psi_{i}\left[y_{i}\left(k_{\mathbf{x}_{i}}\boldsymbol{\kappa}+\boldsymbol{\kappa}_{0}\right)-1+\xi_{i}\right]\geq
0,\ i=1,...,N\text{,}$

and the Karush-Kuhn-Tucker theorem to demonstrate that the primal novel
principal eigenaxis $\boldsymbol{\kappa}$ and the Wolfe-dual novel principal
eigenaxis $\boldsymbol{\psi}$ jointly satisfy an equivalent system of the
fundamental locus equations of binary classification—expressed by Theorem
11.1.

Correspondingly, we demonstrate that the KKT conditions in (12.3) - (12.7)
ensure that the discriminant function
$d\left(\mathbf{s}\right)=k_{\mathbf{s}}\boldsymbol{\kappa}+$
$\boldsymbol{\kappa}_{0}$ is the solution of an equivalent system of the
fundamental locus equations of binary classification expressed by Theorem
11.1, so that the discriminant function satisfies the fundamental __
statistical laws—expressed by Theorem 11.1—that the minimum risk binary
classification system $k_{\mathbf{s}}\boldsymbol{\kappa}+$
$\boldsymbol{\kappa}_{0}\overset{\omega_{1}}{\underset{\omega_{2}}{\gtrless}}0$
is subject to.

###  12.6 The Fundamental Unknowns

The fundamental unknowns associated with the primal optimization problem in
(12.1) are the scale factors $\psi_{i}$ for $N$ scaled unit feature vectors
$\frac{k_{\mathbf{x}_{i}}}{\left\|k_{\mathbf{x}_{i}}\right\|}:$
$\left\\{\psi_{i}\frac{k_{\mathbf{x}_{i}}}{\left\|k_{\mathbf{x}_{i}}\right\|}\right\\}_{i=1}^{N}$
that determine the structure of the geometric locus of the Wolfe-dual novel
principal eigenaxis $\boldsymbol{\psi}$, so that the magnitude of each Wolfe-
dual principal eigenaxis component
$\psi_{i}\frac{k_{\mathbf{x}_{i}}}{\left\|k_{\mathbf{x}_{i}}\right\|}$ that
lies on $\boldsymbol{\psi}$ has a certain positive value $\psi_{i}>0$, and the
direction of the principal eigenaxis component is the direction of the feature
vector $k_{\mathbf{x}_{i}}$.

It will be seen that the value of each active scale factor $\psi_{i}>0$
determines a likelihood value and a likely location—both of which are
normalized relative to length—for a correlated extreme point that is located
within either an overlapping region or near a tail region of distributions of
the $N$ feature vectors $\mathbf{x}$ in the collection of training data
$\left(\mathbf{x}_{1}\mathbf{,}y_{1}\right),\ldots,\left(\mathbf{x}_{N}\mathbf{,}y_{N}\right)\in\mathbb{R}^{d}\times
Y,Y=\left\\{\pm 1\right\\}$.

It will also be seen that the value of each active scale factor $\psi_{i}>0$
determines the magnitude of a scaled extreme vector that lies on the geometric
locus of the primal novel principal eigenaxis $\boldsymbol{\kappa}$ —as well
as a likelihood value and a likely location for the extreme vector.

Since the primal optimization problem in (12.1) is a convex optimization
problem, it follows that the inequalities in (12.6) and (12.7) must only hold
for certain values of the primal variables $\boldsymbol{\kappa}$ and
$\boldsymbol{\kappa}_{0}$ and the Wolfe-dual variables $\psi_{i}$ [80]. We
demonstrate that the KKT conditions in (12.3) - (12.7) restrict the magnitudes
and the eigenenergies exhibited by all of the principal eigenaxis components
on both $\boldsymbol{\psi}$ and $\boldsymbol{\kappa}$ in such a manner that
the minimum risk binary classification system
$k_{\mathbf{s}}\boldsymbol{\kappa}+$
$\boldsymbol{\kappa}_{0}\overset{\omega_{1}}{\underset{\omega_{2}}{\gtrless}}0$
satisfies a state of statistical equilibrium—so that the total allowed
eigenenergy $\left\|Z|\boldsymbol{\kappa}\right\|_{\min_{c}}^{2}$ and the
expected risk
$\mathfrak{R}_{\mathfrak{\min}}\left(Z|\left\|\boldsymbol{\kappa}\right\|_{\min_{c}}^{2}\right)$
exhibited by the system are jointly minimized within the decision space
$Z=Z_{1}\cup Z_{2}$ of the system.

###  12.7 The Wolfe-dual Eigenenergy Functional

Substituting the expressions for $\boldsymbol{\kappa}$ and $\psi_{i}$ in
(12.3) and (12.4) into the primal eigenenergy functional in (12.2) and
simplifying the resulting expression determines the dual eigenenergy
functional of a minimum risk binary classification system
$k_{\mathbf{s}}\boldsymbol{\kappa}+$
$\boldsymbol{\kappa}_{0}\overset{\omega_{1}}{\underset{\omega_{2}}{\gtrless}}0$,
also known as the Wolfe dual problem

(12.8)
$\max\Xi_{\psi_{i}}\left(\psi_{i}\right)=\sum\nolimits_{i=1}^{N}\psi_{i}-\sum\nolimits_{i,j=1}^{N}\psi_{i}\psi_{j}y_{i}y_{j}\frac{k_{\mathbf{x}_{i}}k_{\mathbf{x}_{j}}+\delta_{ij}/C}{2}\text{,}$

where $\delta_{ij}$ is the Kronecker $\delta$ defined as unity for $i=j$ and
$0$ otherwise, at which point the scale factors $\psi_{i}$ for the $N$ scaled
unit feature vectors
$\left\\{\psi_{i}\frac{k_{\mathbf{x}_{i}}}{\left\|k_{\mathbf{x}_{i}}\right\|}\right\\}_{i=1}^{N}$
that determine the structure of the geometric locus of the Wolfe-dual novel
principal eigenaxis $\boldsymbol{\psi}$ are subject to the constraints
$\psi_{i}\geq 0$ and $\sum\nolimits_{i=1}^{N}y_{i}\psi_{i}=0$, where
$y_{i}=\left\\{\pm 1\right\\}$.

The dual eigenenergy functional in (12.8) can be written in vector notation by
letting
$\mathbf{Q}\triangleq\epsilon\mathbf{I}+\widetilde{\mathbf{X}}\widetilde{\mathbf{X}}^{T}$,
where $\epsilon\ll 1$,
$\widetilde{\mathbf{X}}\triangleq\mathbf{D}_{y}\mathbf{X}$, $\mathbf{D}_{y}$
is an $N\times N$ diagonal matrix of class membership statistics $y_{i}$, and
the $N\times d$ matrix $\widetilde{\mathbf{X}}$ is a matrix of $N$ labeled
reproducing kernels for $N$ feature vectors $\mathbf{x\in}$ $\mathbb{R}^{d}$

$\widetilde{\mathbf{X}}=\begin{pmatrix}y_{1}k_{\mathbf{x}_{1}},&y_{2}k_{\mathbf{x}_{2}},&\ldots,&y_{N}k_{\mathbf{x}_{N}}\end{pmatrix}^{T}\text{.}$

Call the dual eigenenergy functional in (12.8) “the Wolfe-dual eigenenergy
functional of a minimum risk binary classification system.” Accordingly, the
matrix version of the Wolfe-dual eigenenergy functional of a minimum risk
binary classification system $k_{\mathbf{s}}\boldsymbol{\kappa}+$
$\boldsymbol{\kappa}_{0}\overset{\omega_{1}}{\underset{\omega_{2}}{\gtrless}}0$
is written as

(12.9)
$\max\Xi_{\boldsymbol{\psi}}\left(\boldsymbol{\psi}\right)=\mathbf{1}^{T}\boldsymbol{\psi}-\frac{\boldsymbol{\psi}^{T}\mathbf{Q}\boldsymbol{\psi}}{2}\text{,}$

at which point the structure and behavior and properties of the Wolfe-dual
novel principal eigenaxis $\boldsymbol{\psi}$ are symmetrically and
equivalently related to the structure and behavior and properties of the
primal novel principal eigenaxis $\boldsymbol{\kappa}$, such that the
geometric locus of the Wolfe-dual novel principal eigenaxis
$\boldsymbol{\psi}$ is subject to the constraints
$\boldsymbol{\psi}^{T}\mathbf{y}=0$ and $\psi_{i}\geq 0$, where $y_{i}\in
Y=\left\\{\pm 1\right\\}$, such that the inequalities $\psi_{i}>0$ only hold
for certain values of $\psi_{i}$.

###  12.8 Estimate of a Joint Covariance Matrix

The symmetric matrix
$\mathbf{Q}\triangleq\epsilon\mathbf{I}+\widetilde{\mathbf{X}}\widetilde{\mathbf{X}}^{T}$
of the random quadratic form
$\boldsymbol{\psi}^{T}\mathbf{Q}\boldsymbol{\psi}$ in the Wolfe-dual
eigenenergy functional of (12.9) provides an estimate of a joint covariance
matrix, such that each element
$y_{i}\left\|k_{\mathbf{x}_{i}}\right\|y_{j}\left\|k_{\mathbf{x}_{j}}\right\|\cos\theta_{k_{\mathbf{x}_{i}}k_{\mathbf{x}_{j}}}$
of the matrix $\mathbf{Q}$ provides an estimate of joint variabilities between
coordinates
$\left\\{\left\|k_{\mathbf{x}_{i}}\right\|\cos\mathbb{\alpha}_{\mathbf{e}_{i}k_{\mathbf{x}_{i}}}\right\\}_{i=1}^{d}$
and
$\left\\{\left\|k_{\mathbf{x}_{j}}\right\|\cos\mathbb{\alpha}_{\mathbf{e}_{i}k_{\mathbf{x}_{j}}}\right\\}_{i=1}^{d}$
of certain feature vectors $k_{\mathbf{x}_{i}}$ and $k_{\mathbf{x}_{j}}$, so
that each element
$y_{i}y_{j}\left\|k_{\mathbf{x}_{i}}\right\|\left\|k_{\mathbf{x}_{j}}\right\|\cos\theta_{k_{\mathbf{x}_{i}}k_{\mathbf{x}_{j}}}$
of the joint covariance matrix $\mathbf{Q}$ where $y_{i}y_{j}=-1$ describes
differences between joint variabilities of feature vectors
$k_{\mathbf{x}_{i}}$ and $k_{\mathbf{x}_{j}}$ that belong to different pattern
classes, at which point each element
$\left\|k_{\mathbf{x}_{i}}\right\|\left\|k_{\mathbf{x}_{j}}\right\|\cos\theta_{k_{\mathbf{x}_{i}}k_{\mathbf{x}_{j}}}$
of the joint covariance matrix $\mathbf{Q}$ is correlated with the distance
$\left\|k_{\mathbf{x}_{i}}-k_{\mathbf{x}_{j}}\right\|$ between the loci of
certain feature vectors $k_{\mathbf{x}_{i}}$ and $k_{\mathbf{x}_{j}}$.

Recall that Theorem 7.1 is an existence theorem that guarantees the existence
of an exclusive principal eigen-coordinate system—which is the principal part
of an equivalent representation of a correlated quadratic form—such that the
exclusive principal eigen-coordinate system is the solution of an equivalent
form of the vector algebra locus equation of the geometric locus of a certain
quadratic curve or surface—at which point the principal eigenaxis of the
geometric locus of the quadratic curve or surface satisfies the geometric
locus of the quadratic curve or surface in terms of its _total allowed
eigenenergy_.

Moreover, the novel principal eigen-coordinate transform method expressed by
Theorem 7.1 reveals that the shape and the fundamental property exhibited by
the geometric locus of the quadratic curve or surface are both determined by
the exclusive principal eigen-coordinate system, such that the eigenvalues of
the symmetric matrix of the correlated quadratic form modulate the
eigenenergies exhibited by the components of the exclusive principal eigen-
coordinate system—which is the principal eigenaxis of the geometric locus of
the quadratic curve or surface, so that the principal eigenaxis satisfies the
geometric locus of the quadratic curve or surface in terms of its total
allowed eigenenergy; and the uniform property exhibited by all of the points
that lie on the geometric locus of the quadratic curve or surface is the total
allowed eigenenergy exhibited by the principal eigenaxis of the geometric
locus of the quadratic curve or surface.

The objective of the Wolfe-dual eigenenergy functional in (12.9) is _led_ by
the _guarantees_ provided by Theorem 7.1.

###  12.9 Objective of the Wolfe-dual Eigenenergy Functional

We realize that the Wolfe-dual eigenenergy functional in (12.9) constitutes a
convex programming problem, so that $\boldsymbol{\psi}$ and
$\boldsymbol{\kappa}$ are both subject to certain geometrical and statistical
conditions—expressed by the Karush-Kuhn-Tucker conditions in (12.3) -
(12.7)—inside the Wolfe-dual principal eigenspace of $\boldsymbol{\psi}$ and
$\boldsymbol{\kappa}$. Moreover, the strong duality theorem [24, 47, 48, 54]
provides us with a guarantee that the structure and behavior and properties
exhibited by the geometric locus of the Wolfe-dual novel principal eigenaxis
$\boldsymbol{\psi}$ are symmetrically and equivalently related to the
structure and behavior and properties exhibited by the geometric locus of the
primal novel principal eigenaxis $\boldsymbol{\kappa}$.

Accordingly, let $\psi_{i\ast}>0$ be an active scale factor associated with
the Wolfe-dual eigenenergy functional

$\max\Xi_{\boldsymbol{\psi}}\left(\boldsymbol{\psi}\right)=\mathbf{1}^{T}\boldsymbol{\psi}-\frac{\boldsymbol{\psi}^{T}\mathbf{Q}\boldsymbol{\psi}}{2}\text{,}$

at which point the geometric locus of the Wolfe-dual novel principal eigenaxis
$\boldsymbol{\psi}$ is subject to the constraints
$\boldsymbol{\psi}^{T}\mathbf{y}=0$ and $\psi_{i\ast}>0$, and let
$k_{\mathbf{x}_{i\ast}}$ be an extreme vector that is used to construct the
joint covariance matrix $\mathbf{Q}$ of the random quadratic form
$\boldsymbol{\psi}^{T}\mathbf{Q}\boldsymbol{\psi}$.

Also, let the geometric locus of the Wolfe-dual novel principal eigenaxis
$\boldsymbol{\psi}$ be symmetrically and equivalently related to the principal
eigenvector $\boldsymbol{\psi}_{\max}$ of the joint covariance matrix
$\mathbf{Q}$ and the inverted joint covariance matrix $\mathbf{Q}^{-1}$
associated with the pair of random quadratic forms
$\boldsymbol{\psi}^{T}\mathbf{Q}\boldsymbol{\psi}$ and
$\boldsymbol{\psi}^{T}\mathbf{Q}^{-1}\boldsymbol{\psi}$, so that the
geometrical and statistical structure and the statistical behavior and
properties exhibited by both $\boldsymbol{\psi}$ and
$\boldsymbol{\psi}_{\max}$ are determined by the values of the active scale
factors $\psi_{i\ast}>0$ for normalized extreme vectors
$\frac{k_{\mathbf{x}_{i\ast}}}{\left\|k_{\mathbf{x}_{i\ast}}\right\|}$.

Finally, let the Wolfe-dual novel principal eigenaxis $\boldsymbol{\psi}$ and
the primal novel principal eigenaxis $\boldsymbol{\kappa}$ be subject to the
Karush-Kuhn-Tucker conditions in (12.3) - (12.7)—inside the Wolfe-dual
principal eigenspace of $\boldsymbol{\psi}$ and $\boldsymbol{\kappa}$, so that
the structure and behavior and properties exhibited by the Wolfe-dual novel
principal eigenaxis $\boldsymbol{\psi}$ are symmetrically and equivalently
related to the structure and behavior and properties exhibited by the primal
novel principal eigenaxis $\boldsymbol{\kappa}$.

Using the above notation and assumptions, along with the guarantees provided
by the strong duality theorem and the novel principal eigen-coordinate
transform method expressed by Theorem 7.1 and Corollaries 7.2 \- 7.3, it
follows that the objective of the Wolfe-dual eigenenergy functional is to
_find_ the _principal_ _eigenvector_ $\boldsymbol{\psi}_{\max}$ of the joint
covariance matrix $\mathbf{Q}$ and the inverted joint covariance matrix
$\mathbf{Q}^{-1}$ associated with the pair of random quadratic forms
$\boldsymbol{\psi}^{T}\mathbf{Q}\boldsymbol{\psi}$ and
$\boldsymbol{\psi}^{T}\mathbf{Q}^{-1}\boldsymbol{\psi}$, where the joint
covariance matrix $\mathbf{Q}$ is associated with the random quadratic form
$\boldsymbol{\psi}^{T}\mathbf{Q}\boldsymbol{\psi}$ in the Wolfe-dual
eigenenergy functional
$\max\Xi_{\boldsymbol{\psi}}\left(\boldsymbol{\psi}\right)=\mathbf{1}^{T}\boldsymbol{\psi}-\frac{\boldsymbol{\psi}^{T}\mathbf{Q}\boldsymbol{\psi}}{2}$
of (12.9), wherein $\boldsymbol{\psi}^{T}\mathbf{y}=0$ and $\psi_{i\ast}>0$,
and $y_{i}\in Y=\left\\{\pm 1\right\\}$, so that the structure and behavior
and properties of the principal eigenvector $\boldsymbol{\psi}_{\max}$ of
$\mathbf{Q}$ and $\mathbf{Q}^{-1}$ are symmetrically and equivalently related
to the structure and behavior and properties of the primal novel principal
eigenaxis $\boldsymbol{\kappa}$ in such a manner that the geometric locus of
the novel principal eigenaxis $\boldsymbol{\kappa}$ is the principal part of
an equivalent representation of the pair of random quadratic forms
$\boldsymbol{\psi}^{T}\mathbf{Q}\boldsymbol{\psi}$ and
$\boldsymbol{\psi}^{T}\mathbf{Q}^{-1}\boldsymbol{\psi}$, such that the
geometric locus of the novel principal eigenaxis $\boldsymbol{\kappa}$ is the
principal eigenaxis of the geometric locus of the decision boundary of a
minimum risk binary classification system, at which point the geometric locus
of the novel principal eigenaxis $\boldsymbol{\kappa}$ satisfies the geometric
locus of the decision boundary of the system in terms of a critical minimum
eigenenergy $\left\|\boldsymbol{\kappa}\right\|_{\min_{c}}^{2}$ and a minimum
expected risk
$\mathfrak{R}_{\mathfrak{\min}}\left(\left\|\boldsymbol{\kappa}\right\|_{\min_{c}}^{2}\right)$,
such that the uniform properties exhibited by all of the points that lie on
the geometric locus of the decision boundary are the critical minimum
eigenenergy $\left\|\boldsymbol{\kappa}\right\|_{\min_{c}}^{2}$ and the
minimum expected risk
$\mathfrak{R}_{\mathfrak{\min}}\left(\left\|\boldsymbol{\kappa}\right\|_{\min_{c}}^{2}\right)$
exhibited by the geometric locus of the novel principal eigenaxis
$\boldsymbol{\kappa}$.

It will be seen that the machine learning algorithm being examined
_transforms_ the random quadratic form
$\boldsymbol{\psi}^{T}\mathbf{Q}\boldsymbol{\psi}$ in the Wolfe-dual
eigenenergy functional
$\max\Xi\left(\boldsymbol{\psi}\right)=\mathbf{1}^{T}\boldsymbol{\psi}-\boldsymbol{\psi}^{T}\mathbf{Q}\boldsymbol{\psi/}2$
of a minimum risk binary classification system
$k_{\mathbf{s}}\boldsymbol{\kappa}+\mathbf{\kappa}_{0}\overset{\omega_{1}}{\underset{\omega_{2}}{\gtrless}}0$—that
is subject to the constraints $\boldsymbol{\psi}^{T}\mathbf{y}=0$ and
$\psi_{i\ast}>0$— _into_ a geometric locus of a novel principal eigenaxis
$\boldsymbol{\kappa}=\boldsymbol{\kappa}_{1}-\boldsymbol{\kappa}_{2}$ of the