<s>[INST] optimize distance from point to plane with Lagrange method I am trying to optimize distance from point to plane using Lagrange multiplier.
Usually for such problems you are given specific point like (1,2,3) in 3D, and then an exact plane which is just the subject of Lagrange. But what I have here doesn't specify values for point and plane.
It says problem happens in a D-dimensional space. It denotes the point as X, the plane as (wT)x+b=0, where wT is the transpose of w (which is just a n-by-1 matrix I suppose), and requires me to use Lagrange to optimize the distance. The final result should be expressed with w, b and X. Without specific values I am really lost in how to approach such kind of problem. Any suggestions?
 [/INST] I think what you are looking for is kind of this
Distance from point to plane
\begin{equation*}
D(\mathbf{x}_{0},\mathbf{P}) = \frac{\vert \mathbf{w}^{T} \mathbf{x}_{0} + \mathbf{b} \vert}
{\Vert \mathbf{w} \Vert_{2}}
\end{equation*}
Let 
\begin{equation*}
\mathbf{P} = \lbrace \mathbf{x} \in \mathbb{R}^{n} / \mathbf{w}^{T} \mathbf{x} + \mathbf{b} = 0\rbrace
\end{equation*}
So
\begin{equation*}
D(\mathbf{x}_0, \mathbf{P}) = \inf_{\mathbf{x} \in \mathbf{P} }  \Vert \mathbf{x} - \mathbf{x}_{0} \Vert_{2} 
\end{equation*}
In order to get the minimum value satisfying above, it is needed to calculate de Lagragian 
\begin{equation*}
\mathit{L}(\mathbf{x}, \lambda) = \Vert \mathbf{x} - \mathbf{x}_{0} \Vert_{2} + \lambda ( \mathbf{w}^{T} \mathbf{x} + \mathbf{b} ) \hspace{0.3cm} \text{where} \hspace{0.3cm} \mathbf{x}^T \mathbf{x} = \Vert \mathbf{x} \Vert_{2}^{2}
\end{equation*}
Finding partial derivatives it yields:
\begin{equation*}
\frac{\partial \mathit{L}(\mathbf{x}, \lambda)}{\partial \mathbf{x}}
= \frac{\mathbf{x} - \mathbf{x}_{0}}{ \Vert \mathbf{x} - \mathbf{x}_{0} \Vert_{2}}
+ \lambda \mathbf{w} = 0
\end{equation*}
\begin{equation*}
\frac{\partial \mathit{L}(\mathbf{x}, \lambda)}{\partial \lambda}
= \mathbf{w}^{T} \mathbf{x} + \mathbf{b} = 0
\end{equation*}
Focusing on the second equation we have
\begin{equation*}
\mathbf{w}^{T} \mathbf{x} + \mathbf{b} = 0 \hspace{0.3cm} \text{so} \hspace{0.3cm}
\mathbf{w}^{T} ( \mathbf{x} - \mathbf{x}_{0} ) = - ( \mathbf{w}^{T} \mathbf{x}_{0} + \mathbf{b} )
\end{equation*}
In the first derivative we make a right internal product of $\mathbf{w}^{T}$
\begin{equation*}
\frac{\mathbf{w}^{T}(\mathbf{x} - \mathbf{x}_{0})}{ \Vert \mathbf{x} - \mathbf{x}_{0} \Vert_{2}}    
+ \lambda \mathbf{w}^{T} \mathbf{w} = 0
\end{equation*}
from above, we have
\begin{equation*}
-\frac{( \mathbf{w}^{T} \mathbf{x}_{0} + \mathbf{b} )}{ \Vert \mathbf{x} - \mathbf{x}_{0} \Vert_{2}}    
+ \lambda \mathbf{w}^{T} \mathbf{w} = 0
\end{equation*}
from which
\begin{equation*}
\lambda^{\star} = \frac{( \mathbf{w}^{T} \mathbf{x}_{0} + \mathbf{b} )}{ \Vert \mathbf{w} \Vert_{2}^{2} \Vert \mathbf{x} - \mathbf{x}_{0} \Vert_{2}}    
\end{equation*}
Replacing $\lambda$ in the first equation, it yields;
\begin{equation*}
\frac{\mathbf{x} - \mathbf{x}_{0}}{ \Vert \mathbf{x} - \mathbf{x}_{0} \Vert_{2}}
+ \frac{( \mathbf{w}^{T} \mathbf{x}_{0} + \mathbf{b} )}{ \Vert \mathbf{w} \Vert_{2}^{2} \Vert \mathbf{x} - \mathbf{x}_{0} \Vert_{2}} \mathbf{w} = 0
\end{equation*}
So
\begin{equation*}
\mathbf{x} - \mathbf{x}_{0} = - \frac{( \mathbf{w}^{T} \mathbf{x}_{0} + \mathbf{b} )}{ \Vert \mathbf{w} \Vert_{2}^{2}} \mathbf{w} \hspace{0.3cm} \text{or}\hspace{0.3cm} 
\mathbf{x}^{\star} = \mathbf{x}_{0} - \frac{( \mathbf{w}^{T} \mathbf{x}_{0} + \mathbf{b} )}{ \Vert \mathbf{w} \Vert_{2}^{2}} \mathbf{w}
\end{equation*}
Therefore
\begin{equation*}
\Vert \mathbf{x} - \mathbf{x}_{0} \Vert_{2} =  \frac{\vert \mathbf{w}^{T} \mathbf{x}_{0} + \mathbf{b} \vert}{ \Vert \mathbf{w} \Vert_{2}} 
\end{equation*}
</s>