– it has to come at the cost of failure for other pairs of patches. A possible explanation for this cost is shown in Figure \[lossex2\]. As a result, we introduce a modified hinge embedding loss with threshold $t$ that stops the network from minimizing $L_2$ distances too much:

$$\label{Eq2}
l_t(p_1,p_2) =  \begin{cases}  max (0, L_2(p_1,p_2)- t), ~~~~~~~~~~~~~~~ p_1 \equiv p_2 \\ max(0,m - (L_2(p_1,p_2)- t)),~~~ p_1 \neq p_2    \end{cases}$$

We add $t$ also to the second equation to keep the “virtual decision boundary” at $m/2$. This is not necessary but makes comparison between different $t$ values fairer.

![ If a sample is pushed (blue arrow), although it is clearly on the correct side of the decision boundary other samples also move due to weight change. If most samples are classified correctly beforehand, this creates more false decision boundary crossings than correct ones. $l_h$ performs the unnecessary push, $l_t$ not. []{data-label="lossex2"}](img/lossexample3.pdf){width="1.0\linewidth"}

As our goal is a network that creates features with the property $L_2(p_1,p^+_2)<L_2(p_1,p^-_2)$ one might argue that it is better to train this property directly. A known function to do this is a gap based loss [@han2015matchnet; @wohlhart2015learning], that only keeps a gap in the $L_2$ distance between matching 