increased (see [@simo2015discriminative] for a more detailed description).

Siamese architectures can be strongly speed up at testing time as neighboring patches in the image share convolutions. Details on how the speedup works are described in our supplementary material. The network that we used for our experiments is shown in Table \[arch\]. Similar to [@brown2011discriminative], we use Tanh nonlinearity layers as we also have found them to outperform ReLU for Siamese based patch feature creation.

Loss Function and Batch Selection {#s31}
---------------------------------

The most common loss function for Siamese network based feature creation is the hinge embedding loss: $$l_h(p_1,p_2) =  \begin{cases} L_2(p_1,p_2), ~~~~~~~~~~~~~~~~~~~~~~~~ p_1 \equiv p_2 \\ max(0,m - L_2(p_1,p_2)),~ p_1 \neq p_2    \end{cases}$$ $$L_2(p_1,p_2) = || D(p_1) - D (p_2) ||_2$$

It tries to minimize the $L_2$ distance of matching patches and to increase the $L_2$ distance of non-matching patches above $m$. An architectural flaw which is not or only poorly treated by existing loss functions is the fact that the loss pushes feature distances between matching patches unlimitedly to zero ($ L_2(p_1,p^+_2) \rightarrow 0$). We think that training up to very small $L_2$ distances for patches that differ due to effects like rotation or motion blur is very costly 