Patch Matching for Optical Flow with Thresholded Hinge Embedding Loss'
---

Introduction
============

In recent years, variants of the PatchMatch [@barnes2009patchmatch] approach showed not only to be useful for nearest neighbor field estimation, but also for the more challenging problem of large displacement optical flow estimation. So far, most top performing methods like Deep Matching [@weinzaepfel2013deepflow] or Flow Fields [@bailer2015flow] strongly rely on robust multi-scale matching strategies, while they still use engineered features (data terms) like SIFTFlow [@liu2008sift] for the actual matching.

On the other hand, works like [@simo2015discriminative; @zagoruyko2015learning] demonstrated the effectiveness of features based on Convolutional Neural Network (CNNs) for matching patches. However, these works did not validate the performance of their features using an actual patch matching approach like *PatchMatch* or *Flow Fields* that matches all pixels between image pairs. Instead, they simply treat matching patches as a classification problem between a predefined set of patches.

This ignores many practical issues. For instance, it is important that CNN based features are not only able to distinguish between different patch positions, but the position should also be determined accurately. Furthermore, the top performing CNN architectures are very slow when used for patch matching as it requires matching several 