[@Mikolajczyk2005A], Symbench [@Snavely2012Image], Heinly [@Heinly2012Comparative], and AdelaideRMF [@Wong2011Dynamic]. Exemplar images from these datasets and a brief summarization of their inherited nuisances are shown in Fig. \[fig:dataset\] and Table \[tab:dataset\], respectively.

[lccc]{} Dataset & Challenges & Matching pairs\
VGG [@Mikolajczyk2005A] & Zoom, rotation, blur, viewpoint change, & 40\
& light change and JPEG compression &\
Symbench [@Snavely2012Image] & Light change, & 46\
& different rendering styles &\
Heinly [@Heinly2012Comparative] & Zoom and rotation & 29\
AdelaideRMF [@Wong2011Dynamic] & Multi-structures, & 38\
& viewpoint change &\

**The VGG dataset [@Mikolajczyk2005A].** VGG is a hybrid dataset involving eight scenes. Each scene consists of six images with the first image being the reference one with respect to the others. Challenges including blur, viewpoint change, zoom, rotation, light change, and JPEG compression exist in this dataset. The ground-truth is the homography matrix $\mathbf{H}$, indicating that the transformation between two images on each scene satisfies the plane homographic constraint.

**The Symbench dataset [@Snavely2012Image].** The Symbench dataset is composed of 46 image pairs. Each pair includes the same object with light change or different rendering styles. The homographic transformation $\mathbf{H}$ of each image pair is given as the ground-truth.

**The Heinly dataset [@Heinly2012Comparative].** The Heinly dataset comprises 