lead to a significant decrease in accuracy. Hence, the ability to handle outliers is essential for performing P$n$P algorithms in practical applications. The most common outliers handling mechanism is to combine a P$n$P ($n=$ 3 or 4) algorithm [@nister2004minimal][@gao2003complete][@josephson2009pose] with the RANSAC-based scheme [@fischler1981random] to eliminate outliers, and then perform a more accurate P$n$P algorithm with the remaining inliers to refine the result. A number of very fast closed-form P3P [@kneip2011novel] or P4P [@bujnak2008general] algorithms have been proposed. However, their RANSAC combination scheme still needs many trials until the selected three or four 3D-2D correspondences are all inliers, which results in a high time complexity. Hence, the computational speed decreases significantly as the percentage of outliers increases. To reduce the time complexity, one natural idea is to utilize the P$n$P algorithm with a smaller $n$. However, when $n=$ 1 or 2, the P$n$P problem has infinitely many solutions, which makes the conventional RANSAC-based scheme infeasible.

To the best of our knowledge, except for RANSAC-based methods, the only P$n$P method that addresses outliers is REPP$n$P [@ferraz2014very] proposed by Ferraz *et al*., which is the state-of-the-art P$n$P method that is robust to outliers. REPP$n$P integrates an outlier rejection mechanism with camera pose 