<p>I have found an approximate solution in <a href="https://arxiv.org/pdf/1611.09813v5.pdf" rel="nofollow noreferrer">https://arxiv.org/pdf/1611.09813v5.pdf</a></p>
Find camera translation given 3d world coordinates and 2d image points
<p>I only know my camera rotation <code>R</code>, but without the translational component <code>T</code>. Intrinsics are known, and I have a set of 3D points <code>P</code> with their 2D correspondences <code>Q</code>. Is there a method to compute <code>T</code> directly, and at best is differentiable in <code>P</code>.</p>