title: Fisheye calibration OpenCV Python
body: <p>I am using a fisheye camera and I would like to calibrate it and correct its barrel distortion using OpenCV or any other library in Python. I have tried different methods using the classic chessboard pattern through different images.</p>
  <p>I have been following this approach:</p>
  <pre><code>    # Prepare object points, like (0,0,0), (1,0,0), (2,0,0), ...,(7,5,0)
      object_p = np.zeros((self.__ny*self.__nx,3),np.float32)
      object_p[:,:2] = np.mgrid[0:self.__nx,0:self.__ny].T.reshape(-1,2) # s,y coordinates
  
  for image_filename in calibration_filenames:
          # Read in each image
          image = cv2.imread(image_filename)
          image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB) # RGB is standard in matlibplot
  
          # Convert to grayscale
          gray = cv2.cvtColor(image, cv2.COLOR_RGB2GRAY)
  
          # Find the chessboard corners
          ret, corners = cv2.findChessboardCorners(gray, (self.__nx, self.__ny), None)
  
          # If found, draw corners
          if ret == True:
              # Store the corners found in the current image
              object_points.append(object_p) # how it should look like
              image_points.append(corners) # how it looks like
  
              # Draw and display the corners
              cv2.drawChessboardCorners(image, (self.__nx, self.__ny), corners, ret)
              plt.figure()
              plt.imshow(image)
              plt.show()
  
      # Do the calibration
      ret, mtx, dist, rvecs, tvecs = cv2.calibrateCamera(object_points, image_points, shape, None, None)
  </code></pre>
  <p>And then this to correct my image:</p>
  <pre><code>cv2.undistort(image, mtx, dist, None, mtx)
  </code></pre>
  <p>The main problem is that, though correction performs good with some images, is very susceptible to camera location. A variation of cm in the camera will result in very different corrections (see below).
  <a href="https://i.stack.imgur.com/jZPAo.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/jZPAo.jpg" alt="enter image description here" /></a>
  <a href="https://i.stack.imgur.com/O5kLW.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/O5kLW.jpg" alt="enter image description here" /></a></p>
  <p>Any idea of how could I control this zoom effect to show maintain the grid area always in my corrected image?</p>
  <p>I am aware that this question may be similar to the one in <a href="https://stackoverflow.com/questions/13013513/fisheye-wide-angle-lens-calibration-in-opencv">here</a>.</p>