title: Matlab Stereo Camera Calibration Scene Reconstruction Error
body: <p>I am trying to use the Computer Vision System Toolbox to calibrate the pair of cameras below in order to be able to generate a 3-D point cloud of a vehicle at a range between 1 to 5m.  The output image size was approximately 1 MB per image for the checkerboard calibration images and the checkerboard square size was 25 mm.  The cameras used were a pair of SJ4000 HD1080P cameras.  The cameras were placed as parallel to each other as possible with no angle in the vertical axis.  The checkboard calibration was done with the aid of a bright light and whiteboard.  The mean error per pixel using the stereo camera calibrator code was 3.31 with 31/32 successful pairings.  The approximate distance to the checkerboard was 30 cm and the distance between the cameras was 20 cm.
  <a href="https://i.stack.imgur.com/owxeg.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/owxeg.jpg" alt="enter image description here"></a>
  The problem I am encountering upon rectification is during the 3D Reconstruction of the scene.  The figure below is what was outputted.  I’m not sure if there is a parameter that is missing in the camera setup or if there is something that is missing / needs to be added in the script.  Below is the code that was used for the Stereo anaglyph and scene reconstruction that was adapted from the Matlab Stereo Camera Calibration tutorial.</p>
  
  <pre><code>% Read in the stereo pair of images.
  I1 = imread('left.jpg');
  I2 = imread('right.jpg');
  
  % Rectify the images.
  [J1, J2] = rectifyStereoImages(I1, I2, stereoParams);
  
  % Display the images before rectification.
  figure;
  imshow(stereoAnaglyph(I1, I2), 'InitialMagnification', 50);
  title('Before Rectification');
  
  % Display the images after rectification.
  figure;
  imshow(stereoAnaglyph(J1, J2), 'InitialMagnification', 50);
  title('After Rectification');
  % 
  % Compute Disparity for 3-D Reconstruction 
  % The distance in pixels between corresponding points in the rectified images is called disparity. 
  % The disparity is used for 3-D reconstruction, because it is proportional to the distance between the cameras and the 3-D world point.
  disparityMap = disparity(rgb2gray(J1), rgb2gray(J2));
  figure;
  imshow(disparityMap, [0, 64], 'InitialMagnification', 50);
  colormap('jet');
  colorbar;
  title('Disparity Map');
  
  %Reconstruct the 3-D Scene
  %Reconstruct the 3-D world coordinates of points corresponding to each pixel     from the disparity map.
  
  point3D = reconstructScene(disparityMap, stereoParams);
  
  % Convert from millimeters to meters.
  point3D = point3D / 1000;
  
  % Visualize the 3-D Scene
  % Plot points between 3 and 7 meters away from the camera.
  z = point3D(:, :, 3);
  zdisp = z;
  point3Ddisp = point3D;
  point3Ddisp(:,:,3) = zdisp;
  showPointCloud(point3Ddisp, J1, 'VerticalAxis', 'Y',...
      'VerticalAxisDir', 'Down' );
  xlabel('X');
  ylabel('Y');
  zlabel('Z');
  </code></pre>
  
  <p>I have included the images of the Scene Reconstruction, Disparity Map, Mean Error Per Pixel and After Rectification.  The version of Matlab I am using is R2014b Student Edition purchased from the Matlab Website.</p>
  
  <p><a href="https://i.stack.imgur.com/Ewqmy.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/Ewqmy.png" alt="Scene Reconstruction"></a></p>
  
  <p><a href="https://i.stack.imgur.com/IdXrZ.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/IdXrZ.png" alt="Mean Error per Pixel"></a></p>
  
  <p><a href="https://i.stack.imgur.com/UqUO1.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/UqUO1.jpg" alt="After Rectification"></a></p>
  
  <p><a href="https://i.stack.imgur.com/SgbVM.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/SgbVM.jpg" alt="Disparity Map"></a></p>