title: Reading Image Stream from RCCC Bayer Camera Sensor in Ubuntu
body: <p>I am working with a LI-AR0820 GMSL2 camera which uses the On-Semi AR0820 sensor that captures images in a 12-Bit RCCC Bayer format. I want to read the real-time image stream from the camera and turn it into a grayscale image (using <a href="https://www.analog.com/media/en/technical-documentation/application-notes/EE358.pdf" rel="nofollow noreferrer">this</a> demosaicing algorithm) and then feed it into an object detection algorithm. However, since OpenCV does not support the RCCC format I can't use the VideoCapture class to get image data from the camera. I am looking for something similar to get the streamed image data in an array-like format so that I can further manipulate it. Any ideas?</p>
  
  <p>I'm running Ubuntu 18.04 with OpenCV 3.2.0 and Python 3.7.1.</p>
  
  <p><strong>EDIT.</strong> I am using the code <a href="https://stackoverflow.com/questions/25799264/characterizing-my-raw-camera-output">here</a>.</p>
  
  <pre><code>#include &lt;vector&gt;
  #include &lt;iostream&gt;
  #include &lt;stdio.h&gt;
  #include &lt;opencv2/opencv.hpp&gt;
  #include &lt;opencv2/highgui/highgui.hpp&gt;
  
  int main() {
      // Each pixel is made up of 16 bits, with the high 4 bits always equal to 0
      unsigned char bytes[2];
  
      // Hold the data in a vector
      std::vector&lt;unsigned short int&gt; data;
  
      // Read the camera data
      FILE *fp = fopen("test.raw","rb");
      while(fread(bytes, 2, 1, fp) != 0) {
          // The data comes in little-endian, so shift the second byte right and concatenate the first byte
          data.push_back(bytes[0] | (bytes[1] &lt;&lt; 8));
      }
  
      // Make a matrix 1280x720 with 16 bits of unsigned integers
      cv::Mat imBayer = cv::Mat(720, 1280, CV_16U);
  
      // Make a matrix to hold RGB data
      cv::Mat imRGB;
  
      // Copy the data in the vector into a nice matrix
      memmove(imBayer.data, data.data(), data.size()*2);
  
      // Convert the GR Bayer pattern into RGB, putting it into the RGB matrix!
      cv::cvtColor(imBayer, imRGB, CV_BayerGR2RGB);
  
      cv::namedWindow("Display window", cv::WINDOW_AUTOSIZE);
      // *15 because the image is dark
      cv::imshow("Display window", 15*imRGB);
  
      cv::waitKey(0);
  
      return 0;
  }
  </code></pre>
  
  <p>There are two problems with the code. First, I have to get a raw image file using fswebcam and then use the code above to read the raw file and display the image. I want to be able to access the /dev/video1 node and directly read the raw data from there instead of having to first save it and then read it separately. Second, OpenCV does not support the RCCC Bayer format so I have to come up with a demosaicing method.</p>
  
  <p>The camera outputs serialized data through a Coax cable, so I use a Deser board with USB 3.0 connection to connect the camera to my laptop. The setup can be seen <a href="https://leopardimaging.com/product/li-ar0820-gmsl2/" rel="nofollow noreferrer">here</a>.</p>