title: Display Color and Depth using OpenNI and OpenCV
body: <p>I've been posting quite alot recently about accessing depth images from a Kinect camera using OpenNI and OpenCV.</p>
  
  <p>Following some tutorials and advice from some other posts I have been able to write this script for showing the 2D color stream of the camera (if the second part is commented out), and the depth stream, which is not yet working:</p>
  
  <pre><code>#include &lt;opencv2/opencv.hpp&gt;
  #include &lt;opencv2\highgui\highgui.hpp&gt;
  #include &lt;opencv2\imgproc\imgproc.hpp&gt;
  #include &lt;OpenNI.h&gt;
  
  int main()
  {
      openni::Device device;
  
      openni::VideoStream  color;
      openni::VideoStream depth;
  
      openni::VideoFrameRef depthFrame;
      openni::VideoFrameRef colorFrame;
  
      openni::Status rc = openni::STATUS_OK;
  
      rc = openni::OpenNI::initialize();
      rc = device.open(openni::ANY_DEVICE);
  
      rc = color.create(device, openni::SENSOR_COLOR);
      rc = color.start();
      rc = depth.create(device, openni::SENSOR_DEPTH);
      rc = depth.start();
  
      cv::Mat framecolor;
      cv::Mat framedepth;
  
      while (true)
      {
          color.readFrame(&amp;colorFrame);
          const openni::RGB888Pixel* imageBuffer = (const openni::RGB888Pixel*)colorFrame.getData();
  
          framecolor.create(colorFrame.getHeight(), colorFrame.getWidth(), CV_8UC3);
          memcpy(framecolor.data, imageBuffer, 3 * colorFrame.getHeight()*colorFrame.getWidth() * sizeof(uint8_t));
  
          cv::cvtColor(framecolor, framecolor, CV_BGR2RGB); //this will put colors right
          cv::imshow("framecolor", framecolor);
          ////////////////////////////Second Part/////////////////////////////////////
          depthFrame.getVideoMode();
          const openni::DepthPixel* imageBuffer2 = (const openni::DepthPixel*)depthFrame.getData();
  
          framedepth.create(depthFrame.getHeight(), depthFrame.getWidth(), CV_16U);
          memcpy(framecolor.data, imageBuffer2, 3 * depthFrame.getHeight()*depthFrame.getWidth() * sizeof(uint16_t));
          framedepth.convertTo(framedepth, CV_8U);
          cv::imshow("framedepth", framedepth);
  
          if (cvWaitKey(30)&gt;=0)
          {
              break;
          }
      }
      cv::destroyAllWindows();
      return 0;
  }
  </code></pre>
  
  <p>The error im getting takes me to the openNI.g file, with the following:</p>
  
  <p><a href="https://i.stack.imgur.com/copI6.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/copI6.png" alt="enter image description here"></a></p>
  
  <p>I'm guessing that it has something to do with the file types or pixel formats. there is however alot of option and i'm not even certain that this method of doing it will even work. </p>
  
  <p>Can anyone help with the required filetypes or procedures?</p>