title: Import OpenCV Mat into C++ Tensorflow without copying
body: <p>My goal is to run a TensorFlow model in real time to control a vehicle from a learned model.  Our vehicle system uses ROS (Robot Operating System) which is tied closely to OpenCV.  So, I receive an OpenCV Mat containing the image of interest from ROS.</p>
  
  <pre><code>    cv::Mat cameraImg;
  </code></pre>
  
  <p>I would like to create a Tensorflow Tensor directly from the data in this OpenCV matrix to avoid the expense of copying the matrix line-by-line.  Using the answer to <a href="https://stackoverflow.com/questions/36044197/how-do-i-pass-an-opencv-mat-into-a-c-tensorflow-graph">This Question</a> I have managed to get the forward pass of the network working with the following code:</p>
  
  <pre><code>cameraImg.convertTo(cameraImg, CV_32FC3);
  
  Tensor inputImg(DT_FLOAT, TensorShape({1,inputheight,inputwidth,3}));
  auto inputImageMapped = inputImg.tensor&lt;float, 4&gt;();
  auto start = std::chrono::system_clock::now();
  //Copy all the data over
  for (int y = 0; y &lt; inputheight; ++y) {
      const float* source_row = ((float*)cameraImg.data) + (y * inputwidth * 3);
      for (int x = 0; x &lt; inputwidth; ++x) {
          const float* source_pixel = source_row + (x * 3);
          inputImageMapped(0, y, x, 0) = source_pixel[2];
          inputImageMapped(0, y, x, 1) = source_pixel[1];
          inputImageMapped(0, y, x, 2) = source_pixel[0];
      }
  }
  auto end = std::chrono::system_clock::now();
  </code></pre>
  
  <p>However, using this method the copy to the tensor takes between 80ms and 130ms, while the entire forward pass (for a 10-layer convolutional network) only takes 25ms.  </p>
  
  <p>Looking at <a href="https://www.tensorflow.org/versions/r0.10/api_docs/cc/ClassTensor.html" rel="noreferrer">the tensorflow documentation</a>, it appears there is a Tensor constructor that takes an allocator.  However, I have not been able to find any Tensorflow or Eigen documentation relating to this functionality or the <a href="https://eigen.tuxfamily.org/dox/group__TutorialMapClass.html" rel="noreferrer">Eigen Map class</a> as it relates to Tensors.</p>
  
  <p>Does anyone have any insight into how this code can be sped up, ideally by re-using my OpenCV memory?</p>
  
  <p><strong>EDIT:</strong>
  I have successfully implemented what @mrry suggested, and can re-use the memory allocated by OpenCV.  I have opened <a href="https://github.com/tensorflow/tensorflow/issues/8033" rel="noreferrer">github issue 8033</a> requesting this be added to the tensorflow source tree.  My method isn't that pretty, but it works.</p>
  
  <p>It is still very difficult to compile an external library and link it to the libtensorflow.so library.  Potentially the <a href="https://github.com/cjweeks/tensorflow-cmake" rel="noreferrer">tensorflow cmake library</a> will help with this, I have not yet tried it.</p>