<hr>

<blockquote>
  <p>Did I am suppose to use a pre-train model?</p>
</blockquote>

<p>Yes you should, unless you are super confident that you can find a working model directly by urself. </p>

<hr>

<blockquote>
  <p>But the is no pre-train model using lidar image</p>
</blockquote>

<p>First I`m pretty sure there are LIDAR based network .e.g </p>

<blockquote>
  <p>L Caltagirone , LIDAR-Camera Fusion for Road Detection Using Fully
  Convolutional ... arxiv, ‎2018</p>
</blockquote>

<p>Second, even if there is no open source implementation for direct LIDAR-based, You can always convert the LIDAR to the depth image. For Depth image based CNN, there are hundreds of implementation for segmentation and detection. </p>

<hr>

<blockquote>
  <p>How am I suppose to do it?</p>
</blockquote>

<p>First, you can place them side by side parallel, for RGB and depth/LIDAR 3d pointcloud. Feed them separately</p>

<p>Second, you can also combine them by merging the input to 4D tensor and transfer the initial weight to the single model. At last perform transfer learning in your given dataset. </p>

<hr>

<blockquote>
  <p>best CNN algorithm?</p>
</blockquote>

<p>Totally depends on your task and hardware. Do you need best in processing speed or best in accuracy? Define your "best", please. </p>

<p>ALso Are you using it for autonomous car or for in-house nurse care system? different CNN system customizes the weight for different purposes. </p>

<p>Generally, for real-time multiple object detection using a cheap PC e.g DJI manifold, I would suggest Yolo-tiny</p>
Using Lidar images and Camera images to perform object detection
<p>I obtain depth &amp; reflectance maps from Lidar (2D images) and I have also camera images (2D images). Image have the same size.</p>

<p>I want to use CNN to perform object detection using both images. It is a sort of "fusion CNN"</p>

<p>How am I suppose to do it? Did I am suppose to use a pre-train model? But the is no pre-train model using lidar images..</p>

<p>Which is the best CNN algorithm to do it? ie for performing fusion of modalities for object detection</p>

<p>Thanks you in advance</p>