<p>I studied this in my research. </p>

<p>The first is you have to calibrate 2 sensors to know their extrinsic. There are a few open source packages you can play with which I listed Below</p>

<p>The Second is fuse the data. The simple way is just based on calibration transform and use the tf to send. The complicated way is to deply pipelines such as depth image to LIDAR alignment and depth map variance estimation and fusion. You can choose to do it ez way like easiler landmark included EKF estimation, or you can follow CMU Zhangji`s Visual-LIDAR-Inertial fusion work for the direct 3D feature to LIDAR alignment. The choice is urs</p>

<p>(1)
<a href="https://i.stack.imgur.com/mGCk1.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/mGCk1.png" alt="enter image description here"></a>
<a href="http://wiki.ros.org/velo2cam_calibration" rel="nofollow noreferrer">http://wiki.ros.org/velo2cam_calibration</a></p>

<p>Guindel, C., Beltrán, J., Martín, D. and García, F. (2017). Automatic Extrinsic Calibration for Lidar-Stereo Vehicle Sensor Setups. IEEE International Conference on Intelligent Transportation Systems (ITSC), 674–679.</p>

<p>Pros. Pretty accurate and ez to use package. Cons. you have to made rigid cut board. </p>

<p>(2) <a href="https://github.com/ankitdhall/lidar_camera_calibration" rel="nofollow noreferrer">https://github.com/ankitdhall/lidar_camera_calibration</a>
<a href="https://i.stack.imgur.com/UoAax.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/UoAax.png" alt="enter image description here"></a></p>

<p><a href="https://i.stack.imgur.com/LQup0.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/LQup0.png" alt="enter image description here"></a></p>

<p>LiDAR-Camera Calibration using 3D-3D Point correspondences, arXiv 2017</p>

<p>Pros. Ez to use, Ez to make the hardware. Cons May is not so accurate</p>

<p>There were couple of others I listed In my thesis, I`ll go back and check for it and update here. If I remember</p>
3D Stereo_camera and 2D LiDAR pointclouds data fusion
<p>I have two separate pointclouds(<code>type= sensor_msgs/PointCloud2</code>) from two different sensors, a <strong><em>3D stereo camera</em></strong> and a <strong><em>2D LiDAR</em></strong>. I wanted to know how can I fuse these two pointclouds if the stereo pointcloud is 3D with fix length and a 2D LiDAR pointcloud with variable pointcloud length?</p>

<p>If someone has worked on it please help me, your help will be highly appreciated.
Thanks</p>