{"content":[{"text":"Real-time multi-sensor RGB-D image stitching and noise removal \n Connor Monahan \n cmonahan@wustl.edu","bbox":{"left":0.130881,"top":0.117856,"width":0.738238,"height":0.082771}},{"text":"Abstract \n Low-cost, consumer-oriented depth sensors such as \n the Microsoft Kinect provide an accessible way to \n get quality metric depth data, contrasted with more \n accurate but more expensive LIDAR or time-of- \n flight based sensors. However, these devices by \n themselves suffer from problems such as sparse in- \n formation spaces and limited field of view. This \n project addresses these problems by applying previ- \n ous work in hole filling and image stitching to allow \n for depth maps with wide field of view based on \n multiple depth sensors in different orientations. Ex- \n perimental results demonstrate that with some in- \n corporated improvements, smooth depth maps can \n be generated from multiple source images with min- \n imal tuning.","bbox":{"left":0.121327,"top":0.231533,"width":0.365004,"height":0.254237}},{"text":"Figure 1: Registered Kinect metric depth map","bbox":{"left":0.530569,"top":0.438253,"width":0.331209,"height":0.01782499999999998}},{"text":"1 Introduction \n A normal color image can be augmented with \n depth/distance information at every pixel, forming \n what’s known as an RGB-D image where depth is \n an additional channel. This additional spatial infor- \n mation has been increasingly useful in applications \n of computer vision, including fields such as biomet- \n ric authentication, autonomous mobile robotics, and \n self-driving cars.There exist many methods of com- \n puting a depth map from a scene. Structured light \n sensors, laser scanners, LIDAR systems, time-of-flight \n sensors, and binocular stereo vision cameras produce \n this data tailored to a variety of environments at a \n variety of price points. This paper focuses on the Mi- \n crosoft Kinect sensor, a speckle imaging device with \n small field of view oriented at consumer use in close \n range [5]. Despite its limitations, it remains an attrac- \n tive device for amateur or small scale applications due \n to its low cost on the order of $100 during production \n and $10 on the resale market, especially due to the \n wide variety of possible indoor applications. \n While some limitations cannot be avoided such \n as the restriction to indoor use, we know that we \n can overcome limited fields of view in color cameras \n by applying image stitching algorithms to create a \n panoramic photo [1], thus it should be possible to","bbox":{"left":0.115794,"top":0.499158,"width":0.376074,"height":0.41501299999999997}},{"text":"improve the Kinect depth map by stitching multiple \n views together as well. Solving this problem can thus \n enable the use of cheaper sensors in more complicated \n vision applications. In this project, I will implement \n noise filtering and depth map stitching. I will test \n my implementation using images taken on my Kinect \n sensor, visually comparing the resulting panorama to \n the real environment as well as to characteristics of \n the results shown in [2].","bbox":{"left":0.508139,"top":0.480076,"width":0.376073,"height":0.13858399999999998}},{"text":"2 Background & Related Work \n The algorithm presented in this paper is an adap- \n tation of the work in [2], which investigated depth \n map preprocessing and alignment, color image regis- \n tration, and compositing to solve this problem based \n on pairs of Kinects. That paper in turn first built \n upon the work in [4] who worked on noise reduction \n of Kinect depth maps based on a guided anisotropic \n diffusion method. \n For the second step of stitching together the pro- \n cessed images, they applied the work in [1], which \n solved the stitching problem in color images by com- \n puting features in both images using the SIFT algo- \n rithm [3], performing a dual image keypoint matching \n step using a nearest-neighbor algorithm, and finally \n stitching all the images together using multi-band \n blending based on source images in spherical coordi-","bbox":{"left":0.508139,"top":0.63612,"width":0.376081,"height":0.27805099999999994}},{"text":"1","bbox":{"left":0.49593,"top":0.934083,"width":0.008139000000000007,"height":0.01782499999999998}}],"images_bbox":[{"left":0.526944,"top":0.233133,"width":0.338457,"height":0.196152}],"images_bbox_not_text_overlap":[{"left":0.526944,"top":0.233133,"width":0.338457,"height":0.196152}]}