Here's the simplest possible test case for remap():

import cv2
import numpy as np
inimg = np.arange(2*2).reshape(2,2).astype(np.float32)
inmap = np.array([[0,0],[0,1],[1,0],[1,1]]).astype(np.float32)
outmap = np.array([[10,10],[10,20],[20,10],[20,20]]).astype(np.float32)
outimg = cv2.remap(inimg,inmap,outmap,cv2.INTER_LINEAR)
print "inimg:",inimg
print "inmap:",inmap
print "outmap:",outmap
print "outimg:", outimg

and here's the output:

inimg: [[ 0.  1.]
 [ 2.  3.]]
inmap: [[ 0.  0.]
 [ 0.  1.]
 [ 1.  0.]
 [ 1.  1.]]
outmap: [[ 10.  10.]
 [ 10.  20.]
 [ 20.  10.]
 [ 20.  20.]]
outimg: [[ 0.  0.]
 [ 0.  0.]
 [ 0.  0.]
 [ 0.  0.]]

As you can see, outimg produces 0,0, and it's not even in the correct shape. I expect a 20x20 or 10x10 image with interpolated values from range 0 to 3.

I've read all the documentation. It and everyone on SO states you input an array (a map) of starting points, a map of ending points, and then remap() will put all the values in img into their new positions, interpolating any empty space. I'm doing that, but it just doesn't work. Why? Most examples are for C++. Is it broken in python?

  • I've given an answer but I can expand on it a bit with a more simple example if you answer me the question...what is the output you were expecting there? Obviously an output of shape (2, 2) first off but what values were you expecting? – alkasm Oct 2 '17 at 11:32
  • I'm expecting an output shape of 20x20, since that's the largest value. Most other points are interpolated. – john ktejik Oct 3 '17 at 0:36
  • I'm expecting an output of[[0,0,0,0,0,0,0,0,0,0,.1,.2,.3,.4,.5,.6,.7,.8,.9,1] .... [0,0,0,0,0,0,0,0,0,2,2.1,2.2,2.3,2.4,2.5,2.6,2.7,2.8,2.9,3]] – john ktejik Oct 3 '17 at 0:59


What remap() doesn't do is take the coordinates of your source image, transform the points, and then interpolate. What remap() does do is, for every pixel in the destination image, lookup where it comes from in the source image, and then assigns an interpolated value. It needs to work this way since, in order to interpolate, it needs to look at the values around the source image at each pixel. Let me expand (might repeat myself a bit, but don't take it the wrong way).

From the remap() docs:

map1 – The first map of either (x,y) points or just x values having the type CV_16SC2 , CV_32FC1 , or CV_32FC2 . See convertMaps() for details on converting a floating point representation to fixed-point for speed.

map2 – The second map of y values having the type CV_16UC1 , CV_32FC1 , or none (empty map if map1 is (x,y) points), respectively.

The verbage here on map1 with "the first map of..." is somewhat misleading. Remember, these are strictly the coordinates of where your image gets mapped from...the points are being mapped from src at map_x(x, y), map_y(x, y) and then placed into dst at x, y. And they should be the same shape of the image you want to warp them to. Note the equation shown in the docs:

dst(x,y) =  src(map_x(x,y),map_y(x,y))

Here map_x(x, y) is looking up map_x at the rows and columns given by x, y. Then the image is evaluated at those points. It's looking up the mapped coordinates of x, y in src, and then assigning that value to x, y in dst. If you stare at this long enough, it starts to make some sense. At pixel (0, 0) in the new destination image, I look at map_x and map_y which tell me the location of the corresponding pixel in the source image, and then I can assign an interpolated value at (0, 0) in the destination image by looking at near values in the source. This is sort of the fundamental reason why remap() works this way; it needs to know where a pixel came from so it can see neighboring pixels to interpolate.

Small, contrived example

img = np.uint8(np.random.rand(8, 8)*255)
#array([[230,  45, 153, 233, 172, 153,  46,  29],
#       [172, 209, 186,  30, 197,  30, 251, 200],
#       [175, 253, 207,  71, 252,  60, 155, 124],
#       [114, 154, 121, 153, 159, 224, 146,  61],
#       [  6, 251, 253, 123, 200, 230,  36,  85],
#       [ 10, 215,  38,   5, 119,  87,   8, 249],
#       [  2,   2, 242, 119, 114,  98, 182, 219],
#       [168,  91, 224,  73, 159,  55, 254, 214]], dtype=uint8)

map_y = np.array([[0, 1], [2, 3]], dtype=np.float32)
map_x = np.array([[5, 6], [7, 10]], dtype=np.float32)
mapped_img = cv2.remap(img, map_x, map_y, cv2.INTER_LINEAR)
#array([[153, 251],
#       [124,   0]], dtype=uint8)

So what's happening here? Remember these are the indices of img that will get mapped TO the row and column they are situated at. In this case it's easiest to examine the matrices:

0  1
2  3

5  6
7  10

So the destination image at (0, 0) has the same value as the source image at map_y(0, 0), map_x(0, 0) = 0, 5 and the source image at row 0 and column 5 is 153. Note that in the destination image mapped_img[0, 0] = 153. No interpolation is happening here since my map coordinates are exact integers. Also I included an out-of-bounds index (map_x[1, 1] = 10, which is larger than the image width), and notice that it just gets assigned the value 0 when it's out-of-bounds.

Full use-case example

Here's a full-fledged code example, using a ground truth homography, warping the pixel locations manually, and using remap() to then map the image from the transformed points. Note here that my homography transforms true_dst to src. Thus, I make a set of however many points I want, and then calculate where those points lie in the source image by transforming with the homography. Then remap() is used to look up those points in the source image, and map them into the destination image.

import numpy as np
import cv2

# read images
true_dst = cv2.imread("img1.png")
src = cv2.imread("img2.png")

# ground truth homography from true_dst to src
H = np.array([
    [8.7976964e-01,   3.1245438e-01,  -3.9430589e+01],
    [-1.8389418e-01,   9.3847198e-01,   1.5315784e+02],
    [1.9641425e-04,  -1.6015275e-05,   1.0000000e+00]])

# create indices of the destination image and linearize them
h, w = true_dst.shape[:2]
indy, indx = np.indices((h, w), dtype=np.float32)
lin_homg_ind = np.array([indx.ravel(), indy.ravel(), np.ones_like(indx).ravel()])

# warp the coordinates of src to those of true_dst
map_ind = H.dot(lin_homg_ind)
map_x, map_y = map_ind[:-1]/map_ind[-1]  # ensure homogeneity
map_x = map_x.reshape(h, w).astype(np.float32)
map_y = map_y.reshape(h, w).astype(np.float32)

# remap!
dst = cv2.remap(src, map_x, map_y, cv2.INTER_LINEAR)
blended = cv2.addWeighted(true_dst, 0.5, dst, 0.5, 0)
cv2.imshow('blended.png', blended)

Remap for warping

Images and ground truth homographies from the Visual Geometry Group at Oxford.

  • A lot here to digest. I appreciate the thought and depth put into it. Your homography example is EXACTLY what I am trying to do, just with different images and challenges. Point is I need to know how to use remap() In your code, "map_ind = H.dot(lin_homg_ind) map_x, map_y = map_ind[:-1]/map_ind[-1]" (apologies, SO doesn't allow formatting in comments) - THATS what I need to understand. – john ktejik Oct 3 '17 at 0:52
  • src(map_x(x,y),map_y(x,y)) - this says to me that map_x provides points to take from src and put into dst[x,y] - which is exactly what I'm trying to do. ? – john ktejik Oct 3 '17 at 0:55
  • @johnktejik "this says to me that map_x provides points to take from src and put into dst[x,y] - which is exactly what I'm trying to do." Yes, precisely. map_x[1, 0] holds the coordinate from src that gets put into dst[1, 0]. Btw you can do inline code using backticks; `this` produces this. Feel free to send me an email (on the website in my profile) and I'd be happy to help you out with specific problems in implementation. I've used these functions a lot. – alkasm Oct 3 '17 at 3:36
  • @johnktejik added the short code example that we discussed over email. Cheers! – alkasm Oct 3 '17 at 20:53
  • 1
    @johnktejik I expanded on it well since I haven't seen another post go into detail on this function, so I think it will prove to be helpful in the future when people try to use remap(). Anyways, the larger code example was already sitting in a file. :) – alkasm Oct 4 '17 at 1:17

Your Answer

