title: Find median for an image array for background estimation fast in Swift/ObjectiveC
body: <p>I need to estimate the background across multiple frames of a stationary video (not-moving camera). I have a number of frames and want to calculate the median for each pixel across all frames (usually 10 to 100 frames). I was able to do that with brute force but the performance is just awful (it takes 30-120 seconds to calculate). In <code>Python</code> with <code>NumPy</code> I can achieve this in a single <code>np.median</code> call:</p>
  <pre class="lang-py prettyprint-override"><code>medianFrames = [im1, im2, im3, im4]
  medianFrame = np.median(medianFrames, axis=0).astype(dtype=np.uint8)   
  </code></pre>
  <p>In Objective-C the algorithm is below, very slow because it enumerates each pixel, creates an array for each component (R,G,B), and then calculates median value. It works but it's super slow (uses <code>OpenCV</code> <code>Mat</code> for image manipulation):</p>
  <pre><code>   for (int i = 0; i &lt; result.rows; i++) {
          for (int j = 0; j &lt; result.cols; j++) {
              NSMutableArray *elements_B = [NSMutableArray arrayWithCapacity:arr.count];
              NSMutableArray *elements_G = [NSMutableArray arrayWithCapacity:arr.count];
              NSMutableArray *elements_R = [NSMutableArray arrayWithCapacity:arr.count];
              for(int frameIndex = 0; frameIndex &lt; arr.count; frameIndex++) {
                  Mat frame = matArray[frameIndex];
                  int B = frame.at&lt;Vec3b&gt;(i, j)[0];
                  int G = frame.at&lt;Vec3b&gt;(i, j)[1];
                  int R = frame.at&lt;Vec3b&gt;(i, j)[2];
                  elements_B[frameIndex] = [NSNumber numberWithInt:B];
                  elements_G[frameIndex] = [NSNumber numberWithInt:G];
                  elements_R[frameIndex] = [NSNumber numberWithInt:R];
              }
              
              NSArray *sortedB = [elements_B sortedArrayUsingSelector:@selector(compare:)];
              NSUInteger middleB = [sortedB count] / 2;
              NSNumber *medianB = [sortedB objectAtIndex:middleB];
              
              result.at&lt;Vec3b&gt;(i,j)[0] = medianB.intValue;
              
              NSArray *sortedG = [elements_G sortedArrayUsingSelector:@selector(compare:)];
              NSUInteger middleG = [sortedG count] / 2;
              NSNumber *medianG = [sortedG objectAtIndex:middleG];
              
              result.at&lt;Vec3b&gt;(i,j)[1] = medianG.intValue;
              
              NSArray *sortedR = [elements_R sortedArrayUsingSelector:@selector(compare:)];
              NSUInteger middleR = [sortedR count] / 2;
              NSNumber *medianR = [sortedR objectAtIndex:middleR];
              
              result.at&lt;Vec3b&gt;(i,j)[2] = medianR.intValue;
          }
      }
  </code></pre>
  <p>The real bottleneck is an enumeration of each pixel across each image and calculating the median value. What is the best way to process multiple images and execute pixel-based math operations efficiently, as <code>NumPy</code> does?</p>