title: iOS Heart rate detection Algorithm
body: <p>I'm trying to implement heart beat recording functionality in an app i'm developing.</p>
  
  <p>The preferred method of doing this is by using the iPhone's camera with the light on, having the user place their finger on the lens, and detecting fluctuations in the video feed, which correspond to the user's heart.</p>
  
  <p>I found a very good starting point with the following stack overflow question 
  <a href="https://stackoverflow.com/questions/9274027/detecting-heart-rate-using-the-camera">here</a></p>
  
  <p>The question provides useful code to plot a heart beat time graph.</p>
  
  <p>It shows how to start an AVCaptureSession and turn the camera's light on like so:</p>
  
  <pre><code>session = [[AVCaptureSession alloc] init];
  
  AVCaptureDevice* camera = [AVCaptureDevice defaultDeviceWithMediaType:AVMediaTypeVideo];
  if([camera isTorchModeSupported:AVCaptureTorchModeOn]) {
      [camera lockForConfiguration:nil];
      camera.torchMode=AVCaptureTorchModeOn;
      //  camera.exposureMode=AVCaptureExposureModeLocked;
      [camera unlockForConfiguration];
  }
  // Create a AVCaptureInput with the camera device
  NSError *error=nil;
  AVCaptureInput* cameraInput = [[AVCaptureDeviceInput alloc] initWithDevice:camera error:&amp;error];
  if (cameraInput == nil) {
      NSLog(@"Error to create camera capture:%@",error);
  }
  
  // Set the output
  AVCaptureVideoDataOutput* videoOutput = [[AVCaptureVideoDataOutput alloc] init];
  
  // create a queue to run the capture on
  dispatch_queue_t captureQueue=dispatch_queue_create("catpureQueue", NULL);
  
  // setup our delegate
  [videoOutput setSampleBufferDelegate:self queue:captureQueue];
  
  // configure the pixel format
  videoOutput.videoSettings = [NSDictionary dictionaryWithObjectsAndKeys:[NSNumber numberWithUnsignedInt:kCVPixelFormatType_32BGRA], (id)kCVPixelBufferPixelFormatTypeKey,
                               nil];
  videoOutput.minFrameDuration=CMTimeMake(1, 10);
  
  // and the size of the frames we want
  [session setSessionPreset:AVCaptureSessionPresetLow];
  
  // Add the input and output
  [session addInput:cameraInput];
  [session addOutput:videoOutput];
  
  // Start the session
  [session startRunning];
  </code></pre>
  
  <p>Self in this example must be an <code>&lt;AVCaptureVideoDataOutputSampleBufferDelegate&gt;</code>
  And will therefore have to implement the following method to obtain raw camera data:</p>
  
  <pre><code>- (void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection {
  static int count=0;
  count++;
  // only run if we're not already processing an image
  // this is the image buffer
  CVImageBufferRef cvimgRef = CMSampleBufferGetImageBuffer(sampleBuffer);
  // Lock the image buffer
  CVPixelBufferLockBaseAddress(cvimgRef,0);
  // access the data
  int width=CVPixelBufferGetWidth(cvimgRef);
  int height=CVPixelBufferGetHeight(cvimgRef);
  // get the raw image bytes
  uint8_t *buf=(uint8_t *) CVPixelBufferGetBaseAddress(cvimgRef);
  size_t bprow=CVPixelBufferGetBytesPerRow(cvimgRef);
  float r=0,g=0,b=0;
  for(int y=0; y&lt;height; y++) {
      for(int x=0; x&lt;width*4; x+=4) {
          b+=buf[x];
          g+=buf[x+1];
          r+=buf[x+2];
          //          a+=buf[x+3];
      }
      buf+=bprow;
  }
  r/=255*(float) (width*height);
  g/=255*(float) (width*height);
  b/=255*(float) (width*height);
  
  float h,s,v;
  
  RGBtoHSV(r, g, b, &amp;h, &amp;s, &amp;v);
  
  // simple highpass and lowpass filter 
  
  static float lastH=0;
  float highPassValue=h-lastH;
  lastH=h;
  float lastHighPassValue=0;
  float lowPassValue=(lastHighPassValue+highPassValue)/2;
  
  lastHighPassValue=highPassValue;
  
      //low pass value can now be used for basic heart beat detection
  
  
  }
  </code></pre>
  
  <p>RGB is converted to HSV and it is Hue that is monitored for fluctuations.</p>
  
  <p>And RGB to HSV is implemented as follows</p>
  
  <pre><code>void RGBtoHSV( float r, float g, float b, float *h, float *s, float *v ) {
  float min, max, delta; 
  min = MIN( r, MIN(g, b )); 
  max = MAX( r, MAX(g, b )); 
  *v = max;
  delta = max - min; 
  if( max != 0 )
      *s = delta / max;
  else {
      // r = g = b = 0 
      *s = 0; 
      *h = -1; 
      return;
  }
  if( r == max )
      *h = ( g - b ) / delta; 
  else if( g == max )
      *h=2+(b-r)/delta;
  else 
      *h=4+(r-g)/delta; 
  *h *= 60;
  if( *h &lt; 0 ) 
      *h += 360;
  }
  </code></pre>
  
  <p>The low pass value calculated in <code>capureOutput:</code> initially provides erratic data, but then stabilises to the following:</p>
  
  <pre><code>2013-11-04 16:18:13.619 SampleHeartRateApp[1743:1803] -0.071218
  2013-11-04 16:18:13.719 SampleHeartRateApp[1743:1803] -0.050072
  2013-11-04 16:18:13.819 SampleHeartRateApp[1743:1803] -0.011375
  2013-11-04 16:18:13.918 SampleHeartRateApp[1743:1803] 0.018456
  2013-11-04 16:18:14.019 SampleHeartRateApp[1743:1803] 0.059024
  2013-11-04 16:18:14.118 SampleHeartRateApp[1743:1803] 0.052198
  2013-11-04 16:18:14.219 SampleHeartRateApp[1743:1803] 0.078189
  2013-11-04 16:18:14.318 SampleHeartRateApp[1743:1803] 0.046035
  2013-11-04 16:18:14.419 SampleHeartRateApp[1743:1803] -0.113153
  2013-11-04 16:18:14.519 SampleHeartRateApp[1743:1803] -0.079792
  2013-11-04 16:18:14.618 SampleHeartRateApp[1743:1803] -0.027654
  2013-11-04 16:18:14.719 SampleHeartRateApp[1743:1803] -0.017288
  </code></pre>
  
  <p>An example of the erratic data provided initially is here:</p>
  
  <pre><code>2013-11-04 16:17:28.747 SampleHeartRateApp[1743:3707] 17.271435
  2013-11-04 16:17:28.822 SampleHeartRateApp[1743:1803] -0.049067
  2013-11-04 16:17:28.922 SampleHeartRateApp[1743:1803] -6.524201
  2013-11-04 16:17:29.022 SampleHeartRateApp[1743:1803] -0.766260
  2013-11-04 16:17:29.137 SampleHeartRateApp[1743:3707] 9.956407
  2013-11-04 16:17:29.221 SampleHeartRateApp[1743:1803] 0.076244
  2013-11-04 16:17:29.321 SampleHeartRateApp[1743:1803] -1.049292
  2013-11-04 16:17:29.422 SampleHeartRateApp[1743:1803] 0.088634
  2013-11-04 16:17:29.522 SampleHeartRateApp[1743:1803] -1.035559
  2013-11-04 16:17:29.621 SampleHeartRateApp[1743:1803] 0.019196
  2013-11-04 16:17:29.719 SampleHeartRateApp[1743:1803] -1.027754
  2013-11-04 16:17:29.821 SampleHeartRateApp[1743:1803] 0.045803
  2013-11-04 16:17:29.922 SampleHeartRateApp[1743:1803] -0.857693
  2013-11-04 16:17:30.021 SampleHeartRateApp[1743:1803] 0.061945
  2013-11-04 16:17:30.143 SampleHeartRateApp[1743:1803] -0.701269
  </code></pre>
  
  <p>The low pass value goes positive whenever there is a heart beat. So I tried a very simple live detection algorithm which basically looks at the current value, and sees if it is positive, it also looks at the previous value, if negative it detects negative going to positive and plays a beep sound.</p>
  
  <p>The problem with this is the data isn't always as perfect as the above, sometimes there's anomalous positive readings in amongst negative readings and vice versa.</p>
  
  <p>A graph of the low pass value in time looks like this:
  <img src="https://i.stack.imgur.com/S00Ug.png" alt="enter image description here"></p>
  
  <p>Interestingly the above anomaly is quite common, if I record a graph for a while i'll see a very similar shaped anomaly multiple times.</p>
  
  <p>In my very simple beat detection algorithm, if an anomaly as shown above occurs the counted number of beats in the detection period (10 seconds) can shoot up by 4 or 5 beats. This makes the calculated BPM very inaccurate. But as simple as it is it does work around 70% of the time.</p>
  
  <p>To combat this problem I tried the following.</p>
  
  <p>1.Started recording last 3 low pass values in an array</p>
  
  <p>2.Then looked to see whether or not the middle value had two smaller values surrounding it before and after. (Basic peak detection)</p>
  
  <p>3.Counted this scenario as a beat and added it to the running total of beats in a given time.</p>
  
  <p>This method is however just as vulnerable to the anomalies as any other. And actually seemed to be a worse method. (When playing live beeps after detection they seemed far more erratic than the positive to negative algorithm)</p>
  
  <p><strong>My question is can you help me come up with an algorithm that can reliably detect when a heart beat occurs with reasonable accuracy.</strong></p>
  
  <p>Another problem I realise that i'm going to have to address is detecting whether or not a user's finger is on the lens.</p>
  
  <p>I thought about detecting erratic low pass values but the problem there is the low pass filter accounts for erratic values and smooths them out over time. So help there would be appreciated too.</p>
  
  <p>Thanks for your time.</p>