title: iOS: Audio and Video tap from Airplay
body: <p>I have made a video player that is analyzing the realtime audio and video tracks from the video that is currently playing. The videos are stored on the iOS device (in the Apps Documents directory).</p>
  
  <p>This all works fine. I use MTAudioProcessingTap in order to get all the audio samples and do some FFT, and I am analyzing the video by just copy'ing the pixel buffers from the currently played CMTime (the AVPlayer currentTime property). As I said, this works fine.</p>
  
  <p>But now I want to support Airplay. Just the airplay itself is not difficult, but my taps stop working as soon as Airplay is toggled and the video is playing on the ATV. Somehow, the MTAudioProcessingTap won't process and the pixelbuffers are all empty... I can't get to the data.</p>
  
  <p>Is there any way to get to this data ?</p>
  
  <p>In order to get the pixel buffers, I just fire an event every few milli-sec and retrieving the player's currentTime. Then:</p>
  
  <pre><code>CVPixelBufferRef imageBuffer = [videoOutput copyPixelBufferForItemTime:time itemTimeForDisplay:nil];
  CVPixelBufferLockBaseAddress(imageBuffer,0);
  
  uint8_t *tempAddress = (uint8_t *) CVPixelBufferGetBaseAddress(imageBuffer);
  
  size_t bytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer);
  size_t height = CVPixelBufferGetHeight(imageBuffer);
  
  CVPixelBufferUnlockBaseAddress(imageBuffer,0);
  </code></pre>
  
  <p>Where <code>tempAddress</code> is my pixelbuffer, and <code>videoOutput</code> is an instance of <code>AVPlayerItemVideoOutput</code>.</p>
  
  <p>For audio, I use:</p>
  
  <pre><code>AVMutableAudioMixInputParameters *inputParams = [AVMutableAudioMixInputParameters audioMixInputParametersWithTrack:audioTrack];
  
  // Create a processing tap for the input parameters
  MTAudioProcessingTapCallbacks callbacks;
  
  callbacks.version = kMTAudioProcessingTapCallbacksVersion_0;
  callbacks.clientInfo = (__bridge void *)(self);
  callbacks.init = init;
  callbacks.prepare = prepare;
  callbacks.process = process;
  callbacks.unprepare = unprepare;
  callbacks.finalize = finalize;
  
  MTAudioProcessingTapRef tap;
  OSStatus err = MTAudioProcessingTapCreate(kCFAllocatorDefault, &amp;callbacks,
                                            kMTAudioProcessingTapCreationFlag_PostEffects, &amp;tap);
  if (err || !tap) {
      NSLog(@"Unable to create the Audio Processing Tap");
      return;
  }
  
  inputParams.audioTapProcessor = tap;
  
  // Create a new AVAudioMix and assign it to our AVPlayerItem
  AVMutableAudioMix *audioMix = [AVMutableAudioMix audioMix];
  audioMix.inputParameters = @[inputParams];
  playerItem.audioMix = audioMix;
  </code></pre>
  
  <p>Regards,
  Niek</p>