title: FFMPEG API - Recording video and audio - Syncing problems
body: <p>I'm developing an app which is able to record video from a webcam and audio from a microphone. I've been using QT but unfortunately the camera module does not work on windows which led me to use ffmpeg to record the video/audio.</p>
  
  <p>My Camera module is now working well besides a slight problem with syncing. The audio and video sometimes end up out of sync by a small difference (less than 1 second I'd say, although it might be worse with longer recordings). </p>
  
  <p>When I encode the frames I add the PTS in the following way (which I took from the <a href="https://www.ffmpeg.org/doxygen/trunk/muxing_8c-example.html" rel="nofollow">muxing.c</a> example):</p>
  
  <ul>
  <li>For the video frames I increment the PTS one by one (starting at 0).</li>
  <li>For the audio frames I increment the PTS by the <code>nb_samples</code> of the audio frame (starting at 0).</li>
  </ul>
  
  <p>I am saving the file at 25 fps and asking for the camera to give me 25 fps (which it can). I am also converting the video frames to the <code>YUV420P</code> format. For the audio frames conversion I need to use a <code>AVAudioFifo</code> because the microfone sends bigger samples than the mp4 stream supports, so I have to split them in chuncks. I used the <a href="https://www.ffmpeg.org/doxygen/2.2/transcode_aac_8c-example.html" rel="nofollow">transcode.c</a> example for this.</p>
  
  <p>I am out of ideas in what I should do to sync the audio and video. Do I need to use a clock or something to correctly sync up both streams?</p>
  
  <p>The full code is too big to post here but should it be necessary I can add it to github for example.</p>
  
  <p>Here is the code for writing a frame:</p>
  
  <pre><code>int FFCapture::writeFrame(const AVRational *time_base, AVStream *stream, AVPacket *pkt) {
      /* rescale output packet timestamp values from codec to stream timebase */
      av_packet_rescale_ts(pkt, *time_base, stream-&gt;time_base);
      pkt-&gt;stream_index = stream-&gt;index;
      /* Write the compressed frame to the media file. */
      return av_interleaved_write_frame(oFormatContext, pkt);
  }
  </code></pre>
  
  <p>Code for getting the elapsed time:</p>
  
  <pre><code>qint64 FFCapture::getElapsedTime(qint64 *previousTime) {
      qint64 newTime = timer.elapsed();
      if(newTime &gt; *previousTime) {
          *previousTime = newTime;
          return newTime;
      }
      return -1;
  }
  </code></pre>
  
  <p>Code for adding the PTS (video and audio stream, respectively):</p>
  
  <pre><code>qint64 time = getElapsedTime(&amp;previousVideoTime);
  if(time &gt;= 0) outFrame-&gt;pts = time;
  //if(time &gt;= 0) outFrame-&gt;pts = av_rescale_q(time, outStream.videoStream-&gt;codec-&gt;time_base, outStream.videoStream-&gt;time_base);
  
  qint64 time = getElapsedTime(&amp;previousAudioTime);
  if(time &gt;= 0) {
      AVRational aux;
      aux.num = 1;
      aux.den = 1000;
      outFrame-&gt;pts = time;
      //outFrame-&gt;pts = av_rescale_q(time, aux, outStream.audioStream-&gt;time_base);
  }
  </code></pre>