src/media/audio/audio_core/mix_stage.cc
// We compose the effects of clock reconciliation into our sample-rate-conversion step size, but
// only for streams that use neither the 'optimal' clock, nor the clock we designate as driving our
// hardware-rate-adjustments. We apply this micro-SRC via an intermediate "slew away the error"
// rate-correction factor driven by a PID control. Why use a PID? Sources do not merely chase the
// other clock's rate -- they chase its position. Note that even if we don't adjust our rate, we
// still want a composed transformation for offsets.
//
// Calculate the composed dest-to-src transformation and update the mixer's bookkeeping for
// step_size etc. These are the only deliverables for this method.
We compose the effects of clock reconciliation into our sample-rate-conversion step size, but
only for streams that use neither the 'optimal' clock, nor the clock we designate as driving our
hardware-rate-adjustments. We apply this micro-SRC via an intermediate "slew away the error"
rate-correction factor driven by a PID control. Why use a PID. Sources do not merely chase the
other clock's rate -- they chase its position. Note that even if we don't adjust our rate, we
still want a composed transformation for offsets.

Calculate the composed dest-to-src transformation and update the mixer's bookkeeping for
step_size etc. These are the only deliverables for this method.
We compose the effects of clock reconciliation into our sample - rate - conversion step size but only for streams that use neither the ' optimal ' clock nor the clock we designate as driving our hardware - rate - adjustments . We apply this micro - SRC via an intermediate " slew away the error " rate - correction factor driven by a PID control . Why use a PID . Sources do not merely chase the other clock ' s rate -- they chase its position . Note that even if we don ' t adjust our rate we still want a composed transformation for offsets . Calculate the composed dest - to - src transformation and update the mixer ' s bookkeeping for step_size etc . These are the only deliverables for this method .
void MixStage::ReconcileClocksAndSetStepSize(Mixer::SourceInfo& info,
                                             Mixer::Bookkeeping& bookkeeping,
                                             ReadableStream& stream) {
  constexpr zx::duration kMaxErrorThresholdDuration = zx::msec(2);
  TRACE_DURATION("audio", "MixStage::ReconcileClocksAndSetStepSize");
   Ensure the mappings from source-frame to source-ref-time and monotonic-time are up-to-date.
  auto snapshot = stream.ReferenceClockToFixed();
  info.source_ref_clock_to_frac_source_frames = snapshot.timeline_function;
  if (info.source_ref_clock_to_frac_source_frames.subject_delta() == 0) {
    info.clock_mono_to_frac_source_frames = TimelineFunction();
    info.dest_frames_to_frac_source_frames = TimelineFunction();
    bookkeeping.step_size = 0;
    bookkeeping.rate_modulo = 0;   we need not also clear rate_mod and pos_mod
    return;
  }
  FX_DCHECK(stream.reference_clock().is_valid());
  FX_DCHECK(reference_clock().is_valid());
   Ensure the mappings from source-frame to monotonic-time is up-to-date.
  auto source_ref_clock_to_clock_mono = stream.reference_clock().ref_clock_to_clock_mono();
  auto frac_source_frame_to_clock_mono =
      source_ref_clock_to_clock_mono * info.source_ref_clock_to_frac_source_frames.Inverse();
  info.clock_mono_to_frac_source_frames = frac_source_frame_to_clock_mono.Inverse();
   Assert we can map from local monotonic-time to fractional source frames.
  FX_DCHECK(info.clock_mono_to_frac_source_frames.rate().reference_delta());
   UpdateDestTrans
   Ensure the mappings from dest-frame to monotonic-time is up-to-date.
   We should only be here if we have a valid mix job. This means a job which supplies a valid
   transformation from reference time to destination frames (based on dest frame rate).
  FX_DCHECK(cur_mix_job_.dest_ref_clock_to_frac_dest_frame.rate().reference_delta());
  if (cur_mix_job_.dest_ref_clock_to_frac_dest_frame.rate().subject_delta() == 0) {
    info.dest_frames_to_frac_source_frames = TimelineFunction();
    bookkeeping.step_size = 0;
    bookkeeping.rate_modulo = 0;   we need not also clear rate_mod and pos_mod
    return;
  }
  auto dest_frames_to_dest_ref_clock =
      ReferenceClockToIntegralFrames(cur_mix_job_.dest_ref_clock_to_frac_dest_frame).Inverse();
   Compose our transformation from local monotonic-time to dest frames.
  auto dest_frames_to_clock_mono =
      reference_clock().ref_clock_to_clock_mono() * dest_frames_to_dest_ref_clock;
   ComposeDestToSource
   Compose our transformation from destination frames to source fractional frames.
   Combine the job-supplied dest transformation, with the renderer-supplied mapping of
   monotonic-to-source-subframe, to produce a transformation which maps from dest frames to
   fractional source frames.
  info.dest_frames_to_frac_source_frames =
      info.clock_mono_to_frac_source_frames * dest_frames_to_clock_mono;
   ComputeFrameRateConversionRatio
   Determine the appropriate TimelineRate for step_size. This based exclusively on the frame rates
   of the source and destination. If we happen to be applying "micro-SRC" for this source, then
   that will be included subsequently as a correction factor.
  TimelineRate frac_src_frames_per_dest_frame =
      dest_frames_to_dest_ref_clock.rate() * info.source_ref_clock_to_frac_source_frames.rate();
   SynchronizeClocks
   If client clock and device clock differ, reconcile them.
   For start dest frame, measure [predicted - actual] error (in frac_src) since last mix. Do this
   even if clocks are same on both sides, as this allows us to perform an initial sync-up between
   running position accounting and the initial clock transforms (even those with offsets).
  auto curr_dest_frame = cur_mix_job_.start_pts_of;
  if (info.next_dest_frame != curr_dest_frame || !info.running_pos_established) {
    info.running_pos_established = true;
     Set new running positions, based on the E2E clock (not just from step_size)
    auto prev_running_dest_frame = info.next_dest_frame;
    auto prev_running_frac_src_frame = info.next_frac_source_frame;
    info.ResetPositions(curr_dest_frame, bookkeeping);
    FX_LOGS(DEBUG) << "Running dest position is discontinuous (expected " << prev_running_dest_frame
                   << ", actual " << curr_dest_frame << ") updating running source position from "
                   << prev_running_frac_src_frame.raw_value() << " to "
                   << info.next_frac_source_frame.raw_value();
     Also should reset the PID controls in the relevant clocks.
    reference_clock().ResetRateAdjustmentTuning(curr_dest_frame);
    stream.reference_clock().ResetRateAdjustmentTuning(curr_dest_frame);
  } else if (reference_clock() == stream.reference_clock()) {
     Same clock on both sides can occur when multiple MixStages are connected serially (should
     both be device clocks). Don't synchronize: use frac_src_frames_per_dest_frame as-is.
  } else if (reference_clock().is_device_clock() && stream.reference_clock().is_device_clock()) {
     We are synchronizing two device clocks. Unfortunately, we know they aren't the same.
     To enable this scenario in the future, at least one must be adjustable in hardware (it will
     be the "follower"); the other must be marked as hardware_controlling.
    FX_DCHECK(reference_clock() == stream.reference_clock())
        << "Cannot reconcile two different device clocks: clock routing error";
  } else {
     MeasureClockError
     Between upstream (source) and downstream (dest) clocks, we should have one device clock
     and one client clock. Remember capture mix, where usual roles are reversed.
     They can't BOTH be client clocks (although we could handle this, if routing existed).
    FX_DCHECK(reference_clock().is_device_clock() || stream.reference_clock().is_device_clock())
        << "Cannot reconcile two client clocks. No device clock: clock routing error";
     MeasureClockError
     Measure the error in src_frac_pos
    auto max_error_frac =
        info.source_ref_clock_to_frac_source_frames.rate().Scale(kMaxErrorThresholdDuration.get());
    auto curr_src_frac_pos =
        Fixed::FromRaw(info.dest_frames_to_frac_source_frames(curr_dest_frame));
    info.frac_source_error = info.next_frac_source_frame - curr_src_frac_pos;
     AdjustClock
       If mix is continuous since then, report the latest error to the AudioClock
       AudioClock feeds errors to its internal PID, producing a running correction factor....
       ... If clock is hardware-controlling, apply correction factor to the hardware.
       ... If clock is optimal, apply correction factor directly to the client clock.
       ... Otherwise, expose this error correction factor directly to MixStage
    AudioClock& device_clock =
        stream.reference_clock().is_device_clock() ? stream.reference_clock() : reference_clock();
    AudioClock& client_clock =
        stream.reference_clock().is_device_clock() ? reference_clock() : stream.reference_clock();
    if (std::abs(info.frac_source_error.raw_value()) > max_error_frac) {
       Source error exceeds our threshold
       Reset the rate adjustment process altogether and allow a discontinuity
      info.next_frac_source_frame = curr_src_frac_pos;
      FX_LOGS(DEBUG) << "frac_source_error: out of bounds (" << info.frac_source_error.raw_value()
                     << " vs. limit +/-" << max_error_frac << "), resetting next_frac_src to "
                     << info.next_frac_source_frame.raw_value();
       Reset the PID controls, in the relevant clocks.
      client_clock.ResetRateAdjustmentTuning(curr_dest_frame);
      device_clock.ResetRateAdjustmentTuning(curr_dest_frame);
    } else {
       No error is too small to worry about; handle them all.
      FX_LOGS(TRACE) << "frac_source_error: tuning reference clock at dest " << curr_dest_frame
                     << " for " << info.frac_source_error.raw_value();
      if (client_clock.is_adjustable()) {
         Adjust client_clock, the 'flexible' clock that we have provided to the client
      } else if (device_clock.is_adjustable() && client_clock.controls_hardware_clock()) {
         Adjust device_clock's hardware clock rate based on the frac_source_error
      } else {
        client_clock.TuneRateForError(info.frac_source_error, curr_dest_frame);
         Using this rate adjustment factor, adjust step_size, so future src_positions converge to
         what these two clocks require. The product might exceed uint64/uint64: allow reduction.
        TimelineRate micro_src_factor = client_clock.rate_adjustment();
        frac_src_frames_per_dest_frame =
            TimelineRate::Product(frac_src_frames_per_dest_frame, micro_src_factor, false);
      }
    }
  }
   SetStepSize
   Convert the TimelineRate into [step_size, denominator, rate_modulo] as usual.
  FX_DCHECK(frac_src_frames_per_dest_frame.reference_delta());
  int64_t tmp_step_size = frac_src_frames_per_dest_frame.Scale(1);
  FX_DCHECK(tmp_step_size >= 0);
  FX_DCHECK(tmp_step_size <= std::numeric_limits<uint32_t>::max());
  auto old_denominator = bookkeeping.denominator;
  bookkeeping.step_size = static_cast<uint32_t>(tmp_step_size);
  bookkeeping.denominator = frac_src_frames_per_dest_frame.reference_delta();
  bookkeeping.rate_modulo = frac_src_frames_per_dest_frame.subject_delta() -
                            (bookkeeping.denominator * bookkeeping.step_size);
   If the denominator is changing, update the source position modulos.
  if (old_denominator != bookkeeping.denominator) {
     Reinterpret previous per-job and long-running src_pos_mod values for the new denominator
    if (bookkeeping.src_pos_modulo) {
      __uint128_t tmp_src_pos_modulo =
          static_cast<__uint128_t>(bookkeeping.src_pos_modulo) * bookkeeping.denominator;
      bookkeeping.src_pos_modulo = tmp_src_pos_modulo / old_denominator;
    }
    if (info.next_src_pos_modulo) {
      __uint128_t tmp_next_src_pos_modulo =
          static_cast<__uint128_t>(info.next_src_pos_modulo) * bookkeeping.denominator;
      info.next_src_pos_modulo = tmp_next_src_pos_modulo / old_denominator;
    }
  }
   Otherwise, preserve the previous source position modulo values
}
void MixStage :: ReconcileClocksAndSetStepSize ( Mixer :: SourceInfo & info , Mixer :: Bookkeeping & bookkeeping , ReadableStream & stream ) { constexpr zx :: duration kMaxErrorThresholdDuration = zx :: msec ( 2 ) ; TRACE_DURATION ( " " , " " ) ; auto snapshot = stream . ReferenceClockToFixed ( ) ; info . source_ref_clock_to_frac_source_frames = snapshot . timeline_function ; if ( info . source_ref_clock_to_frac_source_frames . subject_delta ( ) == 0 ) { info . clock_mono_to_frac_source_frames = TimelineFunction ( ) ; info . dest_frames_to_frac_source_frames = TimelineFunction ( ) ; bookkeeping . step_size = 0 ; bookkeeping . rate_modulo = 0 ; return ; } FX_DCHECK ( stream . reference_clock ( ) . is_valid ( ) ) ; FX_DCHECK ( reference_clock ( ) . is_valid ( ) ) ; auto source_ref_clock_to_clock_mono = stream . reference_clock ( ) . ref_clock_to_clock_mono ( ) ; auto frac_source_frame_to_clock_mono = source_ref_clock_to_clock_mono * info . source_ref_clock_to_frac_source_frames . Inverse ( ) ; info . clock_mono_to_frac_source_frames = frac_source_frame_to_clock_mono . Inverse ( ) ; FX_DCHECK ( info . clock_mono_to_frac_source_frames . rate ( ) . reference_delta ( ) ) ; FX_DCHECK ( cur_mix_job_ . dest_ref_clock_to_frac_dest_frame . rate ( ) . reference_delta ( ) ) ; if ( cur_mix_job_ . dest_ref_clock_to_frac_dest_frame . rate ( ) . subject_delta ( ) == 0 ) { info . dest_frames_to_frac_source_frames = TimelineFunction ( ) ; bookkeeping . step_size = 0 ; bookkeeping . rate_modulo = 0 ; return ; } auto dest_frames_to_dest_ref_clock = ReferenceClockToIntegralFrames ( cur_mix_job_ . dest_ref_clock_to_frac_dest_frame ) . Inverse ( ) ; auto dest_frames_to_clock_mono = reference_clock ( ) . ref_clock_to_clock_mono ( ) * dest_frames_to_dest_ref_clock ; info . dest_frames_to_frac_source_frames = info . clock_mono_to_frac_source_frames * dest_frames_to_clock_mono ; TimelineRate frac_src_frames_per_dest_frame = dest_frames_to_dest_ref_clock . rate ( ) * info . source_ref_clock_to_frac_source_frames . rate ( ) ; auto curr_dest_frame = cur_mix_job_ . start_pts_of ; if ( info . next_dest_frame != curr_dest_frame || ! info . running_pos_established ) { info . running_pos_established = true ; auto prev_running_dest_frame = info . next_dest_frame ; auto prev_running_frac_src_frame = info . next_frac_source_frame ; info . ResetPositions ( curr_dest_frame , bookkeeping ) ; FX_LOGS ( DEBUG ) << " " << prev_running_dest_frame << " " << curr_dest_frame << " " << prev_running_frac_src_frame . raw_value ( ) << " " << info . next_frac_source_frame . raw_value ( ) ; reference_clock ( ) . ResetRateAdjustmentTuning ( curr_dest_frame ) ; stream . reference_clock ( ) . ResetRateAdjustmentTuning ( curr_dest_frame ) ; } else if ( reference_clock ( ) == stream . reference_clock ( ) ) { } else if ( reference_clock ( ) . is_device_clock ( ) && stream . reference_clock ( ) . is_device_clock ( ) ) { FX_DCHECK ( reference_clock ( ) == stream . reference_clock ( ) ) << " " ; } else { FX_DCHECK ( reference_clock ( ) . is_device_clock ( ) || stream . reference_clock ( ) . is_device_clock ( ) ) << " " ; auto max_error_frac = info . source_ref_clock_to_frac_source_frames . rate ( ) . Scale ( kMaxErrorThresholdDuration . get ( ) ) ; auto curr_src_frac_pos = Fixed :: FromRaw ( info . dest_frames_to_frac_source_frames ( curr_dest_frame ) ) ; info . frac_source_error = info . next_frac_source_frame - curr_src_frac_pos ; AudioClock & device_clock = stream . reference_clock ( ) . is_device_clock ( ) ? stream . reference_clock ( ) : reference_clock ( ) ; AudioClock & client_clock = stream . reference_clock ( ) . is_device_clock ( ) ? reference_clock ( ) : stream . reference_clock ( ) ; if ( std :: abs ( info . frac_source_error . raw_value ( ) ) > max_error_frac ) { info . next_frac_source_frame = curr_src_frac_pos ; FX_LOGS ( DEBUG ) << " " << info . frac_source_error . raw_value ( ) << " " << max_error_frac << " " << info . next_frac_source_frame . raw_value ( ) ; client_clock . ResetRateAdjustmentTuning ( curr_dest_frame ) ; device_clock . ResetRateAdjustmentTuning ( curr_dest_frame ) ; } else { FX_LOGS ( TRACE ) << " " << curr_dest_frame << " " << info . frac_source_error . raw_value ( ) ; if ( client_clock . is_adjustable ( ) ) { } else if ( device_clock . is_adjustable ( ) && client_clock . controls_hardware_clock ( ) ) { } else { client_clock . TuneRateForError ( info . frac_source_error , curr_dest_frame ) ; TimelineRate micro_src_factor = client_clock . rate_adjustment ( ) ; frac_src_frames_per_dest_frame = TimelineRate :: Product ( frac_src_frames_per_dest_frame , micro_src_factor , false ) ; } } } FX_DCHECK ( frac_src_frames_per_dest_frame . reference_delta ( ) ) ; int64_t tmp_step_size = frac_src_frames_per_dest_frame . Scale ( 1 ) ; FX_DCHECK ( tmp_step_size >= 0 ) ; FX_DCHECK ( tmp_step_size <= std :: numeric_limits < uint32_t > :: max ( ) ) ; auto old_denominator = bookkeeping . denominator ; bookkeeping . step_size = static_cast < uint32_t > ( tmp_step_size ) ; bookkeeping . denominator = frac_src_frames_per_dest_frame . reference_delta ( ) ; bookkeeping . rate_modulo = frac_src_frames_per_dest_frame . subject_delta ( ) - ( bookkeeping . denominator * bookkeeping . step_size ) ; if ( old_denominator != bookkeeping . denominator ) { if ( bookkeeping . src_pos_modulo ) { __uint128_t tmp_src_pos_modulo = static_cast < __uint128_t > ( bookkeeping . src_pos_modulo ) * bookkeeping . denominator ; bookkeeping . src_pos_modulo = tmp_src_pos_modulo / old_denominator ; } if ( info . next_src_pos_modulo ) { __uint128_t tmp_next_src_pos_modulo = static_cast < __uint128_t > ( info . next_src_pos_modulo ) * bookkeeping . denominator ; info . next_src_pos_modulo = tmp_next_src_pos_modulo / old_denominator ; } } }
We compose the effects of clock reconciliation into our sample-rate-conversion step size, but
only for streams that use neither the 'optimal' clock, nor the clock we designate as driving our
hardware-rate-adjustments.
We compose the effects of clock reconciliation into our sample - rate - conversion step size but only for streams that use neither the ' optimal ' clock nor the clock we designate as driving our hardware - rate - adjustments .
// UpdateSourceTrans // // Ensure the mappings from source-frame to source-ref-time and monotonic-time are up-to-date. // we need not also clear rate_mod and pos_mod // Ensure the mappings from source-frame to monotonic-time is up-to-date. // Assert we can map from local monotonic-time to fractional source frames. // UpdateDestTrans // // Ensure the mappings from dest-frame to monotonic-time is up-to-date. // We should only be here if we have a valid mix job. This means a job which supplies a valid // transformation from reference time to destination frames (based on dest frame rate). // we need not also clear rate_mod and pos_mod // Compose our transformation from local monotonic-time to dest frames. // ComposeDestToSource // // Compose our transformation from destination frames to source fractional frames. // // Combine the job-supplied dest transformation, with the renderer-supplied mapping of // monotonic-to-source-subframe, to produce a transformation which maps from dest frames to // fractional source frames. // ComputeFrameRateConversionRatio // // Determine the appropriate TimelineRate for step_size. This based exclusively on the frame rates // of the source and destination. If we happen to be applying "micro-SRC" for this source, then // that will be included subsequently as a correction factor. // SynchronizeClocks // // If client clock and device clock differ, reconcile them. // For start dest frame, measure [predicted - actual] error (in frac_src) since last mix. Do this // even if clocks are same on both sides, as this allows us to perform an initial sync-up between // running position accounting and the initial clock transforms (even those with offsets). // Set new running positions, based on the E2E clock (not just from step_size) // Also should reset the PID controls in the relevant clocks. // Same clock on both sides can occur when multiple MixStages are connected serially (should // both be device clocks). Don't synchronize: use frac_src_frames_per_dest_frame as-is. // We are synchronizing two device clocks. Unfortunately, we know they aren't the same. // To enable this scenario in the future, at least one must be adjustable in hardware (it will // be the "follower"); the other must be marked as hardware_controlling. // MeasureClockError // // Between upstream (source) and downstream (dest) clocks, we should have one device clock // and one client clock. Remember capture mix, where usual roles are reversed. // They can't BOTH be client clocks (although we could handle this, if routing existed). // MeasureClockError // // Measure the error in src_frac_pos // AdjustClock // //   If mix is continuous since then, report the latest error to the AudioClock //   AudioClock feeds errors to its internal PID, producing a running correction factor.... //   ... If clock is hardware-controlling, apply correction factor to the hardware. //   ... If clock is optimal, apply correction factor directly to the client clock. //   ... Otherwise, expose this error correction factor directly to MixStage // Source error exceeds our threshold // Reset the rate adjustment process altogether and allow a discontinuity // Reset the PID controls, in the relevant clocks. // No error is too small to worry about; handle them all. // Adjust client_clock, the 'flexible' clock that we have provided to the client // Adjust device_clock's hardware clock rate based on the frac_source_error // Using this rate adjustment factor, adjust step_size, so future src_positions converge to // what these two clocks require. The product might exceed uint64/uint64: allow reduction. // SetStepSize // // Convert the TimelineRate into [step_size, denominator, rate_modulo] as usual. // If the denominator is changing, update the source position modulos. // Reinterpret previous per-job and long-running src_pos_mod values for the new denominator // Otherwise, preserve the previous source position modulo values