title: Getting stuttering during rendering of my DirectShow filter despite output file being "smooth"
body: <p>I have a DirectShow application written in Delphi 6 using the DSPACK component library.  I have two filter graphs that cooperate with each other.</p>
  
  <p>The <em>primary</em> filter graph has this structure:</p>
  
  <ol>
  <li>Capture Filter with 100 ms buffer size.</li>
  <li>(connected to) A Sample Grabber Filter.</li>
  </ol>
  
  <p>The "secondary" filter graph has this structure.</p>
  
  <ol>
  <li>Custom Push Source Filter that accepts audio directly to an audio buffer storehouse it manages.</li>
  <li>(connected to) A Render Filter.</li>
  </ol>
  
  <p>The Push Source Filter uses an Event to control delivery of audio.  Its FillBuffer() command waits on the Event.  The Event is signaled when new audio data is added to the buffer.  </p>
  
  <p>When I run the filter graphs, I hear tiny "gaps" in the audio.  Usually I associate that condition with an improperly constructed audio buffers that are not filled in or have "gaps" in them.  But as a test I added a Tee Filter and connected a WAV Dest Filter followed by a File Writer filter.  When I examine the output WAV file, it is perfectly smooth and contiguous.  In other words, the gaps I hear from the speaker are not evident in the output file.</p>
  
  <p>This would indicate that the although the audio from the Capture Filter is propagating successfully, the delivery of the audio buffers is getting periodic interference.  The "gaps" I hear are not 10 times a second but more like 2 or 3 times a second, sometimes with short periods of no gaps at all.  So it is not happening every buffer or I would hear gaps at 10 times a second.</p>
  
  <p>My first guess would be that its a locking problem, but I have a time-out set on the Event of 150 ms and if that happens an Exception is thrown.  No Exceptions are being thrown.  I also have a time-out of 40 ms set on <em>every</em> Critical Section that is used in the application and <em>none</em> of those are triggering either.  I checked my OutputDebugString() dumps and the time between the <em>non-signaled</em> (blocked) and <em>signaled</em> (unblocked) occurrences shows a fairly constant pattern alternating between 94 ms and 140 ms.  In other words, the FillBuffer() call in my Push Source Filter stays blocked for 94 ms, then 140 ms, and repeats.  Note the durations drift a bit but its pretty regular.  That pattern seems consistent with a thread waiting on the Capture Filter to dump its audio buffer to into the Push Source Filter on a 100 ms interval, given the vagaries of Windows thread switching.</p>
  
  <p>I <em>think</em> I am using double-buffering in my Push Source Filter, so my belief is that if none of the locking mechanisms are taking a combined time of 200 ms or more, I should not be interrupting the audio stream.  But I can't think of anything else other than a locking problem that would cause these symptoms.  I have included the code from my DecideBufferSize() method in my Push Source Filter below in case I am doing something wrong.  Although it is somewhat lengthy, I have also included the FillBuffer() call below to show how I am generating timestamps, in case that might be having an effect.</p>
  
  <p>What else could be causing my audio stream to the Render Filter to stutter despite all the audio buffers being delivered intact?</p>
  
  <p><strong>Question</strong>: Do I have to implement double-buffering myself?  I figured DirectShow render filters do that for you, otherwise the other filter graphs I created without my custom Push Source Filter wouldn't have worked properly.  But perhaps since I am creating another lock/unlock situation in the filter graph I need to add my own layer of double buffering?  I would like to avoid that of course to avoid additional latency so if there is another fix for my situation I'd like to know.</p>
  
  <pre><code>function TPushSourcePinBase_wavaudio.DecideBufferSize(Allocator: IMemAllocator; Properties: PAllocatorProperties): HRESULT;
  var
      // pvi: PVIDEOINFOHEADER;
      errMsg: string;
      Actual: ALLOCATOR_PROPERTIES;
      sampleSize, numBytesPerBuffer: integer;
      // ourOwnerFilter: TPushSourceFilterBase_wavaudio;
  begin
      if (Allocator = nil) or (Properties = nil) then
      begin
          Result := E_POINTER;
          // =========================== EXIT POINT ==============
          Exit;
      end; // if (Allocator = nil) or (Properties = nil) then
  
      FFilter.StateLock.Lock;
      try
          // Allocate enough space for the desired amount of milliseconds
          //  we want to buffer (approximately).
          numBytesPerBuffer := Trunc((FOurOwnerFilter.WaveFormatEx.nAvgBytesPerSec / 1000) * FBufferLatencyMS);
  
          // Round it up to be an even multiple of the size of a sample in bytes.
          sampleSize := bytesPerSample(FOurOwnerFilter.WaveFormatEx);
  
          // Round it down to the nearest increment of sample size.
          numBytesPerBuffer := (numBytesPerBuffer div sampleSize) * sampleSize;
  
          if gDebug then OutputDebugString(PChar(
              '(TPushSourcePinBase_wavaudio.DecideBufferSize) Resulting buffer size for audio is: ' + IntToStr(numBytesPerBuffer)
          ));
  
          // Sanity check on the buffer size.
          if numBytesPerBuffer &lt; 1 then
          begin
              errMsg := '(TPushSourcePinBase_wavaudio.DecideBufferSize) The calculated number of bytes per buffer is zero or less.';
  
              if gDebug then OutputDebugString(PChar(errMsg));
              MessageBox(0, PChar(errMsg), 'PushSource Play Audio File filter error', MB_ICONERROR or MB_OK);
  
              Result := E_FAIL;
              // =========================== EXIT POINT ==============
              Exit;
          end;
  
          // --------------- Do the buffer allocation -----------------
  
          // Ensure a minimum number of buffers
          if (Properties.cBuffers = 0) then
              Properties.cBuffers := 2;
  
          Properties.cbBuffer := numBytesPerBuffer;
  
          Result := Allocator.SetProperties(Properties^, Actual);
  
          if Failed(Result) then
              // =========================== EXIT POINT ==============
              Exit;
  
          // Is this allocator unsuitable?
          if (Actual.cbBuffer &lt; Properties.cbBuffer) then
              Result := E_FAIL
          else
              Result := S_OK;
  
      finally
          FFilter.StateLock.UnLock;
      end; // try()
  end;
  
  // *******************************************************
  
  
  // This is where we provide the audio data.
  function TPushSourcePinBase_wavaudio.FillBuffer(Sample: IMediaSample): HResult;
      // Given a Wave Format and a Byte count, convert the Byte count
      //  to a REFERENCE_TIME value.
      function byteCountToReferenceTime(waveFormat: TWaveFormat; numBytes: LongInt): REFERENCE_TIME;
      var
          durationInSeconds: Extended;
      begin
          if waveFormat.nAvgBytesPerSec &lt;= 0 then
              raise Exception.Create('(TPushSourcePinBase_wavaudio.FillBuffer::byteCountToReferenceTime) Invalid average bytes per second value found in the wave format parameter: ' + IntToStr(waveFormat.nAvgBytesPerSec));
  
          // Calculate the duration in seconds given the audio format and the
          //  number of bytes requested.
          durationInSeconds := numBytes / waveFormat.nAvgBytesPerSec;
  
          // Convert it to increments of 100ns since that is the unit value
          //  for DirectShow timestamps (REFERENCE_TIME).
          Result :=
              Trunc(durationInSeconds * REFTIME_ONE_SECOND);
      end;
  
      // ---------------------------------------------------------------
  
      function min(v1, v2: DWord): DWord;
      begin
          if v1 &lt;= v2 then
              Result := v1
          else
              Result := v2;
      end;
  
      // ---------------------------------------------------------------
  
  var
      pData: PByte;
      cbData: Longint;
      pwfx: PWaveFormat;
      aryOutOfDataIDs: TDynamicStringArray;
      intfAudFiltNotify: IAudioFilterNotification;
      i: integer;
      errMsg: string;
      bIsShuttingDown: boolean;
  
      // MSDN: The REFERENCE_TIME data type defines the units for reference times
      //  in DirectShow. Each unit of reference time is 100 nanoseconds.
      Start, Stop: REFERENCE_TIME;
      durationInRefTime, ofsInRefTime: REFERENCE_TIME;
      wfOutputPin: TWaveFormat;
  
      aryDebug: TDynamicByteArray;
  begin
      aryDebug := nil;
  
      if (Sample = nil) then
      begin
          Result := E_POINTER;
          // =========================== EXIT POINT ==============
          Exit;
      end; // if (Sample = nil) then
  
      // Quick lock to get sample size.
      FSharedState.Lock;
      try
          cbData := Sample.GetSize;
      finally
          // Don't want to have our filter state locked when calling
          //  isEnoughDataOrBlock() since that call can block.
          FSharedState.UnLock;
      end; // try
  
      aryOutOfDataIDs := nil;
      intfAudFiltNotify := nil;
  
      // This call will BLOCK until have enough data to satisfy the request
      //  or the buffer storage collection is freed.
      if FOurOwnerFilter.bufferStorageCollection.isEnoughDataOrBlock(cbData, bIsShuttingDown) then
      begin
          // If we are shutting down, just exit with S_FALSE as the return to
          //   tell the caller we are done streaming.
          if bIsShuttingDown then
          begin
              Result := S_FALSE;
  
              // =========================== EXIT POINT ==============
              exit;
          end; // if bIsShuttingDown then
  
          // Re-acquire the filter state lock.
          FSharedState.Lock;
  
          try
              // Get the data and return it.
  
              // Access the sample's data buffer
              cbData := Sample.GetSize;
              Sample.GetPointer(pData);
  
              // Make sure this format matches the media type we are supporting.
              pwfx := AMMediaType.pbFormat;       // This is the format that our Output pin is set to.
              wfOutputPin := waveFormatExToWaveFormat(FOurOwnerFilter.waveFormatEx);
  
              if not isEqualWaveFormat(pwfx^, wfOutputPin) then
              begin
                  Result := E_FAIL;
  
                  errMsg :=
                      '(TPushSourcePinBase_wavaudio.FillBuffer) The wave format of the incoming media sample does not match ours.'
                      + CRLF
                      + ' &gt; Incoming sample: ' + waveFormatToString(pwfx^)
                      + CRLF
                      + ' &gt; Our output pin: ' + waveFormatToString(wfOutputPin);
                  OutputDebugString(PChar(errMsg));
  
                  postComponentLogMessage_error(errMsg, FOurOwnerFilter.FFilterName);
  
                  MessageBox(0, PChar(errMsg), 'PushSource Play Audio File filter error', MB_ICONERROR or MB_OK);
  
                  Result := E_FAIL;
  
                  // =========================== EXIT POINT ==============
                  exit;
              end; // if not isEqualWaveFormatEx(pwfx^, FOurOwnerFilter.waveFormatEx) then
  
              // Convert the Byte index into the WAV data array into a reference
              //  time value in order to offset the start and end timestamps.
              ofsInRefTime := byteCountToReferenceTime(pwfx^, FWaveByteNdx);
  
              // Convert the number of bytes requested to a reference time vlaue.
              durationInRefTime := byteCountToReferenceTime(pwfx^, cbData);
  
              // Now I can calculate the timestamps that will govern the playback
              //  rate.
              Start := ofsInRefTime;
              Stop := Start + durationInRefTime;
  
              {
              OutputDebugString(PChar(
                  '(TPushSourcePinBase_wavaudio.FillBuffer) Wave byte index, start time, stop time: '
                  + IntToStr(FWaveByteNdx)
                  + ', '
                  + IntToStr(Start)
                  + ', '
                  + IntToStr(Stop)
              ));
              }
  
              Sample.SetTime(@Start, @Stop);
  
              // Set TRUE on every sample for uncompressed frames
              Sample.SetSyncPoint(True);
  
              // Check that we're still using audio
              Assert(IsEqualGUID(AMMediaType.formattype, FORMAT_WaveFormatEx));
  
  {
  // Debugging.
  FillChar(pData^, cbData, 0);
  SetLength(aryDebug, cbData);
  if not FOurOwnerFilter.bufferStorageCollection.mixData(@aryDebug[0], cbData, aryOutOfDataIDs) then
  }
              // Grab the requested number of bytes from the audio data.
              if not FOurOwnerFilter.bufferStorageCollection.mixData(pData, cbData, aryOutOfDataIDs) then
              begin
                  // We should not have had any partial copies since we
                  //  called isEnoughDataOrBlock(), which is not supposed to
                  //  return TRUE unless there is enough data.
                  Result := E_FAIL;
  
                  errMsg := '(TPushSourcePinBase_wavaudio.FillBuffer) The mix-data call returned FALSE despite our waiting for sufficient data from all participating buffer channels.';
                  OutputDebugString(PChar(errMsg));
  
                  postComponentLogMessage_error(errMsg, FOurOwnerFilter.FFilterName);
  
                  MessageBox(0, PChar(errMsg), 'PushSource Play Audio File filter error', MB_ICONERROR or MB_OK);
  
                  Result := E_FAIL;
  
                  // =========================== EXIT POINT ==============
                  exit;
              end; // if not FOurOwnerFilter.bufferStorageCollection.mixData(pData, cbData, aryOutOfDataIDs) then
  
              // ------------- OUT OF DATA NOTIFICATIONS -----------------
  
              {
                  WARNING:  TBufferStorageCollection automatically posts
                  AudioFilterNotification messages to any buffer storage
                  that has a IRequestStep user data interface attached to
                  it!.
              }
  
              if FOurOwnerFilter.wndNotify &gt; 0 then
              begin
                  // ----- Post Audio Notification to Filter level notify handle ---
                  if Length(aryOutOfDataIDs) &gt; 0 then
                  begin
                      for i := Low(aryOutOfDataIDs) to High(aryOutOfDataIDs) do
                      begin
                          // Create a notification and post it.
                          intfAudFiltNotify := TAudioFilterNotification.Create(aryOutOfDataIDs[i], afnOutOfData);
  
                          // ourOwnerFilter.intfNotifyRequestStep.triggerResult(intfAudFiltNotify);
                          PostMessageWithUserDataIntf(FOurOwnerFilter.wndNotify, WM_PUSH_SOURCE_FILTER_NOTIFY, intfAudFiltNotify);
                      end; // for()
                  end; // if Length(aryOutOfDataIDs) &gt; 0 then
              end; // if FOurOwnerFilter.wndNotify &gt; 0 then
  
              // Advance the Wave Byte index by the number of bytes requested.
              Inc(FWaveByteNdx, cbData);
  
              Result := S_OK;
          finally
              FSharedState.UnLock;
          end; // try
      end
      else
      begin
          // Tell DirectShow to stop streaming with us.  Something has
          //  gone seriously wrong with the audio streams feeding us.
          errMsg := '(TPushSourcePinBase_wavaudio.FillBuffer) Time-out occurred while waiting for sufficient data to accumulate in our audio buffer channels.';
          OutputDebugString(PChar(errMsg));
  
          postComponentLogMessage_error(errMsg, FFilter.filterName);
          MessageBox(0, PChar(errMsg), 'PushSource Play Audio File filter error', MB_ICONERROR or MB_OK);
  
          Result := E_FAIL;
      end;
  end;
  </code></pre>