Answers.Body: <p>You can have timestamp gaps in an audio stream in Matroska-family or MOV/MP4 format files. That's how Google records and stores WebRTC recordings. There won't be zero signal audio data, rather there will be no packets corresponding to the gap and timestamps of the packets before and after the gap will indicate audio position.</p>
  <p>However, it is not the encoder's job or feature in ffmpeg to drop those sections. You will need to use the <a href="http://www.ffmpeg.org/ffmpeg-filters.html#silenceremove" rel="nofollow noreferrer">silenceremove</a> filter with option <code>timestamp=copy</code> (6.1+ or git master &gt; May 2023 needed).</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-12-09T09:52:37.367Z
Answers.Text: Would `silenceremove` produce such output with gaps? Or would it "collapse" the gaps and make the wall-clock-duration shorter? This was my misunderstanding while originally reading its documentation which led my to asking this question.
Answers.UserDisplayName: Vadim Kantorov
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-12-09T10:45:39.597Z
Answers.Text: silenceremove can only remove data; it doesn't consolidate gaps.
Answers.UserDisplayName: Gyan
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-12-09T10:58:31.363Z
Answers.Text: `silenceremove` does not affect any timestamps and they stay original (as if no gaps were introduced), right? My goal is to have the encoded file have the same duration as before encoding (and have silence imputed by zeros at decoding time). Do you know if gaps are supported by `webm`/`mka`? Am I correct that `ogg` does not support gaps? Would you have an advice of how to instruct `ffmpeg` to encode separately `.wav` streams / in a decoupled mode (for phone call recordings, there are long periods of silence if one considers every channel separately)?
Answers.UserDisplayName: Vadim Kantorov
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-12-14T14:46:56.523Z
Answers.Text: I tested the `silenceremove` solution. Unfortunately, it does not create `gaps` and it changes the duration of the audio - this is not what I need. I'd like to simply drop the silent packets from the stream, but preserve the timestamps and keep the audio duration and have the dropped packets be replaced by digital silence at decoding time.
Answers.UserDisplayName: Vadim Kantorov
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-12-14T14:47:08.800Z
Answers.Text: Here is a repro: `ffmpeg -y -f lavfi -i sine=f=440:d=5 -f lavfi -i anullsrc=d=3600:cl=mono -f lavfi -i sine=f=880:d=5 -filter_complex "[0][1][2]concat=n=3:v=0:a=1" 5sec_silent1h_5sec.wav;
  ffmpeg -y -i 5sec_silent1h_5sec.wav -acodec libopus 5sec_silent1h_5sec.opus;
  ffmpeg -y -i 5sec_silent1h_5sec.wav -acodec libopus -af silenceremove=window=0:detection=peak:stop_mode=all:start_mode=all:stop_periods=-1:stop_threshold=0 5sec_5sec.mkv`
Answers.UserDisplayName: Vadim Kantorov
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-12-14T14:48:01.717Z
Answers.Text: The good duration of `5sec_silent1h_5sec.opus` and bad duration of `5sec_5sec.mkv` are confirmed by `ffprobe` (1h10s and 10s respectively)
Answers.UserDisplayName: Vadim Kantorov
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-12-14T16:14:39.537Z
Answers.Text: Updated answer. See at the end.
Answers.UserDisplayName: Gyan
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-12-14T16:19:59.643Z
Answers.Text: Thanks for helping out with this :) I've just tried your updated proposal using ffmpeg 6.1 static bits from https://johnvansickle.com/ffmpeg/, the duration turns correct, but it has the same leakage/wrong pts problem as my attempts with `aselect` filter :(
Answers.UserDisplayName: Vadim Kantorov
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-12-14T17:15:33.260Z
Answers.Text: I converted the input `5sec_silent1h_5sec.wav` to be 48khz (in my original code above it was 44.1khz), but it made no difference - still leakage
Answers.UserDisplayName: Vadim Kantorov
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-12-15T05:42:57.583Z
Answers.Text: Probably to do with the attack/release used by the filter. Will have to debug when I have time.
Answers.UserDisplayName: Gyan
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-12-15T12:00:11.173Z
Answers.Text: Thank you! However, this leakage is quite long - a full second :( I hope my repro above will help!
Answers.UserDisplayName: Vadim Kantorov
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-12-15T14:06:14.563Z
Answers.Text: Also, maybe the leakage is related to some silenceremove filter options I used above?
  
  But given the strangeness of such leakage, I suspect that somehow some packets of the 880hz-segment get incorrect timestamps assigned which forces them to be played before the 1h-long silence/gap
Answers.UserDisplayName: Vadim Kantorov
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-12-09T04:31:38.507Z
Answers.LastActivityDate: 2023-12-14T16:14:09.757Z
Answers.LastEditDate: 2023-12-14T16:14:09.757Z
Answers.UserDisplayName: Gyan
Body: <p>Does <code>webm</code>/<code>mka</code>/<code>mkv</code>/<code>mp4</code> support discontinuous audio streams?</p>
  <p>I'm wondering, can <code>ffmpeg</code> transcode <code>.wav</code>-audio into <code>opus</code>-encapsulated-into-<code>webm</code>, but filter out silent frames - especially full seconds of silence, completely drop them from the stream to save space? (maybe via <a href="https://ffmpeg.org/ffmpeg-bitstream-filters.html" rel="nofollow noreferrer">https://ffmpeg.org/ffmpeg-bitstream-filters.html</a> ?)</p>
  <p>The goal of all this is to very efficiently represent near-silence in phone-call-recording files for archival purposes (i.e. by completely omitting it from the stream and having it replaced by digital zeros at audio decoding time).</p>
  <p>As far as I understand <code>opus</code>-encapsulated-into-<code>ogg</code> does not support such timestamp discontinuities. <code>opus</code>-encapsulated-into-<code>rtp</code> assumingly does support this, but I'm wondering if more &quot;offline&quot; formats such as <code>webm</code> and whether it's possible to produce them directly without passing by <code>rtp</code> format first.</p>
  <p>These frames could then be replaced by zeros at audio decoding time by some form of PLC or DTS gaps replacement by zero signal.</p>
  <p>Thanks!</p>
  <hr />
  <p><strong>UPD1</strong>: Unfortunately, <code>silenceremove</code> filter does not work as I need it. The way I used it below, the audio duration after <code>silenceremove</code> is shortened by the amount of silence. This is not what I am looking for. I am looking to preserve the timestamps, have the silence imputed by digital silence at decoding time and preserve the original audio duration (as reported by <code>ffprobe</code> or audio players)</p>
  <pre class="lang-shell prettyprint-override"><code>ffmpeg -y -f lavfi -i sine=f=440:d=5 -f lavfi -i anullsrc=d=3600:cl=mono -f lavfi -i sine=f=880:d=5 -filter_complex &quot;[0][1][2]concat=n=3:v=0:a=1&quot; 5sec_silent1h_5sec.wav
  
  ffmpeg -y -i 5sec_silent1h_5sec.wav -acodec libopus 5sec_silent1h_5sec.opus
  
  ffmpeg -y -i 5sec_silent1h_5sec.wav -acodec libopus -af silenceremove=window=0:detection=peak:stop_mode=all:start_mode=all:stop_periods=-1:stop_threshold=0 5sec_5sec.mkv
  </code></pre>
  <p>The good duration of 5sec_silent1h_5sec.opus and bad duration of 5sec_5sec.mkv are confirmed by ffprobe (1h10s and 10s respectively).</p>
  <p>Is there a way to adjust the filters/command line arguments to preserve the audio duration?</p>
  <hr />
  <p><strong>UPD2</strong>
  The following somewhat works, although ~1s of 880hz audio leaks into the first segment (before the long silence segment), no matter how I adjust the <code>between</code> bounds - very strange!</p>
  <p><code>ffmpeg -y -i 5sec_silent1h_5sec.wav -acodec libopus -af &quot;aselect='between(t,0,4)+between(t,3606,3610)'&quot; 5sec_silent1h_5sec_with_leakage.mkv</code></p>
  <p>If this leakage can be fixed, then this can be used <a href="https://superuser.com/a/1659916/813257">together</a> <a href="https://stackoverflow.com/questions/74418102/more-precision-from-ffmpeg-silencedetect">with</a> <code>silencedetect</code> filter.</p>
  <p>But of course it would be nicer to get the output in a single invokation of ffmpeg.</p>
  <p>The example output file with &quot;leakage&quot;: <a href="https://1drv.ms/u/s!Apx8USiTtrYmq9Q4QaETF2OJWdKGQA?e=ACTmMg" rel="nofollow noreferrer">https://1drv.ms/u/s!Apx8USiTtrYmq9Q4QaETF2OJWdKGQA?e=ACTmMg</a></p>
  <hr />
  <p><strong>UPD3</strong>: On a recent ffmpeg 6.1 adding <code>timestamp=copy</code> filter option also produces a file with a correct duration, but with leakage <code>ffmpeg -y -i 5sec_silent1h_5sec.wav -acodec libopus -af silenceremove=window=0:detection=peak:stop_mode=all:start_mode=all:stop_periods=-1:stop_threshold=0:timestamp=copy 5sec_1hsilence_5sec_with_leakage.mkv</code>. So it seems that timestamps are still not copied exactly.</p>
  
Tags: <ffmpeg><audio><matroska>
Title: Possible to force FFmpeg to omit silence in audio files and produce discontinuous `webm`/`mka`?
UserDisplayName: Vadim Kantorov
ThreadText: # Possible to force FFmpeg to omit silence in audio files and produce discontinuous `webm`/`mka`?
  
  (Asked by: username_0 on 2023-12-09)
  
  Does `webm`/`mka`/`mkv`/`mp4` support discontinuous audio streams?
  
  I'm wondering, can `ffmpeg` transcode `.wav`-audio into `opus`-encapsulated-into-`webm`, but filter out silent frames - especially full seconds of silence, completely drop them from the stream to save space? (maybe via <https://ffmpeg.org/ffmpeg-bitstream-filters.html> ?)
  
  The goal of all this is to very efficiently represent near-silence in phone-call-recording files for archival purposes (i.e. by completely omitting it from the stream and having it replaced by digital zeros at audio decoding time).
  
  As far as I understand `opus`-encapsulated-into-`ogg` does not support such timestamp discontinuities. `opus`-encapsulated-into-`rtp` assumingly does support this, but I'm wondering if more "offline" formats such as `webm` and whether it's possible to produce them directly without passing by `rtp` format first.
  
  These frames could then be replaced by zeros at audio decoding time by some form of PLC or DTS gaps replacement by zero signal.
  
  Thanks!
  
  ---
  
  **UPD1**: Unfortunately, `silenceremove` filter does not work as I need it. The way I used it below, the audio duration after `silenceremove` is shortened by the amount of silence. This is not what I am looking for. I am looking to preserve the timestamps, have the silence imputed by digital silence at decoding time and preserve the original audio duration (as reported by `ffprobe` or audio players)
  
  ```shell
  ffmpeg -y -f lavfi -i sine=f=440:d=5 -f lavfi -i anullsrc=d=3600:cl=mono -f lavfi -i sine=f=880:d=5 -filter_complex "[0][1][2]concat=n=3:v=0:a=1" 5sec_silent1h_5sec.wav
  
  ffmpeg -y -i 5sec_silent1h_5sec.wav -acodec libopus 5sec_silent1h_5sec.opus
  
  ffmpeg -y -i 5sec_silent1h_5sec.wav -acodec libopus -af silenceremove=window=0:detection=peak:stop_mode=all:start_mode=all:stop_periods=-1:stop_threshold=0 5sec_5sec.mkv
  
  ```
  
  The good duration of 5sec\_silent1h\_5sec.opus and bad duration of 5sec\_5sec.mkv are confirmed by ffprobe (1h10s and 10s respectively).
  
  Is there a way to adjust the filters/command line arguments to preserve the audio duration?
  
  ---
  
  **UPD2**
  The following somewhat works, although ~1s of 880hz audio leaks into the first segment (before the long silence segment), no matter how I adjust the `between` bounds - very strange!
  
  `ffmpeg -y -i 5sec_silent1h_5sec.wav -acodec libopus -af "aselect='between(t,0,4)+between(t,3606,3610)'" 5sec_silent1h_5sec_with_leakage.mkv`
  
  If this leakage can be fixed, then this can be used [together](https://superuser.com/a/1659916/813257) [with](https://stackoverflow.com/questions/74418102/more-precision-from-ffmpeg-silencedetect) `silencedetect` filter.
  
  But of course it would be nicer to get the output in a single invokation of ffmpeg.
  
  The example output file with "leakage": <https://1drv.ms/u/s!Apx8USiTtrYmq9Q4QaETF2OJWdKGQA?e=ACTmMg>
  
  ---
  
  **UPD3**: On a recent ffmpeg 6.1 adding `timestamp=copy` filter option also produces a file with a correct duration, but with leakage `ffmpeg -y -i 5sec_silent1h_5sec.wav -acodec libopus -af silenceremove=window=0:detection=peak:stop_mode=all:start_mode=all:stop_periods=-1:stop_threshold=0:timestamp=copy 5sec_1hsilence_5sec_with_leakage.mkv`. So it seems that timestamps are still not copied exactly.
  
  **Tags:** ffmpeg, audio, matroska
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-12-09. Score: 0
  
  You can have timestamp gaps in an audio stream in Matroska-family or MOV/MP4 format files. That's how Google records and stores WebRTC recordings. There won't be zero signal audio data, rather there will be no packets corresponding to the gap and timestamps of the packets before and after the gap will indicate audio position.
  
  However, it is not the encoder's job or feature in ffmpeg to drop those sections. You will need to use the [silenceremove](http://www.ffmpeg.org/ffmpeg-filters.html#silenceremove) filter with option `timestamp=copy` (6.1+ or git master > May 2023 needed).
  
  > **Comments:**
  > **username_0**: Would `silenceremove` produce such output with gaps? Or would it "collapse" the gaps and make the wall-clock-duration shorter? This was my misunderstanding while originally reading its documentation which led my to asking this question.  
  > **username_1**: silenceremove can only remove data; it doesn't consolidate gaps.  
  > **username_0**: `silenceremove` does not affect any timestamps and they stay original (as if no gaps were introduced), right? My goal is to have the encoded file have the same duration as before encoding (and have silence imputed by zeros at decoding time). Do you know if gaps are supported by `webm`/`mka`? Am I correct that `ogg` does not support gaps? Would you have an advice of how to instruct `ffmpeg` to encode separately `.wav` streams / in a decoupled mode (for phone call recordings, there are long periods of silence if one considers every channel separately)?  
  > **username_0**: I tested the `silenceremove` solution. Unfortunately, it does not create `gaps` and it changes the duration of the audio - this is not what I need. I'd like to simply drop the silent packets from the stream, but preserve the timestamps and keep the audio duration and have the dropped packets be replaced by digital silence at decoding time.  
  > **username_0**: Here is a repro: `ffmpeg -y -f lavfi -i sine=f=440:d=5 -f lavfi -i anullsrc=d=3600:cl=mono -f lavfi -i sine=f=880:d=5 -filter_complex "[0][1][2]concat=n=3:v=0:a=1" 5sec_silent1h_5sec.wav; ffmpeg -y -i 5sec_silent1h_5sec.wav -acodec libopus 5sec_silent1h_5sec.opus; ffmpeg -y -i 5sec_silent1h_5sec.wav -acodec libopus -af silenceremove=window=0:detection=peak:stop_mode=all:start_mode=all:stop_periods=-1:stop_threshold=0 5sec_5sec.mkv`  
  > **username_0**: The good duration of `5sec_silent1h_5sec.opus` and bad duration of `5sec_5sec.mkv` are confirmed by `ffprobe` (1h10s and 10s respectively)  
  > **username_1**: Updated answer. See at the end.  
  > **username_0**: Thanks for helping out with this :) I've just tried your updated proposal using ffmpeg 6.1 static bits from https://johnvansickle.com/ffmpeg/, the duration turns correct, but it has the same leakage/wrong pts problem as my attempts with `aselect` filter :(  
  > **username_0**: I converted the input `5sec_silent1h_5sec.wav` to be 48khz (in my original code above it was 44.1khz), but it made no difference - still leakage  
  > **username_1**: Probably to do with the attack/release used by the filter. Will have to debug when I have time.  
  > **username_0**: Thank you! However, this leakage is quite long - a full second :( I hope my repro above will help!  
  > **username_0**: Also, maybe the leakage is related to some silenceremove filter options I used above?  But given the strangeness of such leakage, I suspect that somehow some packets of the 880hz-segment get incorrect timestamps assigned which forces them to be played before the 1h-long silence/gap  
  
  ---
  
  