Body: <p>I am streaming video from a camera to a process that monitors the stream for specific events. I am able to connect to the camera, download the m3u8 playlist, fetch the individual AVI video segments and start working on them.</p>
  <p>However, I have noticed that the individual video segments have repeating frames, as if the streaming is inserting some kind of delay for synchronisation purposes.</p>
  <p>Unfortunately, there is nothing in the video segments, or the m3u8 file to denote some kind of sync offset and how much that offset might be so that I can seek to that frame and start processing each segment from its particular offset.</p>
  <p>When I try to access the video segments via ffmpeg, it reports a range of frames as &quot;missing&quot;. Python libraries such as <a href="https://github.com/PyAV-Org/PyAV" rel="nofollow noreferrer">PyAV</a> or <a href="http://soft-matter.github.io/pims/v0.6.1/index.html" rel="nofollow noreferrer">pims</a> do not complain for missing frames but I suspect that these repeated frames (appearing as a long pause) at the beginning of the video segments might be filled in by the libraries themselves if they detect some kind of error with the file.</p>
  <p>I have also gone through <a href="https://datatracker.ietf.org/doc/html/rfc8216" rel="nofollow noreferrer">rfc8216</a> which does mention &quot;discontinuities&quot; but these are marked at the level of segment. My problem here is not with discontinuities from segment to segment, but discontinuities WITHIN the content of a given video segment.</p>
  <p>Is there something I may be missing here? What is the origin of these repeated frames and how would I be able to obtain a number of video segments from the camera that when played back, do not suffer from these long intermitent pauses?</p>
  <p><strong>EDIT:</strong></p>
  <p><a href="https://pastebin.com/xPiufmmP" rel="nofollow noreferrer">Here is</a> the output of <code>ffprobe</code> with <code>-report</code> added. The bit that I am a bit worried about is:</p>
  <pre><code>    Input #0, mpegts, from 'stream_0.avi':
        Duration: 00:00:05.89, start: 58029.795556, bitrate: 346 kb/s
  </code></pre>
  <p>...provided that, that <code>start</code> is an offset into the file where the &quot;useful&quot; content starts (?).</p>
  <p><strong>EDIT2:</strong></p>
  <p><a href="https://pastebin.com/TZCwR6g7" rel="nofollow noreferrer">Here</a> is the <code>ffmpeg</code> report. I think the <code>cur_dts</code> errors are &quot;filled&quot; in automatically with the value of the first available frame.</p>
  <p>The point here is to get some kind of &quot;handle&quot; to these offsets so that I can take them into account when going through the frames across segments and effectively go through an &quot;un-interrupted&quot; sequence of frames just as it would be received from the camera.</p>
  <p>Indeed, the files are not plain AVI, they are MPEG-TS.</p>
  <p><strong>EDIT3:</strong>
  <a href="https://pastebin.com/sDWCCBDY" rel="nofollow noreferrer">Here</a> is the newest report <em>on a different capture segment</em>. The previously captured segments were indeed showing as covering an interval of 6 seconds but carried just 2 frames.</p>
  <p>These &quot;corner cases&quot; are indicative of the kind of information I would need at the point of processing, just like a video player does.</p>
  <p>So, the point here is not so much &quot;just playing the video&quot; or getting access to its frames in the right way, but also being aware of all these little issues which would affect the processing as well. For example, here we have 2 &quot;samples&quot; (2 frames) out of a 6 second window. If this is a glitch of the camera or the network, I would like to get a hint about it because it would help with the processing.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-06-06T05:14:26.967Z
Comments.Text: *When I try to access the video segments via ffmpeg, it reports a range of frames as "missing"* --> Rerun with -report added. Share link to report file.
Comments.UserDisplayName: Gyan
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-06-07T15:03:58.767Z
Comments.Text: @Gyan thank you, please see update
Comments.UserDisplayName: A_A
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-06-08T04:39:34.117Z
Comments.Text: That's a ffprobe report, not ffmpeg. You have to run `ffmpeg -i SEGMENT -f null - -report` .`BTW, your input has extension AVI but is actually MPEG-TS. The non-zero start time is perfectly normal.
Comments.UserDisplayName: Gyan
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-06-09T09:49:21.847Z
Comments.Text: @Gyan Please see "EDIT2". I am already finding this helpful, so at some point, please make sure you post this in some form of an answer. Your *"...MPEG-TS.."* comment was helpful in finding [TS-DUCK](https://tsduck.io/) and more importantly the ["Intro to MPEG-TS"](https://tsduck.io/download/docs/mpegts-introduction.pdf) which I am going through at the moment. I was handed AVI files, no library complained when opening them, I did not think twice about it :)
Comments.UserDisplayName: A_A
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-06-11T15:17:33.497Z
Comments.Text: @Gyan Any ideas about the `ffmpeg` report?
Comments.UserDisplayName: A_A
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-06-11T15:50:08.347Z
Comments.Text: Only 2 video frames are shown as decoded. Run `ffmpeg -copyts -i SEGMENT -an -vf showinfo -f null - -report` and share report.
Comments.UserDisplayName: Gyan
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-06-12T08:09:12.900Z
Comments.Text: @Gyan Thanks, this is useful. There were indeed 2 frames in that segment which may have been some kind of a glitch (?). Please see EDIT3. This is more representative of the kinds of errors I got before (from ffmpeg) that were showing that a number of frames were missing.
Comments.UserDisplayName: A_A
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-06-16T08:19:31.850Z
Comments.Text: @Gyan The missing frames seem to be genuine. That is, even if you discarded the long(ish) periods of time where a single frame is repeated, the remaining segments are not consecutive. I did a longer capture to confirm this and there are random discontinuities throughout. Which means that the frames are genuinely missing from the stream and nothing can be done about it :( Except if there is something else I may have missed here?
Comments.UserDisplayName: A_A
Tags: <ffmpeg><streaming><camera><sync>
Title: Offsets in HLS video segments
UserDisplayName: A_A
ThreadText: # Offsets in HLS video segments
  
  (Asked by: username_0 on 2024-06-05)
  
  I am streaming video from a camera to a process that monitors the stream for specific events. I am able to connect to the camera, download the m3u8 playlist, fetch the individual AVI video segments and start working on them.
  
  However, I have noticed that the individual video segments have repeating frames, as if the streaming is inserting some kind of delay for synchronisation purposes.
  
  Unfortunately, there is nothing in the video segments, or the m3u8 file to denote some kind of sync offset and how much that offset might be so that I can seek to that frame and start processing each segment from its particular offset.
  
  When I try to access the video segments via ffmpeg, it reports a range of frames as "missing". Python libraries such as [PyAV](https://github.com/PyAV-Org/PyAV) or [pims](http://soft-matter.github.io/pims/v0.6.1/index.html) do not complain for missing frames but I suspect that these repeated frames (appearing as a long pause) at the beginning of the video segments might be filled in by the libraries themselves if they detect some kind of error with the file.
  
  I have also gone through [rfc8216](https://datatracker.ietf.org/doc/html/rfc8216) which does mention "discontinuities" but these are marked at the level of segment. My problem here is not with discontinuities from segment to segment, but discontinuities WITHIN the content of a given video segment.
  
  Is there something I may be missing here? What is the origin of these repeated frames and how would I be able to obtain a number of video segments from the camera that when played back, do not suffer from these long intermitent pauses?
  
  **EDIT:**
  
  [Here is](https://pastebin.com/xPiufmmP) the output of `ffprobe` with `-report` added. The bit that I am a bit worried about is:
  
  ```
      Input #0, mpegts, from 'stream_0.avi':
        Duration: 00:00:05.89, start: 58029.795556, bitrate: 346 kb/s
  
  ```
  
  ...provided that, that `start` is an offset into the file where the "useful" content starts (?).
  
  **EDIT2:**
  
  [Here](https://pastebin.com/TZCwR6g7) is the `ffmpeg` report. I think the `cur_dts` errors are "filled" in automatically with the value of the first available frame.
  
  The point here is to get some kind of "handle" to these offsets so that I can take them into account when going through the frames across segments and effectively go through an "un-interrupted" sequence of frames just as it would be received from the camera.
  
  Indeed, the files are not plain AVI, they are MPEG-TS.
  
  **EDIT3:**
  [Here](https://pastebin.com/sDWCCBDY) is the newest report *on a different capture segment*. The previously captured segments were indeed showing as covering an interval of 6 seconds but carried just 2 frames.
  
  These "corner cases" are indicative of the kind of information I would need at the point of processing, just like a video player does.
  
  So, the point here is not so much "just playing the video" or getting access to its frames in the right way, but also being aware of all these little issues which would affect the processing as well. For example, here we have 2 "samples" (2 frames) out of a 6 second window. If this is a glitch of the camera or the network, I would like to get a hint about it because it would help with the processing.
  
  **Tags:** ffmpeg, streaming, camera, sync
  
  > **Comments:**
  > **username_1**: *When I try to access the video segments via ffmpeg, it reports a range of frames as "missing"* --> Rerun with -report added. Share link to report file.  
  > **username_0**: @username_1 thank you, please see update  
  > **username_1**: That's a ffprobe report, not ffmpeg. You have to run `ffmpeg -i SEGMENT -f null - -report` .`BTW, your input has extension AVI but is actually MPEG-TS. The non-zero start time is perfectly normal.  
  > **username_0**: @username_1 Please see "EDIT2". I am already finding this helpful, so at some point, please make sure you post this in some form of an answer. Your *"...MPEG-TS.."* comment was helpful in finding [TS-DUCK](https://tsduck.io/) and more importantly the ["Intro to MPEG-TS"](https://tsduck.io/download/docs/mpegts-introduction.pdf) which I am going through at the moment. I was handed AVI files, no library complained when opening them, I did not think twice about it :)  
  > **username_0**: @username_1 Any ideas about the `ffmpeg` report?  
  > **username_1**: Only 2 video frames are shown as decoded. Run `ffmpeg -copyts -i SEGMENT -an -vf showinfo -f null - -report` and share report.  
  > **username_0**: @username_1 Thanks, this is useful. There were indeed 2 frames in that segment which may have been some kind of a glitch (?). Please see EDIT3. This is more representative of the kinds of errors I got before (from ffmpeg) that were showing that a number of frames were missing.  
  > **username_0**: @username_1 The missing frames seem to be genuine. That is, even if you discarded the long(ish) periods of time where a single frame is repeated, the remaining segments are not consecutive. I did a longer capture to confirm this and there are random discontinuities throughout. Which means that the frames are genuinely missing from the stream and nothing can be done about it :( Except if there is something else I may have missed here?  
  
  ---
  
  ## 0 answers
  
  