Answers.Body: <p>It turns out it is impossible to live stream with the RPi hardware acceleration using ffmpeg.
  For ffmpeg to configure the hardware acceleration to issue SPS/PPS frames during the stream and not only at the beginning of it, it needs to use a <a href="https://github.com/raspberrypi/firmware/issues/242" rel="nofollow noreferrer">specific API of the raspberry firmware</a> which it does not at the moment. </p>
  
  <p><a href="http://frozen.ca/streaming-raw-h-264-from-a-raspberry-pi/" rel="nofollow noreferrer">Raspivid</a> and <a href="http://gstreamer-devel.966125.n4.nabble.com/omxh264enc-and-stream-format-string-avc-td4671543.html" rel="nofollow noreferrer">gstreamer</a> does that though. So turn to thoses solution and forget about ffmpeg.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-04-21T17:45:43.920Z
Answers.Text: If this is the issue, then you may be able to stream the omx encode to a pipe in mpegts format and then use a 2nd ffmpeg instance to pipe to rtp. The MPEG-TS muxer will insert xPS in-band.
Answers.UserDisplayName: Gyan
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-04-21T16:01:00.940Z
Answers.LastActivityDate: 2017-04-21T16:01:00.940Z
Answers.UserDisplayName: Luke Skywalker
Answers.Body: <p>Here my FFmpeg patch to enable inline header on OMX hardware encoder.</p>
  
  <p>libavcodec/omx.c
  OMX_SetParameter(s->handle, OMX_IndexParamBrcmVideoAVCInlineHeaderEnable, &amp;vid_param_inline_header);
  where vid_param_inline_header is a OMX_CONFIG_PORTBOOLEANTYPE struct.</p>
  
  <p><a href="https://github.com/Serveurperso/FFmpeg/commit/6e410823e63103342e8fc3407ff6698808d4d1ef" rel="nofollow noreferrer">https://github.com/Serveurperso/FFmpeg/commit/6e410823e63103342e8fc3407ff6698808d4d1ef</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-12-10T18:35:07.633Z
Answers.Text: Consider submitting a patch to ffmpeg-devel but make it an option instead of hardcoded.
Answers.UserDisplayName: Gyan
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-12-10T21:26:46.533Z
Answers.Text: Yes but there is a last problem:( only when I enable OMX InlineHeader the latency grow over the time.There is a strange time calculation problem inside the muxer. I get a lot of "application provided invalid, non monotonically increasing dts to muxer in stream xxxx/0" and it always freeze at 3335.
Answers.UserDisplayName: Serveurperso
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-12-10T18:10:56.190Z
Answers.LastActivityDate: 2019-12-10T18:10:56.190Z
Answers.UserDisplayName: Serveurperso
Body: <p>Trying to stream a webcam through rtp with a raspberry pi.</p>
  
  <p>Cating the SDP file on my desktop, and launching ffplay:</p>
  
  <pre><code>ffplay sdp.out
  </code></pre>
  
  <p>then streaming out this way:</p>
  
  <pre><code>ffmpeg -i /dev/video0 -c:v h264_omx -f rtp rtp://192.168.0.101:1234
  </code></pre>
  
  <p>works perfectly. But If I launch ffmpeg first THEN ffplay, ffplay stutters some error message about missing PPS and never displays any frame:</p>
  
  <pre><code>[h264 @ 0x7fd97c0008c0] decode_slice_header error
  [h264 @ 0x7fd97c0008c0] no frame!
  [h264 @ 0x7fd97c0008c0] non-existing PPS 0 referenced
  </code></pre>
  
  <p>After looking for a while I understand some intermediary frame might be missing. However, using the <code>-g</code> option this way does not change anything:</p>
  
  <pre><code>ffmpeg -i /dev/video0 -c:v h264_omx -g 30 -f rtp rtp://192.168.0.101:1234
  </code></pre>
  
  <p>It is a problem for me because I am using the raspberry as a securiy camera. My desktop can be restarted from time to time and I need ffplay to be able to catch after the streaming is started.</p>
  
  <p>Any help is appreciated.</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-03-03T08:57:56.383Z
Comments.Text: Rather than a intermediary frame being missing, it's about head frames. When ffplay starts, if the first frame it receives is not a keyframe, then that frame needs a previous (missing) frame to get decoded. That previous frame may be a I-frame not queued to be ever shown or a P frame referred to by a B-frame and would have been shown after the B-frame had it been received.
Comments.UserDisplayName: Gyan
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-03-03T09:04:52.357Z
Comments.Text: I'm confused. Does it mean there is no solution to this problem (it's the way rdp works, the listener has to be started and then the flow started) or there would be a way to regularly insert  a keyframe for the listener to be able to catch the stream in the middle.
Comments.UserDisplayName: Luke Skywalker
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-03-03T09:08:57.920Z
Comments.Text: Does the stream not become smooth after a few seconds?
Comments.UserDisplayName: Gyan
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-03-03T09:09:54.287Z
Comments.Text: No, ffplay keeps outputing error message for ever. It is never able to display any frame.
Comments.UserDisplayName: Luke Skywalker
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-03-03T09:35:49.060Z
Comments.Text: Does replacing `-g 30` with `-force_key_frames "expr:gte(t,n_forced)"` work?
Comments.UserDisplayName: Gyan
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-03-03T10:00:39.363Z
Comments.Text: Indeed, thanks to your suggestion I found another interesting question about this: http://superuser.com/questions/908280/what-is-the-correct-way-to-fix-keyframes-in-ffmpeg-for-dash. So I'll try `force_key_frames` and also check the result with the forensic method mentioned in the question and let you know.
Comments.UserDisplayName: Luke Skywalker
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-03-03T18:00:26.113Z
Comments.Text: I tried your solution to no avail. ffplay is still not able to display anything if not started first. If I dump the output into a file, I see that intermediate frames are generated:`./ffmpeg -i /dev/video0 -c:v h264_omx -force_key_frames 'expr:gte(t,n_forced)' -o file.avi`
  and then `./ffprobe -i file.avi -select_streams v -show_frames -of csv -show_entries frame=pict_type`. O see some `frame,I` in the middle of the `frame,P`. So why would ffplay not able to decode those frame? Is this a limitation of the rdp protocol somehow?
Comments.UserDisplayName: Luke Skywalker
Tags: <ffmpeg><streaming>
Title: h264_omx and non-existing PPS 0 referenced
UserDisplayName: Luke Skywalker
ThreadText: # h264_omx and non-existing PPS 0 referenced
  
  (Asked by: username_0 on 2017-03-03)
  
  Trying to stream a webcam through rtp with a raspberry pi.
  
  Cating the SDP file on my desktop, and launching ffplay:
  
  ```
  ffplay sdp.out
  
  ```
  
  then streaming out this way:
  
  ```
  ffmpeg -i /dev/video0 -c:v h264_omx -f rtp rtp://192.168.0.101:1234
  
  ```
  
  works perfectly. But If I launch ffmpeg first THEN ffplay, ffplay stutters some error message about missing PPS and never displays any frame:
  
  ```
  [h264 @ 0x7fd97c0008c0] decode_slice_header error
  [h264 @ 0x7fd97c0008c0] no frame!
  [h264 @ 0x7fd97c0008c0] non-existing PPS 0 referenced
  
  ```
  
  After looking for a while I understand some intermediary frame might be missing. However, using the `-g` option this way does not change anything:
  
  ```
  ffmpeg -i /dev/video0 -c:v h264_omx -g 30 -f rtp rtp://192.168.0.101:1234
  
  ```
  
  It is a problem for me because I am using the raspberry as a securiy camera. My desktop can be restarted from time to time and I need ffplay to be able to catch after the streaming is started.
  
  Any help is appreciated.
  
  **Tags:** ffmpeg, streaming
  
  > **Comments:**
  > **username_1**: Rather than a intermediary frame being missing, it's about head frames. When ffplay starts, if the first frame it receives is not a keyframe, then that frame needs a previous (missing) frame to get decoded. That previous frame may be a I-frame not queued to be ever shown or a P frame referred to by a B-frame and would have been shown after the B-frame had it been received.  
  > **username_0**: I'm confused. Does it mean there is no solution to this problem (it's the way rdp works, the listener has to be started and then the flow started) or there would be a way to regularly insert  a keyframe for the listener to be able to catch the stream in the middle.  
  > **username_1**: Does the stream not become smooth after a few seconds?  
  > **username_0**: No, ffplay keeps outputing error message for ever. It is never able to display any frame.  
  > **username_1**: Does replacing `-g 30` with `-force_key_frames "expr:gte(t,n_forced)"` work?  
  > **username_0**: Indeed, thanks to your suggestion I found another interesting question about this: http://superuser.com/questions/908280/what-is-the-correct-way-to-fix-keyframes-in-ffmpeg-for-dash. So I'll try `force_key_frames` and also check the result with the forensic method mentioned in the question and let you know.  
  > **username_0**: I tried your solution to no avail. ffplay is still not able to display anything if not started first. If I dump the output into a file, I see that intermediate frames are generated:`./ffmpeg -i /dev/video0 -c:v h264_omx -force_key_frames 'expr:gte(t,n_forced)' -o file.avi` and then `./ffprobe -i file.avi -select_streams v -show_frames -of csv -show_entries frame=pict_type`. O see some `frame,I` in the middle of the `frame,P`. So why would ffplay not able to decode those frame? Is this a limitation of the rdp protocol somehow?  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_0 on 2017-04-21. Score: 2 | Accepted answer
  
  It turns out it is impossible to live stream with the RPi hardware acceleration using ffmpeg.
  For ffmpeg to configure the hardware acceleration to issue SPS/PPS frames during the stream and not only at the beginning of it, it needs to use a [specific API of the raspberry firmware](https://github.com/raspberrypi/firmware/issues/242) which it does not at the moment.
  
  [Raspivid](http://frozen.ca/streaming-raw-h-264-from-a-raspberry-pi/) and [gstreamer](http://gstreamer-devel.966125.n4.nabble.com/omxh264enc-and-stream-format-string-avc-td4671543.html) does that though. So turn to thoses solution and forget about ffmpeg.
  
  > **Comments:**
  > **username_1**: If this is the issue, then you may be able to stream the omx encode to a pipe in mpegts format and then use a 2nd ffmpeg instance to pipe to rtp. The MPEG-TS muxer will insert xPS in-band.  
  
  ---
  
  ### Answer by username_2 on 2019-12-10. Score: 0
  
  Here my FFmpeg patch to enable inline header on OMX hardware encoder.
  
  libavcodec/omx.c
  OMX\_SetParameter(s->handle, OMX\_IndexParamBrcmVideoAVCInlineHeaderEnable, &vid\_param\_inline\_header);
  where vid\_param\_inline\_header is a OMX\_CONFIG\_PORTBOOLEANTYPE struct.
  
  <https://github.com/username_2/FFmpeg/commit/6e410823e63103342e8fc3407ff6698808d4d1ef>
  
  > **Comments:**
  > **username_1**: Consider submitting a patch to ffmpeg-devel but make it an option instead of hardcoded.  
  > **username_2**: Yes but there is a last problem:( only when I enable OMX InlineHeader the latency grow over the time.There is a strange time calculation problem inside the muxer. I get a lot of "application provided invalid, non monotonically increasing dts to muxer in stream xxxx/0" and it always freeze at 3335.  
  
  ---
  
  