Answers.Body: <p>So, after some (long) time - it is a problem with the old version of <code>ffmpeg</code> - which is the version bundled with Ubuntu 22.04.</p>
  <p>Using a self-compiled version of <code>ffmpeg</code> version 6.0 with CUDA Extensions. I can run the following command line, and everything is working perfectly.</p>
  <pre class="lang-none prettyprint-override"><code>/home/richja/bin/ffmpeg -y -report -hide_banner -loglevel verbose 
  -hwaccel cuda -hwaccel_output_format cuda 
  -i movie.MP4 
  -f rawvideo -framerate 10.0 -s 1920x1080 -pix_fmt rgba -i - 
  -filter_complex [0:v]scale_cuda=format=yuv420p[mp4_stream];[1:v]format=yuva420p,hwupload[overlay_stream];[mp4_stream][overlay_stream]overlay_cuda 
  -vcodec h264_nvenc 
  -rc:v cbr -b:v 20M -bf:v 3 -profile:v main -spatial-aq true -movflags faststart overlay.MP4
  </code></pre>
  <p>The performance of this setup is frankly incredible, and can render overlays at more than 12x real-time.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-05-17T14:54:35.077Z
Answers.LastActivityDate: 2023-05-17T14:54:35.077Z
Answers.UserDisplayName: time4tea
Body: <p>Background: I'm trying to overlay a full frame image on top of an MP4, in ffmpeg, using as much hardware-assistance as possible. The inputs to ffmpeg are:</p>
  <ul>
  <li>a stream from an MP4 file - with 1 audio and 1 video stream</li>
  <li>frames from some software, which are supplied as raw RGBA frames, at the same frame size as the movie.</li>
  </ul>
  <p>I struggled for some time to get <em>something</em> working and through various iterations, ended up with this:</p>
  <pre class="lang-none prettyprint-override"><code>ffmpeg -y -report -hide_banner -loglevel verbose 
  -hwaccel cuda -hwaccel_output_format yuv420p 
  -i movie.mp4 
  -f rawvideo -framerate 10.0 -s 1024x576 -pix_fmt rgba -i - 
  -filter_complex [0:v]hwupload[mp4_stream];[1:v]format=yuva420p,hwupload[overlay_stream];[mp4_stream][overlay_stream]overlay_cuda[out] 
  -map [out] -vcodec h264_nvenc 
  -rc:v cbr -b:v 20M -bf:v 3 -profile:v main -spatial-aq true -movflags faststart output.mp4
  </code></pre>
  <p>I picked <code>yuv420p</code> because looking at the overlay code ( <a href="https://github.com/FFmpeg/FFmpeg/blob/4976b102d89787d59a3aaa438b76c62aec86ad5a/libavfilter/vf_overlay_cuda.c#L94" rel="nofollow noreferrer">https://github.com/FFmpeg/FFmpeg/blob/4976b102d89787d59a3aaa438b76c62aec86ad5a/libavfilter/vf_overlay_cuda.c#L94</a> ) - you can only overlay an image with alpha on top of an image of this type.</p>
  <p>However, this results in a very green image! - although the overlay (the date, in this case) seems to have been overlaid with the right alpha etc.</p>
  <p><a href="https://i.sstatic.net/P3DiQ.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/P3DiQ.jpg" alt="a green image" /></a></p>
  <p>I was wondering if the blend mode or some other aspect of the overlay was causing a problem, so attempted to reduce the problem size. I thought that perhaps just processing the video -  movie-&gt;decode-&gt;encode - might shed some light, so I tried this:</p>
  <pre class="lang-none prettyprint-override"><code>ffmpeg -y -report -hide_banner -loglevel verbose 
  -hwaccel cuda -hwaccel_output_format yuv420p -i movie.mp4 
  -f rawvideo -framerate 10.0 -s 1024x576 -pix_fmt rgba -i - 
  -filter_complex [0:v]hwupload[mp4_stream] 
  -map [mp4_stream] -vcodec h264_nvenc 
  -rc:v cbr -b:v 20M -bf:v 3 -profile:v main -spatial-aq true -movflags faststart output.mp4
  </code></pre>
  <p>This is essentially the same (I hope) but just ignoring the overlay frames - still green.</p>
  <p><em>However</em>, replacing the <code>yuv420p</code> with <code>nv12</code>: <code>-hwaccel cuda -hwaccel_output_format nv12 </code> results in a correctly coloured image. This is no use for my actual use case, as nv12 does not support alpha overlays.</p>
  <p><a href="https://i.sstatic.net/1x2Ca.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/1x2Ca.png" alt="a correctly coloured image" /></a></p>
  <p>Versions:</p>
  <p><code>ffmpeg version 4.4.2-0ubuntu0.22.04.1</code></p>
  <p><code>Hardware: NVIDIA Corporation GA106 [GeForce RTX 3060 Lite Hash Rate] (rev a1)</code></p>
  <p><code>Kernel: 5.15.0-71-generic #78-Ubuntu SMP</code></p>
  <p><code>Driver: 525.105.17</code></p>
  <p>How to get the output the correct colour, while still using yuv420p, so I can overlay properly on top of it.</p>
  <p>I have an inkling that this is due to some planes of the YUV file being zero? But how to fix this in the command line <code>ffmpeg</code> - I'm not sure. Thanks!</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-05-17T10:37:16.463Z
Comments.Text: I am wondering if it is the version of ffmpeg - I can see that scale_cuda was modified to add the option "format" in what appears to be 8.0.103 -(https://github.com/FFmpeg/FFmpeg/commit/62dc5df941f5e196164c151691e4274195523e95)  but my ffmpeg appears to be using `7.110.100` - is it just too old?
Comments.UserDisplayName: time4tea
Tags: <ffmpeg><nvidia>
Title: Using ffmpeg nvidia/cuda overlay processing results in green output - how to get the correct colours?
UserDisplayName: time4tea
ThreadText: # Using ffmpeg nvidia/cuda overlay processing results in green output - how to get the correct colours?
  
  (Asked by: username_0 on 2023-05-17)
  
  Background: I'm trying to overlay a full frame image on top of an MP4, in ffmpeg, using as much hardware-assistance as possible. The inputs to ffmpeg are:
  
  * a stream from an MP4 file - with 1 audio and 1 video stream
  * frames from some software, which are supplied as raw RGBA frames, at the same frame size as the movie.
  
  I struggled for some time to get *something* working and through various iterations, ended up with this:
  
  ```none
  ffmpeg -y -report -hide_banner -loglevel verbose 
  -hwaccel cuda -hwaccel_output_format yuv420p 
  -i movie.mp4 
  -f rawvideo -framerate 10.0 -s 1024x576 -pix_fmt rgba -i - 
  -filter_complex [0:v]hwupload[mp4_stream];[1:v]format=yuva420p,hwupload[overlay_stream];[mp4_stream][overlay_stream]overlay_cuda[out] 
  -map [out] -vcodec h264_nvenc 
  -rc:v cbr -b:v 20M -bf:v 3 -profile:v main -spatial-aq true -movflags faststart output.mp4
  
  ```
  
  I picked `yuv420p` because looking at the overlay code ( <https://github.com/FFmpeg/FFmpeg/blob/4976b102d89787d59a3aaa438b76c62aec86ad5a/libavfilter/vf_overlay_cuda.c#L94> ) - you can only overlay an image with alpha on top of an image of this type.
  
  However, this results in a very green image! - although the overlay (the date, in this case) seems to have been overlaid with the right alpha etc.
  
  [![a green image](https://i.sstatic.net/P3DiQ.jpg)](https://i.sstatic.net/P3DiQ.jpg)
  
  I was wondering if the blend mode or some other aspect of the overlay was causing a problem, so attempted to reduce the problem size. I thought that perhaps just processing the video - movie->decode->encode - might shed some light, so I tried this:
  
  ```none
  ffmpeg -y -report -hide_banner -loglevel verbose 
  -hwaccel cuda -hwaccel_output_format yuv420p -i movie.mp4 
  -f rawvideo -framerate 10.0 -s 1024x576 -pix_fmt rgba -i - 
  -filter_complex [0:v]hwupload[mp4_stream] 
  -map [mp4_stream] -vcodec h264_nvenc 
  -rc:v cbr -b:v 20M -bf:v 3 -profile:v main -spatial-aq true -movflags faststart output.mp4
  
  ```
  
  This is essentially the same (I hope) but just ignoring the overlay frames - still green.
  
  *However*, replacing the `yuv420p` with `nv12`: `-hwaccel cuda -hwaccel_output_format nv12`  results in a correctly coloured image. This is no use for my actual use case, as nv12 does not support alpha overlays.
  
  [![a correctly coloured image](https://i.sstatic.net/1x2Ca.png)](https://i.sstatic.net/1x2Ca.png)
  
  Versions:
  
  `ffmpeg version 4.4.2-0ubuntu0.22.04.1`
  
  `Hardware: NVIDIA Corporation GA106 [GeForce RTX 3060 Lite Hash Rate] (rev a1)`
  
  `Kernel: 5.15.0-71-generic #78-Ubuntu SMP`
  
  `Driver: 525.105.17`
  
  How to get the output the correct colour, while still using yuv420p, so I can overlay properly on top of it.
  
  I have an inkling that this is due to some planes of the YUV file being zero? But how to fix this in the command line `ffmpeg` - I'm not sure. Thanks!
  
  **Tags:** ffmpeg, nvidia
  
  > **Comments:**
  > **username_0**: I am wondering if it is the version of ffmpeg - I can see that scale_cuda was modified to add the option "format" in what appears to be 8.0.103 -(https://github.com/FFmpeg/FFmpeg/commit/62dc5df941f5e196164c151691e4274195523e95)  but my ffmpeg appears to be using `7.110.100` - is it just too old?  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2023-05-17. Score: 0
  
  So, after some (long) time - it is a problem with the old version of `ffmpeg` - which is the version bundled with Ubuntu 22.04.
  
  Using a self-compiled version of `ffmpeg` version 6.0 with CUDA Extensions. I can run the following command line, and everything is working perfectly.
  
  ```none
  /home/richja/bin/ffmpeg -y -report -hide_banner -loglevel verbose 
  -hwaccel cuda -hwaccel_output_format cuda 
  -i movie.MP4 
  -f rawvideo -framerate 10.0 -s 1920x1080 -pix_fmt rgba -i - 
  -filter_complex [0:v]scale_cuda=format=yuv420p[mp4_stream];[1:v]format=yuva420p,hwupload[overlay_stream];[mp4_stream][overlay_stream]overlay_cuda 
  -vcodec h264_nvenc 
  -rc:v cbr -b:v 20M -bf:v 3 -profile:v main -spatial-aq true -movflags faststart overlay.MP4
  
  ```
  
  The performance of this setup is frankly incredible, and can render overlays at more than 12x real-time.
  
  
  ---
  
  