Answers.Body: <p>The problem was about the overlay filter which is too slow. I've changed for a combination of <code>hstack</code> and <code>vstack</code> as you can see here :</p>
  
  <pre><code>ffmpeg \
  -hwaccel cuvid -c:v h264_cuvid -i bbb_sunflower_2160p_30fps_normal.mp4 \
  -hwaccel cuvid -c:v h264_cuvid -i bbb_sunflower_2160p_30fps_normal.mp4 \
  -hwaccel cuvid -c:v h264_cuvid -i bbb_sunflower_2160p_30fps_normal.mp4 \
  -hwaccel cuvid -c:v h264_cuvid -i bbb_sunflower_2160p_30fps_normal.mp4 \
  -filter_complex " \
  [0:v] scale_npp=1280:720, hwdownload, format=nv12 [upperleft]; \
  [1:v] scale_npp=1280:720, hwdownload, format=nv12 [upperright]; \
  [2:v] scale_npp=1280:720, hwdownload, format=nv12 [lowerleft]; \
  [3:v] scale_npp=1280:720, hwdownload, format=nv12 [lowerright]; \
  [upperleft][upperright] hstack [upperline]; \
  [lowerleft][lowerright] hstack [lowerline]; \
  [upperline][lowerline] vstack [mosaic]; \
  [mosaic] hwupload_cuda" \
  -c:v h264_nvenc -preset llhp -profile:v baseline -f matroska pipe:1 | ffplay -i -
  </code></pre>
  
  <p>It works very well and I even can build a 1080p mosaic using <code>lossless</code> preset and <code>high444p</code> profile.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-10-25T12:26:43.900Z
Answers.LastActivityDate: 2018-10-25T12:26:43.900Z
Answers.UserDisplayName: c.censier
Body: <p>I'm using a NVIDIA Quadro P4000 and my config is Intel i7 2.4 GHz with 8 Go of RAM. I'm working on Ubuntu 16.04, I've installed NVIDIA drivers using apt repo aswell as CUDA. I'm using FFmpeg in version 3.3.</p>
  
  <p>I've managed to display a mosaic of 4 x 4K videos (same videos for test purpose) with FFmpeg, using hardware acceleration (<a href="https://video.stackexchange.com/questions/25061/ffmpeg-mosaic-with-hardware-acceleration-on-quadro-p4000">previous post</a>), using this command line :</p>
  
  <pre><code>ffmpeg \
  -hwaccel cuvid -c:v h264_cuvid -i bbb_sunflower_2160p_30fps_normal.mp4 \
  -hwaccel cuvid -c:v h264_cuvid -i bbb_sunflower_2160p_30fps_normal.mp4 \
  -hwaccel cuvid -c:v h264_cuvid -i bbb_sunflower_2160p_30fps_normal.mp4 \
  -hwaccel cuvid -c:v h264_cuvid -i bbb_sunflower_2160p_30fps_normal.mp4 \
  -filter_complex "nullsrc=size=2560x1440 [base]; \
  [0:v] scale_npp=1280:720, hwdownload, format=nv12 [upperleft]; \
  [1:v] scale_npp=1280:720, hwdownload, format=nv12 [upperright]; \
  [2:v] scale_npp=1280:720, hwdownload, format=nv12 [lowerleft]; \
  [3:v] scale_npp=1280:720, hwdownload, format=nv12 [lowerright]; \
  [base][upperleft] overlay=shortest=1:eval=init [tmp1]; \
  [tmp1][upperright] overlay=shortest=1:eval=init:x=1280 [tmp2]; \
  [tmp2][lowerleft] overlay=shortest=1:eval=init:y=720 [tmp3]; \
  [tmp3][lowerright] overlay=shortest=1:eval=init:x=1280:y=720, hwupload_cuda" \
  -c:v h264_nvenc -preset llhp -profile:v baseline -f matroska pipe:1 | ffplay -i -
  </code></pre>
  
  <p>Now my issue is about performance, videos are played at 80% of their original speed, so close to my goal...</p>
  
  <p>As I understand from now, this manipulation has several steps :</p>
  
  <ol>
  <li>The four videos are decoded by the GPU.</li>
  <li>Scaling to 720p is also performed by the GPU.</li>
  <li>The four flux are downloaded in the CPU and formated in NV12.</li>
  <li>Mosaic is made by the CPU using overlay filter.</li>
  <li>Mosaic video flux is uploaded in the GPU.</li>
  <li>The GPU encodes it in H.264.</li>
  </ol>
  
  <p>After this I'm not sure if the encoded video is retrieved by the CPU and then piped to ffplay or if it's stays in the GPU and directly displayed on the monitor.</p>
  
  <p>Moreover, I'd like ffplay to use hardware acceleration but I didn't manage to force it to use <code>h264_cuvid</code>.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-10-24T10:56:48.020Z
Comments.Text: ffplay s/w h264 decoder gives me 300 fps for a 1440p stream on a Ivy Bridge i5 so that's not the bottleneck. Check the presets for nvenc.
Comments.UserDisplayName: Gyan
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-10-24T12:07:53.207Z
Comments.Text: I've tried all presets and all profiles but I still got speed around 0.8, sometime worse
Comments.UserDisplayName: c.censier
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-10-24T12:11:56.893Z
Comments.Text: What's the fps speed if you replace `-c:v h264_nvenc -preset llhp -profile:v baseline -f matroska pipe:1` with `-an -f null -`?
Comments.UserDisplayName: Gyan
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-10-24T12:18:30.673Z
Comments.Text: I get a speed around 0.9
Comments.UserDisplayName: c.censier
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-10-24T12:31:13.617Z
Comments.Text: ok, what about `ffmpeg -hwaccel cuvid -c:v h264_cuvid -i bbb_sunflower_2160p_30fps_normal.mp4 -hwaccel cuvid -c:v h264_cuvid -i bbb_sunflower_2160p_30fps_normal.mp4 -hwaccel cuvid -c:v h264_cuvid -i bbb_sunflower_2160p_30fps_normal.mp4 -hwaccel cuvid -c:v h264_cuvid -i bbb_sunflower_2160p_30fps_normal.mp4 -map 0:v -map 1:v -map 2:v -map 3:v -f null -` ?
Comments.UserDisplayName: Gyan
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-10-24T12:42:04.743Z
Comments.Text: @Gyan I get a speed around 1.3 so as Paul B. Mahol said, the overlay filter slows the thing down. Is there another way of building a mosaic, can the gpu directly doing it ?
Comments.UserDisplayName: c.censier
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-10-24T12:43:54.230Z
Comments.Text: hstack the top videos and bottom videos and then vstack the result
Comments.UserDisplayName: Gyan
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-10-24T13:18:08.087Z
Comments.Text: Aaaaannnnnd it's working, even in 1080p ! Thank you again Gyan, you're a real life saver.
Comments.UserDisplayName: c.censier
Tags: <ffmpeg><nvidia>
Title: Improve performance on 4 x 4K mosaic with FFmpeg
UserDisplayName: c.censier
ThreadText: # Improve performance on 4 x 4K mosaic with FFmpeg
  
  (Asked by: username_0 on 2018-10-24)
  
  I'm using a NVIDIA Quadro P4000 and my config is Intel i7 2.4 GHz with 8 Go of RAM. I'm working on Ubuntu 16.04, I've installed NVIDIA drivers using apt repo aswell as CUDA. I'm using FFmpeg in version 3.3.
  
  I've managed to display a mosaic of 4 x 4K videos (same videos for test purpose) with FFmpeg, using hardware acceleration ([previous post](https://video.stackexchange.com/questions/25061/ffmpeg-mosaic-with-hardware-acceleration-on-quadro-p4000)), using this command line :
  
  ```
  ffmpeg \
  -hwaccel cuvid -c:v h264_cuvid -i bbb_sunflower_2160p_30fps_normal.mp4 \
  -hwaccel cuvid -c:v h264_cuvid -i bbb_sunflower_2160p_30fps_normal.mp4 \
  -hwaccel cuvid -c:v h264_cuvid -i bbb_sunflower_2160p_30fps_normal.mp4 \
  -hwaccel cuvid -c:v h264_cuvid -i bbb_sunflower_2160p_30fps_normal.mp4 \
  -filter_complex "nullsrc=size=2560x1440 [base]; \
  [0:v] scale_npp=1280:720, hwdownload, format=nv12 [upperleft]; \
  [1:v] scale_npp=1280:720, hwdownload, format=nv12 [upperright]; \
  [2:v] scale_npp=1280:720, hwdownload, format=nv12 [lowerleft]; \
  [3:v] scale_npp=1280:720, hwdownload, format=nv12 [lowerright]; \
  [base][upperleft] overlay=shortest=1:eval=init [tmp1]; \
  [tmp1][upperright] overlay=shortest=1:eval=init:x=1280 [tmp2]; \
  [tmp2][lowerleft] overlay=shortest=1:eval=init:y=720 [tmp3]; \
  [tmp3][lowerright] overlay=shortest=1:eval=init:x=1280:y=720, hwupload_cuda" \
  -c:v h264_nvenc -preset llhp -profile:v baseline -f matroska pipe:1 | ffplay -i -
  
  ```
  
  Now my issue is about performance, videos are played at 80% of their original speed, so close to my goal...
  
  As I understand from now, this manipulation has several steps :
  
  1. The four videos are decoded by the GPU.
  2. Scaling to 720p is also performed by the GPU.
  3. The four flux are downloaded in the CPU and formated in NV12.
  4. Mosaic is made by the CPU using overlay filter.
  5. Mosaic video flux is uploaded in the GPU.
  6. The GPU encodes it in H.264.
  
  After this I'm not sure if the encoded video is retrieved by the CPU and then piped to ffplay or if it's stays in the GPU and directly displayed on the monitor.
  
  Moreover, I'd like ffplay to use hardware acceleration but I didn't manage to force it to use `h264_cuvid`.
  
  **Tags:** ffmpeg, nvidia
  
  > **Comments:**
  > **username_1**: ffplay s/w h264 decoder gives me 300 fps for a 1440p stream on a Ivy Bridge i5 so that's not the bottleneck. Check the presets for nvenc.  
  > **username_0**: I've tried all presets and all profiles but I still got speed around 0.8, sometime worse  
  > **username_1**: What's the fps speed if you replace `-c:v h264_nvenc -preset llhp -profile:v baseline -f matroska pipe:1` with `-an -f null -`?  
  > **username_0**: I get a speed around 0.9  
  > **username_1**: ok, what about `ffmpeg -hwaccel cuvid -c:v h264_cuvid -i bbb_sunflower_2160p_30fps_normal.mp4 -hwaccel cuvid -c:v h264_cuvid -i bbb_sunflower_2160p_30fps_normal.mp4 -hwaccel cuvid -c:v h264_cuvid -i bbb_sunflower_2160p_30fps_normal.mp4 -hwaccel cuvid -c:v h264_cuvid -i bbb_sunflower_2160p_30fps_normal.mp4 -map 0:v -map 1:v -map 2:v -map 3:v -f null -` ?  
  > **username_0**: @username_1 I get a speed around 1.3 so as Paul B. Mahol said, the overlay filter slows the thing down. Is there another way of building a mosaic, can the gpu directly doing it ?  
  > **username_1**: hstack the top videos and bottom videos and then vstack the result  
  > **username_0**: Aaaaannnnnd it's working, even in 1080p ! Thank you again username_1, you're a real life saver.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2018-10-25. Score: 2 | Accepted answer
  
  The problem was about the overlay filter which is too slow. I've changed for a combination of `hstack` and `vstack` as you can see here :
  
  ```
  ffmpeg \
  -hwaccel cuvid -c:v h264_cuvid -i bbb_sunflower_2160p_30fps_normal.mp4 \
  -hwaccel cuvid -c:v h264_cuvid -i bbb_sunflower_2160p_30fps_normal.mp4 \
  -hwaccel cuvid -c:v h264_cuvid -i bbb_sunflower_2160p_30fps_normal.mp4 \
  -hwaccel cuvid -c:v h264_cuvid -i bbb_sunflower_2160p_30fps_normal.mp4 \
  -filter_complex " \
  [0:v] scale_npp=1280:720, hwdownload, format=nv12 [upperleft]; \
  [1:v] scale_npp=1280:720, hwdownload, format=nv12 [upperright]; \
  [2:v] scale_npp=1280:720, hwdownload, format=nv12 [lowerleft]; \
  [3:v] scale_npp=1280:720, hwdownload, format=nv12 [lowerright]; \
  [upperleft][upperright] hstack [upperline]; \
  [lowerleft][lowerright] hstack [lowerline]; \
  [upperline][lowerline] vstack [mosaic]; \
  [mosaic] hwupload_cuda" \
  -c:v h264_nvenc -preset llhp -profile:v baseline -f matroska pipe:1 | ffplay -i -
  
  ```
  
  It works very well and I even can build a 1080p mosaic using `lossless` preset and `high444p` profile.
  
  
  ---
  
  