Adding blurred background to portrait videos with ffmpeg I have a lot of videos that are either 720x1280 or 1280x720. I want to convert all of them to 1280x720, but for the ones that are portrait I would like to show the full video with the remaining space to the left and right being a darkened, blurred version of the video.

I think this is pretty close to that, but it seems to go on forever (maybe because the color keeps providing frames?)

ffmpeg -i video.mp4 -lavfi "\
    color=color=black@.3:size=720x405[dark];
    [0:v]crop=720:405[blurbase];
    [blurbase]boxblur=lr='min(h,w)/20':lp=1:cr='min(cw,ch)/20':cp=1[blurred];
    [blurred][dark]overlay[darkened];
    [darkened]scale=1280:720[bg];
    [0:v]scale=-1:720[fg];
    [bg][fg]overlay=(W-w)/2:(H-h)/2" \
    -vb 800K video_out.mp4


You can see that it just starts adding duplicate frames after it hits the end of the video (in the logging below the video is ~24 sec):

Stream mapping:
  Stream #0:0 (h264) -&gt; crop (graph 0)
  Stream #0:0 (h264) -&gt; scale (graph 0)
  overlay (graph 0) -&gt; Stream #0:0 (libx264)
  Stream #0:1 -&gt; #0:1 (aac (native) -&gt; aac (native))
...
Past duration 0.999992 too large    2256kB time=00:00:24.33 bitrate= 759.4kbits/s dup=1 drop=0 speed=1.36x
Past duration 0.799995 too large
frame= 3177 fps=101 q=-1.0 Lsize=    3797kB time=00:01:45.80 bitrate= 294.0kbits/s dup=407 drop=0 speed=3.37x


How do I fix the issue of the encoding not completing when the end of the video file is reached?

Also, is there a way to conditionally only do the blurred background work if the video is portrait so landscape videos don't spend a lot of unnecessary CPU on blurring the video?
Limit the color output to a few frames. In the first overlay, it will repeat the last frame of the color once that stream ends. So,   color=black@.3:size=720x405:d=1[dark];



Filters can't be executed conditionally, so you will need to probe the input in a separate command beforehand and then execute a command with the filters or one without.

The command below

ffprobe in.mp4 -show_entries stream=display_aspect_ratio -of compact=p=0:nk=1 -v 0


will produce an output like this:

16:9


Just wrap a script to check that value.