Answers.Body: <h1>TL;DR</h1>
  <h2>Correct Approach</h2>
  <ol>
  <li><strong>Pick up the necessary scenes</strong> of videos with <code>ffmpeg</code> 's <strong>time related arguments</strong> <code>-ss</code> in-point, <code>-t</code> duration and <code>-to</code> out-point before the <code>-i</code> input argument and encode each such scene with your desired CRF and with VFR (see question above) as a <strong>standalone intermediary clip</strong>.</li>
  <li>Then <code>-f concat</code> your scenes (same dimension, same codec, variable framerate (inherently different!), different CRF) together and use <code>-movflags faststart</code> to be streaming compatible.</li>
  </ol>
  <ul>
  <li>Full ffmpeg command lines for steps 1+2 in detail section below.</li>
  </ul>
  <h2>Playback of video file</h2>
  <ul>
  <li><p>From ffmpeg 5.1.2:** on macOS 11 Big Sur</p>
  </li>
  <li><p>✅ <strong>QuickLook</strong>, <strong>QuickTime Player</strong>  10.5, <strong>mpv</strong> 0.35.0 — Play back fine.</p>
  </li>
  <li><p>❌ <strong>VLC</strong>: The variant with intro and main part both having VFR does not play back fine! Audio remains in sync throughout the video. But during the intro the portrait photo slide in gets stuck along the way and then from that position slides out later, and then when the main part starts, the video stream recovers and resumes correctly.</p>
  </li>
  <li><p> <strong>Maximum compatibility solution</strong>: Intro part with fixed frame rate and start with variable frame rates only in the main part.</p>
  </li>
  </ul>
  <h3>To visually debug/inspect your resulting video files</h3>
  <p>ℹ️ <a href="https://github.com/mpv-player/mpv/issues/3428#issuecomment-240394048" rel="nofollow noreferrer">mpv can display frame numbers in its On Screen Display</a></p>
  <ul>
  <li>To your <code>~/.config/mpv/mpv.conf</code> add the following line: <code>osd-status-msg=${playback-time/full} / ${duration} (${percent-pos}%)\nframe: ${estimated-frame-number} / ${estimated-frame-count}</code></li>
  <li>Play back your video file and press <a href="https://mpv.io/manual/stable/#keyboard-control" rel="nofollow noreferrer">hotkey</a> <code>⇧-O</code> to toggle OSD on/off.</li>
  </ul>
  <h2>Wrong Approach</h2>
  <ol>
  <li>Encode full video with VFR once as CR18, once as CR28.</li>
  <li>Concat intro with inpoint/outpoint from CR18, followed by main part as CR28 with inpoint being outro's outpoint.</li>
  </ol>
  <p>❌ <strong>Fail</strong>: Concat's inpoint/outpoint/duration functions seem to not work correctly with VFR sources. Result:</p>
  <ul>
  <li>Video track: Intro played twice. That by itself is already inacceptable.</li>
  <li>Worse: Audio plays continously, so after the outro is out of sync.</li>
  </ul>
  <h1>Details</h1>
  <h2>Correct Approach</h2>
  <h4>Creating the intro</h4>
  <pre class="lang-sh prettyprint-override"><code>ffmpeg \
  -to 00:06.300 \ # The intro lasts til 6secs and .300 fractions
  -i VID.m4v \
  \
  -movflags faststart \
  -c:v libx264 -preset veryslow -crf 18 \
  -vf mpdecimate -vsync vfr \
  \
  VID-intro-CR18-vfr.mp4
  </code></pre>
  <h4>Creating the main part</h4>
  <pre class="lang-sh prettyprint-override"><code>ffmpeg \
  -ss 00:06.300 \  # The main part starts when the intro ends.
  -t 30s \         # Duration limited to -t 30s (seconds) during testing. Omit for real run.
  -i VID.m4v \
  \
  -movflags faststart \
  -c:v libx264 -preset veryslow -crf 28 \
  -vf mpdecimate -vsync vfr \
  \
  VID-main-from-intro-endpoint-for-30secs-CR28-vfr.mp4
  </code></pre>
  <h3>Creating concatenation instructions</h3>
  <p>assembly.txt</p>
  <pre><code>file VID-intro-CR18-vfr.mp4
  file VID-main-from-intro-endpoint-CR28-vfr.mp4
  </code></pre>
  <h3>Running concatenation</h3>
  <pre class="lang-sh prettyprint-override"><code>ffmpeg -f concat -i assembly.txt -c copy -movflags faststart \
  VID--intro-CR18-vfr--main-CR28-vfr.mp4
  </code></pre>
  <p>Resulting file plays back fine.</p>
  <p>Processing throws only two &quot;Non-monotonous DTS in output stream&quot; warning messages.
  That seems to be the timecode correction before/after the stitching point between intro and main part:</p>
  <pre><code>[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fd44a005580] Auto-inserting h264_mp4toannexb bitstream filter
  [mp4 @ 0x7fd44a007b00] Non-monotonous DTS in output stream 0:0; previous: 95589, current: 25445; changing to 95590. This may result in incorrect timestamps in the output file.
  [mp4 @ 0x7fd44a007b00] Non-monotonous DTS in output stream 0:1; previous: 278528, current: 277830; changing to 278529. This may result in incorrect timestamps in the output file.
  [mp4 @ 0x7fd44a007b00] Starting second pass: moving the moov atom to the beginning of the file
  frame=  467 fps=0.0 q=-1.0 Lsize=    1898kB time=00:00:36.32 bitrate= 428.0kbits/s speed=1.04e+03x
  video:1311kB audio:566kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 1.112216%
  </code></pre>
  <p>With a high probability your <a href="https://ffmpeg.org/ffmpeg-formats.html#Syntax:%7E:text=inpoint%20timestamp" rel="nofollow noreferrer">timecode will get some auto correction</a> at the stitching point(s), as the cuts most likely will not occur at a keyframes but at interframes. I guess the warning message just told us that.</p>
  <h2>Wrong Approach</h2>
  <h3>Concatenation instructions by copying out the scenes from full length VFR videos encoded at different CRFs</h3>
  <p>assembly.txt</p>
  <pre><code>file VID-CR18-vfr.mp4
  inpoint 00:00
  outpoint 00:06.300
  
  file VID-CR28-vfr.mp4
  inpoint 00:06.300
  </code></pre>
  <p>The resulting file has issues (see above).</p>
  <p>The warning message &quot;Non-monotonous DTS in output stream&quot; occurs ~450 times for a 49min video. Exceprt (with &quot;...&quot; omissions):</p>
  <pre><code>[mp4 @ 0x7fecd573b440] Non-monotonous DTS in output stream 0:1; previous: 441344, current: 0; changing to 441345. This may result in incorrect timestamps in the output file.
  [mp4 @ 0x7fecd573b440] Non-monotonous DTS in output stream 0:1; previous: 441345, current: 1024; changing to 441346. This may result in incorrect timestamps in the output file.
  [mp4 @ 0x7fecd573b440] Non-monotonous DTS in output stream 0:1; previous: 441346, current: 2048; changing to 441347. This may result in incorrect timestamps in the output file.
  [mp4 @ 0x7fecd573b440] Non-monotonous DTS in output stream 0:1; previous: 441347, current: 3072; changing to 441348. This may result in incorrect timestamps in the output file.
  
  ...
  
  [mp4 @ 0x7fecd573b440] Non-monotonous DTS in output stream 0:1; previous: 441364, current: 20480; changing to 441365. This may result in incorrect timestamps in the output file.
  [mp4 @ 0x7fecd573b440] Non-monotonous DTS in output stream 0:1; previous: 441365, current: 21504; changing to 441366. This may result in incorrect timestamps in the output file.
  [mp4 @ 0x7fecd573b440] Non-monotonous DTS in output stream 0:0; previous: 95077, current: -7835; changing to 95078. This may result in incorrect timestamps in the output file.
  [mp4 @ 0x7fecd573b440] Non-monotonous DTS in output stream 0:1; previous: 441366, current: 22528; changing to 441367. This may result in incorrect timestamps in the output file.
  [mp4 @ 0x7fecd573b440] Non-monotonous DTS in output stream 0:1; previous: 441367, current: 23552; changing to 441368. This may result in incorrect timestamps in the output file.
  
  ...
  
  [mp4 @ 0x7fecd573b440] Non-monotonous DTS in output stream 0:1; previous: 441385, current: 41984; changing to 441386. This may result in incorrect timestamps in the output file.
  [mp4 @ 0x7fecd573b440] Non-monotonous DTS in output stream 0:1; previous: 441386, current: 43008; changing to 441387. This may result in incorrect timestamps in the output file.
  [mp4 @ 0x7fecd573b440] Non-monotonous DTS in output stream 0:0; previous: 95078, current: -155; changing to 95079. This may result in incorrect timestamps in the output file.
  [mp4 @ 0x7fecd573b440] Non-monotonous DTS in output stream 0:1; previous: 441387, current: 44032; changing to 441388. This may result in incorrect timestamps in the output file.
  [mp4 @ 0x7fecd573b440] Non-monotonous DTS in output stream 0:1; previous: 441388, current: 45056; changing to 441389. This may result in incorrect timestamps in the output file.
  [mp4 @ 0x7fecd573b440] Non-monotonous DTS in output stream 0:0; previous: 95079, current: 357; changing to 95080. This may result in incorrect timestamps in the output file.
  [mp4 @ 0x7fecd573b440] Non-monotonous DTS in output stream 0:1; previous: 441389, current: 46080; changing to 441390. This may result in incorrect timestamps in the output file.
  
  ...
  
  [mp4 @ 0x7fecd573b440] Non-monotonous DTS in output stream 0:1; previous: 441773, current: 439296; changing to 441774. This may result in incorrect timestamps in the output file.
  [mp4 @ 0x7fecd573b440] Non-monotonous DTS in output stream 0:1; previous: 441774, current: 440320; changing to 441775. This may result in incorrect timestamps in the output file.
  [mp4 @ 0x7fecd573b440] Non-monotonous DTS in output stream 0:1; previous: 441775, current: 441344; changing to 441776. This may result in incorrect timestamps in the output file.
  frame=38540 fps=38539 q=-1.0 size=   56832kB time=00:42:38.23 bitrate= 182.0kbits/s speed=2.56e+03x  frame=44130 fps=38101 q=-1.0 Lsize=   68207kB time=00:49:27.67 bitrate= 188.3kbits/s speed=2.56e+03x    
  video:20486kB audio:45963kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 2.645229%
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-07-05T13:59:54.587Z
Answers.LastActivityDate: 2023-07-05T13:59:54.587Z
Answers.UserDisplayName: porg
Body: <h2>Source Material</h2>
  <ul>
  <li>Having many scenes which essentially are still frames (for many seconds)</li>
  <li>Commonly found in: Slideshows, Screen Recording or Software Mockup/Prototype</li>
  </ul>
  <p>✅ This kind of source material <a href="https://superuser.com/questions/908295/ffmpeg-libx264-how-to-specify-a-variable-frame-rate-but-with-a-maximum/1792778#1792778">wonderfully shrinks in file size without loosing any extra quality</a> simply by using <code>ffmpeg</code> with <code>-vf -mpdecimate -vsync vfr</code> which decimates duplicate (=still) frames and in consequence makes a variable frame rate video from it.</p>
  <h2>But what if I have a few scenes which need more/less CFR ?</h2>
  <ul>
  <li>The H.264 video throughout the majority of its runtime looks totally fine at CRF 28 (lowest recommended quality)</li>
  <li>But in the intro I slide in/out a portrait photo, which requires significantly more intra and inter frame compression as in the rest of the video (prototype of a flat design software, compresses really well).</li>
  </ul>
  <p> How do I encode my intro at CR28, and the rest at CR18, and still get a variable frame rate in my video?</p>
  
Tags: <ffmpeg><encoding><video-quality><framerate><h.264>
Title: How to encode scenes with different CRF into a H.264 MP4 video with a variable frame rate?
UserDisplayName: porg
ThreadText: # How to encode scenes with different CRF into a H.264 MP4 video with a variable frame rate?
  
  (Asked by: username_0 on 2023-07-05)
  
  ## Source Material
  
  * Having many scenes which essentially are still frames (for many seconds)
  * Commonly found in: Slideshows, Screen Recording or Software Mockup/Prototype
  
  ✅ This kind of source material [wonderfully shrinks in file size without loosing any extra quality](https://superuser.com/questions/908295/ffmpeg-libx264-how-to-specify-a-variable-frame-rate-but-with-a-maximum/1792778#1792778) simply by using `ffmpeg` with `-vf -mpdecimate -vsync vfr` which decimates duplicate (=still) frames and in consequence makes a variable frame rate video from it.
  
  ## But what if I have a few scenes which need more/less CFR ?
  
  * The H.264 video throughout the majority of its runtime looks totally fine at CRF 28 (lowest recommended quality)
  * But in the intro I slide in/out a portrait photo, which requires significantly more intra and inter frame compression as in the rest of the video (prototype of a flat design software, compresses really well).
  
  How do I encode my intro at CR28, and the rest at CR18, and still get a variable frame rate in my video?
  
  **Tags:** ffmpeg, encoding, video-quality, framerate, h.264
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2023-07-05. Score: 0
  
  # TL;DR
  
  ## Correct Approach
  
  1. **Pick up the necessary scenes** of videos with `ffmpeg` 's **time related arguments** `-ss` in-point, `-t` duration and `-to` out-point before the `-i` input argument and encode each such scene with your desired CRF and with VFR (see question above) as a **standalone intermediary clip**.
  2. Then `-f concat` your scenes (same dimension, same codec, variable framerate (inherently different!), different CRF) together and use `-movflags faststart` to be streaming compatible.
  
  * Full ffmpeg command lines for steps 1+2 in detail section below.
  
  ## Playback of video file
  
  * From ffmpeg 5.1.2:\*\* on macOS 11 Big Sur
  * ✅ **QuickLook**, **QuickTime Player** 10.5, **mpv** 0.35.0 — Play back fine.
  * ❌ **VLC**: The variant with intro and main part both having VFR does not play back fine! Audio remains in sync throughout the video. But during the intro the portrait photo slide in gets stuck along the way and then from that position slides out later, and then when the main part starts, the video stream recovers and resumes correctly.
  * **Maximum compatibility solution**: Intro part with fixed frame rate and start with variable frame rates only in the main part.
  
  ### To visually debug/inspect your resulting video files
  
  ℹ️ [mpv can display frame numbers in its On Screen Display](https://github.com/mpv-player/mpv/issues/3428#issuecomment-240394048)
  
  * To your `~/.config/mpv/mpv.conf` add the following line: `osd-status-msg=${playback-time/full} / ${duration} (${percent-pos}%)\nframe: ${estimated-frame-number} / ${estimated-frame-count}`
  * Play back your video file and press [hotkey](https://mpv.io/manual/stable/#keyboard-control) `⇧-O` to toggle OSD on/off.
  
  ## Wrong Approach
  
  1. Encode full video with VFR once as CR18, once as CR28.
  2. Concat intro with inpoint/outpoint from CR18, followed by main part as CR28 with inpoint being outro's outpoint.
  
  ❌ **Fail**: Concat's inpoint/outpoint/duration functions seem to not work correctly with VFR sources. Result:
  
  * Video track: Intro played twice. That by itself is already inacceptable.
  * Worse: Audio plays continously, so after the outro is out of sync.
  
  # Details
  
  ## Correct Approach
  
  #### Creating the intro
  
  ```sh
  ffmpeg \
  -to 00:06.300 \ # The intro lasts til 6secs and .300 fractions
  -i VID.m4v \
  \
  -movflags faststart \
  -c:v libx264 -preset veryslow -crf 18 \
  -vf mpdecimate -vsync vfr \
  \
  VID-intro-CR18-vfr.mp4
  
  ```
  
  #### Creating the main part
  
  ```sh
  ffmpeg \
  -ss 00:06.300 \  # The main part starts when the intro ends.
  -t 30s \         # Duration limited to -t 30s (seconds) during testing. Omit for real run.
  -i VID.m4v \
  \
  -movflags faststart \
  -c:v libx264 -preset veryslow -crf 28 \
  -vf mpdecimate -vsync vfr \
  \
  VID-main-from-intro-endpoint-for-30secs-CR28-vfr.mp4
  
  ```
  
  ### Creating concatenation instructions
  
  assembly.txt
  
  ```
  file VID-intro-CR18-vfr.mp4
  file VID-main-from-intro-endpoint-CR28-vfr.mp4
  
  ```
  
  ### Running concatenation
  
  ```sh
  ffmpeg -f concat -i assembly.txt -c copy -movflags faststart \
  VID--intro-CR18-vfr--main-CR28-vfr.mp4
  
  ```
  
  Resulting file plays back fine.
  
  Processing throws only two "Non-monotonous DTS in output stream" warning messages.
  That seems to be the timecode correction before/after the stitching point between intro and main part:
  
  ```
  [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fd44a005580] Auto-inserting h264_mp4toannexb bitstream filter
  [mp4 @ 0x7fd44a007b00] Non-monotonous DTS in output stream 0:0; previous: 95589, current: 25445; changing to 95590. This may result in incorrect timestamps in the output file.
  [mp4 @ 0x7fd44a007b00] Non-monotonous DTS in output stream 0:1; previous: 278528, current: 277830; changing to 278529. This may result in incorrect timestamps in the output file.
  [mp4 @ 0x7fd44a007b00] Starting second pass: moving the moov atom to the beginning of the file
  frame=  467 fps=0.0 q=-1.0 Lsize=    1898kB time=00:00:36.32 bitrate= 428.0kbits/s speed=1.04e+03x
  video:1311kB audio:566kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 1.112216%
  
  ```
  
  With a high probability your [timecode will get some auto correction](https://ffmpeg.org/ffmpeg-formats.html#Syntax:%7E:text=inpoint%20timestamp) at the stitching point(s), as the cuts most likely will not occur at a keyframes but at interframes. I guess the warning message just told us that.
  
  ## Wrong Approach
  
  ### Concatenation instructions by copying out the scenes from full length VFR videos encoded at different CRFs
  
  assembly.txt
  
  ```
  file VID-CR18-vfr.mp4
  inpoint 00:00
  outpoint 00:06.300
  
  file VID-CR28-vfr.mp4
  inpoint 00:06.300
  
  ```
  
  The resulting file has issues (see above).
  
  The warning message "Non-monotonous DTS in output stream" occurs ~450 times for a 49min video. Exceprt (with "..." omissions):
  
  ```
  [mp4 @ 0x7fecd573b440] Non-monotonous DTS in output stream 0:1; previous: 441344, current: 0; changing to 441345. This may result in incorrect timestamps in the output file.
  [mp4 @ 0x7fecd573b440] Non-monotonous DTS in output stream 0:1; previous: 441345, current: 1024; changing to 441346. This may result in incorrect timestamps in the output file.
  [mp4 @ 0x7fecd573b440] Non-monotonous DTS in output stream 0:1; previous: 441346, current: 2048; changing to 441347. This may result in incorrect timestamps in the output file.
  [mp4 @ 0x7fecd573b440] Non-monotonous DTS in output stream 0:1; previous: 441347, current: 3072; changing to 441348. This may result in incorrect timestamps in the output file.
  
  ...
  
  [mp4 @ 0x7fecd573b440] Non-monotonous DTS in output stream 0:1; previous: 441364, current: 20480; changing to 441365. This may result in incorrect timestamps in the output file.
  [mp4 @ 0x7fecd573b440] Non-monotonous DTS in output stream 0:1; previous: 441365, current: 21504; changing to 441366. This may result in incorrect timestamps in the output file.
  [mp4 @ 0x7fecd573b440] Non-monotonous DTS in output stream 0:0; previous: 95077, current: -7835; changing to 95078. This may result in incorrect timestamps in the output file.
  [mp4 @ 0x7fecd573b440] Non-monotonous DTS in output stream 0:1; previous: 441366, current: 22528; changing to 441367. This may result in incorrect timestamps in the output file.
  [mp4 @ 0x7fecd573b440] Non-monotonous DTS in output stream 0:1; previous: 441367, current: 23552; changing to 441368. This may result in incorrect timestamps in the output file.
  
  ...
  
  [mp4 @ 0x7fecd573b440] Non-monotonous DTS in output stream 0:1; previous: 441385, current: 41984; changing to 441386. This may result in incorrect timestamps in the output file.
  [mp4 @ 0x7fecd573b440] Non-monotonous DTS in output stream 0:1; previous: 441386, current: 43008; changing to 441387. This may result in incorrect timestamps in the output file.
  [mp4 @ 0x7fecd573b440] Non-monotonous DTS in output stream 0:0; previous: 95078, current: -155; changing to 95079. This may result in incorrect timestamps in the output file.
  [mp4 @ 0x7fecd573b440] Non-monotonous DTS in output stream 0:1; previous: 441387, current: 44032; changing to 441388. This may result in incorrect timestamps in the output file.
  [mp4 @ 0x7fecd573b440] Non-monotonous DTS in output stream 0:1; previous: 441388, current: 45056; changing to 441389. This may result in incorrect timestamps in the output file.
  [mp4 @ 0x7fecd573b440] Non-monotonous DTS in output stream 0:0; previous: 95079, current: 357; changing to 95080. This may result in incorrect timestamps in the output file.
  [mp4 @ 0x7fecd573b440] Non-monotonous DTS in output stream 0:1; previous: 441389, current: 46080; changing to 441390. This may result in incorrect timestamps in the output file.
  
  ...
  
  [mp4 @ 0x7fecd573b440] Non-monotonous DTS in output stream 0:1; previous: 441773, current: 439296; changing to 441774. This may result in incorrect timestamps in the output file.
  [mp4 @ 0x7fecd573b440] Non-monotonous DTS in output stream 0:1; previous: 441774, current: 440320; changing to 441775. This may result in incorrect timestamps in the output file.
  [mp4 @ 0x7fecd573b440] Non-monotonous DTS in output stream 0:1; previous: 441775, current: 441344; changing to 441776. This may result in incorrect timestamps in the output file.
  frame=38540 fps=38539 q=-1.0 size=   56832kB time=00:42:38.23 bitrate= 182.0kbits/s speed=2.56e+03x  frame=44130 fps=38101 q=-1.0 Lsize=   68207kB time=00:49:27.67 bitrate= 188.3kbits/s speed=2.56e+03x    
  video:20486kB audio:45963kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 2.645229%
  
  ```
  
  
  ---
  
  