Answers.Body: <p>I've noticed that sometimes either the encoder or FFmpeg itself likes to drop frames they deem as duplicates, possibly with the intent that a decoder will just display the same frame multiple times on playback and thus saving some bitrate by not having to encode the same frame twice (or more). This doesn't play nice with calculating VMAF or other metrics afterwards due to desynchronization of frames between the reference and target files, leading to invalid scores because different frames are being compared.</p>
  <p>To get around this, I've started adding <code>-fps_mode passthrough</code> to my encoding commands (<code>-vsync 0</code> for FFmpeg versions prior to 5.1). This way all frames are passed on as-is.</p>
  <p>To see if this is the issue affecting you, simply count the number of frames in your files. This can be done with ffprobe:</p>
  <blockquote>
  <p>ffprobe -v error -count_frames -select_streams v:0 -show_entries stream=nb_read_frames -of default=nokey=1:noprint_wrappers=1 -threads auto input.mp4</p>
  </blockquote>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-02-27T11:55:47.157Z
Answers.LastActivityDate: 2023-02-27T11:55:47.157Z
Answers.UserDisplayName: veikk0
Body: <p>I've seen similar issues but can't seem to come to a resolution myself.
  I have a 4K rip that is in hevc, 10bit depth, which my older phone can't play since it lacks GPU acceleration. Therefore, I'm reencoding it on ffmpeg to a YUV420p scheme. I've done this by GPU acceleration and without, at multiple qualities and bitrates. I figured I'd measure each output against the original video and simply use the one with the higher VMAF score as the one I use to reencode some other 4K videos with the same 10bit depth.</p>
  <p>However, the VMAF score for every output, whether CPU or GPU encoded, 1M or 10M bitrate, is somewhere around 32.2%. This is despite the fact that the higher bitrate ones generally look pretty decent. After looking around, I saw that some people recommend using a filter for <code>setpts=PTS-STARTPTS</code> to sync up the frames, since maybe the encoding process desynced some frames? This still doesn't help.</p>
  <p>Here's where it gets weird. I looked in the log when running the VMAF filter to pick one frame in particular and test it. I found one frame where the VMAF score reported a score of 0.0000:</p>
  <pre><code>    &lt;frame frameNum=&quot;487&quot; integer_motion2=&quot;7.639184&quot; integer_motion=&quot;9.133736&quot; integer_adm2=&quot;0.516201&quot; integer_adm_scale0=&quot;0.902589&quot; integer_adm_scale1=&quot;0.619710&quot; integer_adm_scale2=&quot;0.445753&quot; integer_adm_scale3=&quot;0.428904&quot; integer_vif_scale0=&quot;0.443259&quot; integer_vif_scale1=&quot;0.513409&quot; integer_vif_scale2=&quot;0.532767&quot; integer_vif_scale3=&quot;0.533751&quot; vmaf=&quot;0.000000&quot; /&gt;
  </code></pre>
  <p>Then in ffmpeg I ran a command to extract this frame in both source and encoded with something like:</p>
  <pre><code>.\ffmpeg.exe -i &quot;(sourceVid)&quot; -vf &quot;select=eq(n\,487)&quot; -vframes 1 source.png&quot;
  </code></pre>
  <p>to just get a sample of the frame that VMAF has ranked at 0. Looking at the two pngs, they seem identical to the eye. When running the two pngs through VMAF in ffmpeg, they report a score of 98, which is what I actually expect and is obviously not 0.</p>
  <p>This is driving me mad. I can't tell why VMAF would keep reporting a much lower rate than expected. It does seem like an issue with frame syncing, but I can't see a way to resolve it. Any help would be appreciated</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-10-20T13:48:05.087Z
Comments.Text: Have you checked this https://stackoverflow.com/questions/67598772/right-way-to-use-vmaf-with-ffmpeg and is it any help?
Comments.UserDisplayName: AutoBaker
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-10-20T13:48:19.370Z
Comments.Text: Please also see: https://github.com/Netflix/vmaf/blob/master/resource/doc/ffmpeg.md
Comments.UserDisplayName: AutoBaker
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-10-20T17:37:11.863Z
Comments.Text: I checked both of these and it didn't help, but, after swapping the sources i'm getting better scores (75-99). I'm not sure why this is working, since now my sources are backward. Code is `ffmpeg -i "distorted.mkv" -i "reference.mkv" -frames:v 2000 -filter_complex '[0:v]setpts=PTS-STARTPTS[a];[1:v]setpts=PTS-STARTPTS[b];[b][a]libvmaf=model_path="model.json"'`, whereas before a and b were swapped, as I thought would be correct (distorted comes first). I'm not sure why this is happening, but it seems when the sources are back-to-front from what they should be, VMAF scores are more reasonable
Comments.UserDisplayName: Mdd M
Tags: <ffmpeg>
Title: FFMPEG VMAF consistently low rating despite quality encode
UserDisplayName: Mdd M
ThreadText: # FFMPEG VMAF consistently low rating despite quality encode
  
  (Asked by: username_0 on 2022-10-20)
  
  I've seen similar issues but can't seem to come to a resolution myself.
  I have a 4K rip that is in hevc, 10bit depth, which my older phone can't play since it lacks GPU acceleration. Therefore, I'm reencoding it on ffmpeg to a YUV420p scheme. I've done this by GPU acceleration and without, at multiple qualities and bitrates. I figured I'd measure each output against the original video and simply use the one with the higher VMAF score as the one I use to reencode some other 4K videos with the same 10bit depth.
  
  However, the VMAF score for every output, whether CPU or GPU encoded, 1M or 10M bitrate, is somewhere around 32.2%. This is despite the fact that the higher bitrate ones generally look pretty decent. After looking around, I saw that some people recommend using a filter for `setpts=PTS-STARTPTS` to sync up the frames, since maybe the encoding process desynced some frames? This still doesn't help.
  
  Here's where it gets weird. I looked in the log when running the VMAF filter to pick one frame in particular and test it. I found one frame where the VMAF score reported a score of 0.0000:
  
  ```
      <frame frameNum="487" integer_motion2="7.639184" integer_motion="9.133736" integer_adm2="0.516201" integer_adm_scale0="0.902589" integer_adm_scale1="0.619710" integer_adm_scale2="0.445753" integer_adm_scale3="0.428904" integer_vif_scale0="0.443259" integer_vif_scale1="0.513409" integer_vif_scale2="0.532767" integer_vif_scale3="0.533751" vmaf="0.000000" />
  
  ```
  
  Then in ffmpeg I ran a command to extract this frame in both source and encoded with something like:
  
  ```
  .\ffmpeg.exe -i "(sourceVid)" -vf "select=eq(n\,487)" -vframes 1 source.png"
  
  ```
  
  to just get a sample of the frame that VMAF has ranked at 0. Looking at the two pngs, they seem identical to the eye. When running the two pngs through VMAF in ffmpeg, they report a score of 98, which is what I actually expect and is obviously not 0.
  
  This is driving me mad. I can't tell why VMAF would keep reporting a much lower rate than expected. It does seem like an issue with frame syncing, but I can't see a way to resolve it. Any help would be appreciated
  
  **Tags:** ffmpeg
  
  > **Comments:**
  > **username_1**: Have you checked this https://stackoverflow.com/questions/67598772/right-way-to-use-vmaf-with-ffmpeg and is it any help?  
  > **username_1**: Please also see: https://github.com/Netflix/vmaf/blob/master/resource/doc/ffmpeg.md  
  > **username_0**: I checked both of these and it didn't help, but, after swapping the sources i'm getting better scores (75-99). I'm not sure why this is working, since now my sources are backward. Code is `ffmpeg -i "distorted.mkv" -i "reference.mkv" -frames:v 2000 -filter_complex '[0:v]setpts=PTS-STARTPTS[a];[1:v]setpts=PTS-STARTPTS[b];[b][a]libvmaf=model_path="model.json"'`, whereas before a and b were swapped, as I thought would be correct (distorted comes first). I'm not sure why this is happening, but it seems when the sources are back-to-front from what they should be, VMAF scores are more reasonable  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2023-02-27. Score: 1
  
  I've noticed that sometimes either the encoder or FFmpeg itself likes to drop frames they deem as duplicates, possibly with the intent that a decoder will just display the same frame multiple times on playback and thus saving some bitrate by not having to encode the same frame twice (or more). This doesn't play nice with calculating VMAF or other metrics afterwards due to desynchronization of frames between the reference and target files, leading to invalid scores because different frames are being compared.
  
  To get around this, I've started adding `-fps_mode passthrough` to my encoding commands (`-vsync 0` for FFmpeg versions prior to 5.1). This way all frames are passed on as-is.
  
  To see if this is the issue affecting you, simply count the number of frames in your files. This can be done with ffprobe:
  
  > ffprobe -v error -count\_frames -select\_streams v:0 -show\_entries stream=nb\_read\_frames -of default=nokey=1:noprint\_wrappers=1 -threads auto input.mp4
  
  
  ---
  
  