Answers.Body: <p>Looks to be an encoder deficit, probably due to the rapid transition in the alpha from 1 to 0 in the border regions. Outputting to VP9 with alpha produces a clean result as does outputting to prores without alpha. Outputting with an uniform alpha i.e. whole alpha plane reset to 0 or 0.5 or 1 also produces a clean result. It's not a scaler issue as rawvideo with yuva444p10le also produces clean result.</p>
  
  <p>Looking at the source code, I see this line, which may be relevant:</p>
  
  <pre><code>// todo alpha quantisation for high quants
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-12-05T06:16:46.273Z
Answers.Text: Should I raise a bug report, or is it redundant, given that it's already a 'todo' item
Answers.UserDisplayName: stib
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-12-05T06:17:54.090Z
Answers.Text: Raise a bug report, as that may raise priority.
Answers.UserDisplayName: Gyan
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-12-07T13:36:12.397Z
Answers.Text: Link to [bug report](https://trac.ffmpeg.org/ticket/5995).
Answers.UserDisplayName: Gyan
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-01T17:45:41.140Z
Answers.Text: @stib The bug report is marked as [fixed](https://trac.ffmpeg.org/ticket/5995#comment:9). Can you verify?
Answers.UserDisplayName: Gyan
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-03T04:01:58.640Z
Answers.Text: Thanks for the heads-up. It appears to be working, but I haven't fully tested it yet.
Answers.UserDisplayName: stib
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-10-03T00:31:41.803Z
Answers.Text: Seems to be fixed now.
Answers.UserDisplayName: stib
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-12-05T06:13:58.113Z
Answers.LastActivityDate: 2016-12-05T06:13:58.113Z
Answers.UserDisplayName: Gyan
Body: <p><strong>EDIT: This bug has been fixed. If you're experiencing the problem below, try updating ffmpeg</strong></p>
  
  <p>I need to supply video with alpha as a prores4444 encoded file. I've been trying to use ffmpeg, but for once it seems to be letting me down (I know, gobsmacked!).</p>
  
  <p>I'm using the prores_ks codec and following the intructions on the <a href="http://ffmpeg.org/ffmpeg-codecs.html#Private-Options-for-prores_002dks" rel="nofollow noreferrer">man page</a> this is my command, testing on a QTRLE encoded file with 8-bit RGBA:</p>
  
  <pre><code>ffmpeg -i '.\alpha test.mov'  -c:v prores_ks -profile:v 4 -quant_mat 'hq' -pix_fmt yuva444p10le 'prorestest2.mov'
  </code></pre>
  
  <p>The output is:</p>
  
  <pre><code>ffmpeg version N-82664-g801b5c1 Copyright (c) 2000-2016 the FFmpeg developers
    built with gcc 5.4.0 (GCC)
    configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-dxva2 --enable-libmfx --enable-nvenc --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-lzma --enable-decklink --enable-zlib
    libavutil      55. 41.101 / 55. 41.101
    libavcodec     57. 66.108 / 57. 66.108
    libavformat    57. 58.101 / 57. 58.101
    libavdevice    57.  2.100 / 57.  2.100
    libavfilter     6. 67.100 /  6. 67.100
    libswscale      4.  3.101 /  4.  3.101
    libswresample   2.  4.100 /  2.  4.100
    libpostproc    54.  2.100 / 54.  2.100
  Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '.\alpha test.mov':
    Metadata:
      major_brand     : qt
      minor_version   : 537199360
      compatible_brands: qt
      creation_time   : 2016-12-12T03:17:14.000000Z
    Duration: 00:00:10.00, start: 0.000000, bitrate: 406275 kb/s
      Stream #0:0(eng): Video: qtrle (rle  / 0x20656C72), bgra(progressive), 1920x1080, 405436 kb/s, SAR 1:1 DAR 16:9, 25 fps, 25 tbr, 25 tbn, 25 tbc (default)
      Metadata:
        creation_time   : 2016-12-12T03:17:14.000000Z
        handler_name    : Apple Alias Data Handler
        encoder         : Animation
        timecode        : 00:00:00:00
      Stream #0:1(eng): Data: none (tmcd / 0x64636D74), 0 kb/s (default)
      Metadata:
        creation_time   : 2016-12-12T03:17:14.000000Z
        handler_name    : Apple Alias Data Handler
        timecode        : 00:00:00:00
  File 'prorestest2.mov' already exists. Overwrite ? [y/N] y
  Output #0, mov, to 'prorestest2.mov':
    Metadata:
      major_brand     : qt
      minor_version   : 537199360
      compatible_brands: qt
      encoder         : Lavf57.58.101
      Stream #0:0(eng): Video: prores (prores_ks) (ap4h / 0x68347061), yuva444p10le, 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 25 fps, 12800 tbn, 25 tbc (default)
      Metadata:
        creation_time   : 2016-12-12T03:17:14.000000Z
        handler_name    : Apple Alias Data Handler
        timecode        : 00:00:00:00
        encoder         : Lavc57.66.108 prores_ks
  Stream mapping:
    Stream #0:0 -&gt; #0:0 (qtrle (native) -&gt; prores (prores_ks))
  Press [q] to stop, [?] for help
  frame=  250 fps=5.5 q=-0.0 Lsize=  186122kB time=00:00:09.96 bitrate=153082.2kbits/s speed=0.218x
  video:186119kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.001761%
  </code></pre>
  
  <p>It all looks good, unitl you see the results. Here is a frame from the original, exported as a png (using djv):
  <a href="https://i.sstatic.net/KpMMP.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/KpMMP.png" alt="enter image description here"></a></p>
  
  <p>…and here is the prores encoded version. There's blockiness and ringing artefacts that are most apparent in areas that have partial alpha:</p>
  
  <p><a href="https://i.sstatic.net/XUbk5.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/XUbk5.png" alt="prores444 encoded version"></a></p>
  
  <p>It's most apparent if you look at the colour channels individually:</p>
  
  <p><a href="https://i.sstatic.net/Ngh3m.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/Ngh3m.png" alt="enter image description here"></a></p>
  
  <p>I've tried the different <code>-quant_mat</code> settings (in the docs it looks like there should be 6, but if I try <code>-quant_mat 5</code> I get an out of range error. I also tried using the string version, i.e. <code>-quant_mat 'hq'</code>. Another thing I tried was changing the number of alpha bits using <code>-alpha_bits 16</code> and <code>-alpha_bits 8</code>. The problem was worse with 16 bit alpha.</p>
  
  <p>So is it something I'm doing or is it the codec not working properly?</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2016-12-02T05:24:55.093Z
Comments.Text: Can you provide one frame from the original as a PNG?
Comments.UserDisplayName: Gyan
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2016-12-04T23:14:17.013Z
Comments.Text: I've edited the question with pngs from the movie instead of screenshots. The blockiness is harder to see against white.
Comments.UserDisplayName: stib
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2016-12-05T16:39:29.510Z
Comments.Text: @stib Did you also test `prores_aw`? It's another ProRes encoder. I didn't try or look at the source code: got to install drywall instead...
Comments.UserDisplayName: llogan
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2016-12-05T23:45:18.223Z
Comments.Text: oh that's easy: `cd ~/drywall; ./configure && make && sudo make install`
Comments.UserDisplayName: stib
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2016-12-06T00:02:37.550Z
Comments.Text: Gave it a try, no alpha. `ffmpeg -i '.\alpha test.mov'  -c:v prores_aw -profile:v 3 -pix_fmt yuva444p10le 'prores_aw-test.mov'`  →  `Incompatible pixel format 'yuva444p10le' for codec 'prores_aw', auto-selecting format 'yuv422p10le'`
Comments.UserDisplayName: stib
Tags: <ffmpeg><encoding><prores>
Title: Bad quality encodes to Prores4444 with alpha from ffmpeg
UserDisplayName: stib
ThreadText: # Bad quality encodes to Prores4444 with alpha from ffmpeg
  
  (Asked by: username_0 on 2016-12-02)
  
  **EDIT: This bug has been fixed. If you're experiencing the problem below, try updating ffmpeg**
  
  I need to supply video with alpha as a prores4444 encoded file. I've been trying to use ffmpeg, but for once it seems to be letting me down (I know, gobsmacked!).
  
  I'm using the prores\_ks codec and following the intructions on the [man page](http://ffmpeg.org/ffmpeg-codecs.html#Private-Options-for-prores_002dks) this is my command, testing on a QTRLE encoded file with 8-bit RGBA:
  
  ```
  ffmpeg -i '.\alpha test.mov'  -c:v prores_ks -profile:v 4 -quant_mat 'hq' -pix_fmt yuva444p10le 'prorestest2.mov'
  
  ```
  
  The output is:
  
  ```
  ffmpeg version N-82664-g801b5c1 Copyright (c) 2000-2016 the FFmpeg developers
    built with gcc 5.4.0 (GCC)
    configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-dxva2 --enable-libmfx --enable-nvenc --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-lzma --enable-decklink --enable-zlib
    libavutil      55. 41.101 / 55. 41.101
    libavcodec     57. 66.108 / 57. 66.108
    libavformat    57. 58.101 / 57. 58.101
    libavdevice    57.  2.100 / 57.  2.100
    libavfilter     6. 67.100 /  6. 67.100
    libswscale      4.  3.101 /  4.  3.101
    libswresample   2.  4.100 /  2.  4.100
    libpostproc    54.  2.100 / 54.  2.100
  Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '.\alpha test.mov':
    Metadata:
      major_brand     : qt
      minor_version   : 537199360
      compatible_brands: qt
      creation_time   : 2016-12-12T03:17:14.000000Z
    Duration: 00:00:10.00, start: 0.000000, bitrate: 406275 kb/s
      Stream #0:0(eng): Video: qtrle (rle  / 0x20656C72), bgra(progressive), 1920x1080, 405436 kb/s, SAR 1:1 DAR 16:9, 25 fps, 25 tbr, 25 tbn, 25 tbc (default)
      Metadata:
        creation_time   : 2016-12-12T03:17:14.000000Z
        handler_name    : Apple Alias Data Handler
        encoder         : Animation
        timecode        : 00:00:00:00
      Stream #0:1(eng): Data: none (tmcd / 0x64636D74), 0 kb/s (default)
      Metadata:
        creation_time   : 2016-12-12T03:17:14.000000Z
        handler_name    : Apple Alias Data Handler
        timecode        : 00:00:00:00
  File 'prorestest2.mov' already exists. Overwrite ? [y/N] y
  Output #0, mov, to 'prorestest2.mov':
    Metadata:
      major_brand     : qt
      minor_version   : 537199360
      compatible_brands: qt
      encoder         : Lavf57.58.101
      Stream #0:0(eng): Video: prores (prores_ks) (ap4h / 0x68347061), yuva444p10le, 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 25 fps, 12800 tbn, 25 tbc (default)
      Metadata:
        creation_time   : 2016-12-12T03:17:14.000000Z
        handler_name    : Apple Alias Data Handler
        timecode        : 00:00:00:00
        encoder         : Lavc57.66.108 prores_ks
  Stream mapping:
    Stream #0:0 -> #0:0 (qtrle (native) -> prores (prores_ks))
  Press [q] to stop, [?] for help
  frame=  250 fps=5.5 q=-0.0 Lsize=  186122kB time=00:00:09.96 bitrate=153082.2kbits/s speed=0.218x
  video:186119kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.001761%
  
  ```
  
  It all looks good, unitl you see the results. Here is a frame from the original, exported as a png (using djv):
  [![enter image description here](https://i.sstatic.net/KpMMP.png)](https://i.sstatic.net/KpMMP.png)
  
  …and here is the prores encoded version. There's blockiness and ringing artefacts that are most apparent in areas that have partial alpha:
  
  [![prores444 encoded version](https://i.sstatic.net/XUbk5.png)](https://i.sstatic.net/XUbk5.png)
  
  It's most apparent if you look at the colour channels individually:
  
  [![enter image description here](https://i.sstatic.net/Ngh3m.png)](https://i.sstatic.net/Ngh3m.png)
  
  I've tried the different `-quant_mat` settings (in the docs it looks like there should be 6, but if I try `-quant_mat 5` I get an out of range error. I also tried using the string version, i.e. `-quant_mat 'hq'`. Another thing I tried was changing the number of alpha bits using `-alpha_bits 16` and `-alpha_bits 8`. The problem was worse with 16 bit alpha.
  
  So is it something I'm doing or is it the codec not working properly?
  
  **Tags:** ffmpeg, encoding, prores
  
  > **Comments:**
  > **username_1**: Can you provide one frame from the original as a PNG?  
  > **username_0**: I've edited the question with pngs from the movie instead of screenshots. The blockiness is harder to see against white.  
  > **username_2**: @username_0 Did you also test `prores_aw`? It's another ProRes encoder. I didn't try or look at the source code: got to install drywall instead...  
  > **username_0**: oh that's easy: `cd ~/drywall; ./configure && make && sudo make install`  
  > **username_0**: Gave it a try, no alpha. `ffmpeg -i '.\alpha test.mov'  -c:v prores_aw -profile:v 3 -pix_fmt yuva444p10le 'prores_aw-test.mov'`  →  `Incompatible pixel format 'yuva444p10le' for codec 'prores_aw', auto-selecting format 'yuv422p10le'`  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2016-12-05. Score: 2 | Accepted answer
  
  Looks to be an encoder deficit, probably due to the rapid transition in the alpha from 1 to 0 in the border regions. Outputting to VP9 with alpha produces a clean result as does outputting to prores without alpha. Outputting with an uniform alpha i.e. whole alpha plane reset to 0 or 0.5 or 1 also produces a clean result. It's not a scaler issue as rawvideo with yuva444p10le also produces clean result.
  
  Looking at the source code, I see this line, which may be relevant:
  
  ```
  // todo alpha quantisation for high quants
  
  ```
  
  > **Comments:**
  > **username_0**: Should I raise a bug report, or is it redundant, given that it's already a 'todo' item  
  > **username_1**: Raise a bug report, as that may raise priority.  
  > **username_1**: Link to [bug report](https://trac.ffmpeg.org/ticket/5995).  
  > **username_1**: @username_0 The bug report is marked as [fixed](https://trac.ffmpeg.org/ticket/5995#comment:9). Can you verify?  
  > **username_0**: Thanks for the heads-up. It appears to be working, but I haven't fully tested it yet.  
  > **username_0**: Seems to be fixed now.  
  
  ---
  
  