Answers.Body: <p>The answer here is &quot;scalability&quot;.</p>
  <p><a href="https://en.wikipedia.org/wiki/Scalable_Video_Coding" rel="nofollow noreferrer">https://en.wikipedia.org/wiki/Scalable_Video_Coding</a></p>
  <p>SVC has been available as an extension to H.264 for years.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-09-13T22:37:22.497Z
Answers.LastActivityDate: 2021-09-13T22:37:22.497Z
Answers.UserDisplayName: Brad
Body: <p>JPEG has a progressive enhancement mode where a full frame can be downloaded quickly at low quality, and then the quality enhanced as the file is continually loaded.</p>
  
  <p><a href="https://i.sstatic.net/SpGmy.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/SpGmy.jpg" alt="Progressive JPEG Example"></a></p>
  
  <p><em>(Example image from <a href="http://promokit.eu/prestahop-blog/image-optimization-to-speed-up-your-site/" rel="nofollow noreferrer">http://promokit.eu/prestahop-blog/image-optimization-to-speed-up-your-site/</a>)</em></p>
  
  <p>I'm wondering if there is a widely compatible video codec that enables the same functionality.</p>
  
  <p>If the codec were structured this way, it should be technically possible for a player to download a first-pass low-quality stream, begin playback immediately, and then download second-pass and third-pass quality enhancing streams if bandwidth is available to do so, combining them on playback for form progressively higher quality frames.</p>
  
  <p>On the server, there would be segment files for each quality stream, similar to how HLS works today.</p>
  
  <p><a href="https://i.sstatic.net/sfa4S.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/sfa4S.jpg" alt="HLS chunking diagram"></a></p>
  
  <p><em>(Diagram from <a href="http://www.streamingmedia.com/Articles/Editorial/What-Is-.../What-is-HLS-(HTTP-Live-Streaming)-78221.aspx" rel="nofollow noreferrer">http://www.streamingmedia.com/Articles/Editorial/What-Is-.../What-is-HLS-(HTTP-Live-Streaming)-78221.aspx</a>)</em></p>
  
  <p>The key difference here is that with HLS, entire separate copies of the stream are stored in each stream's segments.  If I want a mid-quality stream, I read from the mid-quality files only.  If I want low-quality, I read from the low-quality files only.</p>
  
  <p>What I'm proposing with progressive enhancement is that all players would always read the low-quality (first pass) stream files, and <strong>additionally</strong> the mid-quality (second pass) stream files if they wanted that mid-quality stream.  For high quality, they would read all three.  <strong>This should greatly reduce the storage overhead</strong> of the stream since the total disk usage to support low, mid, and high quality streams should be only slightly more than would be required for just the high quality stream.  This means that the upload bandwidth required on the source end is much less as well, since the stream is effectively uploaded once (plus some overhead) for all three qualities.  I imagine there would be overall a bit less CPU used in the encoding process since I'm only really making one stream (albeit in an unusual structure) rather than having to run three independent codecs simultaneously where much of the codec work is likely duplicated.</p>
  
  <p>I have read about some proprietary codecs in random tech news articles, but never found anything concrete.  I also know that a few companies did audio codecs this way when audio streaming was booming in the mid/late 90s, but that's apples and oranges.</p>
  
  <h2>Questions</h2>
  
  <ul>
  <li>Is there a codec available that allows for progressive enhancement and segementation?</li>
  <li>Is there a way to do this with a widely implemented codec like VP8 or VP9?</li>
  </ul>
  
  <p>I suppose there is some hacky stuff I could do with MJPEG, since MJPEG is basically JPEG progressive enhancement under the hood, but I'd rather find a better codec if it exists.</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2016-04-18T19:34:22.283Z
Comments.Text: Not quite the same thing but look up [Scalable Video Coding](https://en.wikipedia.org/wiki/Scalable_Video_Coding). [+](http://www.nojitter.com/post/240160699/vidyogoogle-announcement-of-vp9-svc-for-webrtc-why-its-important)
Comments.UserDisplayName: Gyan
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2016-04-27T14:10:57.087Z
Comments.Text: Wouldn't the problem be that the player would have to download each stream entirely before it could play? Though I guess you could download GOPs so that you download GOP1-low, GOP1-mid, GOP1-hi, then GOP2-low, GOP2-mid etc., only downloading what you can in the time available.
Comments.UserDisplayName: stib
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2016-04-27T16:39:19.997Z
Comments.Text: @stib Segmentation would be used, similar to how it is used in HLS, so that a few seconds of video would be encoded in each chunk.  You're right, each chunk of each quality stream would have to be downloaded to decode the desired quality for that chunk of time.  And, the client would automatically adjust between downloading 1, 2, or 3 chunks per segment as it has bandwidth to do so.  For HLS, the client decides which bitrate chunk to download, somewhat dynamically.  This would be the same, except that there's 3 requests for the high quality stream instead of one.
Comments.UserDisplayName: Brad
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2016-04-27T21:54:30.847Z
Comments.Text: Quick! get a patent!
Comments.UserDisplayName: stib
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2016-04-27T23:06:06.750Z
Comments.Text: @stib I think that would require me to actually make such a codec.  Math is hard.  :-)  Although, I could probably actually implement it in MJPEG since MJPEG already does the heavy lifting...
Comments.UserDisplayName: Brad
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-09-13T22:12:00.547Z
Comments.Text: @Brad did you ever find any angles on this other than MJPEG?
Comments.UserDisplayName: awidgery
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-09-13T22:35:45.700Z
Comments.Text: @awidgery Yes!  It turns out this has been a thing all along.  The keyword here is, "scalability".  See also:  https://en.wikipedia.org/wiki/Scalable_Video_Coding  Also, it's my understanding that SHVC is to be used for Next Gen/ATSC 3.0 TV where some layers will be broadcast, with enhancement layers via internet, to save on the broadcast bandwidth will still providing the higher quality video.
Comments.UserDisplayName: Brad
Tags: <video><codec><mpeg4><vp9>
Title: Video codec for progressive enhancement
UserDisplayName: Brad
ThreadText: # Video codec for progressive enhancement
  
  (Asked by: username_0 on 2016-04-18)
  
  JPEG has a progressive enhancement mode where a full frame can be downloaded quickly at low quality, and then the quality enhanced as the file is continually loaded.
  
  [![Progressive JPEG Example](https://i.sstatic.net/SpGmy.jpg)](https://i.sstatic.net/SpGmy.jpg)
  
  *(Example image from <http://promokit.eu/prestahop-blog/image-optimization-to-speed-up-your-site/>)*
  
  I'm wondering if there is a widely compatible video codec that enables the same functionality.
  
  If the codec were structured this way, it should be technically possible for a player to download a first-pass low-quality stream, begin playback immediately, and then download second-pass and third-pass quality enhancing streams if bandwidth is available to do so, combining them on playback for form progressively higher quality frames.
  
  On the server, there would be segment files for each quality stream, similar to how HLS works today.
  
  [![HLS chunking diagram](https://i.sstatic.net/sfa4S.jpg)](https://i.sstatic.net/sfa4S.jpg)
  
  *(Diagram from <http://www.streamingmedia.com/Articles/Editorial/What-Is-.../What-is-HLS-(HTTP-Live-Streaming)-78221.aspx>)*
  
  The key difference here is that with HLS, entire separate copies of the stream are stored in each stream's segments. If I want a mid-quality stream, I read from the mid-quality files only. If I want low-quality, I read from the low-quality files only.
  
  What I'm proposing with progressive enhancement is that all players would always read the low-quality (first pass) stream files, and **additionally** the mid-quality (second pass) stream files if they wanted that mid-quality stream. For high quality, they would read all three. **This should greatly reduce the storage overhead** of the stream since the total disk usage to support low, mid, and high quality streams should be only slightly more than would be required for just the high quality stream. This means that the upload bandwidth required on the source end is much less as well, since the stream is effectively uploaded once (plus some overhead) for all three qualities. I imagine there would be overall a bit less CPU used in the encoding process since I'm only really making one stream (albeit in an unusual structure) rather than having to run three independent codecs simultaneously where much of the codec work is likely duplicated.
  
  I have read about some proprietary codecs in random tech news articles, but never found anything concrete. I also know that a few companies did audio codecs this way when audio streaming was booming in the mid/late 90s, but that's apples and oranges.
  
  ## Questions
  
  * Is there a codec available that allows for progressive enhancement and segementation?
  * Is there a way to do this with a widely implemented codec like VP8 or VP9?
  
  I suppose there is some hacky stuff I could do with MJPEG, since MJPEG is basically JPEG progressive enhancement under the hood, but I'd rather find a better codec if it exists.
  
  **Tags:** video, codec, mpeg4, vp9
  
  > **Comments:**
  > **username_1**: Not quite the same thing but look up [Scalable Video Coding](https://en.wikipedia.org/wiki/Scalable_Video_Coding). [+](http://www.nojitter.com/post/240160699/vidyogoogle-announcement-of-vp9-svc-for-webrtc-why-its-important)  
  > **username_2**: Wouldn't the problem be that the player would have to download each stream entirely before it could play? Though I guess you could download GOPs so that you download GOP1-low, GOP1-mid, GOP1-hi, then GOP2-low, GOP2-mid etc., only downloading what you can in the time available.  
  > **username_0**: @username_2 Segmentation would be used, similar to how it is used in HLS, so that a few seconds of video would be encoded in each chunk.  You're right, each chunk of each quality stream would have to be downloaded to decode the desired quality for that chunk of time.  And, the client would automatically adjust between downloading 1, 2, or 3 chunks per segment as it has bandwidth to do so.  For HLS, the client decides which bitrate chunk to download, somewhat dynamically.  This would be the same, except that there's 3 requests for the high quality stream instead of one.  
  > **username_2**: Quick! get a patent!  
  > **username_0**: @username_2 I think that would require me to actually make such a codec.  Math is hard.  :-)  Although, I could probably actually implement it in MJPEG since MJPEG already does the heavy lifting...  
  > **username_3**: @username_0 did you ever find any angles on this other than MJPEG?  
  > **username_0**: @username_3 Yes!  It turns out this has been a thing all along.  The keyword here is, "scalability".  See also:  https://en.wikipedia.org/wiki/Scalable_Video_Coding  Also, it's my understanding that SHVC is to be used for Next Gen/ATSC 3.0 TV where some layers will be broadcast, with enhancement layers via internet, to save on the broadcast bandwidth will still providing the higher quality video.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2021-09-13. Score: 0 | Accepted answer
  
  The answer here is "scalability".
  
  <https://en.wikipedia.org/wiki/Scalable_Video_Coding>
  
  SVC has been available as an extension to H.264 for years.
  
  
  ---
  
  