Answers.Body: <p>I could imagine that they also use hardware supported transcoding. A company like Google certainly has the resources to make custom FPGA transcoders and then transcoding speed gets blazing.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-04-01T08:38:26.527Z
Answers.Text: This could also be an option yeah. For example I guess that the v8 and v9 codecs were designed to scale optimally with their hardware, but regarding the x264 I think that the best pick is slice like a boss, give each slice to a different thread and rejoin the pieces together at the end.
Answers.UserDisplayName: user3450548
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-04-01T09:12:38.000Z
Answers.Text: A bit of insight about YouTube internals although quite dated: http://multimedia.cx/eggs/googles-youtube-uses-ffmpeg/
Answers.UserDisplayName: Hans Meiser
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2016-04-01T04:10:00.110Z
Answers.LastActivityDate: 2016-04-01T04:10:00.110Z
Answers.UserDisplayName: Hans Meiser
Answers.Body: <p>A common method is split-and-stitch where the file is cut into pieces and sent to multiple servers for transcode. That way you can transcode a file of any length in a fixed amount of time.</p>
  
  <p>Telestreams Episode Engine can do this, but I'm sure Google uses something custom coded. </p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2015-11-18T13:35:24.360Z
Answers.Text: Sounds right: http://www.tvtechnology.com/news/0086/hpa-day--youtubes-video-flow/223783
Answers.UserDisplayName: Gyan
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2015-11-19T21:11:59.463Z
Answers.Text: Great article, @Mulvya. All the internet should have the `edit` function from [se], the text over there badly needs a revision :)
Answers.UserDisplayName: brasofilo
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2015-11-18T12:27:44.397Z
Answers.LastActivityDate: 2015-11-18T12:27:44.397Z
Answers.UserDisplayName: kimgroth
Body: <p>For the bit I read YouTube and Vimeo rely on servers powered by some modified libs i guess like libavcodec and tools like ffmpeg in a serious custom build.</p>
  
  <p>Let's assume a transcode to h264 but the same could be applied to other codecs, I think VP8 and VP9 works in the same way.</p>
  
  <p>If I upload 1h video file on their servers, right after the upload i have all the versions ready... pratically like 1 seconds after.</p>
  
  <p>But, if codecs doesn't scale horizontally in a good way unless you want to trade time vs compression (see h264) and I'm pretty sure this websites care about the size of the videos..</p>
  
  <p>Given also that single threads speed are capped by CPU frequency and actual frequency isn't up to 3.5 ghz generally unless they use overclocked servers under liquid nitrogen :D</p>
  
  <p>How the transcode is done so fast?</p>
  
  <p>They use still multithread with a mix of settings, trading on initial file size, custom builds and everything else to have just the files insta-ready to the user and then continue a second transcode with a slow preset that will be swapped as soon this file will be ready? They start to transcode as soon they receive the file stream (aka: during the upload ?)</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2015-11-15T05:00:31.773Z
Comments.Text: "pratically like 1 seconds after" --> that's not my experience. It's fast but not that fast for me, and the HD versions take somewhat longer. See https://support.google.com/youtube/answer/71674?hl=en and http://greenlimepie.com/how-long-does-it-take-videos-to-process-on-youtube/
Comments.UserDisplayName: Gyan
Tags: <ffmpeg><youtube><transcoding><libav><vimeo>
Title: How is possible for YouTube and Vimeo transcode so fast?
UserDisplayName: user3450548
ThreadText: # How is possible for YouTube and Vimeo transcode so fast?
  
  (Asked by: username_0 on 2015-11-15)
  
  For the bit I read YouTube and Vimeo rely on servers powered by some modified libs i guess like libavcodec and tools like ffmpeg in a serious custom build.
  
  Let's assume a transcode to h264 but the same could be applied to other codecs, I think VP8 and VP9 works in the same way.
  
  If I upload 1h video file on their servers, right after the upload i have all the versions ready... pratically like 1 seconds after.
  
  But, if codecs doesn't scale horizontally in a good way unless you want to trade time vs compression (see h264) and I'm pretty sure this websites care about the size of the videos..
  
  Given also that single threads speed are capped by CPU frequency and actual frequency isn't up to 3.5 ghz generally unless they use overclocked servers under liquid nitrogen :D
  
  How the transcode is done so fast?
  
  They use still multithread with a mix of settings, trading on initial file size, custom builds and everything else to have just the files insta-ready to the user and then continue a second transcode with a slow preset that will be swapped as soon this file will be ready? They start to transcode as soon they receive the file stream (aka: during the upload ?)
  
  **Tags:** ffmpeg, youtube, transcoding, libav, vimeo
  
  > **Comments:**
  > **username_1**: "pratically like 1 seconds after" --> that's not my experience. It's fast but not that fast for me, and the HD versions take somewhat longer. See https://support.google.com/youtube/answer/71674?hl=en and http://greenlimepie.com/how-long-does-it-take-videos-to-process-on-youtube/  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_3 on 2015-11-18. Score: 5 | Accepted answer
  
  A common method is split-and-stitch where the file is cut into pieces and sent to multiple servers for transcode. That way you can transcode a file of any length in a fixed amount of time.
  
  Telestreams Episode Engine can do this, but I'm sure Google uses something custom coded.
  
  > **Comments:**
  > **username_1**: Sounds right: http://www.tvtechnology.com/news/0086/hpa-day--youtubes-video-flow/223783  
  > **username_4**: Great article, @Mulvya. All the internet should have the `edit` function from [se], the text over there badly needs a revision :)  
  
  ---
  
  ### Answer by username_2 on 2016-04-01. Score: 1
  
  I could imagine that they also use hardware supported transcoding. A company like Google certainly has the resources to make custom FPGA transcoders and then transcoding speed gets blazing.
  
  > **Comments:**
  > **username_0**: This could also be an option yeah. For example I guess that the v8 and v9 codecs were designed to scale optimally with their hardware, but regarding the x264 I think that the best pick is slice like a boss, give each slice to a different thread and rejoin the pieces together at the end.  
  > **username_2**: A bit of insight about YouTube internals although quite dated: http://multimedia.cx/eggs/googles-youtube-uses-ffmpeg/  
  
  ---
  
  