Bypassing YouTube video download throttling
&quot;Have you ever tried to download videos from YouTube? I mean manually without relying on software like youtube-dl, yt-dlp or one of &quot;these&quot; websites. It&#x27;s much more complicated than you might think.&quot;<p>This reminds me of some sort of fizzbuzz test.  This is not complicated at all.  There is no need to use the Range header or run Javascript.<p>The short script below does not download anything because there is no need.  It does not use Range headers, it does not run Javascript and it makes only one TCP connection.  With the JSON it fetches, one can simply extract the videoplayback URLs and put them in a locally-hosted HTML page with no Javascript.<p><pre><code>    #!&#x2F;bin&#x2F;sh
    # usage: echo videoId | $0 &lt;-- this will indicate len to use    
    # usage: echo videoId | $0 len | openssl s_client -connect www.youtube.com:443 -ign_eof
    # usage: $0 len &lt; videoId-list | openssl s_client -connect www.youtube.com:443 -ign_eof
    
    (
    while read x;do
    test ${#x} -eq 11||continue
    if test $# -ne 1;then len=${#x};x=$(grep -m1 ^\{ $0|sed &#x27;s&#x2F;\$x&#x2F;&#x2F;&#x27;|wc -c);exec echo usage: ${0##*&#x2F;} $((x+len));fi
    
    cr=$(printf &#x27;\r&#x27;);
    sed &quot;&#x2F;^[a-zA-Z].*: &#x2F;s&#x2F;$&#x2F;$cr&#x2F;;s&#x2F;^$&#x2F;$cr&#x2F;&quot; &lt;&lt; eof 
    POST &#x2F;youtubei&#x2F;v1&#x2F;player?key=AIzaSyA8eiZmM1FaDVjRy-df2KTyQ_vz_yYM39w HTTP&#x2F;1.1
    Host: www.youtube.com
    Content-Type: application&#x2F;json
    Content-Length: $1
    Connection: keep-alive
    
    {&quot;context&quot;: {&quot;client&quot;: {&quot;clientName&quot;: &quot;IOS&quot;, &quot;clientVersion&quot;: &quot;17.33.2&quot; }}, &quot;videoId&quot;: &quot;$x&quot;, &quot;params&quot;: &quot;CgIQBg==&quot;, &quot;playbackContext&quot;: {&quot;contentPlaybackContext&quot;: {&quot;html5Preference&quot;: &quot;HTML5_PREF_WANTS&quot;}}, &quot;contentCheckOk&quot;: true, &quot;racyCheckOk&quot;: true}
    eof
    done
    printf &#x27;\r\n&#x27;
    printf &#x27;GET &#x2F;robots.txt HTTP&#x2F;1.0\r\nHost: www.youtube.com\r\nConnection: close\r\n\r\n&#x27;;
    )
    </code></pre>
For processing the JSON I wrote custom utilities in C that (a) extract videoIds and other useful strings, (b) generate HTTP similar to above, and (c) filter the returned JSON into CSV, SQL or HTML.   For me, these run faster than Python and jq and are easier to edit.  Using these utilities I can also do full searches that return hundreds to thousands of results and I can easily exclude all &quot;suggested&quot; or &quot;recommended&quot; videos.<p>CSV output<p>1666520150,23 Oct 2022 10:15:50 UTC,22,aqz-KE-bpKQ,&quot;Big Buck Bunny 60fps 4K - Official Blender Foundation Short Film&quot;,00:10:35,635,UCSMOQeBJ2RAnuFungnQOxLg,19211597,&quot;Blender&quot;<p>SQL output<p>INSERT INTO t1(ts,utc,itag,vid,title,dur,len,cid,views,author) VALUES(1666520150,&#x27;23 Oct 2022 10:15:50 UTC&#x27;,22,&#x27;aqz-KE-bpKQ&#x27;,&#x27;Big Buck Bunny 60fps 4K - Official Blender Foundation Short Film&#x27;,&#x27;00:10:35&#x27;,635,&#x27;UCSMOQeBJ2RAnuFungnQOxLg&#x27;,19211597,&#x27;Blender&#x27;) ON CONFLICT(vid) DO UPDATE SET views=excluded.views;<p>HTML output<p>Looks just like CSV except vid is a hyperlink