<p dir="auto">Content-Length header is duplicated if specified on Python3, leading to 400 responses:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Python 3.3.0 (default, Apr 30 2013, 06:45:58)
[GCC 4.2.1 Compatible Apple Clang 4.0 ((tags/Apple/clang-421.0.60))] on darwin
Type &quot;help&quot;, &quot;copyright&quot;, &quot;credits&quot; or &quot;license&quot; for more information.
&gt;&gt;&gt; import requests
&gt;&gt;&gt; requests.post(&quot;http://httpbin.org/post&quot;, headers={&quot;Content-Length&quot;: 0})
&lt;Response [400]&gt;"><pre class="notranslate"><code class="notranslate">Python 3.3.0 (default, Apr 30 2013, 06:45:58)
[GCC 4.2.1 Compatible Apple Clang 4.0 ((tags/Apple/clang-421.0.60))] on darwin
Type "help", "copyright", "credits" or "license" for more information.
&gt;&gt;&gt; import requests
&gt;&gt;&gt; requests.post("http://httpbin.org/post", headers={"Content-Length": 0})
&lt;Response [400]&gt;
</code></pre></div>
<p dir="auto">We need to go deeper:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="&gt;&gt;&gt; import six
&gt;&gt;&gt; six.moves.http_client.HTTPConnection.debuglevel = 1
&gt;&gt;&gt; requests.post(&quot;http://httpbin.org/post&quot;, headers={&quot;Content-Length&quot;: 0})
send: b'POST /post HTTP/1.1\r\nHost: httpbin.org\r\nAccept-Encoding: identity\r\nContent-Length: 0\r\nUser-Agent: python-requests/1.2.0 CPython/3.3.0 Darwin/12.3.0\r\nContent-Length: 0\r\nAccept: */*\r\nAccept-Encoding: gzip, deflate, compress\r\n\r\n'
reply: 'HTTP/1.1 400 BAD_REQUEST\r\n'
header: Content-Length header: Connection &lt;Response [400]&gt;"><pre class="notranslate"><code class="notranslate">&gt;&gt;&gt; import six
&gt;&gt;&gt; six.moves.http_client.HTTPConnection.debuglevel = 1
&gt;&gt;&gt; requests.post("http://httpbin.org/post", headers={"Content-Length": 0})
send: b'POST /post HTTP/1.1\r\nHost: httpbin.org\r\nAccept-Encoding: identity\r\nContent-Length: 0\r\nUser-Agent: python-requests/1.2.0 CPython/3.3.0 Darwin/12.3.0\r\nContent-Length: 0\r\nAccept: */*\r\nAccept-Encoding: gzip, deflate, compress\r\n\r\n'
reply: 'HTTP/1.1 400 BAD_REQUEST\r\n'
header: Content-Length header: Connection &lt;Response [400]&gt;
</code></pre></div>
<p dir="auto">That's what happens:</p>
<div class="highlight highlight-source-httpspec notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="POST /post HTTP/1.1
Host: httpbin.org
Accept-Encoding: identity
Content-Length: 0
User-Agent: python-requests/1.2.0 CPython/3.3.0 Darwin/12.3.0
Content-Length: 0
Accept: */*
Accept-Encoding: gzip, deflate, compress"><pre class="notranslate"><span class="pl-k">POST</span><span class="pl-c1"> /post HTTP/1.1</span>
<span class="pl-s"><span class="pl-v">Host:</span> httpbin.org</span>
<span class="pl-s"><span class="pl-v">Accept-Encoding:</span> identity</span>
<span class="pl-s"><span class="pl-v">Content-Length:</span> 0</span>
<span class="pl-s"><span class="pl-v">User-Agent:</span> python-requests/1.2.0 CPython/3.3.0 Darwin/12.3.0</span>
<span class="pl-s"><span class="pl-v">Content-Length:</span> 0</span>
<span class="pl-s"><span class="pl-v">Accept:</span> */*</span>
<span class="pl-s"><span class="pl-v">Accept-Encoding:</span> gzip, deflate, compress</span></pre></div>
<p dir="auto">Two content-lengths, one 400 response :-(<br>
On Python2 everything's OK.</p>
<p dir="auto">Spaces are not allowed in header names: <a href="https://www.rfc-editor.org/rfc/rfc9112.html#name-field-line-parsing" rel="nofollow">https://www.rfc-editor.org/rfc/rfc9112.html#name-field-line-parsing</a><br>
When requests encounters such invalid header names, it will not only throw away the invalid header, but also stops parsing all following headers.<br>
When the invalid header comes before headers such as <code class="notranslate">Content-Length</code>, this can lead to requests getting stuck indefinitely.</p>
<h2 dir="auto">Expected Result</h2>
<p dir="auto">Requests ignores the invalid header (or shows/fixes it; the RFC is only defining behavior for servers and proxies, it is unclear what the correct behavior for clients is and different clients behave differently) and continues parsing of the response.</p>
<h2 dir="auto">Actual Result</h2>
<p dir="auto">Requests ignores the invalid header and all headers after the invalid one. This can lead to requests getting stuck.</p>
<h2 dir="auto">Reproduction Steps</h2>
<ul dir="auto">
<li><code class="notranslate">pip install wptserve</code>: server that allows us to send invalid responses</li>
<li><code class="notranslate">python test.py</code></li>
</ul>
<p dir="auto"><code class="notranslate">test.py</code>:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="import wptserve
from wptserve.handlers import handler, as_is_handler
import requests
from requests.exceptions import ConnectionError
import subprocess

http1 = wptserve.WebTestHttpd(port=8081,
                            routes=[(&quot;*&quot;, &quot;*.asis&quot;, as_is_handler)],
                            use_ssl=False,
                            key_file=None,
                            certificate=None,
                            )
http1.start()

@handler 
def space_after_length(request, response):
    return 200, [(&quot;Content-Length&quot;, 14), (&quot;SPACE &quot;,'space'), (&quot;hidden&quot;, &quot;hidden&quot;)], '&lt;div&gt;ABC&lt;/div&gt;'
http1.router.register(&quot;*&quot;, &quot;/space_after_length&quot;, space_after_length)

@handler 
def space_first(request, response):
    return 200, [(&quot;SPACE &quot;,'space'), (&quot;Content-Length&quot;, 14), (&quot;hidden&quot;, &quot;hidden&quot;)], '&lt;div&gt;ABC&lt;/div&gt;'
http1.router.register(&quot;*&quot;, &quot;/space_first&quot;, space_first)


for path in [&quot;/space_first&quot;, &quot;/space_after_length&quot;]:
    url = f&quot;http://localhost:8081{path}&quot;
    curl = subprocess.check_output([&quot;curl&quot;, &quot;-s&quot;, &quot;-I&quot;, url]).decode(&quot;utf-8&quot;)
    try:
        r = requests.get(url, timeout=1)
        # r = requests.get(url)  # Stuck forever
        headers = r.headers
    except ConnectionError:
        headers = &quot;Hangs forever or timeout&quot;
    finally:
        print(f&quot;{path}\ncurl output:\n{curl}requests output:\n{headers}\n&quot;)
"><pre class="notranslate"><span class="pl-k">import</span> <span class="pl-s1">wptserve</span>
<span class="pl-k">from</span> <span class="pl-s1">wptserve</span>.<span class="pl-s1">handlers</span> <span class="pl-k">import</span> <span class="pl-s1">handler</span>, <span class="pl-s1">as_is_handler</span>
<span class="pl-k">import</span> <span class="pl-s1">requests</span>
<span class="pl-k">from</span> <span class="pl-s1">requests</span>.<span class="pl-s1">exceptions</span> <span class="pl-k">import</span> <span class="pl-v">ConnectionError</span>
<span class="pl-k">import</span> <span class="pl-s1">subprocess</span>

<span class="pl-s1">http1</span> <span class="pl-c1">=</span> <span class="pl-s1">wptserve</span>.<span class="pl-v">WebTestHttpd</span>(<span class="pl-s1">port</span><span class="pl-c1">=</span><span class="pl-c1">8081</span>,
                            <span class="pl-s1">routes</span><span class="pl-c1">=</span>[(<span class="pl-s">"*"</span>, <span class="pl-s">"*.asis"</span>, <span class="pl-s1">as_is_handler</span>)],
                            <span class="pl-s1">use_ssl</span><span class="pl-c1">=</span><span class="pl-c1">False</span>,
                            <span class="pl-s1">key_file</span><span class="pl-c1">=</span><span class="pl-c1">None</span>,
                            <span class="pl-s1">certificate</span><span class="pl-c1">=</span><span class="pl-c1">None</span>,
                            )
<span class="pl-s1">http1</span>.<span class="pl-en">start</span>()

<span class="pl-en">@<span class="pl-s1">handler</span></span> 
<span class="pl-k">def</span> <span class="pl-en">space_after_length</span>(<span class="pl-s1">request</span>, <span class="pl-s1">response</span>):
    <span class="pl-k">return</span> <span class="pl-c1">200</span>, [(<span class="pl-s">"Content-Length"</span>, <span class="pl-c1">14</span>), (<span class="pl-s">"SPACE "</span>,<span class="pl-s">'space'</span>), (<span class="pl-s">"hidden"</span>, <span class="pl-s">"hidden"</span>)], <span class="pl-s">'&lt;div&gt;ABC&lt;/div&gt;'</span>
<span class="pl-s1">http1</span>.<span class="pl-s1">router</span>.<span class="pl-en">register</span>(<span class="pl-s">"*"</span>, <span class="pl-s">"/space_after_length"</span>, <span class="pl-s1">space_after_length</span>)

<span class="pl-en">@<span class="pl-s1">handler</span></span> 
<span class="pl-k">def</span> <span class="pl-en">space_first</span>(<span class="pl-s1">request</span>, <span class="pl-s1">response</span>):
    <span class="pl-k">return</span> <span class="pl-c1">200</span>, [(<span class="pl-s">"SPACE "</span>,<span class="pl-s">'space'</span>), (<span class="pl-s">"Content-Length"</span>, <span class="pl-c1">14</span>), (<span class="pl-s">"hidden"</span>, <span class="pl-s">"hidden"</span>)], <span class="pl-s">'&lt;div&gt;ABC&lt;/div&gt;'</span>
<span class="pl-s1">http1</span>.<span class="pl-s1">router</span>.<span class="pl-en">register</span>(<span class="pl-s">"*"</span>, <span class="pl-s">"/space_first"</span>, <span class="pl-s1">space_first</span>)


<span class="pl-k">for</span> <span class="pl-s1">path</span> <span class="pl-c1">in</span> [<span class="pl-s">"/space_first"</span>, <span class="pl-s">"/space_after_length"</span>]:
    <span class="pl-s1">url</span> <span class="pl-c1">=</span> <span class="pl-s">f"http://localhost:8081<span class="pl-s1"><span class="pl-kos">{</span><span class="pl-s1">path</span><span class="pl-kos">}</span></span>"</span>
    <span class="pl-s1">curl</span> <span class="pl-c1">=</span> <span class="pl-s1">subprocess</span>.<span class="pl-en">check_output</span>([<span class="pl-s">"curl"</span>, <span class="pl-s">"-s"</span>, <span class="pl-s">"-I"</span>, <span class="pl-s1">url</span>]).<span class="pl-en">decode</span>(<span class="pl-s">"utf-8"</span>)
    <span class="pl-k">try</span>:
        <span class="pl-s1">r</span> <span class="pl-c1">=</span> <span class="pl-s1">requests</span>.<span class="pl-en">get</span>(<span class="pl-s1">url</span>, <span class="pl-s1">timeout</span><span class="pl-c1">=</span><span class="pl-c1">1</span>)
        <span class="pl-c"># r = requests.get(url)  # Stuck forever</span>
        <span class="pl-s1">headers</span> <span class="pl-c1">=</span> <span class="pl-s1">r</span>.<span class="pl-s1">headers</span>
    <span class="pl-k">except</span> <span class="pl-v">ConnectionError</span>:
        <span class="pl-s1">headers</span> <span class="pl-c1">=</span> <span class="pl-s">"Hangs forever or timeout"</span>
    <span class="pl-k">finally</span>:
        <span class="pl-en">print</span>(<span class="pl-s">f"<span class="pl-s1"><span class="pl-kos">{</span><span class="pl-s1">path</span><span class="pl-kos">}</span></span><span class="pl-cce">\n</span>curl output:<span class="pl-cce">\n</span><span class="pl-s1"><span class="pl-kos">{</span><span class="pl-s1">curl</span><span class="pl-kos">}</span></span>requests output:<span class="pl-cce">\n</span><span class="pl-s1"><span class="pl-kos">{</span><span class="pl-s1">headers</span><span class="pl-kos">}</span></span><span class="pl-cce">\n</span>"</span>)</pre></div>
<p dir="auto"><code class="notranslate">Output</code>:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="/space_first
curl output:
HTTP/1.1 200 OK
SPACE : space
Content-Length: 14
hidden: hidden
Server: BaseHTTP/0.6 Python/3.9.6
Date: Thu, 29 Sep 2022 13:46:58 GMT

requests output:
Hangs forever or timeout

/space_after_length
curl output:
HTTP/1.1 200 OK
Content-Length: 14
SPACE : space
hidden: hidden
Server: BaseHTTP/0.6 Python/3.9.6
Date: Thu, 29 Sep 2022 13:46:59 GMT

requests output:
{'Content-Length': '14'}"><pre class="notranslate"><code class="notranslate">/space_first
curl output:
HTTP/1.1 200 OK
SPACE : space
Content-Length: 14
hidden: hidden
Server: BaseHTTP/0.6 Python/3.9.6
Date: Thu, 29 Sep 2022 13:46:58 GMT

requests output:
Hangs forever or timeout

/space_after_length
curl output:
HTTP/1.1 200 OK
Content-Length: 14
SPACE : space
hidden: hidden
Server: BaseHTTP/0.6 Python/3.9.6
Date: Thu, 29 Sep 2022 13:46:59 GMT

requests output:
{'Content-Length': '14'}
</code></pre></div>
<h2 dir="auto">System Information</h2>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="$ python -m requests.help"><pre class="notranslate"><code class="notranslate">$ python -m requests.help
</code></pre></div>
<div class="highlight highlight-source-json notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="{
  &quot;chardet&quot;: {
    &quot;version&quot;: null
  },
  &quot;charset_normalizer&quot;: {
    &quot;version&quot;: &quot;2.1.1&quot;
  },
  &quot;cryptography&quot;: {
    &quot;version&quot;: &quot;&quot;
  },
  &quot;idna&quot;: {
    &quot;version&quot;: &quot;3.4&quot;
  },
  &quot;implementation&quot;: {
    &quot;name&quot;: &quot;CPython&quot;,
    &quot;version&quot;: &quot;3.9.6&quot;
  },
  &quot;platform&quot;: {
    &quot;release&quot;: &quot;21.6.0&quot;,
    &quot;system&quot;: &quot;Darwin&quot;
  },
  &quot;pyOpenSSL&quot;: {
    &quot;openssl_version&quot;: &quot;&quot;,
    &quot;version&quot;: null
  },
  &quot;requests&quot;: {
    &quot;version&quot;: &quot;2.28.1&quot;
  },
  &quot;system_ssl&quot;: {
    &quot;version&quot;: &quot;1010111f&quot;
  },
  &quot;urllib3&quot;: {
    &quot;version&quot;: &quot;1.26.12&quot;
  },
  &quot;using_charset_normalizer&quot;: true,
  &quot;using_pyopenssl&quot;: false
}"><pre class="notranslate">{
  <span class="pl-ent">"chardet"</span>: {
    <span class="pl-ent">"version"</span>: <span class="pl-c1">null</span>
  },
  <span class="pl-ent">"charset_normalizer"</span>: {
    <span class="pl-ent">"version"</span>: <span class="pl-s"><span class="pl-pds">"</span>2.1.1<span class="pl-pds">"</span></span>
  },
  <span class="pl-ent">"cryptography"</span>: {
    <span class="pl-ent">"version"</span>: <span class="pl-s"><span class="pl-pds">"</span><span class="pl-pds">"</span></span>
  },
  <span class="pl-ent">"idna"</span>: {
    <span class="pl-ent">"version"</span>: <span class="pl-s"><span class="pl-pds">"</span>3.4<span class="pl-pds">"</span></span>
  },
  <span class="pl-ent">"implementation"</span>: {
    <span class="pl-ent">"name"</span>: <span class="pl-s"><span class="pl-pds">"</span>CPython<span class="pl-pds">"</span></span>,
    <span class="pl-ent">"version"</span>: <span class="pl-s"><span class="pl-pds">"</span>3.9.6<span class="pl-pds">"</span></span>
  },
  <span class="pl-ent">"platform"</span>: {
    <span class="pl-ent">"release"</span>: <span class="pl-s"><span class="pl-pds">"</span>21.6.0<span class="pl-pds">"</span></span>,
    <span class="pl-ent">"system"</span>: <span class="pl-s"><span class="pl-pds">"</span>Darwin<span class="pl-pds">"</span></span>
  },
  <span class="pl-ent">"pyOpenSSL"</span>: {
    <span class="pl-ent">"openssl_version"</span>: <span class="pl-s"><span class="pl-pds">"</span><span class="pl-pds">"</span></span>,
    <span class="pl-ent">"version"</span>: <span class="pl-c1">null</span>
  },
  <span class="pl-ent">"requests"</span>: {
    <span class="pl-ent">"version"</span>: <span class="pl-s"><span class="pl-pds">"</span>2.28.1<span class="pl-pds">"</span></span>
  },
  <span class="pl-ent">"system_ssl"</span>: {
    <span class="pl-ent">"version"</span>: <span class="pl-s"><span class="pl-pds">"</span>1010111f<span class="pl-pds">"</span></span>
  },
  <span class="pl-ent">"urllib3"</span>: {
    <span class="pl-ent">"version"</span>: <span class="pl-s"><span class="pl-pds">"</span>1.26.12<span class="pl-pds">"</span></span>
  },
  <span class="pl-ent">"using_charset_normalizer"</span>: <span class="pl-c1">true</span>,
  <span class="pl-ent">"using_pyopenssl"</span>: <span class="pl-c1">false</span>
}</pre></div>