<h2 dir="auto">Summary</h2>
<p dir="auto">Information can be lost during the parsing of some complex headers response, like for example these ones <a href="https://developers.google.com/search/reference/robots_meta_tag#xrobotstag" rel="nofollow">https://developers.google.com/search/reference/robots_meta_tag#xrobotstag</a></p>
<p dir="auto">Simulating something like this with httpbin gives us: <a href="http://httpbin.org/response-headers?X-Robots-Tag=googlebot:%20nofollow&amp;X-Robots-Tag=otherbot:%20noindex,%20nofollow&amp;X-Robots-Tag=noarchive&amp;X-Robots-Tag=unavailable_after:%2025%20Jun%202010%2015:00:00%20PST" rel="nofollow">http://httpbin.org/response-headers?X-Robots-Tag=googlebot:%20nofollow&amp;X-Robots-Tag=otherbot:%20noindex,%20nofollow&amp;X-Robots-Tag=noarchive&amp;X-Robots-Tag=unavailable_after:%2025%20Jun%202010%2015:00:00%20PST</a></p>
<h2 dir="auto">Expected Result</h2>
<p dir="auto">Something like the httpbin output would be nice:<br>
<code class="notranslate">{ "Content-Length": "227",  "Content-Type": "application/json",  "X-Robots-Tag": [ "googlebot: nofollow",  "otherbot: noindex, nofollow",  "noarchive",  "unavailable_after: 25 Jun 2010 15:00:00 PST" ] }</code></p>
<h2 dir="auto">Actual Result</h2>
<p dir="auto">Instead response headers returned by <em>requests</em> are like this:</p>
<p dir="auto"><code class="notranslate">{'Date': 'Mon, 13 Jul 2020 15:14:21 GMT', 'Content-Type': 'application/json', 'Content-Length': '227', 'Connection': 'keep-alive', 'Server': 'gunicorn/19.9.0', 'X-Robots-Tag': 'googlebot: nofollow, otherbot: noindex, nofollow, noarchive, unavailable_after: 25 Jun 2010 15:00:00 PST', 'Access-Control-Allow-Origin': '*', 'Access-Control-Allow-Credentials': 'true'}</code></p>
<p dir="auto">Note that using <code class="notranslate">raw._original_response.getheaders()</code> is returning more useful results:</p>
<p dir="auto"><code class="notranslate">[('Date', 'Mon, 13 Jul 2020 15:14:21 GMT'), ('Content-Type', 'application/json'), ('Content-Length', '227'), ('Connection', 'keep-alive'), ('Server', 'gunicorn/19.9.0'), ('X-Robots-Tag', 'googlebot: nofollow'), ('X-Robots-Tag', 'otherbot: noindex, nofollow'), ('X-Robots-Tag', 'noarchive'), ('X-Robots-Tag', 'unavailable_after: 25 Jun 2010 15:00:00 PST'), ('Access-Control-Allow-Origin', '*'), ('Access-Control-Allow-Credentials', 'true')]</code></p>
<h2 dir="auto">Reproduction Steps</h2>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="import requests
r=requests.get('http://httpbin.org/response-headers?X-Robots-Tag=googlebot:%20nofollow&amp;X-Robots-Tag=otherbot:%20noindex,%20nofollow&amp;X-Robots-Tag=noarchive&amp;X-Robots-Tag=unavailable_after:%2025%20Jun%202010%2015:00:00%20PST')
print(r.headers)
print(r.raw._original_response.getheaders())"><pre class="notranslate"><span class="pl-k">import</span> <span class="pl-s1">requests</span>
<span class="pl-s1">r</span><span class="pl-c1">=</span><span class="pl-s1">requests</span>.<span class="pl-en">get</span>(<span class="pl-s">'http://httpbin.org/response-headers?X-Robots-Tag=googlebot:%20nofollow&amp;X-Robots-Tag=otherbot:%20noindex,%20nofollow&amp;X-Robots-Tag=noarchive&amp;X-Robots-Tag=unavailable_after:%2025%20Jun%202010%2015:00:00%20PST'</span>)
<span class="pl-en">print</span>(<span class="pl-s1">r</span>.<span class="pl-s1">headers</span>)
<span class="pl-en">print</span>(<span class="pl-s1">r</span>.<span class="pl-s1">raw</span>.<span class="pl-s1">_original_response</span>.<span class="pl-en">getheaders</span>())</pre></div>
<h2 dir="auto">System Information</h2>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="$ python -m requests.help"><pre class="notranslate"><code class="notranslate">$ python -m requests.help
</code></pre></div>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="{
  &quot;chardet&quot;: {
    &quot;version&quot;: &quot;3.0.4&quot;
  },
  &quot;cryptography&quot;: {
    &quot;version&quot;: &quot;&quot;
  },
  &quot;idna&quot;: {
    &quot;version&quot;: &quot;2.9&quot;
  },
  &quot;implementation&quot;: {
    &quot;name&quot;: &quot;CPython&quot;,
    &quot;version&quot;: &quot;3.8.3&quot;
  },
  &quot;platform&quot;: {
    &quot;release&quot;: &quot;16.7.0&quot;,
    &quot;system&quot;: &quot;Darwin&quot;
  },
  &quot;pyOpenSSL&quot;: {
    &quot;openssl_version&quot;: &quot;&quot;,
    &quot;version&quot;: null
  },
  &quot;requests&quot;: {
    &quot;version&quot;: &quot;2.23.0&quot;
  },
  &quot;system_ssl&quot;: {
    &quot;version&quot;: &quot;1010107f&quot;
  },
  &quot;urllib3&quot;: {
    &quot;version&quot;: &quot;1.25.9&quot;
  },
  &quot;using_pyopenssl&quot;: false
}
"><pre class="notranslate"><code class="notranslate">{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "2.9"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.8.3"
  },
  "platform": {
    "release": "16.7.0",
    "system": "Darwin"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.23.0"
  },
  "system_ssl": {
    "version": "1010107f"
  },
  "urllib3": {
    "version": "1.25.9"
  },
  "using_pyopenssl": false
}

</code></pre></div>
<p dir="auto">Hi,</p>
<p dir="auto">I noticed URL encode issue in get() --</p>
<p dir="auto">In [32]: r = requests.get(r'<a href="http://i.ebayimg.com/00/s/ODAzWDEyODA=/$%28KGrHqV,!o8E63YcElkoBPFMhH2vUQ~~60_1.JPG" rel="nofollow">http://i.ebayimg.com/00/s/ODAzWDEyODA=/$%28KGrHqV,!o8E63YcElkoBPFMhH2vUQ~~60_1.JPG</a>')<br>
In [33]: r.status_code<br>
Out[33]: 404</p>
<p dir="auto">In [34]: r = urllib2.urlopen("<a href="http://i.ebayimg.com/00/s/ODAzWDEyODA=/$%28KGrHqV,!o8E63YcElkoBPFMhH2vUQ~~60_1.JPG" rel="nofollow">http://i.ebayimg.com/00/s/ODAzWDEyODA=/$%28KGrHqV,!o8E63YcElkoBPFMhH2vUQ~~60_1.JPG</a>")<br>
In [35]: r.getcode()<br>
Out[35]: 200</p>
<p dir="auto">In Wireshark, I see that it's because requests further encodes the URL (path) to """/00/s/ODAzWDEyODA%3D/%24%28KGrHqV%2C%21o8E63YcElkoBPFMhH2vUQ%7E%7E60_1.JPG"""</p>
<p dir="auto">Is it a bug?</p>