<p dir="auto">Originally reported <a href="https://stackoverflow.com/questions/42966405/scrapy-post-header-has-two-content-length-fields" rel="nofollow">on StackOverflow</a>.</p>
<p dir="auto">HTTP requests with POST method and no request body are sent with 2 <code class="notranslate">Content-Length: 0</code> headers.</p>
<p dir="auto">Reproducible with Scrapy 1.3.3 and Twisted 17.1:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="$ scrapy version -v
Scrapy    : 1.3.3
lxml      : 3.7.3.0
libxml2   : 2.9.3
cssselect : 1.0.1
parsel    : 1.1.0
w3lib     : 1.17.0
Twisted   : 17.1.0
Python    : 2.7.12 (default, Nov 19 2016, 06:48:10) - [GCC 5.4.0 20160609]
pyOpenSSL : 16.2.0 (OpenSSL 1.0.2g  1 Mar 2016)
Platform  : Linux-4.8.0-41-generic-x86_64-with-Ubuntu-16.10-yakkety
$ scrapy shell
&gt;&gt;&gt; fetch(scrapy.Request('http://httpbin.org/post', method='POST'))
2017-03-23 10:58:34 [scrapy.core.engine] DEBUG: Crawled (200) &lt;POST http://httpbin.org/post&gt; (referer: None)"><pre class="notranslate"><code class="notranslate">$ scrapy version -v
Scrapy    : 1.3.3
lxml      : 3.7.3.0
libxml2   : 2.9.3
cssselect : 1.0.1
parsel    : 1.1.0
w3lib     : 1.17.0
Twisted   : 17.1.0
Python    : 2.7.12 (default, Nov 19 2016, 06:48:10) - [GCC 5.4.0 20160609]
pyOpenSSL : 16.2.0 (OpenSSL 1.0.2g  1 Mar 2016)
Platform  : Linux-4.8.0-41-generic-x86_64-with-Ubuntu-16.10-yakkety
$ scrapy shell
&gt;&gt;&gt; fetch(scrapy.Request('http://httpbin.org/post', method='POST'))
2017-03-23 10:58:34 [scrapy.core.engine] DEBUG: Crawled (200) &lt;POST http://httpbin.org/post&gt; (referer: None)
</code></pre></div>
<p dir="auto">(note that httpbin.org/post output does not show duplicate headers, but the Wireshark capture does)</p>
<p dir="auto">Wireshark sniffing:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="POST /post HTTP/1.1
Content-Length: 0
Content-Length: 0
Accept-Language: en
Accept-Encoding: gzip,deflate
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
User-Agent: Scrapy/1.3.3 (+http://scrapy.org)
Host: httpbin.org

HTTP/1.1 200 OK
Connection: keep-alive
Server: gunicorn/19.7.1
Date: Thu, 23 Mar 2017 09:58:34 GMT
Content-Type: application/json
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true
Content-Length: 458
Via: 1.1 vegur

{
  &quot;args&quot;: {}, 
  &quot;data&quot;: &quot;&quot;, 
  &quot;files&quot;: {}, 
  &quot;form&quot;: {}, 
  &quot;headers&quot;: {
    &quot;Accept&quot;: &quot;text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8&quot;, 
    &quot;Accept-Encoding&quot;: &quot;gzip,deflate&quot;, 
    &quot;Accept-Language&quot;: &quot;en&quot;, 
    &quot;Connection&quot;: &quot;close&quot;, 
    &quot;Content-Length&quot;: &quot;0&quot;, 
    &quot;Host&quot;: &quot;httpbin.org&quot;, 
    &quot;User-Agent&quot;: &quot;Scrapy/1.3.3 (+http://scrapy.org)&quot;
  }, 
  &quot;json&quot;: null, 
  &quot;origin&quot;: &quot;89.84.122.217&quot;, 
  &quot;url&quot;: &quot;http://httpbin.org/post&quot;
}"><pre class="notranslate"><code class="notranslate">POST /post HTTP/1.1
Content-Length: 0
Content-Length: 0
Accept-Language: en
Accept-Encoding: gzip,deflate
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
User-Agent: Scrapy/1.3.3 (+http://scrapy.org)
Host: httpbin.org

HTTP/1.1 200 OK
Connection: keep-alive
Server: gunicorn/19.7.1
Date: Thu, 23 Mar 2017 09:58:34 GMT
Content-Type: application/json
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true
Content-Length: 458
Via: 1.1 vegur

{
  "args": {}, 
  "data": "", 
  "files": {}, 
  "form": {}, 
  "headers": {
    "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8", 
    "Accept-Encoding": "gzip,deflate", 
    "Accept-Language": "en", 
    "Connection": "close", 
    "Content-Length": "0", 
    "Host": "httpbin.org", 
    "User-Agent": "Scrapy/1.3.3 (+http://scrapy.org)"
  }, 
  "json": null, 
  "origin": "89.84.122.217", 
  "url": "http://httpbin.org/post"
}
</code></pre></div>
<p dir="auto">This is due to Twisted's <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="199833525" data-permission-text="Title is private" data-url="https://github.com/twisted/twisted/issues/670" data-hovercard-type="pull_request" data-hovercard-url="/twisted/twisted/pull/670/hovercard" href="https://github.com/twisted/twisted/pull/670">twisted/twisted#670</a> since v17.1 and Scrapy's <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="135129222" data-permission-text="Title is private" data-url="https://github.com/scrapy/scrapy/issues/1800" data-hovercard-type="pull_request" data-hovercard-url="/scrapy/scrapy/pull/1800/hovercard" href="https://github.com/scrapy/scrapy/pull/1800">#1800</a> since Scrapy 1.1.0</p>
<h3 dir="auto">Description</h3>
<p dir="auto">When trying to install scrapy 2.3.0. using poetry , poetry fails with a [RecursionError].</p>
<ul dir="auto">
<li>This might be due to dependency cycles in the scrapy package.</li>
</ul>
<h3 dir="auto">Steps to Reproduce</h3>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="mkdir test
cd test
poetry init
poetry add scrapy=2.3.0 #scrapy=2.2.0 works!
poetry lock"><pre class="notranslate"><code class="notranslate">mkdir test
cd test
poetry init
poetry add scrapy=2.3.0 #scrapy=2.2.0 works!
poetry lock
</code></pre></div>
<p dir="auto">fails with:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Updating dependencies
Resolving dependencies... (6.2s)

[RecursionError]
maximum recursion depth exceeded while calling a Python object"><pre class="notranslate"><code class="notranslate">Updating dependencies
Resolving dependencies... (6.2s)

[RecursionError]
maximum recursion depth exceeded while calling a Python object
</code></pre></div>
<p dir="auto"><strong>Expected behavior:</strong> [What you expect to happen]</p>
<p dir="auto">Should install normally, (just like scrapy 2.2.0 works...)</p>
<p dir="auto"><strong>Actual behavior:</strong> [What actually happens]</p>
<p dir="auto">see above</p>
<p dir="auto"><strong>Reproduces how often:</strong> [What percentage of the time does it reproduce?]</p>
<p dir="auto">100%</p>
<h3 dir="auto">Versions</h3>
<p dir="auto">Please paste here the output of executing <code class="notranslate">scrapy version --verbose</code> in the command line.</p>
<p dir="auto">not applicable</p>
<h3 dir="auto">Additional context</h3>
<p dir="auto">this is the relevant project:</p>
<ul dir="auto">
<li><a href="https://github.com/python-poetry/poetry">https://github.com/python-poetry/poetry</a></li>
</ul>
<p dir="auto">Any additional information, configuration, data or output from commands that might be necessary to reproduce or understand the issue. Please try not to include screenshots of code or the command line, paste the contents as text instead. You can use <a href="https://help.github.com/en/articles/creating-and-highlighting-code-blocks">GitHub Flavored Markdown</a> to make the text look better.</p>