<h3 dir="auto">Description</h3>
<p dir="auto">When using scrapy-redis or self-defined dupefilters, the speed of item pipeline will be extremely slow. It's not Redis's problem because when I move the deduplication to DownloadMiddleware (check url finger in process_request, add url finger in process_response), the speed of item pipeline returns to normal. Here are two links recording detailed information.<br>
<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="663590938" data-permission-text="Title is private" data-url="https://github.com/rmax/scrapy-redis/issues/174" data-hovercard-type="issue" data-hovercard-url="/rmax/scrapy-redis/issues/174/hovercard" href="https://github.com/rmax/scrapy-redis/issues/174">rmax/scrapy-redis#174</a><br>
<a href="https://stackoverflow.com/questions/63026873/scarpy-redis-slows-down-item-pipelines" rel="nofollow">https://stackoverflow.com/questions/63026873/scarpy-redis-slows-down-item-pipelines</a></p>
<h3 dir="auto">Steps to Reproduce</h3>
<ol dir="auto">
<li>set DUPEFILTER_CLASS to "project.filename.customize-dupefilter-class"</li>
</ol>
<p dir="auto"><strong>Expected behavior:</strong>  Item pipeline's speed should not varies dramartically</p>
<p dir="auto"><strong>Actual behavior:</strong> Item pipeline's speed is extremely slow</p>
<p dir="auto"><strong>Reproduces how often:</strong> Always</p>
<h3 dir="auto">Versions</h3>
<p dir="auto">Scrapy       : 2.2.0<br>
lxml         : 4.5.2.0<br>
libxml2      : 2.9.10<br>
cssselect    : 1.1.0<br>
parsel       : 1.6.0<br>
w3lib        : 1.22.0<br>
Twisted      : 20.3.0<br>
Python       : 3.7.7 (default, Mar 10 2020, 15:43:33) - [Clang 11.0.0 (clang-1100.0.33.17)]<br>
pyOpenSSL    : 19.1.0 (OpenSSL 1.1.1g  21 Apr 2020)<br>
cryptography : 2.9.2<br>
Platform     : Darwin-19.5.0-x86_64-i386-64bit</p>
<h3 dir="auto">headers lowercase</h3>
<p dir="auto">[scrapy can not work when headers are lowercase ]</p>
<h3 dir="auto">Steps to Reproduce</h3>
<ol dir="auto">
<li>i must use lowercase headers key and I use Twisted mapping headers and it doesn't work</li>
</ol>
<p dir="auto"><strong>Expected behavior:</strong> scrapy run with no exception</p>
<p dir="auto"><strong>Actual behavior:</strong> scrapy and twisted does nothing for headers</p>
<p dir="auto"><strong>Reproduces how often:</strong> [What percentage of the time does it reproduce?]</p>
<h3 dir="auto">Versions</h3>
<p dir="auto">Scrapy       : 2.0.1<br>
lxml         : 4.4.1.0<br>
libxml2      : 2.9.5<br>
cssselect    : 1.1.0<br>
parsel       : 1.5.2<br>
w3lib        : 1.21.0<br>
Twisted      : 19.7.0<br>
Python       : 3.7.4 (tags/v3.7.4:e09359112e, Jul  8 2019, 20:34:20) [MSC v.1916 64 bit (AMD64)]<br>
pyOpenSSL    : 19.0.0 (OpenSSL 1.1.1c  28 May 2019)<br>
cryptography : 2.7<br>
Platform     : Windows-10-10.0.18362-SP0</p>
<h3 dir="auto">Additional context</h3>
<p dir="auto">I try to rewrite the Request and Headers class but it gives me a line <code class="notranslate">Please use app access!</code></p>