<p dir="auto">Sometimes it is needed to parse several variants of item from single page, where only a couple of fields differ. Obvious way to do this is to create base item with all common fields collected and then create new items (for example in a loop) using new ItemLoader instance with base item passed to it.</p>
<p dir="auto">The problem I'm facing now is that ItemLoader.load_item() modifies base item - which is counterintuitive and can result in weird behavior (for example if variant items can have different fields - after those field were added to base item - they would appear in all loaded items).</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="In [5]: item = Item()

In [7]: item['url'] = 'foo'

In [8]: item
Out[8]: {'url': 'foo'}

In [9]: l = ItemLoader(item)

In [12]: l.add_value('category', 'bar')

In [13]: item
Out[13]: {'url': 'foo'}

In [14]: item_copy = l.load_item()

In [15]: item_copy
Out[15]: {'category': 'bar', 'url': 'foo'}

In [16]: item
Out[16]: {'category': 'bar', 'url': 'foo'}

In [17]: id(item)
Out[17]: 49468304

In [18]: id(item_copy)
Out[18]: 49468304"><pre class="notranslate"><code class="notranslate">In [5]: item = Item()

In [7]: item['url'] = 'foo'

In [8]: item
Out[8]: {'url': 'foo'}

In [9]: l = ItemLoader(item)

In [12]: l.add_value('category', 'bar')

In [13]: item
Out[13]: {'url': 'foo'}

In [14]: item_copy = l.load_item()

In [15]: item_copy
Out[15]: {'category': 'bar', 'url': 'foo'}

In [16]: item
Out[16]: {'category': 'bar', 'url': 'foo'}

In [17]: id(item)
Out[17]: 49468304

In [18]: id(item_copy)
Out[18]: 49468304
</code></pre></div>
<p dir="auto">Now I'm using workaround like this to suppress such behavior:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="loader = ItemLoader(selector=sel)
...
item = loader.load_item()
for variant in variants:
    item_copy = item.copy()
    loader = ItemLoader(item_copy)
    ...
    yield loader.load_item()"><pre class="notranslate"><code class="notranslate">loader = ItemLoader(selector=sel)
...
item = loader.load_item()
for variant in variants:
    item_copy = item.copy()
    loader = ItemLoader(item_copy)
    ...
    yield loader.load_item()
</code></pre></div>
<p dir="auto">What do you think about using item copy inside ItemLoader by default?</p>
<p dir="auto">Seen with Scrapy 14.0, Twisted 11.1.0, Python 2.6/2.7. After downgrade to twisted 11.0.0 the error does not show up.</p>
<p dir="auto">In my case, happened in a long running spider which doesn't perform anything unusual, and after the error the crawler hangs up.</p>
<p dir="auto">See the log below:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="2011-11-23 09:32:40-0600 [projects] DEBUG: Crawled (200) &lt;GET http://www.example.net/p/foo&gt; (referer: http://www.example.net/p?page=51903&amp;sort=users)
2011-11-23 09:32:40-0600 [projects] DEBUG: Crawled (200) &lt;GET http://www.example.net/p/bar&gt; (referer: http://www.example.net/p?page=51903&amp;sort=users)
2011-11-23 09:38:10-0600 [projects] INFO: Crawled 89600 pages (at 42 pages/min), scraped 31075 items (at 16 items/min)
2011-11-23 09:38:11-0600 [-] Unhandled Error
    Traceback (most recent call last):
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/scrapy/commands/crawl.py&quot;, line 45, in run
        self.crawler.start()
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/scrapy/crawler.py&quot;, line 76, in start
        reactor.run(installSignalHandlers=False) # blocking call
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py&quot;, line 1169, in run
        self.mainLoop()
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py&quot;, line 1178, in mainLoop
        self.runUntilCurrent()
    --- &lt;exception caught here&gt; ---
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py&quot;, line 800, in runUntilCurrent
        call.func(*call.args, **call.kw)
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/tcp.py&quot;, line 337, in failIfNotConnected
        self.connector.connectionFailed(failure.Failure(err))
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py&quot;, line 1055, in connectionFailed
        self.factory.clientConnectionFailed(self, reason)
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/web/client.py&quot;, line 413, in clientConnectionFailed
        self._disconnectedDeferred.callback(None)
    exceptions.AttributeError: ScrapyHTTPClientFactory instance has no attribute '_disconnectedDeferred'

2011-11-23 09:38:13-0600 [-] Unhandled Error
    Traceback (most recent call last):
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/scrapy/commands/crawl.py&quot;, line 45, in run
        self.crawler.start()
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/scrapy/crawler.py&quot;, line 76, in start
        reactor.run(installSignalHandlers=False) # blocking call
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py&quot;, line 1169, in run
        self.mainLoop()
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py&quot;, line 1178, in mainLoop
        self.runUntilCurrent()
    --- &lt;exception caught here&gt; ---
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py&quot;, line 800, in runUntilCurrent
        call.func(*call.args, **call.kw)
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/tcp.py&quot;, line 337, in failIfNotConnected
        self.connector.connectionFailed(failure.Failure(err))
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py&quot;, line 1055, in connectionFailed
        self.factory.clientConnectionFailed(self, reason)
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/web/client.py&quot;, line 413, in clientConnectionFailed
        self._disconnectedDeferred.callback(None)
    exceptions.AttributeError: ScrapyHTTPClientFactory instance has no attribute '_disconnectedDeferred'

2011-11-23 09:38:13-0600 [-] Unhandled Error
    Traceback (most recent call last):
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/scrapy/commands/crawl.py&quot;, line 45, in run
        self.crawler.start()
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/scrapy/crawler.py&quot;, line 76, in start
        reactor.run(installSignalHandlers=False) # blocking call
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py&quot;, line 1169, in run
        self.mainLoop()
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py&quot;, line 1178, in mainLoop
        self.runUntilCurrent()
    --- &lt;exception caught here&gt; ---
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py&quot;, line 800, in runUntilCurrent
        call.func(*call.args, **call.kw)
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/tcp.py&quot;, line 337, in failIfNotConnected
        self.connector.connectionFailed(failure.Failure(err))
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py&quot;, line 1055, in connectionFailed
        self.factory.clientConnectionFailed(self, reason)
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/web/client.py&quot;, line 413, in clientConnectionFailed
        self._disconnectedDeferred.callback(None)
    exceptions.AttributeError: ScrapyHTTPClientFactory instance has no attribute '_disconnectedDeferred'

2011-11-23 09:38:13-0600 [-] Unhandled Error
    Traceback (most recent call last):
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/scrapy/commands/crawl.py&quot;, line 45, in run
        self.crawler.start()
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/scrapy/crawler.py&quot;, line 76, in start
        reactor.run(installSignalHandlers=False) # blocking call
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py&quot;, line 1169, in run
        self.mainLoop()
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py&quot;, line 1178, in mainLoop
        self.runUntilCurrent()
    --- &lt;exception caught here&gt; ---
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py&quot;, line 800, in runUntilCurrent
        call.func(*call.args, **call.kw)
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/tcp.py&quot;, line 337, in failIfNotConnected
        self.connector.connectionFailed(failure.Failure(err))
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py&quot;, line 1055, in connectionFailed
        self.factory.clientConnectionFailed(self, reason)
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/web/client.py&quot;, line 413, in clientConnectionFailed
        self._disconnectedDeferred.callback(None)
    exceptions.AttributeError: ScrapyHTTPClientFactory instance has no attribute '_disconnectedDeferred'

2011-11-23 09:38:13-0600 [-] Unhandled Error
    Traceback (most recent call last):
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/scrapy/commands/crawl.py&quot;, line 45, in run
        self.crawler.start()
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/scrapy/crawler.py&quot;, line 76, in start
        reactor.run(installSignalHandlers=False) # blocking call
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py&quot;, line 1169, in run
        self.mainLoop()
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py&quot;, line 1178, in mainLoop
        self.runUntilCurrent()
    --- &lt;exception caught here&gt; ---
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py&quot;, line 800, in runUntilCurrent
        call.func(*call.args, **call.kw)
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/tcp.py&quot;, line 337, in failIfNotConnected
        self.connector.connectionFailed(failure.Failure(err))
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py&quot;, line 1055, in connectionFailed
        self.factory.clientConnectionFailed(self, reason)
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/web/client.py&quot;, line 413, in clientConnectionFailed
        self._disconnectedDeferred.callback(None)
    exceptions.AttributeError: ScrapyHTTPClientFactory instance has no attribute '_disconnectedDeferred'

2011-11-23 09:38:13-0600 [-] Unhandled Error
    Traceback (most recent call last):
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/scrapy/commands/crawl.py&quot;, line 45, in run
        self.crawler.start()
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/scrapy/crawler.py&quot;, line 76, in start
        reactor.run(installSignalHandlers=False) # blocking call
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py&quot;, line 1169, in run
        self.mainLoop()
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py&quot;, line 1178, in mainLoop
        self.runUntilCurrent()
    --- &lt;exception caught here&gt; ---
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py&quot;, line 800, in runUntilCurrent
        call.func(*call.args, **call.kw)
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/tcp.py&quot;, line 337, in failIfNotConnected
        self.connector.connectionFailed(failure.Failure(err))
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py&quot;, line 1055, in connectionFailed
        self.factory.clientConnectionFailed(self, reason)
      File &quot;/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/web/client.py&quot;, line 413, in clientConnectionFailed
        self._disconnectedDeferred.callback(None)
    exceptions.AttributeError: ScrapyHTTPClientFactory instance has no attribute '_disconnectedDeferred'

2011-11-23 09:38:58-0600 [projects] INFO: Crawled 89600 pages (at 0 pages/min), scraped 31075 items (at 0 items/min)
2011-11-23 09:39:58-0600 [projects] INFO: Crawled 89600 pages (at 0 pages/min), scraped 31075 items (at 0 items/min)
2011-11-23 09:40:58-0600 [projects] INFO: Crawled 89600 pages (at 0 pages/min), scraped 31075 items (at 0 items/min)"><pre class="notranslate"><code class="notranslate">2011-11-23 09:32:40-0600 [projects] DEBUG: Crawled (200) &lt;GET http://www.example.net/p/foo&gt; (referer: http://www.example.net/p?page=51903&amp;sort=users)
2011-11-23 09:32:40-0600 [projects] DEBUG: Crawled (200) &lt;GET http://www.example.net/p/bar&gt; (referer: http://www.example.net/p?page=51903&amp;sort=users)
2011-11-23 09:38:10-0600 [projects] INFO: Crawled 89600 pages (at 42 pages/min), scraped 31075 items (at 16 items/min)
2011-11-23 09:38:11-0600 [-] Unhandled Error
    Traceback (most recent call last):
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/scrapy/commands/crawl.py", line 45, in run
        self.crawler.start()
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/scrapy/crawler.py", line 76, in start
        reactor.run(installSignalHandlers=False) # blocking call
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py", line 1169, in run
        self.mainLoop()
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py", line 1178, in mainLoop
        self.runUntilCurrent()
    --- &lt;exception caught here&gt; ---
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py", line 800, in runUntilCurrent
        call.func(*call.args, **call.kw)
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/tcp.py", line 337, in failIfNotConnected
        self.connector.connectionFailed(failure.Failure(err))
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py", line 1055, in connectionFailed
        self.factory.clientConnectionFailed(self, reason)
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/web/client.py", line 413, in clientConnectionFailed
        self._disconnectedDeferred.callback(None)
    exceptions.AttributeError: ScrapyHTTPClientFactory instance has no attribute '_disconnectedDeferred'

2011-11-23 09:38:13-0600 [-] Unhandled Error
    Traceback (most recent call last):
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/scrapy/commands/crawl.py", line 45, in run
        self.crawler.start()
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/scrapy/crawler.py", line 76, in start
        reactor.run(installSignalHandlers=False) # blocking call
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py", line 1169, in run
        self.mainLoop()
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py", line 1178, in mainLoop
        self.runUntilCurrent()
    --- &lt;exception caught here&gt; ---
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py", line 800, in runUntilCurrent
        call.func(*call.args, **call.kw)
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/tcp.py", line 337, in failIfNotConnected
        self.connector.connectionFailed(failure.Failure(err))
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py", line 1055, in connectionFailed
        self.factory.clientConnectionFailed(self, reason)
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/web/client.py", line 413, in clientConnectionFailed
        self._disconnectedDeferred.callback(None)
    exceptions.AttributeError: ScrapyHTTPClientFactory instance has no attribute '_disconnectedDeferred'

2011-11-23 09:38:13-0600 [-] Unhandled Error
    Traceback (most recent call last):
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/scrapy/commands/crawl.py", line 45, in run
        self.crawler.start()
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/scrapy/crawler.py", line 76, in start
        reactor.run(installSignalHandlers=False) # blocking call
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py", line 1169, in run
        self.mainLoop()
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py", line 1178, in mainLoop
        self.runUntilCurrent()
    --- &lt;exception caught here&gt; ---
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py", line 800, in runUntilCurrent
        call.func(*call.args, **call.kw)
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/tcp.py", line 337, in failIfNotConnected
        self.connector.connectionFailed(failure.Failure(err))
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py", line 1055, in connectionFailed
        self.factory.clientConnectionFailed(self, reason)
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/web/client.py", line 413, in clientConnectionFailed
        self._disconnectedDeferred.callback(None)
    exceptions.AttributeError: ScrapyHTTPClientFactory instance has no attribute '_disconnectedDeferred'

2011-11-23 09:38:13-0600 [-] Unhandled Error
    Traceback (most recent call last):
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/scrapy/commands/crawl.py", line 45, in run
        self.crawler.start()
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/scrapy/crawler.py", line 76, in start
        reactor.run(installSignalHandlers=False) # blocking call
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py", line 1169, in run
        self.mainLoop()
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py", line 1178, in mainLoop
        self.runUntilCurrent()
    --- &lt;exception caught here&gt; ---
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py", line 800, in runUntilCurrent
        call.func(*call.args, **call.kw)
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/tcp.py", line 337, in failIfNotConnected
        self.connector.connectionFailed(failure.Failure(err))
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py", line 1055, in connectionFailed
        self.factory.clientConnectionFailed(self, reason)
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/web/client.py", line 413, in clientConnectionFailed
        self._disconnectedDeferred.callback(None)
    exceptions.AttributeError: ScrapyHTTPClientFactory instance has no attribute '_disconnectedDeferred'

2011-11-23 09:38:13-0600 [-] Unhandled Error
    Traceback (most recent call last):
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/scrapy/commands/crawl.py", line 45, in run
        self.crawler.start()
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/scrapy/crawler.py", line 76, in start
        reactor.run(installSignalHandlers=False) # blocking call
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py", line 1169, in run
        self.mainLoop()
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py", line 1178, in mainLoop
        self.runUntilCurrent()
    --- &lt;exception caught here&gt; ---
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py", line 800, in runUntilCurrent
        call.func(*call.args, **call.kw)
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/tcp.py", line 337, in failIfNotConnected
        self.connector.connectionFailed(failure.Failure(err))
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py", line 1055, in connectionFailed
        self.factory.clientConnectionFailed(self, reason)
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/web/client.py", line 413, in clientConnectionFailed
        self._disconnectedDeferred.callback(None)
    exceptions.AttributeError: ScrapyHTTPClientFactory instance has no attribute '_disconnectedDeferred'

2011-11-23 09:38:13-0600 [-] Unhandled Error
    Traceback (most recent call last):
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/scrapy/commands/crawl.py", line 45, in run
        self.crawler.start()
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/scrapy/crawler.py", line 76, in start
        reactor.run(installSignalHandlers=False) # blocking call
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py", line 1169, in run
        self.mainLoop()
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py", line 1178, in mainLoop
        self.runUntilCurrent()
    --- &lt;exception caught here&gt; ---
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py", line 800, in runUntilCurrent
        call.func(*call.args, **call.kw)
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/tcp.py", line 337, in failIfNotConnected
        self.connector.connectionFailed(failure.Failure(err))
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/internet/base.py", line 1055, in connectionFailed
        self.factory.clientConnectionFailed(self, reason)
      File "/home/josemanuel/envs/global/lib/python2.7/site-packages/twisted/web/client.py", line 413, in clientConnectionFailed
        self._disconnectedDeferred.callback(None)
    exceptions.AttributeError: ScrapyHTTPClientFactory instance has no attribute '_disconnectedDeferred'

2011-11-23 09:38:58-0600 [projects] INFO: Crawled 89600 pages (at 0 pages/min), scraped 31075 items (at 0 items/min)
2011-11-23 09:39:58-0600 [projects] INFO: Crawled 89600 pages (at 0 pages/min), scraped 31075 items (at 0 items/min)
2011-11-23 09:40:58-0600 [projects] INFO: Crawled 89600 pages (at 0 pages/min), scraped 31075 items (at 0 items/min)
</code></pre></div>