<p>It would be hard to specifically ban crawlers as a practical matter because there is no reliable way to differentiate them from other normal nodes. They can, in fact, just be normal nodes that are instrumented with code to collect the desired data and report it back to an aggregation system.</p>
<p>So then you are left with banning normal nodes. Since all of the value of running a node comes from its connectivity to the P2P network via other nodes, this would not make sense.</p>

CC BY-SA 4.0
2021-05-06T20:19:38.337Z
The groups of nodes I mentioned (bitnodes, dsn.tm.kit.edu etc.), all have public user-agents (named `subver` in `getpeerinfo`), so you can (currently) ban them easily. Although it's possible that they could (start to) pretend to be normal nodes and use Bitcoin Core's `subver`s instead, the core question still remains.
cryptagoras
CC BY-SA 4.0
2021-06-05T18:27:19.023Z
Crawlers will most definitely start imitating real nodes if they'd be banned in this way.
Pieter Wuille
CC BY-SA 4.0
2021-05-06T17:35:01.043Z
2021-06-06T20:53:49.680Z
2021-06-06T20:53:49.680Z
bigjosh
<p>There are multiple entities and open-source software that crawl the Bitcoin P2P network to discover and monitor nodes for various purposes (statistics, surveys, academic papers etc.)</p>
<p>Some of those I am aware of:</p>
<ul>
<li><code>btc_node_scanner</code></li>
<li><code>dsn.tm.kit.edu</code></li>
<li><code>nodes.multiven.io</code></li>
<li><code>bitnodes.earn.com</code></li>
<li><code>bitnodes.io</code></li>
</ul>
<ol>
<li><p>Banning such nodes would free my connection slots for real useful nodes that actually provide consensus sourcing.
But, is there any downside to banning such crawler nodes?</p>
</li>
<li><p>Also, what about banning <code>btc-seeder</code> nodes? Would that affect negatively the health of the P2P network?</p>
</li>
</ol>
<hr />
<p>I think the ideal answer could provide the effects to two cases:</p>
<ul>
<li>banning done by one user and how they are affected</li>
<li>banning done by a grand/meaningful number of users and how the overall network would be affected</li>
</ul>
<p>Thank you</p>

CC BY-SA 4.0
2021-06-06T23:41:19.653Z
Do they actually stay connected as a peer for any significant length of time?  I would have though they just came and went.  Anyway, maybe you could just up your number of slots?  Because I doubt these monitoring nodes use that much network bandwidth ... or do they?
davidbak
<network><nodes><p2p>
Are there any downsides to banning node crawlers?
cryptagoras
# Are there any downsides to banning node crawlers?

(Asked by: username_0 on 2021-05-02)

There are multiple entities and open-source software that crawl the Bitcoin P2P network to discover and monitor nodes for various purposes (statistics, surveys, academic papers etc.)

Some of those I am aware of:

* `btc_node_scanner`
* `dsn.tm.kit.edu`
* `nodes.multiven.io`
* `bitnodes.earn.com`
* `bitnodes.io`

1. Banning such nodes would free my connection slots for real useful nodes that actually provide consensus sourcing.
   But, is there any downside to banning such crawler nodes?
2. Also, what about banning `btc-seeder` nodes? Would that affect negatively the health of the P2P network?

---

I think the ideal answer could provide the effects to two cases:

* banning done by one user and how they are affected
* banning done by a grand/meaningful number of users and how the overall network would be affected

Thank you

**Tags:** network, nodes, p2p

> **Comments:**
> **username_1**: Do they actually stay connected as a peer for any significant length of time?  I would have though they just came and went.  Anyway, maybe you could just up your number of slots?  Because I doubt these monitoring nodes use that much network bandwidth ... or do they?  

---

## 1 answers

### Answer by username_2 on 2021-05-06. Score: 1

It would be hard to specifically ban crawlers as a practical matter because there is no reliable way to differentiate them from other normal nodes. They can, in fact, just be normal nodes that are instrumented with code to collect the desired data and report it back to an aggregation system.

So then you are left with banning normal nodes. Since all of the value of running a node comes from its connectivity to the P2P network via other nodes, this would not make sense.

> **Comments:**
> **username_0**: The groups of nodes I mentioned (bitnodes, dsn.tm.kit.edu etc.), all have public user-agents (named `subver` in `getpeerinfo`), so you can (currently) ban them easily. Although it's possible that they could (start to) pretend to be normal nodes and use Bitcoin Core's `subver`s instead, the core question still remains.  
> **username_3**: Crawlers will most definitely start imitating real nodes if they'd be banned in this way.  

---

