<h2>Why this is hard</h2>
<p>Bitcoin Core is chiefly a tool to synchronize with the current state of the network and to enforce the consensus rules with a self-owned piece of hardware. Beside being on the same page regarding the rules, this requires the node only to keep track of the ledger, the Unspent Transaction Output (UTXO) set. Bitcoin transactions need to be processed only once to that end.<br />
The blockchain being now over 330 GiB, some people are actually choosing to run pruned nodes (that discard old blockchain data after processing it to calculate the UTXO set). Keeping additional indexes around with dozens of GiB in size is a special interest most users don't seem to care for.<br />
In fact, Bitcoin Core does not feature a full address index out of the box. Originally maybe, because address reuse is considered a bad practice, later because other projects offer this, leaving Bitcoin Core developers feeling that they could focus on other things.</p>
<h2>How you do it anyway</h2>
<p>Bitcoin Core has an option to build a full transaction index <code>-txindex</code> (turning it on will require you to start over with synchronization and take about 30 GiB more space). This index is needed if you want to look up arbitrary transactions from the past. Other projects leverage this functionality and build additional indexing capability around it to offer Lightning Network support or Bitcoin explorers. I would recommend that you look into the <a href="https://github.com/mempool/mempool" rel="nofollow noreferrer">Mempool Open Source Project</a> or the <a href="https://github.com/blockstream/esplora" rel="nofollow noreferrer">Esplora Block Explorer</a>. Either of these will allow you to run your own instance of a block explorer with APIs to look up blockchain data on basis of addresses. Note that the Esplora index will take about 1 TiB and apparently takes quite some time to build (H/T @Anonymous).</p>

CC BY-SA 4.0
2021-03-27T15:29:50.337Z
Thank you!  FWIW, I did have `txindex=1` in my `bitcoin.conf` at the time I did the IBD, so I guess I should be good to go?  (BTW, your answer explains something that had been puzzling me: my first attempt at IBD failed for lack of space, even though 350GB had been available when I started running `bitcoind`.  Now I realize that the `txindex=1` in my `bitcoin.conf` came with a much bigger space requirement than that.
kjo
CC BY-SA 4.0
2021-03-27T15:52:09.347Z
Yeah, I think the transaction index adds another 200 GiB or so. If you had it on already, you'd be good to go indeed.
Murch
CC BY-SA 4.0
2021-03-27T15:58:16.570Z
After looking a bit into mempool and esplora, which appear to be GUI/browser-based tools, I realized that I should have spelled out that I want to run queries from the command line and/or programmatically.  I am sorry about this omission!  (I keep forgetting that those like me, who consider graphical interfaces basically useless for any real work, make up a minuscule minority.)  I have now edited my post to make this requirement clear.  (I fear, however, that, given that what I want to do appears to be difficult, this additional requirement may be a deal-breaker...)
kjo
CC BY-SA 4.0
2021-03-27T16:08:30.520Z
Oh, I just found https://github.com/Blockstream/esplora/blob/master/API.md, which you alluded to in your original reply, but I missed in my initial look-over.  (I still have not been able to find much documentation for mempool, though.)
kjo
CC BY-SA 4.0
2021-03-27T16:15:55.883Z
OK, I guess this is it for mempool: https://mempool.space/api
kjo
CC BY-SA 4.0
2021-03-27T16:33:49.187Z
@Murch The esplorer database is over a terabyte and takes many days to produce on fast hardware.
Claris
CC BY-SA 4.0
2021-03-27T16:44:44.943Z
@Anonymous: Yes, I was talking about Bitcoin Core's transaction index.
Murch
CC BY-SA 4.0
2021-03-27T16:45:12.453Z
That’s a couple of gigabytes.
Claris
CC BY-SA 4.0
2021-03-27T16:47:35.480Z
Oh, I misremembered. It seems to be around 30 GiB according to past-you: https://bitcoin.stackexchange.com/a/49715/5406
Murch
CC BY-SA 4.0
2021-03-27T12:52:29.053Z
2021-03-27T16:50:18.953Z
2021-03-27T16:50:18.953Z
Murch
<p>I have written a program that allows you to search for any address regardless if it belongs to your wallet or not, in Python3.</p>
<p>Here is the github link:</p>
<p><a href="https://github.com/ORP967/Bitcoin_Core_RPC_par_address" rel="nofollow noreferrer">https://github.com/ORP967/Bitcoin_Core_RPC_par_address</a></p>
<p>Let me know what you think and if you have any improvements.</p>

CC BY-SA 4.0
2022-01-23T22:35:46.127Z
2022-01-23T22:35:46.127Z
ORP967
<p>I recently started running <code>bitcoind</code> on a local Debian host, and I am still learning how to use the information in the local copy of the bitcoin blockchain.</p>
<p>I would like to extract all the information for one or more public bitcoin addresses currently available in the full node locally<sup> 1</sup>.  <em><strong>EDIT</strong><sup> 2</sup>: I want to do this both from the <code>bash</code> command line and programmatically.  (IOW: no GUI-based tools, please.)</em></p>
<p>I recently <a href="https://bitcoin.stackexchange.com/a/103895/115715">learned</a> that, contrary to my initial expectation, bitcoin core is not suitable for performing this sort of query.</p>
<p>Therefore, I went back to Google, but I've come up empty<sup> 3</sup>.  Hence I'm posting my question here.</p>
<p>Specifically, I would like to know</p>
<ol>
<li>what software (preferably in the form of one or more Debian packages) would I need to install on the local full node that would enable me to extract all the information for one or more public bitcoin addresses currently available in the full node locally;</li>
<li>how exactly would I use this software to perform such queries.</li>
</ol>
<hr />
<p><sup><sup>1</sup> I want to do this without reference to any specific wallet.  Having to add this requirement feels extremely weird to me (like having to add the requirement that answers to my question should work seven days a week).  This is a clear sign to me that there is still at least one pretty gigantic gap and/or confusion in my understanding of bitcoin, because I don't understand why a wallet should be <em>even relevant</em> to the kind of query I want to do.  I would have thought that a wallet would be, at most, an optional piece of information to modify the query, not an essential input for it.  But the fact is that I have not been able to find any way to interrogate the full node's database for specific addresses that does not require specifying a wallet.  This is just mind-boggling to me.</sup></p>
<p><sup><sup>2</sup> I apologize for not mentioning this critical requirement in my original post.  That was a major oversight on my part!</sup></p>
<p><sup><sup>3</sup> I am beginning to realize that the queries I want to perform on a full node's database (and without reference to a wallet) is something practically no one cares to do, and therefore it is hard to find tools for doing them.  Again, this strikes me as a bit weird, and feeds my growing suspicion that I am <em>totally</em> missing the point about bitcoin, blockchains, full nodes, etc.</sup></p>

<address><full-node><database>
How to extract all info for a public address from a local full node's db?
kjo
# How to extract all info for a public address from a local full node's db?

(Asked by: username_0 on 2021-03-27)

I recently started running `bitcoind` on a local Debian host, and I am still learning how to use the information in the local copy of the bitcoin blockchain.

I would like to extract all the information for one or more public bitcoin addresses currently available in the full node locally 1. ***EDIT** 2: I want to do this both from the `bash` command line and programmatically. (IOW: no GUI-based tools, please.)*

I recently [learned](https://bitcoin.stackexchange.com/a/103895/115715) that, contrary to my initial expectation, bitcoin core is not suitable for performing this sort of query.

Therefore, I went back to Google, but I've come up empty 3. Hence I'm posting my question here.

Specifically, I would like to know

1. what software (preferably in the form of one or more Debian packages) would I need to install on the local full node that would enable me to extract all the information for one or more public bitcoin addresses currently available in the full node locally;
2. how exactly would I use this software to perform such queries.

---

1 I want to do this without reference to any specific wallet. Having to add this requirement feels extremely weird to me (like having to add the requirement that answers to my question should work seven days a week). This is a clear sign to me that there is still at least one pretty gigantic gap and/or confusion in my understanding of bitcoin, because I don't understand why a wallet should be *even relevant* to the kind of query I want to do. I would have thought that a wallet would be, at most, an optional piece of information to modify the query, not an essential input for it. But the fact is that I have not been able to find any way to interrogate the full node's database for specific addresses that does not require specifying a wallet. This is just mind-boggling to me.

2 I apologize for not mentioning this critical requirement in my original post. That was a major oversight on my part!

3 I am beginning to realize that the queries I want to perform on a full node's database (and without reference to a wallet) is something practically no one cares to do, and therefore it is hard to find tools for doing them. Again, this strikes me as a bit weird, and feeds my growing suspicion that I am *totally* missing the point about bitcoin, blockchains, full nodes, etc.

**Tags:** address, full-node, database


---

## 2 answers

### Answer by username_1 on 2021-03-27. Score: 1 | Accepted answer

## Why this is hard

Bitcoin Core is chiefly a tool to synchronize with the current state of the network and to enforce the consensus rules with a self-owned piece of hardware. Beside being on the same page regarding the rules, this requires the node only to keep track of the ledger, the Unspent Transaction Output (UTXO) set. Bitcoin transactions need to be processed only once to that end.  
The blockchain being now over 330 GiB, some people are actually choosing to run pruned nodes (that discard old blockchain data after processing it to calculate the UTXO set). Keeping additional indexes around with dozens of GiB in size is a special interest most users don't seem to care for.  
In fact, Bitcoin Core does not feature a full address index out of the box. Originally maybe, because address reuse is considered a bad practice, later because other projects offer this, leaving Bitcoin Core developers feeling that they could focus on other things.

## How you do it anyway

Bitcoin Core has an option to build a full transaction index `-txindex` (turning it on will require you to start over with synchronization and take about 30 GiB more space). This index is needed if you want to look up arbitrary transactions from the past. Other projects leverage this functionality and build additional indexing capability around it to offer Lightning Network support or Bitcoin explorers. I would recommend that you look into the [Mempool Open Source Project](https://github.com/mempool/mempool) or the [Esplora Block Explorer](https://github.com/blockstream/esplora). Either of these will allow you to run your own instance of a block explorer with APIs to look up blockchain data on basis of addresses. Note that the Esplora index will take about 1 TiB and apparently takes quite some time to build (H/T @Anonymous).

> **Comments:**
> **username_0**: Thank you!  FWIW, I did have `txindex=1` in my `bitcoin.conf` at the time I did the IBD, so I guess I should be good to go?  (BTW, your answer explains something that had been puzzling me: my first attempt at IBD failed for lack of space, even though 350GB had been available when I started running `bitcoind`.  Now I realize that the `txindex=1` in my `bitcoin.conf` came with a much bigger space requirement than that.  
> **username_1**: Yeah, I think the transaction index adds another 200 GiB or so. If you had it on already, you'd be good to go indeed.  
> **username_0**: After looking a bit into mempool and esplora, which appear to be GUI/browser-based tools, I realized that I should have spelled out that I want to run queries from the command line and/or programmatically.  I am sorry about this omission!  (I keep forgetting that those like me, who consider graphical interfaces basically useless for any real work, make up a minuscule minority.)  I have now edited my post to make this requirement clear.  (I fear, however, that, given that what I want to do appears to be difficult, this additional requirement may be a deal-breaker...)  
> **username_0**: Oh, I just found https://github.com/Blockstream/esplora/blob/master/API.md, which you alluded to in your original reply, but I missed in my initial look-over.  (I still have not been able to find much documentation for mempool, though.)  
> **username_0**: OK, I guess this is it for mempool: https://mempool.space/api  
> **username_2**: @username_1 The esplorer database is over a terabyte and takes many days to produce on fast hardware.  
> **username_1**: @Anonymous: Yes, I was talking about Bitcoin Core's transaction index.  
> **username_2**: That’s a couple of gigabytes.  
> **username_1**: Oh, I misremembered. It seems to be around 30 GiB according to past-you: https://bitcoin.stackexchange.com/a/49715/5406  

---

### Answer by username_3 on 2022-01-23. Score: 0

I have written a program that allows you to search for any address regardless if it belongs to your wallet or not, in Python3.

Here is the github link:

<https://github.com/username_3/Bitcoin_Core_RPC_par_address>

Let me know what you think and if you have any improvements.


---

