<p>I got it :_)</p>
<p>I used binary.ReadUvarint which not the same as ReadVarint in the Bitcoin Core (<a href="https://github.com/bitcoin/bitcoin/blob/0d509bab45d292caeaf34600e57b5928757c6005/src/serialize.h#L465" rel="nofollow noreferrer">https://github.com/bitcoin/bitcoin/blob/0d509bab45d292caeaf34600e57b5928757c6005/src/serialize.h#L465</a>)</p>
<p>So far I have not been able to explain the reason for this behavior, but i found the right code on the btcd (<a href="https://github.com/btcsuite/btcd/blob/ae5533602c46f4ea14a4da725c19cb3db7d5eb88/blockchain/compress.go#L93" rel="nofollow noreferrer">https://github.com/btcsuite/btcd/blob/ae5533602c46f4ea14a4da725c19cb3db7d5eb88/blockchain/compress.go#L93</a>) and adapted it to my use case.</p>
<pre class="lang-go prettyprint-override"><code>func DeserializeVLQ(buf io.ByteReader) (uint64, int, error) {
    var n uint64
    var size int

    for {
        val, err := buf.ReadByte()
        if err != nil {
            if err == io.EOF {
                break // End of stream is a natural break
            }
            return 0, 0, err // Return any error that occurs
        }

        size++
        n = (n &lt;&lt; 7) | uint64(val&amp;0x7F)
        if val&amp;0x80 != 0x80 {
            break
        }
        n++
    }

    return n, size, nil
}
</code></pre>

CC BY-SA 4.0
2024-04-03T14:46:30.260Z
2024-04-03T14:46:30.260Z
Novak
<p>I have a problem with reading tx out from the Bitcoin Core LevelDB chainstate storage. I know that the structure of the current version of the chainstate is <code>key =&gt; value</code> where the key is a <code>prefix (0x43) | tx_id | coin_n</code> (<a href="https://github.com/bitcoin/bitcoin/blob/0d509bab45d292caeaf34600e57b5928757c6005/src/txdb.cpp#L39" rel="nofollow noreferrer">https://github.com/bitcoin/bitcoin/blob/0d509bab45d292caeaf34600e57b5928757c6005/src/txdb.cpp#L39</a>)  and the value is obfuscated by the obfuscation key through the <code>Coin XOR obfuscation_key</code> value of the <code>Coin</code>.</p>
<p>Ref: <a href="https://github.com/bitcoin/bitcoin/blob/0d509bab45d292caeaf34600e57b5928757c6005/src/txdb.cpp#L68" rel="nofollow noreferrer">https://github.com/bitcoin/bitcoin/blob/0d509bab45d292caeaf34600e57b5928757c6005/src/txdb.cpp#L68</a></p>
<p>So, i have successfully read the prefix, obfuscation key and the txId with the <code>output_n</code></p>
<pre><code>key:
    prefix: [1]byte

(COutPoint)
    txID: [32]byte
    outN: VARINT
</code></pre>
<p>Ref: <a href="https://github.com/bitcoin/bitcoin/blob/0d509bab45d292caeaf34600e57b5928757c6005/src/primitives/transaction.h#L28" rel="nofollow noreferrer">https://github.com/bitcoin/bitcoin/blob/0d509bab45d292caeaf34600e57b5928757c6005/src/primitives/transaction.h#L28</a></p>
<p>The Coin structure (<a href="https://github.com/bitcoin/bitcoin/blob/0d509bab45d292caeaf34600e57b5928757c6005/src/coins.h#L24" rel="nofollow noreferrer">https://github.com/bitcoin/bitcoin/blob/0d509bab45d292caeaf34600e57b5928757c6005/src/coins.h#L24</a>)</p>
<pre><code>code: VARINT((coinbase ? 1 : 0) | (height &lt;&lt; 1))
txOut:  CTxOut (via TxOutCompression)
</code></pre>
<p>Now i stucked on the reading the code. In some values, i have a valid code with true block height and coinbase boolean. But in some values i have invalid data where the block height is not the same with the transaction's block height.</p>
<p>How you can reproduce this to help me?</p>
<p>My input (LevelDB view):</p>
<pre><code>Key                                                                     | Value
0e006f62667573636174655f6b6579                                          | 08f05564aa8317fe24
42                                                                      | bde0dff15a88e0eb3d71c37fd986c4cdfd8916bd0b8d0b13343afef66829284d
4300c4626bf6dc028730d3a1f3951d398a71a164d8a898965d11bb8a699091144c00    | 8f6778aa979ddbbc669c187cbf11ad2e30ca53025f3f3b961e
43047ae5966a25bdc32ab4a258c57e28ffa763fc273c08b38fe85839e99e71402500    | ef6778aa979ddbbc669c187cbf11ad2e30ca53025f3f3b961e
430e3afb85bb66cc8a5edfd96f4aad1e65e4e6cf38f1dcc584c525d0b25b89599c00    | 706e56b68303740168c3add6552bf877fa95fb9d2bcbd6e142bb
</code></pre>
<p>When i am reading the first values and deobfuscate them, i am getting true block height and coinbase. But when i am reading the last value, i am getting this:</p>
<pre><code>~ 706e56b68303740168c3add6552bf877fa95fb9d2bcbd6e142bb XOR f05564aa8317fe24
~ 803b321c00148a259896c97cd63c06530ac09f37a8dc28c5b2ee
</code></pre>
<p>So, when i am trying to read the uvarint from this, i am getting an invalid code with invalid block height:</p>
<pre class="lang-go prettyprint-override"><code>package main

import (
    &quot;bytes&quot;
    &quot;encoding/binary&quot;
    &quot;encoding/hex&quot;
    &quot;log&quot;
)

func main() {
    utxo, _ := hex.DecodeString(&quot;803b321c00148a259896c97cd63c06530ac09f37a8dc28c5b2ee&quot;)
    utxoReader := bytes.NewReader(utxo)

    code, _ := binary.ReadUvarint(utxoReader)
    log.Println(&quot;code&quot;, code)
    
    blockHeight := code &gt;&gt; 1
    log.Println(&quot;blockHeight&quot;, blockHeight)
    
    isCoinbase := code|1 == 1
    log.Println(&quot;isCoinbase&quot;, isCoinbase)
}
</code></pre>
<p>Result</p>
<pre><code>2024/04/03 15:24:06 code 7552
2024/04/03 15:24:06 blockHeight 3776
2024/04/03 15:24:06 isCoinbase false
</code></pre>
<p>Expected result: blockHeight must be <code>93</code>, coinbase <code>false</code>.</p>
<p>Other information. My deobfuscate function code:</p>
<pre class="lang-go prettyprint-override"><code>func deobfuscateValue(obfuscationKey, value []byte) []byte {
    deobfuscatedValue := make([]byte, len(value))

    copy(deobfuscatedValue, value)

    for i, c := range value {
        deobfuscatedValue[i] = c ^ obfuscationKey[i%len(obfuscationKey)]
    }

    return deobfuscatedValue
}
</code></pre>
<p>I am running the private regtest network on the Bitcoin Core v0.21 (i have tried on the 24 version and have the same results). I am use bitcoin-testnet-box docker image to test my program. Also, i have tested on the mainnet, but have the same problem too. My blockchain now have only 105 blocks.</p>
<p>Thank you for helping. I think I missed something simple and just don't know something about the chainstate or C++ realization or Golang :)</p>

CC BY-SA 4.0
2024-04-03T14:29:20.910Z
It seems like i am using binary.ReadUvarint which is not right function to read the code value
Novak
<bitcoin-core><utxo><leveldb><go-lang>
I can't parse the UTXO from the chainstate
Novak
# I can't parse the UTXO from the chainstate

(Asked by: username_0 on 2024-04-03)

I have a problem with reading tx out from the Bitcoin Core LevelDB chainstate storage. I know that the structure of the current version of the chainstate is key => value where the key is a prefix (0x43) | tx_id | coin_n (https://github.com/bitcoin/bitcoin/blob/0d509bab45d292caeaf34600e57b5928757c6005/src/txdb.cpp#L39)  and the value is obfuscated by the obfuscation key through the Coin XOR obfuscation_key value of the Coin.
Ref: https://github.com/bitcoin/bitcoin/blob/0d509bab45d292caeaf34600e57b5928757c6005/src/txdb.cpp#L68
So, i have successfully read the prefix, obfuscation key and the txId with the output_n
key:
    prefix: [1]byte

(COutPoint)
    txID: [32]byte
    outN: VARINT

Ref: https://github.com/bitcoin/bitcoin/blob/0d509bab45d292caeaf34600e57b5928757c6005/src/primitives/transaction.h#L28
The Coin structure (https://github.com/bitcoin/bitcoin/blob/0d509bab45d292caeaf34600e57b5928757c6005/src/coins.h#L24)
code: VARINT((coinbase ? 1 : 0) | (height << 1))
txOut:  CTxOut (via TxOutCompression)

Now i stucked on the reading the code. In some values, i have a valid code with true block height and coinbase boolean. But in some values i have invalid data where the block height is not the same with the transaction's block height.
How you can reproduce this to help me?
My input (LevelDB view):
Key                                                                     | Value
0e006f62667573636174655f6b6579                                          | 08f05564aa8317fe24
42                                                                      | bde0dff15a88e0eb3d71c37fd986c4cdfd8916bd0b8d0b13343afef66829284d
4300c4626bf6dc028730d3a1f3951d398a71a164d8a898965d11bb8a699091144c00    | 8f6778aa979ddbbc669c187cbf11ad2e30ca53025f3f3b961e
43047ae5966a25bdc32ab4a258c57e28ffa763fc273c08b38fe85839e99e71402500    | ef6778aa979ddbbc669c187cbf11ad2e30ca53025f3f3b961e
430e3afb85bb66cc8a5edfd96f4aad1e65e4e6cf38f1dcc584c525d0b25b89599c00    | 706e56b68303740168c3add6552bf877fa95fb9d2bcbd6e142bb

When i am reading the first values and deobfuscate them, i am getting true block height and coinbase. But when i am reading the last value, i am getting this:
~ 706e56b68303740168c3add6552bf877fa95fb9d2bcbd6e142bb XOR f05564aa8317fe24
~ 803b321c00148a259896c97cd63c06530ac09f37a8dc28c5b2ee

So, when i am trying to read the uvarint from this, i am getting an invalid code with invalid block height:
package main

import (
    "bytes"
    "encoding/binary"
    "encoding/hex"
    "log"
)

func main() {
    utxo, _ := hex.DecodeString("803b321c00148a259896c97cd63c06530ac09f37a8dc28c5b2ee")
    utxoReader := bytes.NewReader(utxo)

    code, _ := binary.ReadUvarint(utxoReader)
    log.Println("code", code)
    
    blockHeight := code >> 1
    log.Println("blockHeight", blockHeight)
    
    isCoinbase := code|1 == 1
    log.Println("isCoinbase", isCoinbase)
}

Result
2024/04/03 15:24:06 code 7552
2024/04/03 15:24:06 blockHeight 3776
2024/04/03 15:24:06 isCoinbase false

Expected result: blockHeight must be 93, coinbase false.
Other information. My deobfuscate function code:
func deobfuscateValue(obfuscationKey, value []byte) []byte {
    deobfuscatedValue := make([]byte, len(value))

    copy(deobfuscatedValue, value)

    for i, c := range value {
        deobfuscatedValue[i] = c ^ obfuscationKey[i%len(obfuscationKey)]
    }

    return deobfuscatedValue
}

I am running the private regtest network on the Bitcoin Core v0.21 (i have tried on the 24 version and have the same results). I am use bitcoin-testnet-box docker image to test my program. Also, i have tested on the mainnet, but have the same problem too. My blockchain now have only 105 blocks.
Thank you for helping. I think I missed something simple and just don't know something about the chainstate or C++ realization or Golang :)

**Tags:** bitcoin-core, utxo, leveldb, go-lang

> **Comments:**
> **username_0**: It seems like i am using binary.ReadUvarint which is not right function to read the code value  

---

## 1 answers

### Answer by username_0 on 2024-04-03. Score: 3

I got it :_)
I used binary.ReadUvarint which not the same as ReadVarint in the Bitcoin Core (https://github.com/bitcoin/bitcoin/blob/0d509bab45d292caeaf34600e57b5928757c6005/src/serialize.h#L465)
So far I have not been able to explain the reason for this behavior, but i found the right code on the btcd (https://github.com/btcsuite/btcd/blob/ae5533602c46f4ea14a4da725c19cb3db7d5eb88/blockchain/compress.go#L93) and adapted it to my use case.
func DeserializeVLQ(buf io.ByteReader) (uint64, int, error) {
    var n uint64
    var size int

    for {
        val, err := buf.ReadByte()
        if err != nil {
            if err == io.EOF {
                break // End of stream is a natural break
            }
            return 0, 0, err // Return any error that occurs
        }

        size++
        n = (n << 7) | uint64(val&0x7F)
        if val&0x80 != 0x80 {
            break
        }
        n++
    }

    return n, size, nil
}


---

