<blockquote>
<p>For &quot;num_items = read_varint(s)&quot;, there should be a &quot;result += encode_varint(len(tx_in.witness))&quot;.</p>
</blockquote>
<p>The corresponding expression seems to be the <code>result += int_to_little_endian(len(tx_in.witness), 1)</code>, though that's <strong>WRONG</strong>, as it only works up to length of 252 witness stack elements (for more than 252 elements, varint encoding is more than a single byte encoding). Of course, such long witness stacks are pretty rare, so this probably works almost always in practice.</p>
<blockquote>
<p>For &quot;item_len = read_varint(s)&quot;, there should be a &quot;result += encode_varint(len(item))&quot;.</p>
</blockquote>
<p>The corresponding expression seems to be the first part of <code>result += encode_varint(len(item)) + item</code>, which also serializes the item iteself.</p>
<blockquote>
<p>And for &quot;if item_len == 0:&quot;, I don't know what this if do and where the corresponding serialization logic is.</p>
</blockquote>
<p>It appears that this code supports two ways of encoding the witness stack items: they can either be integers or byte arrays. The Bitcoin script language only has one data type (byte arrays), but there are a number of opcodes that interpret these byte arrays as numbers. The encoding for the number 0 is the empty byte array.</p>
<p>What this code does:</p>
<ul>
<li>On encoding:
<ul>
<li>The number 0 gets serialized as the empty array (<code>result += int_to_little_endian(item, 1)</code>Th). That's correct.</li>
<li>Other integers get serialized as 1-byte encodings of the respective integer. This is <strong>WRONG</strong>. For small integers, the correct encoding is two bytes long.</li>
<li>Byte arrays are serialized as varlen encoding of their length, followed by the byte array itself. This is correct.</li>
</ul>
</li>
<li>On decoding:
<ul>
<li>An encoding of the empty byte array is converted to the number 0.</li>
<li>Everything else is left as a byte array.</li>
</ul>
</li>
</ul>
<p>Overall, this looks like clumsily written that nobody should use in production. It probably works within a few simple cases (never more than 252 stack elements, and no integers other than 0 being encoded), but wouldn't pass rigorous testing.</p>

CC BY-SA 4.0
2022-07-20T15:52:14.807Z
2022-07-20T15:52:14.807Z
Pieter Wuille
<p>I'm reading <a href="https://github.com/jimmysong/programmingbitcoin" rel="nofollow noreferrer">Programming Bitcoin</a>. And in Chapter13, I am confused with how the witness part is serialized and parsed. The auther gives the following method to parse it:</p>
<pre><code>def parse_segwit(cls, s, testnet=False):
    version = little_endian_to_int(s.read(4))
    marker = s.read(2)
    if marker != b'\x00\x01':  
        raise RuntimeError('Not a segwit transaction {}'.format(marker))
    num_inputs = read_varint(s)
    inputs = []
    for _ in range(num_inputs):
        inputs.append(TxIn.parse(s))
    num_outputs = read_varint(s)
    outputs = []
    for _ in range(num_outputs):
        outputs.append(TxOut.parse(s))
    for tx_in in inputs:  
        num_items = read_varint(s)
        items = []
        for _ in range(num_items):
            item_len = read_varint(s)
            if item_len == 0:        # where I can't understand 
                items.append(0)      #
            else:
                items.append(s.read(item_len))
        tx_in.witness = items
    locktime = little_endian_to_int(s.read(4))
    return cls(version, inputs, outputs, locktime, 
               testnet=testnet, segwit=True)
</code></pre>
<p>And the following code to serialize it:</p>
<pre><code>def serialize_segwit(self):
    result = int_to_little_endian(self.version, 4)
    result += b'\x00\x01'  
    result += encode_varint(len(self.tx_ins))
    for tx_in in self.tx_ins:
        result += tx_in.serialize()
    result += encode_varint(len(self.tx_outs))
    for tx_out in self.tx_outs:
        result += tx_out.serialize()
    for tx_in in self.tx_ins:  
        result += int_to_little_endian(len(tx_in.witness), 1)
        for item in tx_in.witness:
            if type(item) == int:
                result += int_to_little_endian(item, 1)
            else:
                result += encode_varint(len(item)) + item
    result += int_to_little_endian(self.locktime, 4)
    return result
</code></pre>
<p>What I can't understand are the last for loops in both methods. In my understanding, these are where segwit part get serialized and deserialized, and these codes should be in one-to-one correspondence. As they do in the first half part. But they don't match in the for loops of the last half part. I mean:</p>
<ul>
<li>For &quot;num_items = read_varint(s)&quot;, there should be a &quot;result += encode_varint(len(tx_in.witness))&quot;.</li>
<li>For &quot;item_len = read_varint(s)&quot;, there should be a &quot;result += encode_varint(len(item))&quot;.</li>
<li>And for &quot;if item_len == 0:&quot;, I don't know what this if do and where the corresponding serialization logic is.</li>
</ul>
<p>So is anything wrong with my understanding of the segwit part logic?</p>

<segregated-witness><programming-bitcoin-book>
How does the SegWit part of a transaction get serialized and parsed?
Shawn C
# How does the SegWit part of a transaction get serialized and parsed?

(Asked by: username_0 on 2022-07-20)

I'm reading Programming Bitcoin. And in Chapter13, I am confused with how the witness part is serialized and parsed. The auther gives the following method to parse it:
def parse_segwit(cls, s, testnet=False):
    version = little_endian_to_int(s.read(4))
    marker = s.read(2)
    if marker != b'\x00\x01':  
        raise RuntimeError('Not a segwit transaction {}'.format(marker))
    num_inputs = read_varint(s)
    inputs = []
    for _ in range(num_inputs):
        inputs.append(TxIn.parse(s))
    num_outputs = read_varint(s)
    outputs = []
    for _ in range(num_outputs):
        outputs.append(TxOut.parse(s))
    for tx_in in inputs:  
        num_items = read_varint(s)
        items = []
        for _ in range(num_items):
            item_len = read_varint(s)
            if item_len == 0:        # where I can't understand 
                items.append(0)      #
            else:
                items.append(s.read(item_len))
        tx_in.witness = items
    locktime = little_endian_to_int(s.read(4))
    return cls(version, inputs, outputs, locktime, 
               testnet=testnet, segwit=True)

And the following code to serialize it:
def serialize_segwit(self):
    result = int_to_little_endian(self.version, 4)
    result += b'\x00\x01'  
    result += encode_varint(len(self.tx_ins))
    for tx_in in self.tx_ins:
        result += tx_in.serialize()
    result += encode_varint(len(self.tx_outs))
    for tx_out in self.tx_outs:
        result += tx_out.serialize()
    for tx_in in self.tx_ins:  
        result += int_to_little_endian(len(tx_in.witness), 1)
        for item in tx_in.witness:
            if type(item) == int:
                result += int_to_little_endian(item, 1)
            else:
                result += encode_varint(len(item)) + item
    result += int_to_little_endian(self.locktime, 4)
    return result

What I can't understand are the last for loops in both methods. In my understanding, these are where segwit part get serialized and deserialized, and these codes should be in one-to-one correspondence. As they do in the first half part. But they don't match in the for loops of the last half part. I mean:

For "num_items = read_varint(s)", there should be a "result += encode_varint(len(tx_in.witness))".
For "item_len = read_varint(s)", there should be a "result += encode_varint(len(item))".
And for "if item_len == 0:", I don't know what this if do and where the corresponding serialization logic is.

So is anything wrong with my understanding of the segwit part logic?

**Tags:** segregated-witness, programming-bitcoin-book


---

## 1 answers

### Answer by username_1 on 2022-07-20. Score: 4 | Accepted answer

> For "num\_items = read\_varint(s)", there should be a "result += encode\_varint(len(tx\_in.witness))".

The corresponding expression seems to be the `result += int_to_little_endian(len(tx_in.witness), 1)`, though that's **WRONG**, as it only works up to length of 252 witness stack elements (for more than 252 elements, varint encoding is more than a single byte encoding). Of course, such long witness stacks are pretty rare, so this probably works almost always in practice.

> For "item\_len = read\_varint(s)", there should be a "result += encode\_varint(len(item))".

The corresponding expression seems to be the first part of `result += encode_varint(len(item)) + item`, which also serializes the item iteself.

> And for "if item\_len == 0:", I don't know what this if do and where the corresponding serialization logic is.

It appears that this code supports two ways of encoding the witness stack items: they can either be integers or byte arrays. The Bitcoin script language only has one data type (byte arrays), but there are a number of opcodes that interpret these byte arrays as numbers. The encoding for the number 0 is the empty byte array.

What this code does:

* On encoding:
  + The number 0 gets serialized as the empty array (`result += int_to_little_endian(item, 1)`Th). That's correct.
  + Other integers get serialized as 1-byte encodings of the respective integer. This is **WRONG**. For small integers, the correct encoding is two bytes long.
  + Byte arrays are serialized as varlen encoding of their length, followed by the byte array itself. This is correct.
* On decoding:
  + An encoding of the empty byte array is converted to the number 0.
  + Everything else is left as a byte array.

Overall, this looks like clumsily written that nobody should use in production. It probably works within a few simple cases (never more than 252 stack elements, and no integers other than 0 being encoded), but wouldn't pass rigorous testing.


---

