How much does the extra pool help with missing transactions for compact block re...

A concern brought up in the context of block content diverging from the default mempool policy is the reduced compact block reconstruction rates. In this context, I have seen the extra pool brought up. A frequent claim is that a sufficiently large extra pool would hold most of the missing transactions and therefore mitigate the hit to the block reconstruction from the node not having a large portion of the block content.

An example for such a claim can be seen in an answer to Implications of OP_RETURN changes in upcoming Bitcoin Core version 30.0:

• Compact Block relay latency: There is certainly a way to keep transactions in memory and not relay them. blockreconstructionextratxn is a config that already implements this with a default of 100 txs, users/small-miners can increase that value as needed if it's a concern. In fact, Bitcoin Knots 29.1 ships with a much higher default and also adds additional config called blockreconstructionextratxnsize which can be used to do same but by allocating memory size instead of just number of transactions.

How effective would we expect the extra pool to be at facilitating fast block reconstruction in the context of a node operating with mempool policies that filter 20–40% of transactions appearing in blocks?

    def test_policy_rejection_types(self):
        """Test that each policy rejection type adds transactions to extra pool."""
        self.log.info("Testing policy rejection types for extra pool...")

        self.restart_node_with_limit(count=100)

        rejection_types = ["dust", "low_fee", "op_return_size", "nonstandard_script"]
        rejected_txs = []

        for rejection_type in rejection_types:
            self.log.info(f"Testing {rejection_type} rejection...")
            tx_info = self.create_policy_rejected_tx(rejection_type)

            tx_obj = tx_from_hex(tx_info['hex'])
            self.segwit_node.send_message(msg_tx(tx_obj))

            rejected_txs.append({
                'type': rejection_type,
                'tx_info': tx_info,
                'txid': tx_info['tx'].hash,
                'wtxid': tx_info['tx'].getwtxid()
            })

        self.segwit_node.sync_with_ping()

        mempool = self.nodes[0].getrawmempool()
        for rejected in rejected_txs:
            assert_equal(rejected['txid'] in mempool, False)
            self.log.info(f"✓ {rejected['type']} transaction rejected from mempool")

        indices = list(range(len(rejected_txs)))
        tx_list = [r['tx_info'] for r in rejected_txs]

        result = self.send_compact_block(tx_list, indices)

        assert_equal(result["missing_indices"], [])
        self.log.info("✓ All rejected transactions are available in extra pool")

    def test_extratxnpool_disabled(self):
        """Test that setting count to 0 disables the extra transaction pool."""
        self.log.info("Testing disabled extra transaction pool (0 capacity)...")

        self.restart_node_with_limit(count=0)
        buffersize = 5
        rejected_txs = self.populate_extra_pool(buffersize)

        indices = list(range(buffersize))
        result = self.send_compact_block(rejected_txs, indices)
        assert_equal(result["missing_indices"], indices)
        self.log.info(f"✓ All {buffersize} transactions are missing (extra txn pool disabled)")

These functional tests prove that extra pool helps in compact block reconstruction. With the increased limit for blockreconstructionextratxn and addition of blockreconstructionextratxnsize in v29.1, 20-40% rejected transactions can be cached in memory and used for block reconstruction.

Note: Extra pool evictions follow FIFO and it stores rejected transactions. So, an attacker can fill the extra pool with irrelevant transactions at no cost. It will affect the memory usage and compact block reconstruction.