This EIP introduces three additional message types into the eth protocol (releasing a new version, eth/65): NewPooledTransactionHashes (0x08) to announce a set of transactions without their content; GetPooledTransactions (0x09) to request a batch of transactions by their announced hash; and PooledTransactions (0x0a) to reply to a transaction request. This permits reducing the bandwidth used for transaction propagation from linear complexity in the number of peers to square root; and also reducing the initial transaction exchange from 10s-100s MB to len(pool) * 32B ~= 128KB.
The eth network protocol has two ways to propagate a newly mined block: it can be broadcast to a peer in its entirety (via NewBlock (0x07) in eth/64 and prior or it can be announced only (via NewBlockHashes (0x01)). This duality allows nodes to do the high-bandwidth broadcasting (10s-100s KB) for a square root number of peers; and the low-bandwidth announcing (10s-100s B) for the remaining linear number of peers. The square root broadcast is enough to reach all well connected nodes, but the linear announce is needed to get across degenerate topologies. This works well.
The eth protocol, however, does not have a similar dual mechanism for propagating transactions, so nodes need to rely on broadcasting (via Transactions (0x02)). To cater for degenerate topologies, transactions cannot be broadcast square rooted, rather they need to be transferred linearly to all peers. With N peers, each node will transfer the same transaction N times (counting both directions), whereas 1 would be enough in a perfect world. This is a significant waste.
A similar issue arises when a new network connection is made between two nodes, as they need to sync up their transaction pools, but the pool is just a soup of dangling transactions. Without a way to deduplicate transactions remotely, each node is forced to naively transfer their entire list of transactions to the other side. With pools containing thousands of transactions, a naive transfer amounts to 10s-100s MB, most of which is useless. There is no better way, however.
This EIP introduces three additional message types into the eth protocol (releasing a new version, eth/65): NewPooledTransactionHashes (0x08) to announce a set of transactions without their content; GetPooledTransactions (0x09) to request a batch of transactions by their announced hash; and PooledTransactions (0x0a) to reply to a transaction request. This permits reducing the bandwidth used for transaction propagation from linear complexity in the number of peers to square root; and also reducing the initial transaction exchange from 10s-100s MB to len(pool) * 32B ~= 128KB.
With transaction throughput (and size) picking up in Ethereum, transaction propagation is the current dominant component of the used network resources. Most of these resources are however wasted, as the eth protocol does not have a mechanism to deduplicate transactions remotely, so the same data is transferred over and over again across all network connections.
This EIP proposes a tiny extension to the eth protocol, which permits nodes to agree on the set of transactions that need to be transferred across a network connection, before doing the costly exchange. This should help reduce the global (operational) bandwidth usage of the Ethereum network by at least an order of magnitude.
Add three new message types to the eth protocol:
NewPooledTransactionHashes (0x08): [hash_0: B_32, hash_1: B_32, ...]
devp2p network packet limit), but 4096 seems a sane chunk (128KB) to avoid a single packet hogging a network connection.GetPooledTransactions (0x09): [hash_0: B_32, hash_1: B_32, ...]
devp2p network packet limit), but the recipient may enforce an arbitrary cap on the reply (size or serving time), which must not be considered a protocol violation. To keep wasted bandwidth down (unanswered hashes), 256 seems like a sane upper limit.PooledTransactions (0x0a): [[nonce: P, receivingAddress: B_20, value: P, ...], ...]
GetPooledTransactions (0x09) message. The items in the list are transactions in the format described in the main Ethereum specification.Q: Why limit GetPooledTransactions (0x09) to retrieving items from the pool?
Apart from the transaction pool, transactions in Ethereum are always bundled together by the hundreds in block bodies and existing network retrievals honor this data layout. Allowing direct access to individual transactions in the database has no actionable use case, but would expose costly database reads into the network.
For transaction propagation purposes there is no reason to allow disk access, as any transaction finalized to disk will be broadcast inside a block anyway, so at worse there is a few hundred millisecond delay when a node gets the transaction.
Block propagation may be made a bit more optimal by transferring the contained transactions on demand only, but that is a whole EIP in itself, so better relax the protocol when all the requirements are known and not in advance. It would probably be enough to maintain a set of transactions included in recent blocks in memory.
Q: Should NewPooledTransactionHashes (0x08) deduplicate from disk?
Similarly to GetPooledTransactions (0x09), NewPooledTransactionHashes (0x08) should also only operate on the transaction pool and should ignore the disk altogether. During healthy network conditions, a transaction will propagate through much faster than it's included in a block, so it will essentially be non-existent that a newly announced transaction is already on disk. By avoiding disk deduplication, we can avoid a DoS griefing by remote transaction announces.
If we want to be really correct and avoid even the slightest data race when deduplicating announcements, we can use the same recently-included-transactions trick that we discussed above to discard announcements that have recently become stale.
Q: Why not reuse Transaction (0x02) instead of a new PooledTransactions (0x0a)?
Originally this EIP reused the existing Transaction (0x02) message as the reply to the GetPooledTransactions (0x09) request. This makes client code more complicated, because nodes constantly gossip Transaction (0x02) messages to each other as broadcasts, so it's hard to match up which of the many messages is the actual reply to the request.
By keeping Transaction (0x02) and PooledTransactions (0x0a) as separate messages, we can also leave the protocol more flexible for future optimizations (e.g. adding request IDs, which are meaningless for gossip broadcasts).
This EIP extends the eth protocol in a backwards incompatible way and requires rolling out a new version, eth/65. However, devp2p supports running multiple versions of the same wire protocol side-by-side, so rolling out eth/65 does not require client coordination, since non-updated clients can keep using eth/64.
This EIP does not change the consensus engine, thus does not require a hard fork.
None.
Copyright and related rights waived via CC0.