EIP-1153: Transient storage opcodes

Add opcodes for manipulating state that behaves almost identically to storage but is discarded after every transaction

Metadata

Status: FinalStandards Track: CoreCreated: 2018-06-15

Authors

Alexey Akhunov (@AlexeyAkhunov), Moody Salem (@moodysalem)

Requires

EIP-2200, EIP-3529

Links

GitHub page Discussion on Ethereum Magicians

Abstract

This proposal introduces transient storage opcodes, which manipulate state that behaves identically to storage, except that transient storage is discarded after every transaction, and TSTORE is not subject to the gas stipend check as defined in EIP-2200. In other words, the values of transient storage are never deserialized from storage or serialized to storage. Thus transient storage is cheaper since it never requires disk access. Transient storage is accessible to smart contracts via 2 new opcodes, TLOAD and TSTORE, where “T” stands for "transient:"

Motivation

Running a transaction in Ethereum can generate multiple nested frames of execution, each created by CALL (or similar) instructions. Contracts can be re-entered during the same transaction, in which case there are more than one frame belonging to one contract. Currently, these frames can communicate in two ways: via inputs/outputs passed via CALL instructions, and via storage updates. If there is an intermediate frame belonging to another untrusted contract, communication via inputs/outputs is not secure. Notable example is a reentrancy lock which cannot rely on the intermediate frame to pass through the state of the lock. Communication via storage (SSTORE/SLOAD) is costly. Transient storage is a dedicated and gas efficient solution to the problem of inter frame communication.

Storage refunds accumulated due to inter frame communication are also limited to 20% of gas spent by a transaction due to EIP-3529 (introduced in the London hard fork). This greatly reduces the refunds for transiently-set storage slots in otherwise low-cost transactions. For example, in order to receive the full refund of one re-entrancy lock, the transaction must spend ~80k gas on other operations.

Language support could be added in relatively easy way. For example, in Solidity, a qualifier transient can be introduced (similar to the existing qualifiers memory and storage, and Java's own transient keyword with a similar meaning). Since the addressing scheme of TSTORE and TLOAD is the same as for SSTORE and SLOAD, code generation routines that exist for storage variables, can be easily generalised to also support transient storage.

Potential use cases enabled or improved by this EIP include:

Reentrancy locks
On-chain computable CREATE2 addresses: constructor arguments are read from the factory contract instead of passed as part of init code hash
Single transaction ERC-20 approvals, e.g. #temporaryApprove(address spender, uint256 amount)
Fee-on-transfer contracts: pay a fee to a token contract to unlock transfers for the duration of a transaction
"Till" pattern: allowing users to perform all actions as part of a callback, and checking the "till" is balanced at the end
Proxy call metadata: pass additional metadata to an implementation contract without using calldata, e.g. values of immutable proxy constructor arguments

These opcodes are more efficient to execute than the SSTORE and SLOAD opcodes because the original value never needs to be loaded from storage (i.e. is always 0). The gas accounting rules are also simpler, since no refunds are required.

Specification

Two new opcodes are added to EVM, TLOAD (0x5c) and TSTORE (0x5d). (Note that previous drafts of this EIP specified the values 0xb3 and 0xb4 for TLOAD and TSTORE respectively to avoid conflict with other EIPs. The conflict has since been removed.)

They use the same arguments on stack as SLOAD (0x54) and SSTORE (0x55).

TLOAD pops one 32-byte word from the top of the stack, treats this value as the address, fetches 32-byte word from the transient storage at that address, and pushes the value on top of the stack.

TSTORE pops two 32-byte words from the top of the stack. The word on the top is the address, and the next is the value. TSTORE saves the value at the given address in the transient storage.

Addressing is the same as SLOAD and SSTORE. i.e. each 32-byte address points to a unique 32-byte word.

Gas cost for TSTORE is the same as a warm SSTORE of a dirty slot (i.e. original value is not new value and is not current value, currently 100 gas), and gas cost of TLOAD is the same as a hot SLOAD (value has been read before, currently 100 gas). Gas cost cannot be on par with memory access due to transient storage's interactions with reverts.

All values in transient storage are discarded at the end of the transaction.

Transient storage is private to the contract that owns it, in the same way as persistent storage. Only owning contract frames may access their transient storage. And when they do, all the frames access the same transient store, in the same way as persistent storage, but unlike memory.

When transient storage is used in the context of DELEGATECALL or CALLCODE, then the owning contract of the transient storage is the contract that issued DELEGATECALL or CALLCODE instruction (the caller) as with persistent storage. When transient storage is used in the context of CALL or STATICCALL, then the owning contract of the transient storage is the contract that is the target of the CALL or STATICCALL instruction (the callee).

If a frame reverts, all writes to transient storage that took place between entry to the frame and the return are reverted, including those that took place in inner calls. This mimics the behavior of persistent storage.

If the TSTORE opcode is called within the context of a STATICCALL, it will result in an exception instead of performing the modification. TLOAD is allowed within the context of a STATICCALL.

The behavior of the opcodes for transient storage differs from the opcodes for storage in that TSTORE does not require gasleft, as defined in EIP-2200, to be less than or equal to the gas stipend (currently 2,300).

Rationale

Another option to solve the problem of inter-frame communication is repricing the SSTORE and SLOAD opcodes to be cheaper for the transient storage use case. This has already been done as of EIP-2200. However, EIP-3529 reduced the maximum refund to only 20% of the transaction gas cost, which means the use of transient storage is severely limited.

Another approach is to keep the refund counter for transient storage separate from the refund counter for other storage uses, and remove the refund cap for transient storage. However, that approach is more complex to implement and understand. For example, the 20% refund cap must be applied to the gas used after subtracting the uncapped gas refund. Otherwise, the refund amount available subject to the 20% refund cap could be increased by executing transient storage writes. Thus it is preferable to have a separate mechanism that does not interact with the refund counter. Future hard forks can remove the complex refund behavior meant to support the transient storage use case, encouraging migration to contracts that are more efficient for the Ethereum clients to execute.

There is a known objection to the word-addressed storage-like interface of the TSTORE and TLOAD opcodes since transient storage is more akin to memory than storage in lifecycle. A byte-addressed memory-like interface is another option. The storage-like word-addressed interface is preferred due to the usefulness of mappings in combination with the transaction-scoped memory region. Often times, you will need to keep transient state with arbitrary keys, such as in the ERC-20 temporary approval use case which uses a mapping of (owner, spender) to allowance. Mappings are difficult to implement using linear memory, and linear memory must also have dynamic gas costs. It is also more complicated to handle reverts with a linear memory. It is possible to have a memory-like interface while the underlying implementation uses a map to allow for storage in arbitrary offsets, but this would result in a third memory-storage hybrid interface that would require new code paths in compilers.

Some think that a unique transaction identifier may obviate the need for transient storage as described in this EIP. This is a misconception: a transaction identifier used in combination with regular storage has all the same issues that motivate this EIP. The two features are orthogonal.

Relative cons of this transient storage EIP:

Does not address transient usages of storage in existing contracts
New code in the clients
New concept for the yellow paper (more to update)

Relative pros of this transient storage EIP:

Transient storage opcodes are considered separately in protocol upgrades and not inadvertently broken (e.g. EIP-3529)
Clients do not need to load the original value
No upfront gas cost to account for non-transient writes
Does not change the semantics of the existing operations
No need to clear storage slots after usage
Simpler gas accounting rules
Future storage designs (e.g. Verkle tree) do not need to account for transient storage refunds

Backwards Compatibility

This EIP requires a hard fork to implement.

Since this EIP does not change behavior of any existing opcodes, it is backwards compatible with all existing smart contracts.

Test Cases

A test suite for this EIP can be found here.

Reference Implementation

Because the transient storage must behave almost identically to storage within the context of a single transaction with regards to revert behavior, it is necessary to be able to revert to a previous state of transient storage within a transaction. At the same time reverts are exceptional cases and loads, stores and returns should be cheap.

A map of current state plus a journal of all changes and a list of checkpoints is recommended. This has the following time complexities:

On entry to a call frame, a call marker is added to the list - O(1)
New values are written to the current state, and the previous value is written to the journal - O(1)
When a call exits successfully, the marker to the journal index of when that call was entered is discarded - O(1)
On revert all entries are reverted up to the last checkpoint, in reverse - O(N) where N = number of journal entries since last checkpoint

The worst case time complexity can be produced by writing the maximum number of keys that can fit in one block, and then reverting. In this case, the client is required to do twice as many writes to apply all the entries in the journal. However, the same case applies to the state journaling implementation of existing clients, and cannot be DOS'd with the following code.

Security Considerations

TSTORE presents a new way to allocate memory on a node with linear cost. In other words, each TSTORE allows the developer to store 32 bytes for 100 gas, excluding any other required operations to prepare the stack. Given 30 million gas, the maximum amount of memory that can be allocated using TSTORE is:

Given the same amount of gas, the maximum amount of memory that can be allocated in a single context by MSTORE is ~3.75MB:

However, if you only spend 1M gas allocating memory in each context, and make calls to reset the memory expansion cost, you can allocate ~700KB per million gas, for a total of ~20MB of memory allocated:

Smart contract developers should understand the lifetime of transient storage variables before use. Because transient storage is automatically cleared at the end of the transaction, smart contract developers may be tempted to avoid clearing slots as part of a call in order to save gas. However, this could prevent further interactions with the contract in the same transaction (e.g. in the case of re-entrancy locks) or cause other bugs, so smart contract developers should be careful to only leave transient storage slots with nonzero values when those slots are intended to be used by future calls within the same transaction. Otherwise, these opcodes behave exactly the same as SSTORE and SLOAD, so all the usual security considerations apply especially in regard to reentrancy risk.

Smart contract developers may also be tempted to use transient storage as an alternative to in-memory mappings. They should be aware that transient storage is not discarded when a call returns or reverts, as is memory, and should prefer memory for these use cases so as not to create unexpected behavior on reentrancy in the same transaction. The necessarily high cost of transient storage over memory should already discourage this usage pattern. Most usages of in-memory mappings can be better implemented with key-sorted lists of entries, and in-memory mappings are rarely required in smart contracts (i.e. the author knows of no known use cases in production).