This standard introduces a flexible on-chain storage pattern that organizes data into structured tables that consist of records with fixed key and value schemas, similar to a traditional database. This storage pattern consists of a unified contract interface for data access, along with a compact binary encoding format for both static and dynamic data types. State changes are tracked through standardized events that enable automatic, schema-aware state replication by off-chain indexers. New tables can be dynamically registered at runtime through a special table that stores schema metadata for all tables, allowing the system to evolve without breaking existing contracts or integrations.
The absence of consistent standards for on-chain data management in smart contracts can lead to rigid implementations, tightly coupled contract logic with off-chain services, and challenges in updating or extending a contract’s data layout without breaking existing integrations.
Using the storage mechanism defined in this ERC provides the following benefits:
A smart contract that implements the interface proposed by this ERC and organizes data in Tables. It emits events for each data operation so that off-chain components can replicate the state of all tables.
A storage structure that holds Records sharing the same Schema.
A piece of data stored in a Table, addressed by one or more keys.
ResourceIdA 32-byte value that uniquely identifies each Table within the Store.
Encoding:
| Bytes (from left to right) | Description |
|---|---|
| 0-1 | Table type identifier |
| 2-31 | Unique identifier |
Table Type Identifiers:
0x7462 ("tb") for on-chain tables0x6f74 ("ot") for off-chain tablesSchemaUsed to represent the layout of Records within a table.
Each Table defines two schemas:
| Byte(s) from left to right | Value | Constraint |
|---|---|---|
| 0-1 | Total byte length of static fields | |
| 2 | Number of static length fields | ≤ (28 - number of dynamic length fields) |
| 3 | Number of dynamic length fields | For the key schema, 0 |
| For the value schema, ≤5 | ||
| 4-31 | Each byte encodes a SchemaType | Dynamic-length types MUST come after all the static-length types. |
SchemaTypeSingle byte that represents the type of a specific static or dynamic field.
Type Encoding:
| Value Range | Type |
|---|---|
0x00 to 0x1F | uint8 to uint256 (increments of 8 bits) |
0x20 to 0x3F | int8 to int256 (increments of 8 bits) |
0x40 to 0x5F | bytes1 to bytes32 |
0x60 | bool |
0x61 | address |
0x62 to 0x81 | uint8[] to uint256[] |
0x82 to 0xA1 | int8[] to int256[] |
0xA2 to 0xC1 | bytes1[] to bytes32[] |
0xC2 | bool[] |
0xC3 | address[] |
0xC4 | bytes |
0xC5 | string |
FieldLayoutEncodes the concrete value Schema information, specifically the total byte length of the static fields, the number of dynamic fields and the length of each static field on its own.
This encoding serves as an optimization for on-chain operations. By having the exact lengths readily available, the Store doesn't need to repeatedly compute or translate the schema definitions into actual field lengths during execution.
| Byte(s) from left to right | Value | Constraint |
|---|---|---|
| 0-1 | Total length of static fields | |
| 2 | Number of static length fields | ≤ (28 - number of dynamic length fields) |
| 3 | Number of dynamic length fields | For the key schema, 0 |
| For the value schema, ≤5 | ||
| 4-31 | Each byte encodes the byte length of the corresponding static field |
EncodedLengthsEncodes the byte length of all the dynamic fields of a specific Record. It is returned by the Store methods when reading a Record, as it is needed for decoding dynamic fields.
| Bytes (from least to most significant) | Type | Description |
|---|---|---|
| 0x00-0x06 | uint56 | Total byte length of dynamic data |
| 0x07-0xB | uint40 | Length of the first dynamic field |
| 0x0C-0x10 | uint40 | Length of the second dynamic field |
| 0x11-0x15 | uint40 | Length of the third dynamic field |
| 0x16-0x1A | uint40 | Length of the fourth dynamic field |
| 0x1B-0x1F | uint40 | Length of the fifth dynamic field |
Record data returned by Store methods and included in Store events uses the following encoding rules.
Schema type structure, which uses 28 bytes (bytes 4 to 31) to define field types, with one byte per field (SchemaType).EncodedLengths) to encode the byte lengths of each dynamic field, instead of encoding each length separately as Solidity’s abi.encode would.abi.encodePacked.bytes, and strings):
EncodedLengths.Suppose a table has the following value schema:
Encoding (Pseudocode):
All Stores MUST implement the following interface.
The return values of both getRecord and getField use the encoding rules previously defined in the Packed Data Encoding section. More specifically, getRecord returns the fully encoded record tuple, and the data returned by getField is encoded using the encoding rules as if the field was being encoded on its own.
This standard defines three core operations for manipulating records in a table: setting, updating, and deleting. For each operation, specific events must be emitted. The implementation details of these operations are left to the discretion of each Store implementation.
The fundamental requirement is that for on-chain tables the Record data retrieved through the Store interface methods at any given block MUST be consistent with the Record data that would be obtained by applying the operations implied by the Store events up to that block. This ensures data integrity and allows for accurate off-chain state reconstruction.
Setting a Record means overwriting all of its fields. This operation can be performed whether the record has been set before or not (the standard does not enforce existence checks).
The Store_SetRecord event MUST be emitted whenever the full data of a record has been overwritten.
Parameters:
| Name | Type | Description |
|---|---|---|
| tableId | ResourceId | The ID of the table where the record is set |
| keyTuple | bytes32[] | An array representing the composite key for the record |
| staticData | bytes | The static data of the record using packed encoding |
| encodedLengths | EncodedLengths | The encoded lengths of the dynamic data of the record |
| dynamicData | bytes | The dynamic data of the record, using custom packed encoding |
Splicing the static data of a Record consists in overwriting bytes of the packed encoded static fields. The total length of static data does not change as it is determined by the table’s value schema.
The Store_SpliceStaticData event MUST be emitted whenever the static data of the Record has been spliced.
Parameters:
| Name | Type | Description |
|---|---|---|
| tableId | ResourceId | The ID of the table where the data is spliced |
| keyTuple | bytes32[] | An array representing the key for the record |
| start | uint48 | The start position in bytes for the splice operation |
| data | bytes | Packed ABI encoding of a tuple with the value's static fields |
Splicing the dynamic data of a Record involves modifying the packed encoded representation of its dynamic fields by removing, replacing, and/or inserting new bytes in place.
The Store_SpliceDynamicData event MUST be emitted whenever the dynamic data of the Record has been spliced.
Parameters:
| Name | Type | Description |
|---|---|---|
| tableId | ResourceId | The ID of the table where the data is spliced |
| keyTuple | bytes32[] | An array representing the composite key for the record |
| dynamicFieldIndex | uint8 | The index of the dynamic field to splice data, relative to the start of the dynamic fields (Dynamic field index = field index - number of static fields) |
| start | uint48 | The start position in bytes for the splice operation |
| deleteCount | uint40 | The number of bytes to delete in the splice operation |
| encodedLengths | EncodedLengths | The resulting encoded lengths of the dynamic data of the record |
| data | bytes | The data to insert into the dynamic data of the record at the start byte |
The Store_DeleteRecord event MUST be emitted whenever the Record has been deleted from the Table.
Parameters:
| Name | Type | Description |
|---|---|---|
| tableId | ResourceId | The ID of the table where the record is deleted |
| keyTuple | bytes32[] | An array representing the composite key for the record |
See the reference implementation section for an example on how to index store events.
Tables tableTo keep track of the information of each table and support registering new tables at runtime, the Store implementation MUST include a special on-chain Tables table, which behaves the same way as other on-chain tables except for the special constraints mentioned below.
The Tables table MUST use the following Schemas:
tableId (ResourceId): ResourceId of the table this record describes.fieldLayout (FieldLayout): encodes the byte length of each static data type in the table.keySchema (Schema): represents the data types of the (composite) key of the table.valueSchema (Schema): represents the data types of the value fields of the table.abiEncodedKeyNames (bytes): ABI encoded string array of key names.abiEncodedFieldNames (bytes): ABI encoded string array of field names.Records stored in the Tables table are considered immutable:
Store MUST emit a single Store_SetRecord event for each table being registered.Store SHOULD NOT emit any other Store events for a Table registered in the Tables table.The Tables table MUST store a record that describes itself before any other table is registered, emitting the corresponding Store_SetRecord event. The record must use the following tableId:
By using a predefined ResourceId and Schema for the Tables table, off-chain indexers can interpret store events for all registered tables. This enables the development of advanced off-chain services that operate on structured data rather than raw encoded data like in the previous indexer implementation example.
While the Store_SetRecord event suffices for tracking the data of each record off-chain, including Splice events (Store_SpliceStaticData and Store_SpliceDynamicData) allows for more efficient partial updates. When only a portion of a record changes, emitting a full SetRecord event would be inefficient because the entire record data would need to be read from storage and emitted. Splice events enable the store to emit only the minimal necessary data for the update, reducing gas consumption. This is particularly important for records with large dynamic fields, as the cost of updating them doesn’t grow with the field’s size.
Arrays of dynamic types (e.g., string[], bytes[]) are intentionally not included as supported SchemaTypes. This restriction enforces a flat data schema, which simplifies the store implementation and enhances efficiency. If users need to store such data structures, they can model them using a separate table with a schema like { index: uint256, data: bytes }, where each array element is represented as an individual record.
Including the FieldLayout in the Tables schema provides an on-chain optimization by precomputing and storing the exact byte lengths of static fields. This eliminates the need to repeatedly compute field lengths and offsets during runtime, which can be gas-intensive. By having this information readily available, the store can perform storage operations more efficiently, while components reading from the store can retrieve it from the Tables table to decode the corresponding records.
Tables tableIncluding a special Tables table provides significant benefits for off-chain indexers. While emitting events for table registration isn't strictly necessary for basic indexers that operate on raw encoded data, doing so makes indexers aware of the schemas used by each table. This awareness enables the development of more advanced, schema-aware indexer APIs (e.g., SQL-like query capabilities), enhancing the utility and flexibility of off-chain data interactions.
By reusing existing Store abstractions for table registration, we also simplify the implementation and eliminate the need for additional, specific table registration events. Indexers can leverage the standard Store events to access schema information, ensuring consistency and reducing complexity.
The following example shows how a simple in-memory indexer can use the Store events to replicate the Store state off-chain. It is important to note that this indexer operates over raw encoded data which is not that useful on its own, but can be improved as we will explain in the next section.
We use TypeScript for this example but it can easily be replicated with other languages.
This standard only defines functions to read from the Store (getRecord, getField, and getFieldLength). The methods for setting or modifying records in the store are left to each specific implementation. Therefore, implementations must provide appropriate access control mechanisms for writing to the store, tailored to their specific use cases.
All data stored within a store is accessible not only off-chain but also on-chain by other smart contracts through the provided read functions (getRecord, getField, and getFieldLength). This differs from the typical behavior of smart contracts, where internal storage variables are private by default and cannot be directly read by other contracts unless explicit getter functions are provided. Thus, developers must be mindful that any data stored in the store is openly accessible to other smart contracts.
Copyright and related rights waived via CC0.