This week, we introduce three changes that significantly speed up data-heavy indexers.

The first change adds the option to control what event data the server sends. By default, the DNA protocol attaches additional data to each event delivered over the stream. This data includes the transaction that emitted the event, together with its receipt. If the same transaction emits multiple events, this data is included multiple times. Additionally, most indexers only need some of the additional data in the transaction's receipt, which leads the DNA streams to deliver much more data than is necessary. In this release, indexers can request the server not to send events' transactions and/or receipts. This reduces the amount of data sent to the client by 2 to 10 times less, which means the client has less data to read and parse, resulting in much better performance. Update your event filters as follows to take advantage of this new feature.

const filter = {
  header: { weak: true },
  events: [
    {
      fromAddress: MY_ADDRESS,
      keys: [TRANSFER_KEY],
      includeTransaction: false, // Don't send transaction data.
      includeReceipt: false, // Don't send receipt data.
    },
  ],
};

For users using the PostgreSQL integration, it's now possible to have multiple indexers synch data to the same table. If your application requires indexing similar but unrelated data (for example, Transfer events for different ERC20 tokens), you can run multiple indexers in parallel to speed up indexing. Previously, having multiple indexers write data to the same table resulted in data loss in case of chain reorganizations (the last indexer to handle the reorg would delete the data of all other indexers). This release adds a new configuration option to add additional column constraints to invalidate queries.

export const config = {
  // ... config
  sinkOptions: {
    invalidate: [
      { column: "token_symbol", value: "ETH" },
    ],
  },
}

Parallel indexing will soon come to the MongoDB integration as well.

Finally, this release includes a new version of the Deno runtime. We changed how data is exchanged between the sink (implemented in Rust) and your script (implemented in Javascript) by taking advantage of the new #[op2] extension macro.