This article has been written in preparation for the May 31st WASM conference organized by Supercolony. You can register to attend the virtual conference for free at this link.
Next-generation Web3 apps require a scalable data infrastructure to handle high transaction throughput and millions of new users. Subsquid is an on-chain data indexing solution that enables developers to build and deploy bespoke scalable APIs for on-chain data through its modular architecture and archival services. With upcoming support for WASM contracts on Shiden/Astar and Gear networks, Subsquid unlocks genuinely new use-cases that leverage the high throughput of WASM runtime together with the simplicity and composability of smart contracts.
What was a really silly idea back in the 2017 ICO era, like Uber or Facebook but on blockchain, is now becoming a lot more realistic thanks to much more advanced SDKs like Substrate. In much the same way, many failed ideas of the dot-com era came to be revised — and ultimately transformed into multi-billion businesses — in the 2000s, as better infrastructure and new development tooling shifted what used to be “outright impossible” to merely “very hard” to build. In this article, we will explore the possibilities for builders of blockchain applications that are unlocked by WASM. We will then discover how Subsquid accelerates the building of Dapps and other essential components of Web3.
Data indexing
Let’s look at why we need an indexing solution in the first place. For example, assume we’re building an NFT marketplace and a website for browsing. Information about the NFT trades and mints is stored on-chain in smart contracts, but it’s not feasible to fetch this information directly from a blockchain node. Indeed, challenges, such as the following, make such a design unusable for the end user:
- The data stored in the contract state is usually highly optimized for storage and is not human-readable;
- Node gRPC requests are slow and hard to scale;
- It’s impossible to filter and aggregate on-chain data.
To address the challenges above, a middleware layer is implemented that serves data to clients in a suitable form. It listens to the relevant transactions and events happening on-chain and transforms it into human-readable high-level entries in a database. The API server can then retrieve and fetch the data tailored to the business logic of the website and the UX design.
Monolithic design vs the modular architecture of Subsquid
A standard approach for indexing and transforming the on-chain data is a monolithic design, as illustrated below:
In the design above an indexing node extracts the data from the blockchain then transforms and loads it into a local database. This has a few major drawbacks:
- The indexing node has to spend significant time decoding the raw data;
- Each node needs a stable gRPC connection, making it hard to scale as the transaction throughput increases;
- Long times to catch up with the blockchain head block make frequent changes impractical and significantly slows down the development.
Subsquid has introduced a modular architecture that can be scaled to thousands of TPS and can support multiple indexing consumers.
First, raw on-chain data is collected and pre-indexed by an Archive optimized for storage and high availability. Downstream consumers then source the data from the Archive bypassing the bottleneck of a blockchain gRPC connection. Another huge advantage is the highly effective filtering and batching capabilities. For example, one can fetch event data for specific events (say, “Balances.Transfers”) that occurred in a block range from 5000000 and 51000000:
WASM: A Preview of Subsquid interfaces
WASM-based smart contracts bring about at least 10x performance improvements compared to EVM owing to native code execution. They enable a new class of performant dApps while retaining composability, the key value proposition of Ethereum and EVM-based chains.
At the same time, the development of a dApp contract doesn’t happen in a vacuum: the contract developer should determine how the contract data is going to be consumed by the UI and also have good tooling for data exploration during the development and testing. Given the significantly increased transaction throughput, it makes it challenging to index on-chain data in a naive traditional way.
Subsquid will be the first solution to offer native indexing of WASM contracts, filling the crucial gap in the infrastructure stack. Here’s a preview of the dev interfaces we at Subsquid are building for WASM.
First, we have created a special class WASMProcessor to the Subsquid SDK and have connected to Archives that support WASM contracts (e.g. Shibuya archive)
Next, we tell the processor that we want to subscribe to the events emitted by a contract deployed by a specific contract (say, a6Yrf6jAPUwjoi5YvvoTE4ES5vYAMpV55ZCsFHtwMFPDx7H)
A wasm-typegen tool provided by Subsquid will generate type-safe interfaces for the events emitted by the contract based on the metadata provided by the developer. This will help to eliminate the boilerplate code for data decoding and conversion,
Finally, we just run the processor:
This will do all the magic of connecting to the Archive, batching the event data and applying the handler logic above. And this is it!
As icing on the cake, one can deploy the processor as a Squid to the Subsquid Hosted Service and get a production-ready GraphQL endpoint.
Concluding remarks
WASM comes as a new technology set to boost the speed of smart contracts and extend the boundaries of what is possible to build. Subsquid gives a helping hand to the new developer geniuses building on WASM by allowing them to focus on the business logic, fully automating the essential parts of ingesting, transforming and serving the application data. If you would like to learn more about what Subsquid offers to developers, feel free to join our Discord Server and review our documentation. You may also get in touch with me directly on Telegram (@dzlzv).
About Subsquid
Subsquid is an on-chain indexing and querying solution that enables Web3 builders to gain access to on-chain data on their own terms. Featuring multilayer architecture and decentralised governance, this is the most developer-friendly and resource-efficient way to build, test, and deploy customised APIs for all blockchain-facing applications.
Twitter | Discord | LinkedIn | Telegram | Github | Newsletter | Subsocial
