Application development

Most applications that run on a Substrate blockchain require some form of front-end or user-facing interface—such as a browser, desktop, mobile, or hardware client—that enables users or other programs to access and modify the data that the blockchain stores. For example, you might develop a browser-based application for interactive gaming or a hardware-specific application to implement a hardware wallet. Different libraries exist to build these types of applications, depending on your needs. This article explains the process of querying a Substrate node and using the metadata it exposes to help you understand how you can use the metadata when creating front-end client applications and using client-specific libraries.

Metadata system

Substrate nodes provide an RPC call, state_getMetadata, that returns a complete description of all the types in the current runtime. Client applications use the metadata to interact with the node, to parse responses, and to format message payloads sent to the node. This metadata includes information about a pallet's storage items, transactions, events, errors, and constants. The current metadata version (V14) differs significantly from its predecessors as it contains much richer type information. If a runtime includes a pallet with a custom type, the type information is included as part of the metadata returned. Polkadot uses V14 metadata starting from runtime spec version 9110 at block number 7229126 and Kusama from runtime spec version 9111, at block number 9625129. This is useful to know for developers who intend to interact with runtimes that use older metadata versions. Refer to this document for a migration guide from V13 to V14.

The current metadata schema uses the scale-info crate to get type information for the pallets in the runtime when you compile a node.

The current implementation of the metadata requires front-end APIs to use the SCALE codec library to encode and decode RPC payloads to send and receive transactions. The following steps summarize how metadata is generated, exposed, and used to make and receive calls from the runtime:

  • Callable pallet functions, as well as types, parameters and documentation are exposed by the runtime.
  • The frame-metadata crate describes the structure in which the information about how to communicate with the runtime will be provided. The information takes the form of a type registry provided by scale-info, as well as information about things like which pallets exist (and what the relevant types in the registry are for each pallet).
  • The scale-info crate is used to annotate types across the runtime, and makes it possible to build a registry of runtime types. This type information is detailed enough that we can use it to find out how to correctly SCALE encode or decode some value for a given type.
  • The structure described in frame-metadata is populated with information from the runtime, and this is then SCALE encoded and made available via the state_getMetadata RPC call.
  • Custom RPC APIs use the metadata interface and provide methods to make calls into the runtime. A SCALE codec library is required to encode and decode calls and data to and from the API.

Every Substrate chain stores the version number of the metadata system they are using, which makes it useful for applications to know how to handle the metadata exposes by a certain block. As previously mentioned, the latest metadata version (V14) provides a major enhancement to the metadata that a chain is able to generate. But what if an application wants to interact with blocks that were created with an earlier version than V14? Well, it would require setting up a front-end interface that follows the older metadata system, whereby custom types would need to be identified and manually included as part of the front-end's code. Learn how to use the desub tool to accomplish this if you needed.

Type information bundled in the metadata gives applications the ability to communicate with nodes across different chains, each of which may each expose different calls, events, types and storage. It also allows libraries to generate almost all of the code needed to communicate with a given Substrate node, giving the possibility for libraries like subxt to generate front-end interfaces that are specific to a target chain.

With this system, any runtime can be queried for its available runtime calls, types and parameters.
The metadata also exposes how a type is expected to be decoded, making it easier for an external application to retrieve and process this information.

Metadata format

Querying the state_getMetadata RPC function will return a vector of SCALE-encoded bytes which is decoded using the frame-metadata and parity-scale-codec libraries.

The hex blob returned by the state_getMetadata RPC depends on the metadata version, however will generally have the following structure:

  • a hard-coded magic number, 0x6d657461, which represents "meta" in plain text.
  • a 32 bit integer representing the version of the metadata format in use, for example 14 or 0x0e in hex.
  • hex encoded type and metadata information. In V14, this part would contain a registry of type information (generated by the scale-info crate). In previous versions, this part contained the number of pallets followed by the metadata each pallet exposes.

Here is a condensed version of decoded metadata for a runtime using the V14 metadata system (generated using subxt):

[
  1635018093, // the magic number
  {
    "V14": {
      // the metadata version
      "types": {
        // type information
        "types": []
      },
      "pallets": [
        // metadata exposes by pallets
      ],
      "extrinsic": {
        // the format of an extrinsic  and its signed extensions
        "ty": 111,
        "version": 4, // the transaction version used to encode and decode an extrinsic
        "signed_extensions": []
      },
      "ty": 125 // the type ID for the system pallet
    }
  }
]

As described above, the integer 1635018093 is a "magic number" that represents "meta" in plain text. The rest of the metadata has two sections: pallets and extrinsic. The pallets section contains information about the runtime's pallets, while the extrinsic section describes the version of extrinsics that the runtime is using. Different extrinsic versions may have different formats, especially when considering signed transactions.

Pallets

Here is a condensed example of a single element in the pallets array:

{
  "name": "System", // name of the pallet, the System pallet for example
  "storage": {
    // storage entries
  },
  "calls": [
    // index for this pallet's call types
  ],
  "event": [
    // index for this pallet's event types
  ],
  "constants": [
    // pallet constants
  ],
  "error": [
    // index for this pallet's error types
  ],
  "index": 0 // the index of the pallet in the runtime
}

Every element contains the name of the pallet that it represents, as well as a storage object, calls array, event array, and error array. If calls or events are empty, they will be represented as null and if constants or errors are empty, they will be represented as an empty array.

Type indices for each item are just u32 integers used to access the type information for that item. For example, the type ID for the calls in the System pallet is 145. Querying the type ID will give you information about the available calls of the system pallet including the documentation for each call. For each field, you can access type information and metadata for:

  • Storage metadata: provides blockchain clients with the information that is required to query the storage RPC to get information for a specific storage item.
  • Call metadata: includes information about the runtime calls are defined by the #[pallet] macro including call names, arguments and documentation.
  • Event metadata: provides the metadata generated by the #[pallet::event] macro, including the name, arguments and documentation for a pallet's events
  • Constants metadata provides metadata generated by the #[pallet::constant] macro, including the name, type and hex encoded value of the constant.
  • Error metadata: provides metadata generated by the #[pallet::error] macro, including the name and documentation for each error type in that pallet.

Note that the IDs used aren't stable over time: they will likely change from one version jump to the next, meaning that developers should avoid relying on fixed type IDs to future proof their applications.

Extrinsic

Exrinsic metadata is generated by the runtime and provides useful information on how a transaction is formatted. The returned decoded metadata contains the transaction version and signed extensions, which looks like this:

      "extrinsic": {
        "ty": 111,
        "version": 4,
        "signed_extensions": [
          {
            "identifier": "CheckSpecVersion",
            "ty": 117,
            "additional_signed": 4
          },
          {
            "identifier": "CheckTxVersion",
            "ty": 118,
            "additional_signed": 4
          },
          {
            "identifier": "CheckGenesis",
            "ty": 119,
            "additional_signed": 9
          },
          {
            "identifier": "CheckMortality",
            "ty": 120,
            "additional_signed": 9
          },
          {
            "identifier": "CheckNonce",
            "ty": 122,
            "additional_signed": 34
          },
          {
            "identifier": "CheckWeight",
            "ty": 123,
            "additional_signed": 34
          },
          {
            "identifier": "ChargeTransactionPayment",
            "ty": 124,
            "additional_signed": 34
          }
        ]
      }

The type system is composite, which means that each type ID contains a reference to some type or to another type ID that gives access to the associated primitive types. For example one type we can encode is a BitVec<Order, Store> type: to decode it properly we need to know what the Order and Store types used were, which can be accessed used the "path" in the decoded JSON for that type ID.

RPC APIs

Substrate comes with the following APIs to interact with a node:

  • AuthorApi: An API to make calls into a full node, including authoring extrinsics and verifying session keys.
  • ChainApi: An API to retrieve block header and finality information.
  • OffchainApi: An API for making RPC calls for offchain workers.
  • StateApi: An API to query information about on-chain state such as runtime version, storage items and proofs.
  • SystemApi: An API to retrieve information about network state, such as connected peers and node roles.

Connecting to a node

Querying a Substrate node can either be done by using a Hypertext Transfer Protocol (HTTP) or WebSocket (WS) based JSON-RPC client. The main advantage of WS (used in most applications) is that a single connection can be reused for many messages to and from a node, whereas a typical HTTP connection allows only for a single message from, and then response to the client at a time. For this reason, if you want to subscribe to some RPC endpoint that could lead to multiple messages being returned to the client, you must use a websocket connection and not an HTTP one. Connecting via HTTP is commonly used for fetching data in offchain workers-learn more about that in Offchain operations.

An alternative (and still experimental) way to connect to a Substrate node is by using Substrate Connect, which allows applications to spawn their own light clients and connect directly to the exposed JSON-RPC end-point. These applications would rely on in-browser local memory to establish a connection with the light client.

Start building

Parity maintains the following libraries built on top of the JSON-RPC API for interacting with a Substrate node:

  • subxt provides a way to create an interface for static front-ends built for specific chains.
  • Polkadot JS API provides a library to build dynamic interfaces for any Substrate built blockchain.
  • Substrate Connect provides a library and a browser extension to build applications that connect directly with an in-browser light client created for its target chain. As a library that uses the Polkadot JS API, Connect is useful for applications that need to connect to multiple chains, providing end users with a single experience when interacting with multiple chains for the same app.

Front-end use cases

NameDescriptionLanguageUse case
Polkadot JS APIA Javascript library for interacting with a Substrate chain.JavascriptApplications that need to dynamically adapt to changes in a node, such as for block explorers or chain-agnostic interfaces.
Polkadot JS extensionAn API for interacting with a browser extension build with the Polkadot JS API.JavascriptBrowser extensions.
Substrate ConnectA library for developers to build applications that act as their own light client for their target chain. It also provides a browser extension designed to connect to multiple chains from a single application (web or desktop browser).JavascriptAny browser application.
subxtShort for "submit extrinsics", subxt is a library that generates a statically typed Rust interface to interact with a node's RPC APIs based on a target chain's metadata.RustBuilding lower level applications, such as non-browser graphic user interfaces, chain-specific CLIs or user facing applications that require type-safe communication between the node and the generated interface, preventing users from constructing transactions with bad inputs or submitting calls that don't exist.
txwrapperA Javascript library for offline generation of Substrate transactions.JavascriptWrite scripts to generate signed transactions to a node, useful for testing and decoding transactions.

Where to go next