: Performance and scalability are the more discussed challenges in the cryptocurrency space, related to both Layer 1 projects (standalone blockchains) and Layer 2 solutions (such as Rollups and Off-chain Channels). However, we do not have standardized measures or benchmarks. Numbers often appear in reports in inconsistent and incomplete ways, making it difficult to accurately compare, and often overshadows what is most important in practice.

Scalability and performance

First, let's define two terms, scalability and performance, which have standard computer science meanings but are often misused in the blockchain context. Performance measures what a system is currently capable of achieving. As we discuss below, performance metrics might include volume per second or median transaction confirmation time. Scalability, on the other hand, measures the ability of a system to improve performance by increasing resources.

This distinction is important: when properly defined, many methods for improving performance do not improve scalability at all. A simple example: Use a more efficient digital signature scheme such as BLS signature, which is about half the size of Schnorr or ECDSA signature. If Bitcoin were to switch from ECDSA to BLS, the number of transactions per block could rise by 20-30%, improving performance overnight. But we can only do this once, and there is no more space-saving signature scheme to switch to (BLS signatures can also be aggregated to save more space, but this is another one-time trick).

In blockchain, some other one-time tricks (like SegWit) are possible, but you need a scalable architecture to achieve continuous performance improvement, that is, adding more resources to improve performance. Building a web server, for example, is the conventional wisdom in many other computer systems. With a few common tricks, you can build a very fast server; But ultimately, you need a multi-server architecture to keep up with the growing demand by constantly adding additional servers.

Understanding this distinction also helps to avoid the problem of "Blockchain X is highly scalable, it can process Y transactions per second!" Common category errors found in such statements. The second statement may be impressive, but it is a performance metric, not a scalability metric. It doesn't talk about the ability to increase performance by adding resources.

Scalability essentially requires leveraging parallelism. In the blockchain world, the extension of Layer 1 seems to require shards or something that looks like shards. The basic concept of sharding: splitting a state into pieces so that different validators can handle it independently -- closely related to the definition of extensibility. There are more options at Layer 2, allowing for the addition of parallel processing -- including off-chain channels, rollup services, and side chains.

Latency and Throughput

Traditionally, the performance of blockchain systems has been evaluated through two dimensions, namely latency and throughput. Latency measures the speed at which individual transactions can be confirmed, while throughput measures the total rate of transactions over time. These axes are applicable to both Tier 1 and Tier 2 systems, as well as many other types of computer systems, such as database query engines and web servers.

Unfortunately, measuring and comparing latency and throughput can be complicated. Moreover, individual users don't really care about throughput (which is a system-wide measure). What they really care about are delays and transaction costs. More specifically, the faster their transactions are confirmed, the better, and the lower the fee, the better. Although many other computer systems are also evaluated on a cost/performance basis, for blockchain systems, transaction fees are a somewhat new performance axis that does not exist in traditional computer systems.

Conclusion

It is difficult to assess performance fairly and accurately. Different people will care about different things. When measuring the performance of a car, some users will care about top speed or acceleration, others about gas mileage, and still others about towing ability. All of these are not hard to evaluate, and neither are blockchains.

The blockchain space is still a long way from this level of standardization. In some areas, we may in the future evaluate the throughput of the system with standardized workloads, or show the latency distribution with standardized graphs. Currently, the best approach for evaluators and builders is to collect and publish as much data as possible, and to describe the evaluation methods in detail so that they can be replicated and compared with other systems.