Table of contents

<aside> 📺 Note that the following two sections of this document are available in video format. There are also answers to questions from the WG on this video. But the video is a little bit outdated.

</aside>

Motivation

Over a considerable period, I have been immersed in the design of various dapp architectures. Each dapp presents its unique challenges to overcome. Regardless of the specific project, a set of recurring questions always arises:

  1. How can I efficiently store over 48GBs of data?
  2. When dealing with multiple canisters, how should data be distributed between them? Especially when the data is indexed, how can we ensure efficient lookup while distributing it among canisters?
  3. How can I execute atomic actions when my data is spread across multiple canisters? How can I maintain transactional integrity in an asynchronous environment?
  4. If one of the subnets experiences an outage, how can I ensure continued access to the remaining data?
  5. How will my users access and modify this data?
  6. How can I seamlessly upgrade all of these canisters?
  7. How can I replenish the resources of these canisters?

If you have ever designed a dapp on the Internet Computer (IC), I'm certain you have asked yourself these same questions.

Personally, I have grown weary of constantly pondering these questions. I no longer want to be burdened by the limitations of the Internet Computer. I want to focus solely on my business challenges.

It has become evident that action must be taken. It would be ideal if there were a tool available that abstracts away the platform limitations and effortlessly handles all aspects of scalable data, freeing me to concentrate exclusively on my business objectives.

This ideal tool would allow me to describe my whole database in a single module. Then describe all possible transactions which could happen to this database.

Then I deploy a single canister - a shard, and the data lives and grows inside this shard, as users start using my dapp.

Shards grow.svg

Then, when it reaches the maximum memory limit, the shard spins up empty copies of itself.