Posts

100M clients. Billions of agents. One simulation.

An introductory HiveShard post about scaling deterministic simulations without splitting the world into isolated shards.

100M clients. Billions of agents. One simulation.

Systems at that scale usually make a trade-off between:

  • one global simulation with relaxed consistency or approximations
  • many smaller, isolated simulations with strong consistency

HiveShard eliminates this trade-off: one large-scale, consistent simulation.

This is achieved by partitioning state across 2D space, where each partition is composed of multiple event-driven HiveShards (e.g. CharacterShard and InfectionShard, as shown in the animation), each consuming events, updating local state, and emitting new events.

Use cases

  • large-scale scientific simulations
  • policy and population modeling
  • defense and strategy systems
  • single-world multiplayer environments

Information exchange happens through message propagation between neighboring partitions on each tick. HiveShards only consume events originating within their own or neighboring partitions, which keeps interactions local and enables the system to scale efficiently.

Client interaction

Client interaction is handled separately from the simulation:

  • clients connect through dedicated boundary components (“Edges”)
  • these control what input enters the simulation and at which tick
  • they also filter and distribute state updates back to clients

This separation allows the simulation to scale independently of client connections.

To address slow or faulty partitions, the architecture is designed for replicated execution with quorum validation. Progress depends on consensus, not individual nodes.

Current status

  • runs in-memory using Xcepto (open source test runtime)
  • designed for Kubernetes-scale deployment with Kafka-backed communication

Open source repository

HiveShard