12 August 2026

The Loki Ring: Scaling Without a Central Leader

by Alex Carvalho

I have been studying Grafana Loki’s architecture, and this is the third post in the series. In the first post, the focus was labels, streams, and chunks. In the second post, we traced a log line’s path through the distributors and ingesters. Now, the core question is: how does Loki decide which ingester gets which log stream, and how does it scale that decision without a central coordinator?

The core rule is:

Consistent hashing on a shared ring allows stateless distributors to make identical routing decisions independently, while memberlist gossip keeps membership eventually consistent.

This coordination model is how Loki achieves massive ingestion scale. It avoids placing a single active orchestrator in the middle of every write, instead relying on mathematical consensus via a shared hash ring.

Why distributed coordination is hard

In a single-instance database, routing is trivial: all writes go to the same process. But as log volume grows, Loki must distribute streams across a pool of ingesters.

Doing this in a large-scale system introduces classic distributed challenges:

Membership changes: Nodes are added to increase capacity or removed during rolling upgrades.
Node failures: Processes crash or network partitions cut them off unexpectedly.
Uneven load: Simple modulo hashing (hash(stream) % N) causes massive data relocation whenever $N$ changes.
Partial knowledge: Distributors need to agree on routing without constantly querying a single source of truth.

Loki solves these problems by combining a hash ring with consistent hashing and eventual consistency.

The hash ring and tokens

At the center of Loki’s coordination is the ring. The ring is a logical circle representing a keyspace of $2^{32} - 1$ integers (ranging from 0 to 4,294,967,295).

       0 (or 2^32)
      /   \
     /     \
    /       \
3/4 ring     1/4 ring
    \       /
     \     /
      \   /
     1/2 ring

Every ingester in the cluster is assigned one or more tokens—randomly generated 32-bit integers that mark their positions on this circular keyspace.

When a distributor receives a log stream:

It hashes the tenant ID and the stream’s exact labels to produce a 32-bit integer.
It looks at the ring to find the first token greater than or equal to that hash value.
The ingester that owns that token is assigned to handle the stream. If the hash is larger than all tokens, it wraps around to the beginning of the circle and goes to the ingester owning the smallest token.

Virtual nodes

To prevent uneven data distribution, Loki uses virtual nodes. Instead of assigning a single token to each ingester, Loki assigns multiple tokens (by default, 128 tokens per ingester) to each node. This scatters an ingester’s presence across different parts of the ring, ensuring that even if one segment receives a high volume of writes, the load is statistically distributed across multiple physical nodes.

Consistent hashing: Minimizing data relocation

If Loki used standard modulo hashing, changing the number of ingesters would shuffle almost every stream to a new node:

hash(stream) % N

When $N$ shifts to $N+1$, almost all keys map to a different index. In a database that buffers chunks in memory, this would trigger catastrophic flushing and cache misses.

Consistent hashing solves this. Because nodes are mapped to points on a circle, adding or removing an ingester only affects the segments of the ring immediately adjacent to the new or removed tokens:

Adding Node D:
Before:  Node A ===> Node B ===> Node C ===> Node A
After:   Node A ===> Node B ===> Node D ===> Node C ===> Node A

Only keys that hash to the segment between Node B and the new Node D are relocated from Node C to Node D. The rest of the ring remains completely unaffected.

Replication and quorum

We established in the second post that Loki replicates writes—typically using a replication factor of 3.

To find the three ingesters for a stream, the distributor hashes the stream label set and maps it to the ring. It selects the ingester owning the first token it encounters, and then continues moving clockwise along the ring to select the next two unique ingesters.

       Stream Hash (*)
            |
            v
   ---> [Token 1420 (Node A)] ---> [Token 2800 (Node B)] ---> [Token 3900 (Node C)]

Once the three target ingesters are identified, the distributor sends the write request to all three in parallel. It only waits for a quorum of successful acknowledgements before returning success to the client:

quorum = floor(replication_factor / 2) + 1

For a replication factor of 3, quorum is 2. This guarantees write survival even if one ingester is restarting or temporarily unreachable, matching the crash-recovery guarantees we discussed regarding the Write-Ahead Log (WAL).

The ingester lifecycler

An ingester does not simply pop in and out of the ring. It transitions through defined lifecycle states managed by a component called the lifecycler:

PENDING: The ingester is starting up and preparing to join.
JOINING: The ingester generates its tokens and registers them in the ring.
ACTIVE: The ingester is actively receiving writes and participating in the ring.
LEAVING: The ingester is shutting down. It stops accepting new streams and prepares to hand off or flush its remaining in-memory chunks.
UNHEALTHY: The node has missed heartbeats, indicating it is offline or unresponsive.

This state machine ensures that transitions—like scaling up or down—occur gracefully without dropping writes.

Gossip under the hood

To make these independent decisions, every distributor must have a copy of the ring. But how is the ring state shared?

Loki supports two backends for distributed state coordination:

Key-Value Stores (Consul or ETCD): Nodes write their status and tokens to a central highly-available store, and other nodes watch for updates.
Memberlist (Gossip): Nodes directly share membership and ring state peer-to-peer using the gossip protocol.

In modern Loki deployments, Memberlist is the recommended choice. By using gossip, the cluster coordinates membership and detects node failures entirely peer-to-peer, eliminating external dependencies.

As studied in the gossip post, this membership list is eventually consistent. A distributor might have a slightly delayed view of the ring if an ingester just joined. However, because replication forwards writes to multiple candidates and quorum only requires 2/3 responses, the write path remains extremely resilient to temporary routing discrepancies.

Failure detection and suspicion

To keep the ring healthy, nodes periodically exchange heartbeats. If a node fails to heartbeat within a configured threshold, the memberlist enters a suspicion state:

Heartbeat missed -> Node marked Suspect -> Wait for confirmation -> Node marked Dead

This prevents transient network blips from immediately triggering massive ring changes. If the suspect node does not verify its health within a grace period, it is marked unhealthy, and distributors stop routing streams to it, allowing the remaining replicas to take over.

Summary

The Loki hash ring is the engine that allows Loki to ingest petabytes of logs without a bottleneck coordinator.

By combining:

Tokens and Virtual Nodes to evenly partition the stream keyspace.
Consistent Hashing to minimize data relocation during scale events.
Memberlist Gossip to distribute the ring state peer-to-peer.
Quorum Writes to tolerate individual node failures.

Loki scales writes horizontally. Distributors act as stateless workers, while the ring provides the mathematical framework that guides their work.

In the next post, we will shift focus from memory and routing to disk, looking at Storage Internals and how Loki writes index and chunk files to cheap object storage.