13 August 2026

Loki Storage Internals: How Loki Stores Petabytes of Logs

by Alex Carvalho

I have been exploring how Grafana Loki manages data under the hood. In the first post, I covered labels, streams, and chunks. In the second post, we traced a log line’s path through the system. In the third post, we looked at the hash ring. Now, we face the storage layer: what exactly is stored when Loki flushes data, and how does it scale storage to petabytes of logs?

The core rule is:

Loki decouples log content from its index, storing compressed log chunks and a highly optimized TSDB index separately on cheap object storage, using compactors and index gateways to make queries and retention efficient.

This storage strategy is what makes Loki extremely cost-effective compared to traditional full-text search systems.

Object storage as the single source of truth

Traditional log search engines (like Elasticsearch or OpenSearch) rely on stateful databases that require high-performance block storage (SSDs). These databases must run complex indexing engines on every node, which is both computationally expensive and operationally fragile at petabyte scale.

Loki takes a different approach: it is designed to treat simple, cheap object storage (like AWS S3, Google Cloud Storage, or Azure Blob Storage) as its single source of truth.

Object storage is virtually infinite, highly durable, and very cheap. However, it is also high-latency and has slow seek times. To make object storage work as a backend for a database, Loki splits all data into two distinct categories:

Chunks: Compressed blocks of raw log entries.
Index files: Compact tables mapping log stream labels to chunk references.

By keeping these two paths separate, Loki can stream huge amounts of compressed log data to cheap storage, while keeping the index small enough to fit in memory or cache.

               [ INGEST PATH ]
                 Ingesters
                /         \
   (Flush Chunks)         (Ship Local Index)
              v             v
       +----------+     +---------------+
       |  Object  |     | Object Store: |
       | Storage: |     | TSDB Indexes  |
       |  Chunks  |     +---------------+
       +----------+             ^
              ^                 | (Fetch & Cache)
              | (Fetch Chunks)  |
              |           +-----------+
              |           |   Index   |
              |           |  Gateway  |
              |           +-----------+
              |                 ^
              |                 | (Label Lookup)
            Querier ------------+
               [ READ PATH ]

   * Compactor (Maintenance):
     - Deduplicates & merges TSDB Indexes in Object Storage.
     - Enforces retention & deletes expired/requested Chunks.

Inside the chunk layout

When an ingester flushes a log stream, it cuts the stream’s in-memory entries into a chunk.

A chunk is a binary block containing:

Metadata Header: Stream labels, chunk time range (start and end timestamps), and internal settings.
Compressed Blocks: Log entries grouped into small blocks (usually 8KB to 256KB raw size) and compressed using algorithms like Snappy (default), gzip, or zstd.
Chunk-Level Index (Entries): A table of offsets and timestamps allowing Loki to seek to specific lines inside the chunk without decompressing the entire file.
Checksums: Used to detect corruption during transfer or storage.
Structured Metadata: Key-value pairs attached directly to log lines to support high-cardinality values without creating new streams.

A simplified layout of a chunk file looks like this:

+--------------------------------------------------------+
|                      CHUNK FILE                        |
+--------------------------------------------------------+
| HEADER: Labels, Time Range, Encryption/Compression Type |
+--------------------------------------------------------+
| COMPRESSED BLOCK 1: [Log lines... (Snappy compressed)] |
| COMPRESSED BLOCK 2: [Log lines... (Snappy compressed)] |
| ...                                                    |
+--------------------------------------------------------+
| INDEX TABLE: [Block 1 Offset, Block 2 Offset...]       |
+--------------------------------------------------------+
| CHECKSUM & STRUCTURED METADATA                         |
+--------------------------------------------------------+

Because chunks are stored sequentially and compressed heavily, they compress raw text logs by up to 5x to 10x. Once written to object storage, chunks are immutable.

The TSDB index format

The chunk files contain the logs, but how does Loki know which chunks to download during a query? This is the job of the index.

Historically, Loki supported multiple index formats (like BoltDB-shipper). Today, the standard and recommended index format is TSDB.

The TSDB index is an inverted index structured specifically for Loki’s label-based queries. It partitions the index by time range (usually 24-hour periods). Inside each TSDB index file, Loki stores:

Postings: Maps a specific label key-value pair (e.g., app="checkout") to a list of matching stream IDs.
Label Index: A list of all unique label names and values.
Chunk Tables: Maps stream IDs to the specific chunk references (chunk ID, start/end time) stored in object storage.

When you run a LogQL query like {app="checkout"} |= "error", Loki performs the following steps:

It queries the TSDB index for app="checkout".
The index returns a list of stream IDs.
For those stream IDs, Loki finds the chunk references overlapping the query’s time range.
The querier downloads only those specific chunk files from object storage and scans them for the string "error".

Why TSDB is efficient

Unlike Elasticsearch, which indexes every single word in your log lines, Loki’s TSDB index only indexes the labels.

Because label cardinality is kept low (as we discussed in the first post), the size of the TSDB index is minuscule—often less than 1% to 2% of the size of the raw log data. This tiny index is extremely cheap to store, ship, and cache.

Shipping and serving the index: Shipper and Gateway

To make TSDB indexes work on top of object storage without a central database, Loki uses two components:

The Index Shipper

Ingesters build TSDB indexes locally on their local disks. Every 15 minutes (by default), the Index Shipper scans these files and uploads (ships) them to object storage. This ensures that even if an ingester crashes, historical index data is safe in object storage.

The Index Gateway

Because queriers need to search the index constantly, downloading raw TSDB files from object storage for every query would be incredibly slow.

Loki solves this with the Index Gateway. The Index Gateway acts as a caching query layer for the index. It downloads the TSDB index files from object storage, stores them on local disks, and keeps hot segments in memory. When a querier needs to resolve labels to chunk references, it calls the Index Gateway instead of reading object storage directly.

The Compactor: Housekeeping at scale

Because ingesters flush index files and chunks independently, object storage quickly accumulates thousands of tiny, fragmented index files.

The Compactor is a singleton component that does the housekeeping. It runs in the background and has three primary duties:

Deduplication and Merging: It downloads many small TSDB index files for a given day, merges duplicate stream entries, cleans up redundant records, and writes out a single, large, optimized index file. This improves index lookup times by up to 10x.
Applying Retention Policies: It enforces retention rules. Loki supports both global retention (e.g., “keep logs for 30 days”) and stream-specific retention (e.g., “keep app="payment" for 90 days, but env="dev" for 7 days”). The compactor scans the index, identifies expired chunks, deletes the chunk files from object storage, and updates the index to remove references to them.
Executing Delete Requests: In compliance with regulations like GDPR, users can request log deletion. Deleting logs in Loki is highly non-trivial because chunks are immutable compressed files containing many log entries. The compactor handles this by downloading the target chunks, extracting and removing the specified log lines, re-compressing the remaining data into new chunks, uploading them, and updating the index.

Compactor Pipeline:
[Small TSDB Files] ---> [Merge & Deduplicate] ---> [Single Clean Index File]
[Expired Chunks]   ---> [Identify & Purge]     ---> [Deleted from Object Store]

Summary

By separating the index from the chunks, Grafana Loki creates a highly efficient, asymmetrical architecture. The write path streams raw compressed logs directly to cheap object storage, while the read path uses a compact, cached TSDB index to selectively download only the chunks relevant to a query.

This layout allows Loki to store petabytes of logs at a fraction of the cost of Elasticsearch.

In the next post, we will look at the other side of this architecture: Reading Billions of Logs and how Loki executes queries across these storage systems without a full-text index.