thanos.io

What is Thanos?

Thanos is an open-source toolkit that turns one or more Prometheus servers into a single, highly available metrics system with long-term storage. It keeps Prometheus as the collector you already run, then adds a global query layer, object-storage-backed retention, and downsampling so older data stays queryable without huge cost.

What it adds to Prometheus

• Global query view: query metrics across multiple Prometheus servers and clusters through one endpoint, while still using Prometheus-compatible tools such as Grafana.
• Unlimited retention: ship metric blocks to object storage — the project lists GCP, S3, Azure, Swift and Tencent COS — instead of being limited by local disk.
• Downsampling and compaction: reduce historical data resolution so long-range queries are faster, and apply retention policies to different data ages.
• High availability: run multiple Prometheus instances and deduplicate overlapping series, so a single server failure does not create a metrics blind spot.

Who it is for

Thanos suits platform and observability teams running Prometheus at a scale where a single server, or a single cluster, is no longer enough. Typical situations: several Prometheus servers in different clusters, a need to keep months or years of metrics, or a requirement to query one Grafana dashboard across all environments.

It is less compelling for a small team with one Prometheus server, modest retention, and no cross-cluster query needs. In that case, native Prometheus storage plus a managed metrics service is usually simpler to operate.

Practical trade-offs

The project is a Cloud Native Computing Foundation incubating project, licensed under Apache 2.0, and was founded by Improbable. Official resources: Thanos for docs and releases, and Prometheus for the underlying server it extends.

A useful next step: list every Prometheus server you run, note where each one stores data today, and decide whether your main pain is cross-cluster querying, retention cost, or query speed on old data. Thanos addresses all three, but the deployment shape changes depending on which one matters most to you.

How does Thanos provide high availability for Prometheus?

Thanos provides high availability by letting multiple Prometheus servers scrape the same targets and then deduplicating their results at query time. If one Prometheus instance fails, another replica is still collecting the same metrics, so dashboards and alerts continue to work.

The key pieces, based on the project's own description:

• Global Query View – queries across multiple Prometheus servers and clusters, so no single server is a query bottleneck or single point of failure.
• Sidecar pattern – each Prometheus runs alongside a Thanos sidecar that uploads metric blocks to object storage and exposes the local data to the query layer.
• Object storage for retention – supports GCP, S3, Azure, Swift and Tencent COS, giving metrics a durable home beyond a single Prometheus disk.
• Downsampling and compaction – historical data is compacted and downsampled, which keeps long-range queries fast as retained data grows.
• Prometheus compatibility – Grafana and other tools that use the Prometheus Query API keep working.

How the pieces fit together

A practical scenario: a team runs two Prometheus replicas per cluster, each scraping the same services. Both sidecars ship blocks to S3. Grafana points at a Thanos Query endpoint, which merges the two replicas and removes duplicates. If one replica dies during an upgrade, the other still serves alerts, and historical data remains queryable from object storage.

A decision criterion: Thanos suits teams already standardized on Prometheus who need cross-cluster querying and retention beyond local disk, and who can operate object storage. For a single small Prometheus with modest retention, the added components may not be worth the operational cost.

Next step: read the Getting Started and Design pages on Thanos to see the sidecar, store, query and compactor components and how they connect.

How do I set up Thanos with my existing Prometheus servers?

Thanos sits alongside your existing Prometheus servers rather than replacing them. Each Prometheus keeps scraping as it does today, and you add a small sidecar process that uploads its data to object storage and exposes it to a central query layer. That design is why the project describes itself as an "open source, highly available Prometheus setup with long term storage capabilities."

The standard architecture

• Prometheus + sidecar (one per server): Prometheus continues local scraping and short-term storage. The sidecar ships completed blocks to object storage and answers queries for recent, not-yet-uploaded data.
• Object storage: The long-term store. Thanos lists GCP, S3, Azure, Swift and Tencent COS as supported backends.
• Store gateway: Serves historical blocks from object storage to queries.
• Querier: Fans a PromQL request out to sidecars and store gateways and merges the results, giving one global query view across clusters.
• Compactor: Runs as a single instance over the bucket, applying downsampling and retention policies.
• Ruler (optional): Evaluates recording and alerting rules against the global view instead of a single Prometheus.

A practical setup path

1. Read the Getting Started and Design pages on Thanos before touching production; the design doc explains the sidecar-versus-receiver trade-off better than any tutorial.
2. Stand up a bucket and give each Prometheus a sidecar pointed at it, with a stable external label per Prometheus (cluster, region, replica).
3. Deploy the store gateway and querier, then point Grafana at the querier's Prometheus-compatible endpoint. Existing dashboards and tools keep working because the query API is compatible.
4. Add the compactor only after uploads are flowing, and run exactly one per bucket.
5. Enable the ruler last, once you trust the global view.

Decisions worth making early

The main trade-offs are operational: object storage becomes a hard dependency for historical queries, the compactor is a single point of coordination, and cross-cluster queries can be slower than local ones. If you run a small single-Prometheus setup, the added components may cost more than they return.

For the concrete next step, work through the Getting Started guide on Thanos and join the community Slack listed there; the maintainers and other operators are the fastest source of answers on bucket configuration and migration order.

What object storage backends does Thanos support for long-term metrics storage?

Thanos supports GCP, S3, Azure, Swift and Tencent COS as object storage backends for long-term metrics storage, according to the project's own feature summary on Thanos.

What this means in practice

Thanos stores metric blocks in an object storage bucket rather than keeping everything on Prometheus servers. A component called the sidecar uploads completed blocks from Prometheus to that bucket, and a store gateway serves them back for queries. Because the bucket is the long-term layer, retention is effectively limited by your storage budget rather than by local disk size.

The five supported backends map to common cloud environments:

• GCP — Google Cloud Storage, natural fit for GKE-based monitoring.
• S3 — AWS S3 and S3-compatible endpoints, the most widely used option in self-hosted setups.
• Azure — Azure Blob Storage for teams standardized on Microsoft's cloud.
• Swift — OpenStack Object Storage, relevant for private cloud or OpenStack deployments.
• Tencent COS — Tencent Cloud Object Storage, useful for workloads in that region.

Choosing a backend

Pick the backend that matches where your Prometheus instances already run and where your team has operational experience. Latency and egress costs matter more than raw capacity: queries that touch old data pull blocks from object storage, so a bucket in a distant region will feel slow. If you run on-premises without OpenStack, an S3-compatible store such as MinIO is a common route, since Thanos speaks the S3 API.

A practical first step is to read the Storage section of the official documentation at Thanos and confirm the exact configuration flags for your chosen provider before provisioning a bucket.

How does Thanos downsampling and compaction improve query performance?

Thanos addresses query performance from two directions: compaction reduces the number of storage blocks that must be read, and downsampling reduces the resolution of older data so each block contains fewer samples to scan. Together they make long-range queries over large historical datasets practical, because the query engine touches far less data for the same time window.

Compaction

Prometheus writes small, time-bounded blocks. Over months, those accumulate into many small files. Thanos compaction merges overlapping blocks into larger ones and applies retention policies, so a query spanning weeks reads a handful of blocks instead of hundreds. Fewer blocks mean fewer index lookups, fewer object-storage requests and less merge work at query time. This matters most for dashboards and alerts that repeatedly scan fixed windows, where per-query overhead compounds.

Downsampling

Downsampling creates reduced-resolution copies of historical data — typically aggregating raw samples into coarser intervals. The exact resolution levels are defined in the project's storage documentation rather than on the landing page. The practical effect for a reader is straightforward: a query covering a year does not need every 15-second sample to show a trend line, so a lower-resolution copy answers it faster and cheaper. Recent data stays at full resolution for precise debugging; older data is served from the downsampled copy.

When each one helps

A concrete scenario

An SRE team runs Prometheus across several clusters and keeps three years of metrics in object storage. Without compaction, a Grafana panel showing a 90-day trend scans a large set of small blocks; with compaction it reads a much smaller set. Extending that same panel to two years would still be slow at raw resolution, so the downsampled copy serves it in a fraction of the time. The team keeps full resolution for the last few weeks so that a latency spike during an incident is still visible.

Choosing your approach

Decide based on query time range, not on total data volume. If most queries cover hours or days, compaction alone may be enough. If users routinely look back months or years, enable downsampling and set retention so that raw data is kept long enough for debugging. Verify the exact resolution levels and retention configuration in the project's storage documentation before committing to a policy, since those settings determine what detail you permanently lose.

Start with Thanos for the design and storage documentation, and check Prometheus for how block layout and the query API work underneath, since Thanos builds directly on those conventions.

What are the trade-offs and operational considerations of running Thanos in production?

Thanos trades operational complexity for scale and retention that a single Prometheus server cannot provide. It is a set of components layered on top of Prometheus, not a drop-in replacement, so the main trade-off is that you gain global querying and effectively unlimited metric history while taking on a distributed system to run, monitor and upgrade.

Where the trade-offs land

• Query performance vs. data volume. The global query view lets you query metrics across many Prometheus servers and clusters from one endpoint, but every query may fan out to multiple stores. Large time ranges are where downsampling and compaction matter most; without them, long-range queries get slow and expensive.
• Storage cost vs. retention. Extending the system with object storage (the project lists GCP, S3, Azure, Swift and Tencent COS) makes long retention affordable, but object storage is not a local disk. Compaction, downsampling and store-gateway-style reads add latency and moving parts compared with a single Prometheus with a local TSDB.
• Compatibility vs. control. Because Thanos speaks the Prometheus Query API, existing tools such as Grafana keep working. That compatibility is a real benefit, but it also means you inherit Prometheus semantics, label cardinality problems and query patterns rather than escaping them.
• High availability vs. consistency. Running multiple Prometheus replicas with Thanos sidecars gives you redundancy, but deduplication is approximate and you should expect small gaps or overlaps rather than a single perfectly consistent source of truth.

Operational considerations

• Component sprawl. Sidecars, queriers, stores, compactors and receive-style ingestion each have their own failure modes. Decide early whether you run a sidecar-per-Prometheus model, a remote-write model, or both, because that choice shapes your network, storage and upgrade process.
• Compaction is the critical path. The compactor rewrites and downsamples blocks in object storage. It is stateful, slow and hard to parallelize safely. Give it dedicated resources, watch its progress and avoid running multiple compactors over the same bucket.
• Cardinality and query cost are your real limits. Object storage makes bytes cheap, not queries. A handful of high-cardinality metrics can still make global queries unusable, so label hygiene and recording rules remain essential.
• Upgrades and version skew. The page lists releases from v0.1 through v0.42 and a tip version, which signals a fast-moving project. Plan rolling upgrades and check compatibility between components, since mixed versions can behave unexpectedly.
• Observability of the observability stack. You need to monitor Thanos itself: query latency, store health, compaction lag and object storage errors. If Thanos is down, you may lose visibility into everything else.
• Governance and support. It is a Cloud Native Computing Foundation incubating project under Apache License 2.0, founded by Improbable, with community channels including Slack and GitHub. That means strong community momentum but no single commercial vendor behind the core project.

A practical way to decide

Start with one question: do you need to query metrics across more than one Prometheus, or retain them far beyond local disk? If the answer is no, a single Prometheus with a well-sized disk is simpler and cheaper. If the answer is yes, pilot Thanos with one cluster and one bucket, run the compactor and querier under load, and measure long-range query latency before committing. A useful next step is to read the official design and storage documentation at Thanos and compare your planned topology against the component model described there.