What SQL and open formats does QuestDB support?

Updated

QuestDB runs a standard SQL engine over time-series data and exposes that same data through open formats — Apache Parquet, Apache Iceberg, and Apache Arrow. If your team already writes SQL and your downstream tooling reads Parquet, Iceberg, or Arrow, you can query live and historical data in one place without converting it into a proprietary format first.

SQL support

QuestDB's SQL engine is the single interface for three jobs that are often split across separate systems:

  • Messaging ingestion — events arrive and land in tables you query with SQL.
  • Stream processing — computations run on live data in milliseconds.
  • Tiered storage — years of history stay queryable through the same engine.

The practical effect is that you don't need a separate query language for the live stream and the archive. The same SQL statement style works across both, which is what lets one team compute intraday risk while another runs historical research against the same record.

QuestDB describes this as "one SQL engine spanning messaging, stream processing, and tiered storage," and positions standard SQL as the thing that "keeps every team connected to the record."

Open formats

QuestDB supports three open formats for reading and sharing data:

Format What it's used for
Apache Parquet Columnar file format for analytics and interchange
Apache Iceberg Open table format for large analytical datasets
Apache Arrow In-memory columnar format for fast data transfer

The site states you can "open the same data through Apache Parquet, Iceberg and Arrow." Arrow also appears in the published performance numbers: 220M rows/sec streamed to Arrow, with the first Arrow batch arriving in 32ms and 500M rows streamed in 2.3s.

Why the combination matters

The design goal is that the live stream and the complete record are unified. In practice that means:

  1. Ingest once — ticks, orders, executions, and transactions are captured at market speed, with durability, deduplication, and out-of-order correction built in.
  2. Compute on live data — intraday risk, P&L, and surveillance conditions are evaluated while the market is moving.
  3. Investigate the full history — TCA, markouts, post-trade analysis, compliance, and research run in SQL, and the same data opens through Parquet, Iceberg, and Arrow.

Because the formats are open, teams using different tools can work from the same underlying record instead of exporting copies into incompatible stores.

What to check before committing

  • SQL dialect specifics — "standard SQL" covers the common surface, but confirm the exact functions and syntax your queries rely on against QuestDB's documentation before migrating production queries.
  • Format access pattern — decide whether you need Parquet/Iceberg for file-level interchange, Arrow for in-memory handoff, or both, since they serve different stages of a pipeline.
  • Workload fit — the published figures (19M rows/sec/server ingestion, 220M rows/sec to Arrow) are benchmark results on QuestDB v10.0; treat them as a reference point and validate against your own data shape and query mix.

If your requirement is simply "SQL I already know, over data I can also read as Parquet, Iceberg, or Arrow," QuestDB is built around exactly that combination.