Skip to main content
Version: Next

Benchmarks

Proxy overhead​

End-to-end overhead is measured by queryflux-bench (cargo build --release --bin queryflux --bin queryflux-bench, then target/release/queryflux-bench). It uses mock backends (a Trino HTTP coordinator and a StarRocks MySQL-wire FE) that answer instantly, 50 warmup queries per path, then 500 timed iterations — direct to the mock vs the same request through QueryFlux's Trino HTTP frontend. Overhead is the difference. Numbers below: median of 3 runs on an Apple M1 Max (10 cores, 32 GB); they vary by machine, so compare runs on the same one.

Trino (mock HTTP coordinator), SELECT 1​

p50p95p99
Direct0.09 ms0.13 ms0.15 ms
Via QueryFlux0.20 ms0.25 ms0.28 ms
Overhead0.12 ms0.13 ms0.13 ms

StarRocks path (mock MySQL FE), SELECT 1​

p50p95p99
Direct (one reused MySQL connection)0.03 ms0.09 ms0.12 ms
Via QueryFlux0.19 ms0.26 ms0.32 ms
Overhead0.16 ms0.17 ms0.20 ms

Overhead per feature (Trino path)​

Each feature scenario runs QueryFlux with one feature turned on; the dashboard-style query is the baseline, with none (QUERYFLUX_BENCH_SCENARIOS picks a subset):

Scenariop50p95p99
Dashboard-style query (CTE, join, filters, GROUP BY, LIMIT)0.09 ms0.12 ms0.12 ms
Translation, cache hit (same SQL repeated)0.11 ms0.12 ms0.12 ms
Translation, cache miss (new SQL every query)1.91 ms2.26 ms2.59 ms
Built-in guards (read_only, row_limit, require_predicate)0.19 ms0.22 ms0.26 ms
Python check(ctx) guard0.19 ms0.23 ms0.29 ms
Router chain (header, tags, regex, Python), none matching0.15 ms0.18 ms0.21 ms
Access control, OPA allows0.22 ms0.28 ms0.35 ms
Access control, OPA adds a row filter (query rewritten)2.47 ms2.92 ms3.67 ms
Result cache hit (no backend call)0.37 ms0.51 ms1.06 ms

The two multi-millisecond rows are sqlglot running on every query: a translation-cache miss (repeated SQL is cached), and the row-filter rewrite, which is not cached yet.

Capacity benchmark​

maxRunningQueries is enforced fleet-wide through capacity leases in Postgres (see Worker mode: shared capacity). Every query acquires a lease before it runs and releases it when it ends, so lease throughput and latency bound how many queries per second a fleet can admit. queryflux-bench capacity measures that path in isolation, with no engines involved.

What it does​

Simulated workers run queries that only acquire a lease, hold it for a configurable time, and release it. A denied acquire is handled as dispatch handles it: the query's lease id is released (dispatch drops its slot guard on every denial, so a denial costs two calls), then the acquire is retried with the same backoff (100 ms doubling to a 2 s cap, configurable).

  • Transports. --transport store calls CapacityStore on a PostgresStore in process: what the control plane does for each worker request, minus HTTP. --transport http goes through the control plane's /worker/capacity/* API (the real worker_api router, served in process on an ephemeral port and backed by Postgres) with the JSON a worker sends; each simulated worker has its own HTTP client. --worker-api-url points it at a running control plane instead. both runs each scenario twice.
  • Load. Closed loop (--concurrency C: C queries always in flight) or open loop (--rate R: Poisson arrivals at R/s, or evenly spaced with --uniform-arrivals; arrivals beyond --max-outstanding are shed and counted). Queries spread round-robin over --clusters, each capped at --cap. --query-duration is 0, fixed:MS or exp:MEAN_MS.
  • Cap-violation checker, every scenario. The harness counts admitted, unreleased queries per cluster (up on grant, down before release, so it can never run ahead of the leases really held). Any grant that takes a cluster above its cap is a violation. It also samples active_lease_count per cluster every --sample-ms on a separate connection pool, and checks no lease is left once every query has released. Any violation makes the command exit with status 1.
  • Postgres load. Transactions/s is the change in xact_commit + xact_rollback from pg_stat_database over the measured window; statements/s comes from pg_stat_statements when the extension is loaded, otherwise it is skipped with a note. Both include the benchmark's own sampling (a few queries per second).

Admission backends sit behind a small AdmissionBackend / WorkerAdmission trait pair in crates/queryflux-bench/src/capacity/backend.rs, so a new admission design (counter rows, quotas, a shared reserve) is one more implementation measured by the same scenarios and the same checker.

Running it​

Use a disposable database: migrations run first, and the benchmark writes and deletes lease rows under unique bench-… names.

# Predefined scenarios, both transports (release build)
make bench-capacity DATABASE_URL=postgres://queryflux@localhost:5432/queryflux_bench

# Longer runs and a JSON report
make bench-capacity DATABASE_URL=… BENCH_ARGS="--duration-secs 30 --json capacity.json"

# A custom scenario: 8 workers, 500 queries/s open loop, 4 clusters capped at 20, 40 ms queries
cargo run --release -p queryflux-bench -- capacity --database-url "$DATABASE_URL" \
--transport http --workers 8 --rate 500 --clusters 4 --cap 20 --query-duration exp:40

queryflux-bench capacity --help lists every flag. For statements/s, start Postgres with shared_preload_libraries=pg_stat_statements and run CREATE EXTENSION pg_stat_statements in the benchmark database. Measure on a Postgres with default durability (fsync and synchronous_commit on): a test instance with fsync=off hides the commit cost that dominates lease latency. On macOS, fsync does not flush the drive cache, so even a durable local instance commits faster than Postgres on a Linux server with a real disk.

Predefined scenarios (--scenario NAME, repeatable, or all). The first four reproduce the shape of the table in the high-throughput plan: maxRunningQueries 10,000 so every acquire is granted, zero-length queries (acquire then release), 4 workers.

ScenarioLoadClustersCapcoordinationPoolSize
serial1 at a time, 1 worker110,00032
c32-one-cluster32 concurrent110,00032
c32-eight-clusters32 concurrent810,00032
c32-eight-clusters-pool232 concurrent810,0002 (the default)
contended32 concurrent, 20 ms queries1832
contended-open300/s Poisson arrivals, 20 ms queries (75% of capacity)1832

--duration-secs (default 10) and --warmup-secs (default 2) apply to every scenario.

Reading the output​

ColumnMeaning
adm/sGrants per second in the measured window
acquire p50/p99/p999One acquire call, request to answer (granted or not)
admit p99Query arrival to grant, including denials and backoff
waited (n, p50/p99)How many queries were denied at least once, and their arrival-to-grant time
release p50/p99One release call
deniedAcquire calls answered "full" (each one is followed by a backoff and a retry)
errorsFailed acquire or release calls (timeouts, pool exhaustion); samples are printed below the table
hand-off p50/p99A query ending while others wait on the same cluster, to the next grant there: how long a freed slot sits idle. Approximate: the next grant consumes the oldest pending release
max in-flightHighest harness count of admitted, unreleased queries on any cluster, against the cap
violationsGrants above the cap plus lease-count samples above the cap. Must be 0
pg xact/s, pg stmt/sPostgres transactions and statements per second during the window

The JSON report (--json FILE, or --json - for stdout with the table on stderr) has the same numbers plus mean and max latencies, retries, shed arrivals, leaked leases, the scenario parameters, and the machine.

Baseline​

Measured 2026-10-19 on an Apple M1 Max (10 cores, 32 GB) against a local Postgres 17.6 with default durability (fsync and synchronous_commit on, pg_stat_statements loaded), 15 s per scenario after 3 s warmup, median of three runs with the range of admissions/s. The machine was busy with other builds during the runs, so the spread is wide. Indicative only: compare runs on the same machine, not against these numbers.

ScenarioTransportAdmissions/s (median, range)Acquire p50 / p99 msRelease p50 msDeniedHand-off p50 / p99 msWaited p99 msViolationsPG xact/sPG stmt/s
serialstore2,360 (1,979-2,394)0.24 / 0.60.140/s--04,70111,764
serialhttp1,349 (945-1,376)0.35 / 2.30.230/s--02,5906,748
c32-one-clusterstore3,267 (2,481-3,407)8.47 / 24.90.220/s--06,73416,341
c32-one-clusterhttp3,121 (2,508-3,411)9.05 / 21.90.260/s--06,31815,611
c32-eight-clustersstore6,793 (6,391-7,713)2.80 / 14.41.090/s--013,31833,978
c32-eight-clustershttp6,874 (5,122-7,375)2.90 / 10.01.200/s--013,70134,400
c32-eight-clusters-pool2store1,763 (1,252-3,908)7.11 / 35.17.070/s--03,7208,843
c32-eight-clusters-pool2http1,396 (1,274-3,339)9.38 / 40.89.390/s--02,8447,011
contendedstore324 (299-343)1.73 / 23.80.6430/s2.6 / 24.51282107301,774
contendedhttp325 (291-336)1.95 / 15.80.7131/s2.9 / 20.61761107211,758
contended-openstore303 (302-304)0.70 / 9.30.31210/s3.8 / 22.8631801,0372,571
contended-openhttp301 (300-301)0.85 / 9.60.46195/s4.1 / 23.8471409972,483

What it shows:

  • One hot cluster tops out around 3,000 admissions/s however many workers ask (the per-cluster advisory lock, COUNT(*) and INSERT), while 8 clusters reach about 6,800/s.
  • The default coordinationPoolSize of 2 caps the whole process at roughly 1,400 to 1,800/s for any number of clusters, with release waiting on the pool as long as acquire.
  • Every admission costs Postgres 2 transactions and 5 statements (BEGIN, advisory lock, count-and-insert, COMMIT, then the release DELETE), plus another release per denial: about 10,000 transactions/s at 5,000 admissions/s.
  • Waiting is polling. In contended, queries that are denied once wait seconds (p99 13 to 18 s) because the 100 ms to 2 s backoff keeps losing to queries that arrive just as a slot frees. In contended-open, at 75% of capacity, about a third of queries are denied at least once and wait up to 3 to 6 s (p99), although a freed slot is reused within about 25 ms (hand-off p99).
  • No cap violations in any run.

These differ from the plan's table (measured through a control plane with Postgres in Docker on macOS, which adds network latency to every call): absolute numbers here are higher, but the shape is the same.

Not covered yet​

Full-stack mode (a control plane plus real workers, DuckDB SELECT 1 queries over Trino HTTP, from an open-loop load generator) is not implemented. It would add per-query worker CPU (translation, fingerprinting) and history ingest to the picture; until then, combine this benchmark with the proxy-overhead numbers above.