The blocker is rarely the engine. It's the clients. Dashboards, notebooks, dbt profiles and application code all speak Snowflake's protocol and Snowflake's SQL. Moving a workload means touching every one of them.

QueryFlux removes that step. Snowflake clients connect to it as if it were Snowflake, and it decides which engine runs each session, Snowflake included.

How it works​

QueryFlux speaks the Snowflake HTTP protocol and the SQL API v2. The Python connector, JDBC drivers and SnowSQL log in to QueryFlux the same way they log in to Snowflake. At login, a router picks a cluster group for the session: by the warehouse, role or user the client asked for. Every query in that session runs on that group, with Snowflake SQL translated to the target engine's dialect by sqlglot.

Snowflake itself stays one of the backends, through the ADBC Snowflake driver, so anything you don't move keeps running exactly where it ran before.

The config​

Turn on the Snowflake frontend, declare both backends, and route by warehouse:

queryflux:
  frontends:
    snowflakeHttp:
      enabled: true
      port: 8443

clusters:
  snowflake:
    engine: adbc
    driver: snowflake
    uri: svc_user@myaccount/analytics/public
    auth:
      type: keyPair
      username: SVC_ACCOUNT
      privateKeyPem: "..."
  trino-1:
    engine: trino
    endpoint: https://trino.internal:8443

clusterGroups:
  snowflake: { maxRunningQueries: 50, members: [snowflake] }
  lakehouse: { maxRunningQueries: 100, members: [trino-1] }

routers:
  - type: pythonScript
    script: |
      def route(query: str, ctx: dict) -> str | None:
          # Sessions that ask for the BI warehouse run on the lakehouse engine.
          if ctx.get("extra", {}).get("snowflake.warehouse") == "BI_WH":
              return "lakehouse"
          return None

routingFallback: snowflake

The router sees the login's warehouse, role and schema (snowflake.warehouse, snowflake.role, snowflake.schema) and the user. Here, sessions on BI_WH go to Trino; every other session stays on Snowflake. Start with one warehouse, watch it in Studio, then move the next.

ADBC drivers aren't bundled in the image; declare the Snowflake driver when you start the container (see ADBC drivers).

Point the clients at QueryFlux​

The only change on the client side is the host:

import snowflake.connector

conn = snowflake.connector.connect(
    account="queryflux",
    user="analyst",
    password="...",
    host="queryflux.internal",
    port=8443,
    warehouse="BI_WH",      # this session will run on Trino
    database="analytics",
    schema="public",
)

Dashboards keep their SQL. Notebooks keep their code. Only the host in the connection settings changes, once.

What has to be true​

Offloading is only as good as the engine receiving the work, so check these before you move a workload:

• The data is readable by the target engine. The usual setup: tables in Iceberg that Snowflake and Trino (or StarRocks) both read. QueryFlux maps Snowflake's database.schema.table to the engine's catalog.schema.table, so name the engine's catalog after the Snowflake database.
• The SQL translates. sqlglot covers most Snowflake syntax, but not all of it. Run the workload's real queries through QueryFlux before switching, and add Python fixup scripts for anything that needs a nudge. Query history keeps both the original and the translated SQL, so you can see exactly what ran.
• Routing is per session. The group is chosen at login and kept for the session; choose by warehouse, role or user, not per query.
• Some Snowflake features aren't proxied. Async execution, multiple statements per request, ALTER SESSION and server-side cancel aren't supported, and TLS is terminated in front of QueryFlux. The Snowflake frontend docs list the details.

What you get​

• A way out of all-or-nothing. Move one warehouse's workload at a time, and move it back by changing one rule.
• The same clients. No dashboard rewrites, no new drivers.
• One record. Every query, on Snowflake or elsewhere, lands in one history with its user, route and timing, so you can compare before and after.
• Your engines, your environment. QueryFlux is open source and runs next to the engines you already operate.

Try it: the Snowflake frontend docs cover configuration and clients; routing and clusters covers the Python router.

In practice, every client still has to choose. Your BI tool talks MySQL to StarRocks; your notebooks talk to Trino over HTTP; each has its own connection string, its own dialect and its own idea of capacity. Moving a dashboard from one engine to the other means touching the dashboard.

This guide puts one SQL endpoint in front of both engines, so clients connect once and QueryFlux sends each query to the engine that should run it. You can run the whole stack on a laptop.

What you'll build​

• An Iceberg REST catalog (Lakekeeper) on S3-compatible storage (RustFS), loaded with TPC-H.
• Trino and StarRocks, both reading the same Iceberg tables.
• QueryFlux in front: Trino HTTP on port 8080, MySQL wire on port 3306, and Studio on port 3000 to see where every query went.

1. Start the stack​

You need Docker and Docker Compose, with enough memory for Trino and StarRocks.

git clone https://github.com/lakeops-org/queryflux.git
cd queryflux/examples/full-stack
docker compose up -d --wait

Load TPC-H into Iceberg through Trino, then register the same catalog in StarRocks:

docker compose --profile loader run --rm -T data-loader
docker compose --profile loader run --rm -T starrocks-catalog-setup

Both engines now see the same eight TPC-H tables under lakekeeper.tpch. For more data, rerun the loader with TPCH_SCALE=sf1.

2. Same table, two engines, one endpoint​

Connect a Trino client to QueryFlux (not to Trino directly):

docker compose exec -it trino trino --server http://queryflux:8080 --user test

SELECT o_orderpriority AS priority,
       count(*) AS orders,
       round(sum(o_totalprice), 2) AS revenue
FROM lakekeeper.tpch.orders
GROUP BY o_orderpriority
ORDER BY orders DESC;

Now connect a MySQL client to QueryFlux on port 3306 and run the same query:

docker compose exec -it starrocks mysql -h queryflux -P 3306 -u root

Same numbers, but StarRocks ran it this time. Open Studio at http://localhost:3000 (admin / admin), go to Queries, and you'll see each query with the engine that ran it and the route it took.

3. How the routing works​

The example's config is short:

clusters:
  trino-1:
    engine: trino
    endpoint: http://trino:8080
  starrocks-1:
    engine: starRocks
    endpoint: mysql://starrocks:9030
    auth: { type: basic, username: root, password: "" }

clusterGroups:
  trino-default:     { maxRunningQueries: 100, members: [trino-1] }
  starrocks-default: { maxRunningQueries: 50,  members: [starrocks-1] }

routers:
  - type: protocolBased
    trinoHttp: trino-default
    mysqlWire: starrocks-default

routingFallback: trino-default

• Clusters are the engines. QueryFlux talks to Trino over HTTP and to StarRocks over the MySQL protocol.
• Cluster groups wrap them, each with its own concurrency limit. When a group is full, queries wait at the proxy instead of piling onto the engine.
• Routers pick the group. Here it's simply the protocol the client used.

4. Route by workload, not by protocol​

Routing by protocol is a start. The real win is routing by what the query is, so a client keeps one connection and still lands on the right engine.

Send queries tagged as dashboards to StarRocks, and keep everything else on Trino. A Trino client tags its queries with the standard client-tags header, for example trino --client-tags dashboard or X-Trino-Client-Tags: dashboard:

routers:
  - type: tags
    rules:
      - tags: { dashboard: null }   # key-only tag, as Trino clients send them
        targetGroup: starrocks-default
  - type: protocolBased
    mysqlWire: starrocks-default
routingFallback: trino-default

Or decide in Python, with the SQL, protocol, user and session in hand:

routers:
  - type: pythonScript
    script: |
      def route(query: str, ctx: dict) -> str | None:
          q = query.upper()
          # Short aggregations over the serving tables: StarRocks.
          if "GROUP BY" in q and "JOIN" not in q:
              return "starrocks-default"
          return None  # fall through to the next router

A Trino client routed to StarRocks still writes Trino SQL: QueryFlux translates the dialect with sqlglot before the query reaches StarRocks.

You can also put both engines in one group and move traffic gradually, for example 10% to StarRocks, then 50%, then all of it, with a weighted strategy, while Studio shows per-engine latency side by side. That's how you migrate a workload between engines without touching a single client.

Why this matters​

Iceberg made the data portable. A single SQL endpoint in front of your engines makes the queries portable:

• Clients connect once, over the protocol they already speak.
• Each workload runs on the engine that fits it, and you can change your mind without a migration project.
• Every query, on every engine, lands in one history with its route, timing and user.

The same setup takes more engines (DuckDB, ClickHouse, Athena, and Snowflake, BigQuery or Databricks through ADBC) as you add them.

Try it: the full walkthrough is in the multi-engine quickstart. Routing options are in routing and clusters.

Agents shouldn't choose engines, and they shouldn't hold warehouse passwords. Something in the query path should decide where each query runs, what it may do, and keep the record.

This post shows how to set that up with QueryFlux using features that ship today.

The setup in one file​

One QueryFlux config gives agents a verified identity, their own entry point, capacity pool, guardrails, and per-user identity on the engine:

The sections below map directly to that file.

1. One endpoint, any agent​

Turn on OIDC (or another verified provider) with auth.required: true so MCP callers cannot connect as anonymous. Agents send a bearer token on every request:

The agent gets tools to run a query, list schemas, describe a table, read the engine's plan, and check or cancel what it started.

Agents that prefer a SQL driver can connect over Trino, PostgreSQL, or MySQL instead — routing, guardrails, impersonation, and audit work the same because they live in the query path, not in the MCP layer.

2. Agents get their own lane​

The protocolBased router sends every MCP query to the agents group, which has its own maxRunningQueries. A burst of exploratory agent queries queues in its own lane instead of starving dashboards in analysts.

MCP's execute_query also accepts an optional engine_hint that can name a cluster group and skip the router. Without a group allow-list, an authenticated agent could hint analysts and pick up that group's capacity without its agent guardrails. The config above sets authorization.allowGroups on each group so only IdP group analysts can reach analysts, and only agents can reach agents.

With Postgres persistence, the limit holds across every QueryFlux replica and worker, and waiting queries are served first come, first served.

3. Guardrails in the query path​

The agents group gets three built-in guards on top of any global ones:

A blocked query returns an error naming the rule that failed, so the agent can rewrite and retry. When the built-ins aren't enough, add a Python guard or call your own webhook — both receive the agent's context.

4. The engine sees the person, not a shared account​

With queryAuth: impersonate, QueryFlux authenticates to Trino with its own service account but runs each query as the verified user from the bearer token. The engine applies that person's permissions and writes that person into its audit log. There is no shared password for an agent to leak or overreach with — and without auth.required, impersonation would only forward anonymous.

Trino and ClickHouse support impersonation. For Snowflake and Trino, QueryFlux can instead exchange the caller's OAuth token for one the engine accepts, or pass the caller's credential through.

For finer control, add row filters and column masks from OPA or Cerbos — they apply to agents exactly as they apply to people.

5. Every query has an owner​

QueryFlux stores agent context when it is present:

• MCP: agent_id defaults from the authenticated user and conversation_id from the MCP session. step_index and tool_call_id are optional — the client must send them (as tool parameters or X-Step-Index / X-Tool-Call-Id headers) if you want a full step-by-step trail.
• Trino HTTP (and other SQL drivers): both X-Agent-Id and X-Conversation-Id are required for the request to count as agentic; otherwise it is treated as a normal client query. Optional headers cover step, tool call, and intent.

In Studio, the Agents and Conversations pages group history by agent and session so you can replay what was recorded: every query, where it ran, how long it took, and which guardrail stopped it.

What this adds up to​

An agent gets one endpoint and the SQL it needs. You keep control of:

• which engine runs each query;
• how much of the cluster agents can use;
• what agents may do;
• whose permissions apply on the engine;
• a record of agent and conversation — plus step/tool metadata when the client supplies it.

It works the same over MCP and over a SQL driver, on the engines you already run.

Apache Iceberg and the broader open table format movement changed the contract between storage and compute. Data stopped belonging to a single engine. Instead of copying the same dataset into five systems to support five workloads, teams keep one copy and let many engines read it: Spark for batch, Trino for interactive SQL, DuckDB for local analysis, StarRocks for serving. One dataset, many readers.

Data became interoperable. Query engines did not.

We solved storage interoperability. Compute is next.​

Most teams still treat query engines as silos, and they pay for it in ways that compound quietly.

Every engine speaks its own wire protocol, has its own SQL dialect quirks, and needs its own connection management. Multiply engines by clients (BI tools, notebooks, application code, CLIs) and you don't get a linear integration problem; you get a combinatorial one. Each pairing needs its own driver, dialect handling, and retry logic. Routing, rate limiting, and observability get reinvented in isolation, over and over.

Cost and performance follow the same pattern​

Cloud analytical systems charge either for compute time or for data scanned, and query shapes skew toward compute-bound or IO-bound work. Each class tends to run cheaper under a different pricing model.

While building QueryFlux we ran the same analytical SQL across engines with different pricing models. A poor match between query shape and billing model inflated cost by 2–5× against a better-matched engine. In a prototype that steered CPU-heavy work to compute-priced backends and scan-heavy work to byte-priced ones, total cost for one representative suite fell by up to about 56%, and some individual queries by up to about 90%.

That was a prototype, not a built-in cost optimizer (today you express that logic as routing rules or a Python router), but the gap it showed is structural.

Performance tells the same story from another angle. Trino is built for federation: a coordinator plans the query, schedules stages across workers, and assembles results. For a query scanning millions of rows across dozens of nodes, that overhead is well spent. For a quick filter or a small aggregation, it isn't. DuckDB runs in-process, with no network round trips and no worker coordination, and the same small query can come back in tens of milliseconds instead of a second.

Route by shape, not by hope​

With QueryFlux, you express “small local queries → DuckDB, everything else → Trino” as configuration instead of application code:

When regex isn't enough, drop to Python for the same decision:

The client (BI tool, notebook, or CLI) sends queries as it always has. QueryFlux picks the engine and translates the dialect. The application doesn't change.

Without one layer in front of the engines, capturing those savings is hard: there is no single place to observe queries, apply routing logic, and dispatch to the right backend. Teams end up locked into one engine, or maintaining bespoke glue that can't reason about cost or capability in one place.

What QueryFlux is​

QueryFlux is an open-source SQL proxy and router, written in Rust: one front door, many engines, explicit routing, automatic dialect translation, and shared capacity limits and observability.

Clients connect over the protocol they already speak:

QueryFlux routes each query to a backend, translates SQL when needed, and streams results back in the client's native format. The frontend never knows which engine it is talking to; the engine adapter never knows which protocol the client used.

Native adapters cover Trino, DuckDB, StarRocks, ClickHouse, Athena, and MySQL. The Apache Arrow ADBC driver manager reaches Snowflake, BigQuery, Databricks, Redshift, PostgreSQL, SQL Server, and more. Adding an engine or client protocol means implementing one well-defined Rust trait.

How routing works​

Routing is a two-stage process:

Group selection. A chain of routers inspects each query in order and returns a cluster group — a named pool of backends. The first match wins; if nothing matches, a fallback group takes the query. Routers can match on client protocol, HTTP headers, SQL patterns, client tags, the user's group, or a Python route(query, ctx) function.

Cluster selection. Within the group, a strategy picks one backend: round-robin, least-loaded, failover in priority order, engine affinity, or weighted. A cluster must be healthy, enabled, and under its concurrency limit. If none is, the query waits in line rather than being dropped or sent to an overloaded backend.

Operators express intent at two levels: which pool of engines fits a class of work, and how traffic spreads across that pool.

Dialect translation as a first-class concern​

Routing to the right engine solves only half the problem. Engines speak slightly different SQL, and expecting clients to know the dialect of whichever engine they were routed to defeats the point.

QueryFlux translates with sqlglot (30+ dialects). Translation runs after routing has picked a cluster and before the SQL reaches it: the source dialect comes from the client protocol, the target from the engine. When they match, translation is skipped. Where sqlglot doesn't cover a case, Python fixup scripts can adjust the parsed query, globally or per group.

A client writing Trino SQL can hit DuckDB or StarRocks without knowing it. The engine becomes an infrastructure decision, not an application one.

Built to operate, not just to route​

From "one copy, many reads" to "one query, many execution paths"​

The lakehouse already embraced one copy of data with many readers. QueryFlux extends the idea to execution: one query interface, many execution paths.

Instead of asking which engine a team should use forever, or which dialect to write in, you can ask which engine fits this workload, whether you can fall back when one struggles, and how to keep application code stable while the engines underneath change.

Why open source​

If the data world moved to open table formats to escape lock-in, it makes no sense to solve compute interoperability with a proprietary black box in the middle. An open-source router is a neutral layer above the engines: a place for the community to add adapters and routing policies, and a system that stays composable instead of becoming another platform to be trapped in.

Open data deserves open compute coordination.

The bigger opportunity: agents​

Once queries can be normalized, understood, and routed across engines, you have a system that decides where work runs and how. That matters for analytics, and just as much for the AI agents and application backends that increasingly depend on live data.

Agents don't use data the way dashboards do. They explore, retry, and reformulate, generating SQL against systems they don't fully understand. If every engine stays a silo, every agent has to learn engine-specific dialects, limits, performance trade-offs, and connection logic. That is a poor foundation.

Sitting above the engines, QueryFlux gives agents one stable interface: an MCP endpoint or any SQL driver. It records which agent and conversation every query came from, and puts the same guardrails and access policies in front of agents as in front of everyone else. The agent says what it wants; QueryFlux decides where it runs.

Read next: Give agents SQL, not the keys to every engine.

One copy of data. Many readers. One query. Many possible engines.