---
title: "How we built a query planner for a real-time query language"
slug: "query-planner"
locale: "en-us"
description: "How we designed and implemented the CQL query planner for maximum performance and parallelism."
published_at: "2026-10-22"
updated_at: "2026-11-04"
image: "https://storage.googleapis.com/croct-assets-b931d070/blog/Query_planner_for_a_real_time_query_language_69fc9e8b2c/Query_planner_for_a_real_time_query_language_69fc9e8b2c.png"
authors: ["Marcos Passos"]
category: "Product"
tags: ["Engineering","Segmentation"]
canonical: "https://blog.croct.com/post/query-planner"
---

# How we built a query planner for a real-time query language

CQL is the language our customers use to describe audiences. A query can read anything we know about a visitor: profile attributes, identity resolved across devices, purchase history, cart contents, the city they are browsing from, the current hour in their timezone.

Most personalization platforms allow far less than that. Many only let a query read what the page already holds, and answer it in the page for that reason. It is a fair trade, because nothing waits on a network and latency stays predictable. We went the other way and made the data cheap to reach instead. Everything a query can read sits in stores we run and keep hot, next to the engine that reads them, because going out to a warehouse or a third-party API during a request would cost seconds and we have tens of milliseconds.

That choice is what makes the language worth writing in. It is also the whole engineering problem.

## The problem

Here is an audience a customer might save:

```cql noLineNumber
user's plan is 'pro' and cart's total > 100
  and user has not ordered a product with category like "running"
```

Three things in that sentence need data that did not arrive with the request. The plan is a profile attribute, the total comes from a cart, and the order test is a query over an event store. We call each one a lookup. A lookup takes 5 to 15 milliseconds, nearly all of it spent waiting on a network, and the request has 50 to 80 milliseconds in total before the page paints and the visitor sees content meant for someone else.

Three lookups one after another is 45 milliseconds at the top of that range. The same three at the same time is 15. That gap is the problem, and our first engine could not close it.

![Each lookup takes about 15 ms. Discovered one at a time, they add up; declared up front, they overlap.](https://storage.googleapis.com/croct-assets-b931d070/blog/Sequential_vs_parallel_1b47700a98/Sequential_vs_parallel_1b47700a98.png "Each lookup takes about 15 ms. Discovered one at a time, they add up; declared up front, they overlap.")

CQL started with an interpreter, which parses a query into a tree and walks it, asking each node for its value. Walk the query above and the fetches can only go one at a time. The walker reads the plan and waits. It reaches the total, discovers it needs the cart, and waits again. It reaches the order test and only then finds the event store. Each fetch starts after the one before it comes back, because the walk is what discovers the dependency.

Fetching all three up front is not the answer either, because the conditions decide what gets read. A visitor who is not on the pro plan makes the query false at the first test, and the cart and the order history are never touched. An engine that eagerly fetched everything would waste two lookups on most requests.

So the problem has two halves. We need to know every lookup before the query runs, and we need to know which of them the query will actually reach.

## The plan

A query is written once and answered millions of times, and its lookups do not change from one visitor to the next. The query above reads the profile every time, and may read the cart and the order history. Those facts belong to the sentence, not to the request, and they were knowable the moment someone pressed save. The interpreter rediscovers them on every request, one at a time, at the worst possible moment.

Our [static analyzer](/post/static-analyzer.md) already walks a query to infer a type for every node and narrow those types through conditions. The same walk tells us which lookups the query reads, and which of them sit behind a condition.

For the audience above it comes back with one certain lookup, the profile, and two possible ones, the cart and the order history. That split answers the second half of the problem. The profile is read on every evaluation, so it always starts. The other two sit behind conditions, so starting them early is a bet, and we come back to how that bet is made.

We prepare each query once, ahead of any request, into a form that carries its plan and runs without the tree. Preparing it costs about five times what answering it once costs, paid the first time we see the query, which is an easy trade for an audience that is saved once and answered for months. We only plan when there are at least two lookups to start, since one gains nothing from being announced early.

## Running the plan

The engine starts every certain lookup before the first instruction runs, each as a separate task, then runs the body. By the time the body needs a value, the fetch is already in flight, so it waits on a result instead of starting a request. The request waits once, for the slowest lookup.

A second effect comes free. A request rarely carries a single query. An experience can target several audiences, and the content it serves can carry several dynamic values, each one its own expression. Those queries overlap heavily in what they read, since several of them want the profile and several want the cart. Because every query declares its lookups up front, the whole batch is merged: the first query to need the profile starts the fetch and the rest wait on it.

Measured against the previous engine, with a data source held at 10 milliseconds per lookup:

![Request time against the previous engine, with a data source held at 10 ms per lookup.](https://storage.googleapis.com/croct-assets-b931d070/blog/Pre_fetch_threshold_0cece695a8/Pre_fetch_threshold_0cece695a8.png "Request time against the previous engine, with a data source held at 10 ms per lookup.")

The first row is the control. One lookup has nothing to overlap with, so the engines tie, and planning only helps a request that makes more than one.

The engine itself got faster on the way. Evaluating a prepared query takes 3.7 microseconds for the median audience against 9.4 for the interpreter, measured across the audiences our customers actually run. Next to a 15 millisecond lookup that is noise, which is why the planning is what matters. The slope matters more than any single row: a query added to a batch used to cost 6.9 milliseconds and now costs 0.4, so a request has largely stopped getting more expensive as it asks for more.

## Learning what to prefetch

For a possible lookup there is no right or wrong, only a bet. Starting it early saves a round trip when the query turns out to need it, and wastes a request when it does not. The same query can go either way on two different sites, so there is nothing in the text to read the answer from.

So the planner *learns* instead. For each conditional lookup it watches how often the queries that could reach it actually do, and decides by that frequency. Above a threshold it starts the lookup as soon as the query begins, and below it the query waits as before. The threshold is tunable. Frequencies fade as traffic moves, so a lookup that stops being needed stops being prefetched.

![How often the queries that could reach a lookup actually do decides whether it starts before the query runs.](https://storage.googleapis.com/croct-assets-b931d070/blog/Request_time_bd3fa28325/Request_time_bd3fa28325.png "How often the queries that could reach a lookup actually do decides whether it starts before the query runs.")

## Where this goes

Planning also makes room for the lookups we are working toward. A model prediction is expensive to compute, so being able to start it alongside everything else is what makes a question like this practical:

```cql noLineNumber
user's likelihood of buying > 0.8 and cart's total > 50
```

All of this is so that someone can ask whatever they need about a visitor, written however they want to write it, without thinking about what it costs to answer. That part is ours.