Appendix 2: The Bugs of CQL

A point-in-time study of defects in the public history of CG/SQL

Study date: October 13, 2026

Public-history window: October 7, 2020 through October 10, 2026

Analyzed endpoint: 957db34adb4337afb4498d9feabea7cdb4f9036a

Repository: ricomariani/CG-SQL-author

How to read this appendix

Read this as a historical snapshot. It is not a live quality dashboard. The counts, file sizes, examples, and conclusions describe the public repository at the analyzed endpoint on October 10, 2026. I did not read later fixes or architectural changes back into the results.

The public history also begins with an already mature compiler. It does not contain CQL’s earlier internal development, when the architecture and defect profile would have been substantially different. This appendix is about the maintenance history of the mature, open-source codebase.

How I made this appendix

I used an AI coding agent to mine the complete public Git history, screen candidate fix commits, measure the source and test corpus, retrieve historical patches, compare them with current code, and produce the first draft. It did a lot of the raw work, so I did not personally research and write every sentence from scratch.

I deliberately left the agent room to read the history and reach its own conclusions because the independent reading was useful. Then I heavily curated it. I added my own domain knowledge, supplied operational and architectural context that Git cannot provide, checked the examples against the source and historical patches, and corrected bad assumptions.

That changed several conclusions:

• I corrected the interpretation of CQL’s fail-fast AST and tree-shape development model;
• I separated observed operational bugs from development-time failures, CI-prevented pushes, and prophylactic audit hardening;
• I added what I remember from sustained production use: roughly 12 real bugs per year;
• I corrected an overbroad interpretation of compiler global state and reentrancy;
• I distinguished incomplete first fixes from additional field failures;
• I audited every code sample and expanded examples that did not show enough setup, operation, or test evidence.

So the agent did a lot of the research and drafting. I checked and rewrote the result, and I own it. The appendix is in my voice and these are my conclusions.

Use this as an engineering study. There is no authoritative incident ledger behind these numbers, so read them with the limits stated below.

Summary

The short version: CG/SQL’s public history looks like a mature compiler with a very specific defect shape. The bugs cluster at boundaries between otherwise well-structured subsystems.

Here is the pattern that matters most:

The code usually gets each local operation right. Bugs recur when one fact must remain true across several compiler phases, representations, backends, or ownership conventions.

I came away with seven ideas that organize the history:

1. Cross-phase semantic invariants - nullability, sensitivity, error state, identifier spelling, and ownership must survive semantic analysis, rewriting, SQL generation, and multiple backends. Novel AST kinds are a special case: the normal path is deliberately fail-fast, so missing handlers are forced out during development.
2. Schema evolution needs an observable-state oracle - create order, drop order, dependency order, declaration order, and quoted-name handling are related but not interchangeable. More importantly, version CRCs and housekeeping cannot be allowed to overrule the physical schema they claim to describe.
3. Global compiler state and C ownership - the intentionally stateful implementation is concise and effective, but nested compilation work, repeated invocation, aliasing, and +1 reference ownership require explicit protocols.
4. Representation boundaries and changing contracts - growable buffers, serialized blobs, offset tables, platform integer widths, and C aliasing/alignment rules produce a disproportionate share of low-level fixes. The two binary serializers had very different starting requirements, which explains why their histories also look different.
5. Rationale comments as design memory - non-obvious transformations, invariants, constraints, ordering rules, ownership exceptions, and warnings against tempting changes preserve the WHY that the C type system cannot express.
6. Coverage-driven fail-fast development plus independent tests - 100% coverage and always-on contracts force new tree shapes through every stage that can see them. Golden output is still only a consistency oracle, so the hardest remaining bugs survived when the compiler and its expected outputs shared the same mistaken assumption.
7. Outputs and examples can become infrastructure - JSON became a stable intermediate contract, and some Python examples eventually replaced large C backends. Their required standard changed when their job changed.

CQL is also a useful counterexample to a fair amount of conventional C advice. It has a 27,000-line semantic-analysis file, hundreds of file-static variables, large macros, and several very long functions. At the same time, the median function is only 14 lines, naming is exceptionally regular, the comment corpus is unusually large and purposeful, contracts are dense and always on, arenas simplify allocation lifetimes, and the test suite compiles and executes generated code through several independent paths.

The apparent contradiction is the interesting part:

CQL succeeds because its departures from conventional structural advice are consistent, and because the compensating mechanisms fit the compiler it actually is.

Scope and method

I examined only the public Git history. The root commit is:

At the analyzed mainline tip, I found:

I screened commit messages conservatively:

• 809 commits contained some possible defect language.
• 284 subjects used stronger defect language.
• 88 commits remained after requiring production-code changes and excluding obvious documentation, formatting, build-only, and coverage-only work.

The number 88 is not a bug count. Commit subjects overcount and undercount in equal-opportunity fashion. Some commits called “fix” are cleanup. Some real fixes are titled “Handle”, “Require”, “Validate”, “Coverage”, or simply state a new invariant. A large audit commit can also contain several independent corrections. Recent audit work also includes theoretical invariant hardening that had no known failing caller. Rapid revert/reapply chains can record one real bug plus a bad attempted fix that CI prevented from progressing. I use the screen for trends and candidate selection, not as an exact defect ledger.

I then inspected representative commits by patch, affected production files, tests, and generated-reference changes. The safety boundary is deliberate: I did not search for new defects, try to trigger unknown failures, or assess exploitability.

The public defect profile

Activity by year

The 2025 number shows why message-based classification needs some humility. An important reentrancy fix was hidden in a commit whose subject began with coverage work. The 2026 screen likewise misses plainly corrective commits titled “Validate”, “Require”, “Handle”, and “Size”. What I can say is that 2026 contains a concentrated correctness-audit spike. Much of that spike came from AI-assisted review, and several findings are prophylactic rather than observed failures. Calling these “exactly ten bugs” would give the heuristic precision it does not have.

Observed bugs versus audit hardening

The late-2026 changes need their own bucket.

For most older fixes, the history and my recollection are consistent with a reported failure, something I personally observed, or a failure found while developing a feature. I treated the recent AI-assisted audit work differently. I let the agent ask, “What assumption here is not enforced?” That is a useful question, and it produces a different kind of change.

Representative late-audit changes added buffer-aliasing contracts and whole-shape representation checks. These are prophylactic hardening, not evidence of field failures; detailed examples appear below.

CI-caught changes are bugs prevented

Git history records what was pushed, not what survived the delivery process. Those are not the same thing.

The cql_multinull object-handling chain is the cleanest example:

1. f97fb8be74dc attempted to fix a real missing runtime case.
2. 5b70a230f899 reverted that attempt 14 minutes later.
3. 64d4305c205e applied the properly constrained fix the next day.

What happened is simple: I pushed the first fix too aggressively without running the full suite. CI caught it immediately and forced the correction. The original missing object case was a real bug. The bad attempted fix was not a second operational bug; it was a bug prevented by the test and delivery system.

This distinction matters to the statistics. A raw fix-commit count can assign two fixes, a revert, and a reapplication to one escaped defect. Rapid CI-driven follow-ups should be collapsed into the originating issue and credited to the prevention machinery, not counted as additional field failures.

Where fixes occurred

The following counts overlap because a commit can affect several phases:

This distribution looks right for a compiler, with one CQL-specific twist. Traditional optimizing compilers often find many wrong-code bugs in optimization passes. CQL has extensive rewrites and code generation, without a GCC/LLVM-style tower of aggressive machine-level optimization passes. CQL’s specific concentration is in semantics-preserving rewrites, schema migration, backend/runtime agreement, generated ownership, and SQLite-facing output–not machine-level optimization.

Is CQL reliable?

Yes.

My recollection from the years of constant use at Meta is about 12 real bugs per year. The public history lines up with that number.

The conservative screen found 88 production-fix candidates over roughly six public years, or about 14.6 candidate commits per year. Using the five complete years from 2021 through 2025 gives 14.8 per year. Candidate commits mix hardening, pre-check-in failures, CI-rejected attempts, cleanup, multi-commit fixes, and bugs hidden by neutral or bundled commit subjects, so the screen cannot establish an incident count.

Given those effects, roughly 12 real bugs per year is entirely plausible. Git cannot prove the exact number; that would require a complete issue and incident ledger. Still, the number is supportable and fits the public evidence.

Even if I set my recollection aside, I judge that rate to be exceptionally low. The system is roughly 83,000 lines of current core C and headers, with a grammar, multiple generators, a runtime, schema migration, several representation formats, and years of sustained production use. About one observed bug a month across that surface is a very strong result.

I do not have a statistically calibrated comparison population, so I cannot attach a percentile to “exceptionally low.” It is my engineering judgment, not a measured statistic.

So yes, the compiler works. The low operational bug rate, fail-fast covered tree walking, rationale-heavy comments and preserved design history, layered execution tests, and long production use all point in the same direction.

Ranked findings

1. Fail-fast tree walking works; cross-phase semantics are harder

The largest defect family cuts across labels such as “parser bug”, “semantic bug”, and “code-generator bug.” The common ingredient is a fact represented in more than one place.

I would separate AST kind dispatch from the rest because CQL is deliberately designed to blow up on a novel tree shape. The semantic expression dispatcher does this:

cql_noexport void sem_expr(ast_node *ast) {
  // These are all the expressions there are, we have to find it in this table
  // or else someone added a new expression type and it isn't supported yet.
  symtab_entry *entry = symtab_find(exprs, ast->type);
  Invariant(entry);
  sem_expr_dispatch *disp = (sem_expr_dispatch*)entry->val;
  disp->func(ast, disp->str);
}

The C expression generator does the same thing after semantic analysis:

// These are all the expressions there are, we have to find it in this table
// or else someone added a new expression type and it isn't supported yet.
symtab_entry *entry = symtab_find(cg_exprs, expr->type);
Invariant(entry);
cg_expr_dispatch *disp = (cg_expr_dispatch*)entry->val;
disp->func(expr, disp->str, is_null, value, pri, disp->pri_new);

The extraction macros then check the detailed child shape at each step. A grammar change that adds or changes a reachable tree shape should therefore produce an immediate assertion at the first phase that has not been updated.

The 100% coverage norm is what makes this useful rather than aspirational. A new parser production is exercised, semantic analysis fails until the new shape is handled, and any shape that survives semantic analysis fails again in codegen until that stage handles it. Development becomes a sequence of concrete failures. You do not have to forecast every registration in advance because the compiler tells you where the next one is.

That works extremely well. In the normal covered path, a missing handler cannot quietly persist.

The persistent examples are the facts that can be handled locally and still be wrong globally:

• a semantic error reported but not recorded in the enclosing node;
• nullability or sensitivity copied from only one operand;
• a quoted identifier handled by static SQL but not dynamic upgrade SQL;
• a rewritten expression preserving its value but not its ownership or type wrapper;
• a logical table name renamed in ordinary SQL expressions but not dot expressions.

The missing registrations were mostly deliberate no-op cases

In May 2022, declare_interface_stmt was a valid AST kind, but the C backend had no no-op registration for it:

   NO_OP_STMT_INIT(declare_const_stmt);
   NO_OP_STMT_INIT(declare_named_type);
   NO_OP_STMT_INIT(declare_proc_no_check_stmt);
   NO_OP_STMT_INIT(declare_select_func_no_check_stmt);
+  NO_OP_STMT_INIT(declare_interface_stmt);

Commit: 3a9f41f21a266ab5377d23d9df3594c9a2a2f9c7

At that point, statement code generation immediately required a registered handler:

symtab_entry *entry = symtab_find(cg_stmts, stmt->type);
Contract(entry);

That commit changed only the registration line; it added no focused regression test. What supports the interpretation is the fail-fast lookup, the crash-specific commit subject, and the missing no-op registration itself.

The same narrow family reappeared in December 2024, when two macro-definition AST kinds needed no-op registration in both C and Lua backends:

diff --git a/sources/cg_c.c b/sources/cg_c.c
@@
   NO_OP_STMT_INIT(expr_macro_def);
   NO_OP_STMT_INIT(stmt_list_macro_def);
   NO_OP_STMT_INIT(query_parts_macro_def);
   NO_OP_STMT_INIT(cte_tables_macro_def);
+  NO_OP_STMT_INIT(select_expr_macro_def);
+  NO_OP_STMT_INIT(select_core_macro_def);

diff --git a/sources/cg_lua.c b/sources/cg_lua.c
@@
   LUA_NO_OP_STMT_INIT(expr_macro_def);
   LUA_NO_OP_STMT_INIT(stmt_list_macro_def);
   LUA_NO_OP_STMT_INIT(query_parts_macro_def);
   LUA_NO_OP_STMT_INIT(cte_tables_macro_def);
+  LUA_NO_OP_STMT_INIT(select_expr_macro_def);
+  LUA_NO_OP_STMT_INIT(select_core_macro_def);

Commit: 868ec5b7

This commit likewise added no dedicated regression test. These definitions had already served their semantic purpose, so the correct backend evidence was the explicit no-op classification rather than generated output.

The details matter here. These nodes were deliberately ignored by codegen. They are the awkward edge of the fail-fast scheme: the correct handler is NO_OP, so there may be no ordinary generated output to make the omission obvious except the assertion itself.

I would not mechanically require every parser AST kind in every backend. Semantic analysis rewrites the tree, some parser shapes disappear, and some semantic-only declarations should never reach code generation as active work. Each phase intentionally sees a different set of shapes.

So I read the history more narrowly than “distributed dispatch tables drift.” Normal reachable shapes fail fast and coverage forces them out. The residual registration misses are mainly intentional no-op classifications or other cases where a phase is supposed to ignore a shape.

Error reporting and error state were separate obligations

In July 2021, assignment compatibility correctly detected a lossy conversion, but the error was attached to the wrong expression and the enclosing update node did not receive error state:

-  if (!sem_verify_assignment(ast, sem_type_left, sem_type_right, left->sem->name)) {
+  if (!sem_verify_assignment(right, sem_type_left, sem_type_right, left->sem->name)) {
+    record_error(ast);
     return;
   }

The focused test checks both the diagnostic and propagation through the update subtree:

-- TEST: update with loss of precision
-- + {update_stmt}: err
-- + {update_list}: err
-- + {update_entry}: err
-- + Error % lossy conversion from type 'LONG_INT' in 1L
-- +1 Error
update foo set id = 1L where id = 2;

Commit: 5084c83c1f751dcfd436253c88bad3fbf1bb85f7

The same distinction surfaced again in December 2023:

   // check if length of columns to update and provided values match
   for ( ; name_list && insert_list; name_list = name_list->right, insert_list = insert_list->right) {}
   if (name_list || insert_list) {
     report_error(ast, "CQL0157: count of columns differs from count of values", NULL);
+    record_error(ast);
+    return;
   }

Commit: 7933769f72b4a7d18e73895eac150fa200378c76

The second bug hid inside batched negative tests. Before the repair, four invalid constructs lived in one procedure and the aggregate +4 error: count could not prove that each construct produced its own error:

-proc test_update_from_shape_errors(like update_test_1)
-begin
-  -- Swapped ordering of columns lead to incompatible types.
-  update update_test_1
-  set (name, id) = (from arguments);
-
-  -- Use of non existent shape in values
-  update update_test_1 set (id, name) = (from cursor_not_exist);
-
-  -- Use of non existent shape in column spec
-  update update_test_1 set (like cursor_not_exist) = (from arguments);
-
-  -- Count of columns differ from count of values
-  update update_test_1 set (id) = (from arguments);
-end;

The repaired tests isolate all four operations and check each exact count:

-- TEST: update from_shape sugar error handling, type mismatch
-- + {update_stmt}: err
-- + {update_list}: err
-- + {update_entry}: err
-- + {dot}: err
-- + {name ARGUMENTS}
-- + {name id}
-- * error: % incompatible types in expression 'name'
-- +1 error:
proc test_update_from_shape_errors0(like update_test_1)
begin
  update update_test_1
  set (name, id) = (from arguments);
end;

-- TEST: update from_shape sugar error handling, FROM invalid
-- + {update_stmt}: err
-- + {insert_list}: err
-- + {name cursor_not_exist}: err
-- * error: % name not found 'cursor_not_exist'
-- +1 error:
proc test_update_from_shape_errors1(like update_test_1)
begin
  update update_test_1 set (id, name) = (from cursor_not_exist);
end;

-- TEST: update from_shape sugar error handling invalid like shape
-- + {update_stmt}: err
-- + {columns_values}: err
-- + {shape_def}: err
-- + {name cursor_not_exist}: err
-- * error: % must be a cursor, proc, table, or view 'cursor_not_exist'
-- +1 error:
proc test_update_from_shape_errors2(like update_test_1)
begin
  update update_test_1 set (like cursor_not_exist) = (from arguments);
end;

-- TEST: update from_shape sugar error handling, missing count
-- + SET (id, name) = (ARGUMENTS.id);
-- + {update_stmt}: err
-- * error: % count of columns differs from count of values
-- +1 error:
proc test_update_from_shape_errors3(id int!)
begin
  update update_test_1 set (id, name) = (from arguments);
end;

The details matter here. This repeated defect says more about the abstraction than the care of the person writing the code. The architecture had two distinct operations, report_error and record_error, and convention was responsible for keeping them paired.

Semantic flags are a lattice, not a single property

The 2021 substr fix combined nullability and sensitivity from all arguments instead of copying flags from only the text operand:

-  // This will give us the same type nullability and sensitivity as the original string
-  sem_t flags = sem_type & (SEM_TYPE_NOTNULL | SEM_TYPE_SENSITIVE);
+  // The result is nonnull if all arguments are nonnull, and sensitive if any
+  // arguments are sensitive.
+  sem_t flags = combine_flags(arg1->sem->sem_type, arg2->sem->sem_type);
+  if (arg3) {
+    flags = combine_flags(flags, arg3->sem->sem_type);
+  }

Commit: 2020413560de1f84369d15dd5f793bde8258e02b

The six focused cases prove that nullable and sensitive flags can originate in the string, start, or count argument:

-- + {name substr}: text
create proc substr_test_nullable_string(t text)
begin
  select substr(t, 1, 2) as t;
end;

-- + {name substr}: text
create proc substr_test_nullable_start(t text not null)
begin
  select substr(t, nullable(1), 2) as t;
end;

-- + {name substr}: text
create proc substr_test_nullable_count(t text not null)
begin
  select substr(t, 1, nullable(2)) as t;
end;

-- + {name substr}: text sensitive
create proc substr_test_sensitive_string(t text @sensitive)
begin
  select substr(t, 1, 2) as t;
end;

-- + {name substr}: text sensitive
create proc substr_test_sensitive_start(t text)
begin
  select substr(t, sensitive(1), 2) as t;
end;

-- + {name substr}: text sensitive
create proc substr_test_sensitive_count(t text)
begin
  select substr(t, 1, sensitive(2)) as t;
end;

Nullability, sensitivity, object kind, shape, and error state all decorate expressions. Every builtin and every rewrite has to preserve or recompute the right subset. That is why semantic bugs can recur in a well-tested compiler: each special case is effectively a tiny compiler pass.

Flow analysis failed on the branch that contained nothing

In December 2021, empty IF branches were not counted as branches. That made an improvement performed by every non-empty branch appear to be performed by every branch:

// Records the presence of an empty branch within the branch group. This is
// equivalent to calling `_flow_push_context_branch` immediately followed by
// `_flow_pop_context_branch`.
cql_noexport void flow_context_branch_group_add_empty_branch() {
  Contract(current_context);
  Contract(current_context->kind == FLOW_CONTEXT_KIND_BRANCH_GROUP);

  current_context->branch_group.branch_count++;
}

Commit: d17a350684151871d4ed78b1126f8d6e1a86cbf9

The semantic callers now record an absent body as a real branch:

   if (stmt_list) {
     FLOW_PUSH_CONTEXT_BRANCH();
     sem_set_notnull_improvements_for_true_condition(expr);
     sem_stmt_list_in_current_flow_context(stmt_list);
     FLOW_POP_CONTEXT_BRANCH();
     if (is_error(stmt_list)) {
       record_error(ast);
       return;
     }
+  } else {
+    flow_context_branch_group_add_empty_branch();
   }

@@
     if (stmt_list) {
       FLOW_PUSH_CONTEXT_BRANCH();
       sem_stmt_list_in_current_flow_context(stmt_list);
       FLOW_POP_CONTEXT_BRANCH();
       if (is_error(stmt_list)) {
         record_error(elsenode);
         record_error(if_alt);
         goto error;
       }
+    } else {
+      flow_context_branch_group_add_empty_branch();
     }

The regression puts the empty branch in each significant position and verifies that all six post-branch variables remain nullable:

-- TEST: An empty branch in an IF or SWITCH prevents the IF or SWITCH from
-- persisting any improvements.
-- + {name x0}: x0: integer variable
-- + {name x1}: x1: integer variable
-- + {name x2}: x2: integer variable
-- + {name x3}: x3: integer variable
-- + {name x4}: x4: integer variable
-- + {name x5}: x5: integer variable
-- - error:
create proc empty_branches_prevent_persisting_improvements()
begin
  declare a int;

  if 0 then
    -- empty
  else if 0 then
    set a := 1;
  else
    set a := 1;
  end if;
  let x0 := a;

  if 0 then
    set a := 1;
  else if 0 then
    -- empty
  else
    set a := 1;
  end if;
  let x1 := a;

  if 0 then
    set a := 1;
  else if 0 then
    set a := 1;
  else
    -- empty
  end if;
  let x2 := a;

  switch 0
  when 1 then
    -- empty
    leave;
  when 2 then
    set a := 1;
  else
    set a := 1;
  end;
  let x3 := a;

  switch 0
  when 1 then
    set a := 1;
  when 2 then
    set a := 1;
  else
    -- empty
    leave;
  end;
  let x4 := a;

  switch three_things.zero all values
  when three_things.zero then
    -- empty
    leave;
  when three_things.one then
    set a := 1;
  when three_things.two then
    set a := 1;
  end;
  let x5 := a;
end;

The empty branch had no data structure, but it was still a semantic case. I like the eventual abstraction because “empty branch” now has an explicit place in the flow protocol.

2. Schema migration and ordering form the hardest persistent subsystem

Schema migration was the most maddening subsystem in the history.

It had high coverage, including the 100% norm, and it was tested relentlessly. The implementation usually did exactly what we intended. The problem was that I did not fully understand all the constraints, so we verified the wrong model with great precision.

There were two related failure families:

1. version CRCs and housekeeping history could disagree with the actual SQLite schema;
2. drop, create, dependency, and recreate order were each individually plausible but not interchangeable.

The first family caused the more consequential operational failures. The second produced ordinary implementation bugs around comparators, loop direction, graph order, backend parity, and quoted names.

The question is which order applies. Schema work has several perfectly legitimate answers:

• declaration order;
• dependency order;
• reverse dependency order;
• drop order;
• create order;
• recreate-group order;
• primary-key column order;
• physical serialized-field order.

Each one is locally reasonable. Trouble starts when code reuses one where another is required.

The first problem was the model, not merely the comparator

The original theory treated schema versions as a durable history. Each version had a CRC, so parallel development could add different work to the same version and the upgrader could tell which exact version contents had already run.

That sounded reasonable and the generated code faithfully implemented it. It was still the wrong abstraction.

Schema regions meant that an old version could legitimately acquire new work when a product subscribed to another region. @unsub and @resub added more historical transitions. The stored facet could say that a later delete or column upgrade had already happened while the physical database had returned to an older state.

This produced “zombie” tables. A newly active old-version step could recreate a table while its later deletion facet remained clean. Commit c5b3bb92 moved tombstoned table drops out of individual versions and ran them whenever any upgrade occurred. The public historical account says that change reduced schema-upgrade failures by 90%.

We then found weird housekeeping states: operators could manually remove schema without tombstones, manually delete tables, or leave old version facets behind. Commit 89660734 deleted facets for version gaps. That reconciled one class of stale metadata, but it still repaired the history rather than asking what was physically present.

The test oracle was also getting stronger. In April 2021, 24410139 made the integration suite execute every nothing -> vj -> vi path and compare the result with a direct install of vi:

echo "Upgrade from nothing to v$j, then to v$i -- must match direct update to v$i"

The validator canonicalized the actual sqlite_master contents. This was much stronger than asking whether generated text matched a golden. At Meta we had the same idea first against real product schemas. The public regression made the principle general and durable.

The decisive change came after we had frequency data. The public history says roughly one database in 100,000 was missing columns even though the corresponding version CRC claimed the upgrade had run. We could see the physical failure, but we could not reproduce the complete historical path that created it.

Commit fe921eac stopped using per-version CRCs to decide whether normal tables and columns needed creation. In schematic form:

before: version CRC changed -> run the version's operations
after:  schema changed -> run idempotent operations guarded by physical existence

Versions became ordering labels. Actual table and column presence came from a cached dictionary built from sqlite_master. Each operation became locally idempotent:

bprintf(&preamble, "  LET result := cql_string_dictionary_find(%s_tables_dict_, table_);\n", global_proc_name);
bprintf(&preamble, "  SET exists_ := result IS NOT NULL and result IS NOT '';\n");

bprintf(&upgrade, "    IF NOT %s_table_exists(cql_compressed('%s')) THEN\n", global_proc_name, target_name);

The direct test was both simpler and faster. The public account reports a 13-15% improvement and a 2.2% reduction in upgrader size for the important case.

The final simplification was deletion. Commit 27f0cb64 removed @resub, its grammar and semantic machinery, historical transition matching, diagnostics, and dedicated tests: 2,136 deleted lines across 49 files. A table is unsubscribed or it is not. Removing @unsub is the resubscription operation.

At Meta the recreate-group policy hook went further and compared actual schema with desired schema before deciding whether destructive recreation was needed. It also had to cope with old orphaned tables, missing tombstones, and other states that should never have existed. The public runtime exposes this hook but deliberately implements the naive policy: it always performs full recreation.

The customer-visible symptom of the internal failures was a full schema reset and a full content download rather than preservation across upgrade. It was not fatal, but it was expensive and disruptive. Public Git records the aggregate failure-rate improvements; the reset and redownload behavior comes from my operational recollection.

This is the cleanest example in the study where 100% coverage did exactly what coverage promises while the system still failed. Every line of the wrong model could execute correctly. The missing oracle was the physical end state.

Drop order was not create order

In December 2021, recreate groups were sorted in the order needed to drop tables, and that same order was reused for creation. Foreign keys can require the reverse:

   CHARBUF_OPEN(recreate);
   CHARBUF_OPEN(update_tables);
+  CHARBUF_OPEN(pending_table_creates);

@@
     table_crc ^= crc_charbuf(&make_table);
-    bindent(&update_tables, &make_table, 4);

+    // Now we have to remember that the tables in the recreate annotations have been
+    // sorted by reverse ordinal, meaning they are in the correct order to DROP
+    // we emit the DROP statements first as normal but the creates have to be
+    // stashed in a pending buffer that accumulates in the reverse order.  When we're
+    // done with the group, then we emit the whole batch of creates in the natural order.
+    CHARBUF_OPEN(temp);
+    bindent(&temp, &make_table, 4);
+    bprintf(&temp, "%s", pending_table_creates.ptr);
+    bclear(&pending_table_creates);
+    bprintf(&pending_table_creates, "%s", temp.ptr);
+    CHARBUF_CLOSE(temp);

@@
     if (i + 1 < count && gname[0] && !Strcasecmp(gname, (note+1)->group_name)) {
       continue;
     }
+    bprintf(&update_tables, "%s", pending_table_creates.ptr);
+    bclear(&pending_table_creates);

@@
+  CHARBUF_CLOSE(pending_table_creates);
   CHARBUF_CLOSE(update_tables);
   CHARBUF_CLOSE(recreate);

Commit: 39551d179f77ce34c0a6678fa9e1a89ae14db156

The group-boundary flush is important: it emits the accumulated creates only after all members of that recreate group have been seen. This commit updated generated golden references, but added no executed foreign-key regression.

In October 2022, recreate-group comparator direction required another fix:

 // Sort the @recreate annotations in place: the order is:
-//  * group name
+//  * group ordinal
 //  * ordinal reversed
 //
+//  We use create order for groups so that we do the CRC check for the
+//  strongest tables first - wherein we recursively drop children in
+//  drop order and also drop any tables in the correct drop order via
+//  the call to cql_rebuild_recreate_group. We also dirty the CRC for
+//  any children we end up dropping, so we want the children (weak tables) to
+//  follow after the parent (strong table) so that we recreate them.
@@
-  // reverse ordinal order
+  // We need group ordinal to be in create order so that we blow away
+  // strongest groups (groups that other groups depend on) first
   if (a1->group_ordinal != a2->group_ordinal) {
-    return (a1->group_ordinal < a2->group_ordinal) ? 1 : -1;
+    return (a1->group_ordinal < a2->group_ordinal) ? -1 : 1;
   }

   // It can't be a tie! ordinal is unique!
   Invariant(a1->ordinal != a2->ordinal);

+  // reverse ordinal
   return (a1->ordinal < a2->ordinal) ? 1 : -1;

Commit: ddd4335d0cad57742814498d1ed5ebbb00e63627

In February 2023, @delete table processing similarly needed reverse order:

   cql_object_ref deleteList =
     _cql_create_upgrader_input_statement_list(deletes, "DROP ");

   cql_code rc = SQLITE_OK;
   // Execute all delete table drops
-  for (cql_int32 i = 0;
-       i < cql_string_list_get_count(deleteList); i++) {
+  // Note deleteList provides table drops in create order, so we must execute them in reverse.
+  for (cql_int32 i = cql_string_list_get_count(deleteList); --i >= 0; ) {
     cql_string_ref delete = cql_string_list_get_string(deleteList, i);
     rc = cql_exec_internal(db, delete);
     if (rc != SQLITE_OK) goto cleanup;
   }

Commit: 3b597671509042abf95c568848efd05fbb0659be

The expanded C test fixture supplied parent/child tables and a create-ordered delete list:

 cql_string_ref tables =
   cql_string_ref_new(
     " CREATE TABLE g1( id INTEGER PRIMARY KEY, name TEXT ); "
     "CREATE TABLE use_g1( id INTEGER PRIMARY KEY REFERENCES g1 (id), name2 TEXT); "
-    "CREATE TABLE foo(y text DEFAULT 'it''s, ('); ");
+    "CREATE TABLE foo(y text DEFAULT 'it''s, ('); "
+    "CREATE TABLE g2( id INTEGER PRIMARY KEY ); "
+    "CREATE TABLE use_g2( id INTEGER PRIMARY KEY REFERENCES g2 (id)); ");
 cql_string_ref indices =
   cql_string_ref_new("CREATE INDEX extra_index ON g1 (id); ");
-cql_string_ref deletes = cql_string_ref_new("DROP TABLE IF EXISTS g2; ");
+cql_string_ref deletes = cql_string_ref_new("DROP TABLE IF EXISTS g2; DROP TABLE IF EXISTS use_g2; ");
@@
 rc = cql_rebuild_recreate_group(db, tables, indices, deletes, &result);
 E(rc == SQLITE_OK, "expected succesful recreate group upgrade\n");

The commit did not complete the correction in Lua. It changed the Lua loop to this:

-  for i = 1, #deleteList do
+  for i = 1, #deleteList, -1 do

Because the start is below the positive-length end while the step is negative, that loop executes zero iterations. Commit 6f2c9498 completed the backend correction:

-  for i = 1, #deleteList, -1 do
+  -- these are deleted or unsubscribed tables
+  -- note that tables in the list are in create order, so we reverse to get drop order
+  for i = #deleteList, 1, -1 do
     rc = cql_exec(db, deleteList[i])
     if rc ~= sqlite3.OK then return rc end
   end

So 3b597671 fixed C drop order and expanded the C fixture, while introducing a zero-iteration Lua loop; 6f2c9498 completed the correction. The fixture checks successful execution, but it does not independently prove the drop order because the referenced tables are empty. This is a multi-backend correction sequence. I remember the drop-order failure as operational. Public Git proves the correction chain but does not independently classify it as a released incident. The Lua loop is an incomplete first fix, not a separately reported field defect.

These are different bugs, but they point to the same weak spot. “Order” lived implicitly in loops, comparators, and buffer accumulation instead of in named order domains.

Recreate groups show what happened after the model matured

I remembered recreate groups as the point where we finally had this under control. The Git history supports that conclusion, with an important qualification: the feature was not flawless on its first patch.

Vaishnavi Mantha’s first visible repository work in July 2022 was the runtime skeleton for cql_rebuild_recreate_group. She then added generated integration, compressed desired-schema inputs, the dependency graph, cycle rejection, partial rebuild status, and recursive dependent-group handling.

The runtime hook began before the decisive October create-plan simplification. The harder partial-rebuild and cross-group dependency work landed after that change, when physical state had become the ordinary create-plan authority.

This was harder than the original recreate feature. A runtime could preserve a group when the actual and desired schemas were compatible, or destructively recreate it otherwise. Foreign-key dependencies between groups meant that a parent recreation could force children to recreate as well.

The feature arrived after the integration suite had already spent more than a year requiring every staged upgrade path to converge on the same canonical physical schema. Dependency tests came with the implementation. They covered named groups, multiple referenced groups, same-group references, and cycle rejection.

Public Git does show several development corrections:

• the default rebuild result needed initialization;
• comparator direction needed a concrete parent/child counterexample;
• creation order had to become a true topological group order;
• dropping a child through its parent had to dirty the child’s saved state;
• migration procedures had to run only after destructive recreation;
• deleted-table drops needed reverse order in both C and Lua.

The topological fix in c5df0113 states the conceptual correction directly:

-// We use create order for groups
+// We use topological order for groups.
+// recreate_group_deps holds (parent->child) edges and has no cycles.

Its regression used the kind of interleaving that declaration order could not solve: one group contained a table that depended on another group, followed by another table from the first group.

Every recreate-group correction I found predates the first public version tag, v1.0.0, on February 25, 2024. I found no later functional recreate-group fix and no public issue or commit describing a released recreate-group incident. That absence cannot prove private deployment history, but it corroborates my recollection that the issues were found before release and that the feature produced no customer incident.

This is the positive counterexample. The earlier migration failures repeatedly revealed missing concepts in the state model. Recreate-group failures were bounded implementation edges around an explicit dependency graph and an end-to-end physical-schema oracle. Each counterexample produced a focused fix and a durable regression rather than another redesign of the subsystem.

Identifier correctness was distributed

Quoted identifiers recur in schema and SQL-generation fixes. The October 2026 schema-drop correction touched all four dynamic emitters:

-  bprintf(decls, "  CALL cql_exec_internal(printf('DROP TABLE IF EXISTS [%%s]', table_name));\n");
+  // SQLite identifiers escape embedded double quotes by doubling them.
+  bprintf(decls, "  CALL cql_exec_internal(printf('DROP TABLE IF EXISTS \"%%s\"', replace(table_name, '\"', '\"\"')));\n");

@@
-    bprintf(output, "    CALL cql_exec_internal(printf('DROP TRIGGER [%%s];', C.name));\n");
+    bprintf(output, "    CALL cql_exec_internal(printf('DROP TRIGGER \"%%s\";', replace(C.name, '\"', '\"\"')));\n");

@@
-    bprintf(output, "    CALL cql_exec_internal(printf('DROP VIEW [%%s];', C.name));\n");
+    bprintf(output, "    CALL cql_exec_internal(printf('DROP VIEW \"%%s\";', replace(C.name, '\"', '\"\"')));\n");

@@
-    bprintf(output, "    CALL cql_exec_internal(printf('DROP INDEX [%%s];', C.name));\n");
+    bprintf(output, "    CALL cql_exec_internal(printf('DROP INDEX \"%%s\";', replace(C.name, '\"', '\"\"')));\n");

Commit: 8500516b9efa7f35e9406210600adb8efce4fb9a

The executable upgrade tests create names containing both ] and ". They exercise a targeted table-drop helper plus an unmanaged view, trigger, and index discovered during exclusive upgrade, then verify the targeted objects are gone and unrelated table data survived:

create proc verify_quoted_table_drops()
begin
  declare names cursor for select 'identifier_table]bracket' name
    union all select 'identifier_table"quote' name;
  loop fetch names
  begin
    call cql_exec_internal(printf('CREATE TABLE "%s"(value)',
      replace(names.name, '"', '""')));
    call test_drop_table_helper(names.name);
    if (select exists(select 1 from sqlite_master where name = names.name)) then
      printf_stderr("ERROR! quoted table %s was not dropped\n", names.name);
      throw;
    end if;
  end;
end;

create proc prepare_quoted_schema_objects()
begin
  create table identifier_host (value int);
  insert into identifier_host values (42);
  declare names cursor for select ']bracket' suffix
    union all select '"quote' suffix;
  loop fetch names
  begin
    let view_name := printf('identifier_view%s', names.suffix);
    let trigger_name := printf('identifier_trigger%s', names.suffix);
    let index_name := printf('identifier_index%s', names.suffix);
    call cql_exec_internal(printf('CREATE VIEW "%s" AS SELECT * FROM identifier_host',
      replace(view_name, '"', '""')));
    call cql_exec_internal(printf(
      'CREATE TRIGGER "%s" AFTER INSERT ON identifier_host BEGIN SELECT 1; END',
      replace(trigger_name, '"', '""')));
    call cql_exec_internal(printf('CREATE INDEX "%s" ON identifier_host(value)',
      replace(index_name, '"', '""')));
  end;
end;

create proc verify_quoted_schema_drops()
begin
  if (select exists(select 1 from sqlite_master
    where name in ('identifier_view]bracket', 'identifier_view"quote',
      'identifier_trigger]bracket', 'identifier_trigger"quote',
      'identifier_index]bracket', 'identifier_index"quote'))) then
    printf_stderr("ERROR! quoted schema objects were not dropped\n");
    throw;
  end if;
  if (select count(*) from identifier_host) != 1 or
     (select value from identifier_host) != 42 then
    printf_stderr("ERROR! schema drops modified unrelated data\n");
    throw;
  end if;
  drop table identifier_host;
end;

The table-drop helper is packaged in the C harness:

static cql_code test_drop_quoted_tables(sqlite3 *db) {
  cql_object_ref saved_dictionary = test_tables_dict_;
  cql_object_retain(saved_dictionary);
  cql_set_created_object_ref(&test_tables_dict_, cql_string_dictionary_create());
  cql_code rc = verify_quoted_table_drops(db);

  cql_set_object_ref(&test_tables_dict_, saved_dictionary);
  cql_object_release(saved_dictionary);
  if (rc != SQLITE_OK) {
    fprintf(stderr, "Quoted table-drop helper regression failed: %s\n", sqlite3_errmsg(db));
  }
  return rc;
}

In main, only the unmanaged schema-object setup and verification are gated to the relevant upgrade boundary:

bool check_identifiers = proposed_version == 4 && current_version != 4;
if (check_identifiers && prepare_quoted_schema_objects(db) != SQLITE_OK) {
  fprintf(stderr, "Unable to prepare quoted schema objects.\n");
  return SQLITE_ERROR;
}

if (upgrade(db, should_use_virtual_tables)) {
  fprintf(stderr, "Unable to upgrade DB.\n");
  return SQLITE_ERROR;
}

if (check_identifiers && verify_quoted_schema_drops(db) != SQLITE_OK) {
  fprintf(stderr, "Quoted schema-drop regression failed.\n");
  return SQLITE_ERROR;
}

After the ordinary post-upgrade version and content checks, the same harness invokes the table helper unconditionally:

if (test_drop_quoted_tables(db) != SQLITE_OK) {
  return SQLITE_ERROR;
}

The lesson is simple:

Static SQL generation and dynamic SQL construction must share one identifier model or one escaping primitive.

Subsystem tests can stop the exact bug from coming back while the same family continues in another emitter.

Rewrites compose badly unless state and syntax are explicit

The December 2024 work on backed tables, upsert, and RETURNING found several issues together:

• rowid reselection where it was not appropriate;
• a lost upsert WHERE;
• a missing ambiguity-avoiding WHERE 1;
• confusion between generated names and quoted user identifiers;
• nested rewrite state not restored correctly.

Commit: cf546d7ed13edb2642abb75addcd6d5b48479c53

Each rewrite makes sense on its own. Once combined, the tree carries the obligations of all of them, and that cross-product is where the failure lived.

The correction was spread across the composing rewrites. The generated insert adds SQLite’s documented ambiguity breaker only for upsert:

              new_ast_select_where(
-                NULL,
+                  in_upsert ? new_ast_opt_where( new_ast_num(NUM_INT, "1")) : NULL,
                new_ast_select_groupby(
                  NULL,
                  new_ast_select_having(
                    NULL,
                    NULL
                  )
                )
              )

The backed-table update stops synthesizing rowid reselection during upsert and preserves the upsert’s own WHERE, rewriting its backed-column references:

-  ast_node *select_stmt = rewrite_select_rowid(
-     backed_table_name,
-     opt_where,
-     opt_orderby,
-     opt_limit);
+  ast_node *select_stmt;
+
+  if (!in_upsert) {
+
+    // this generates the normal where clause for the update statement
+    // WHERE rowid in (SELECT rowid FROM backed) which is what you want
+    // for the pdate case
+
+    select_stmt = rewrite_select_rowid(
+      backed_table_name,
+      opt_where,
+      opt_orderby,
+      opt_limit);
+
+    // for debugging print just the select statement
+    // gen_stmt_list_to_stdout(new_ast_stmt_list(select_stmt, NULL));
+
+    // the new where clause for the update statement is going to be something like
+    // rowid IN (select rowid from selected rows)
+  }
+  else {
+    // in the upsert case, the rows in question are already selected by SQLite itself.
+    // We don't need to do the rowid selection, we just need to rewrite the
+    // update list to use the backing table columns directly and we'll select
+    // the rowid out of excluded.rowid
+    select_stmt = NULL;
+  }
@@
-  ast_node *new_opt_where =
-    new_ast_opt_where(
-     new_ast_in_pred(
-       new_ast_str("rowid"),
-       select_stmt
-     )
-  );
+  ast_node *new_opt_where = NULL;
+
+  if (!in_upsert) {
+    new_opt_where = new_ast_opt_where(
+     new_ast_in_pred(
+       new_ast_str("rowid"),
+       select_stmt
+     )
+    );
+  }
+  else {
+    // the upsert case can have a WHERE clause of its own, it stays as is
+    new_opt_where = opt_where;
+
+    if (opt_where) {
+      AST_REWRITE_INFO_SAVE();
+      AST_REWRITE_INFO_SET(opt_where->lineno, opt_where->filename);
+      rewrite_backed_column_references_in_ast(opt_where, backed_table);
+      AST_REWRITE_INFO_RESET();
+      AST_REWRITE_INFO_RESTORE();
+    }
+  }

The nested rewrite now preserves any already-active rewrite context, and CTE names use the normal quoted-name generator:

+  // this can be called from an existing rewrite in the upsert case, handle both cases
+  AST_REWRITE_INFO_SAVE();
   AST_REWRITE_INFO_SET(root->lineno, root->filename);
-
-  rewrite_blob_column_references(&info, root);
-
+    rewrite_blob_column_references(&info, root);
   AST_REWRITE_INFO_RESET();
+  AST_REWRITE_INFO_RESTORE();

-  EXTRACT_STRING(name, ast->left);
-  if (is_ast_star(ast->right)) {
-    // use the shorter syntax for (*) which is just the name
-    gen_printf("%s", name);
-  }
-  else {
-    gen_printf("%s (", name);
+  EXTRACT_ANY_NOTNULL(name_ast, ast->left);
+  gen_name(name_ast);
+  if (!is_ast_star(ast->right)) {
+    // skip this for foo(*), the shorter syntax is just the name
+    gen_printf(" (");
     gen_name_list(ast->right);
     gen_printf(")");
   }

The expanded runtime test proves that quoted backed-table names, insert/select, delete, update, upsert, WHERE, and RETURNING compose into an executable statement, then checks returned values and exhaustion:

proc make_jsonb_backed_schema()
begin
  [[backing_table]]
  [[jsonb]]
  create table `a backing table`(
    `the key` blob primary key,
    `the value` blob
  );
end;

[[backed_by=`a backing table`]]
create table `a table`(
  `col 1` int primary key,
  `col 2` int
);

TEST_MODERN_ONLY!(verify_returning,
begin
  make_jsonb_backed_schema();
    cursor C for
    with data(a, b) as (values
      (1, 1),
      (2, 4),
      (3, 9),
      (4, 16)
    )
    insert into `a table`(`col 1`, `col 2`)
      select * from data
      returning *;

    let i := 0;
    loop fetch C
    begin
      i += 1;
      EXPECT_EQ!(C.`col 1` * C.`col 1`, C.`col 2`);
    end;
    EXPECT_EQ!(i, 4);

    cursor D for
      delete from `a table` where `col 1` in (2, 3)
      returning *;

    fetch D;
    EXPECT_EQ!(D.`col 1`, 2);
    EXPECT_EQ!(D.`col 2`, 4);
    fetch D;
    EXPECT_EQ!(D.`col 1`, 3);
    EXPECT_EQ!(D.`col 2`, 9);
    fetch D;
    EXPECT!(not D);

    cursor E for select * from `a table`;
    i := 0;
    loop fetch E
    begin
      i += 1;
      EXPECT_EQ!(E.`col 1` * E.`col 1`, E.`col 2`);
    end;
    EXPECT_EQ!(i, 2);

    cursor F for
      update `a table` set `col 2` = `col 1` * 1000
      where `col 1` = 4
      returning *;

    fetch F;
    EXPECT_EQ!(F.`col 1`, 4);
    EXPECT_EQ!(F.`col 2`, 4000);
    fetch F;
    EXPECT!(not F);

    cursor G for
      insert into `a table`(`col 1`, `col 2`) values (1, 1), (5, 25)
      on conflict(`col 1`)
      do update set `col 2` = 1000 where `col 1` <= 4
      returning *;

    fetch G;
    EXPECT_EQ!(G.`col 1`, 1);
    EXPECT_EQ!(G.`col 2`, 1000);

    fetch G;
    EXPECT_EQ!(G.`col 1`, 5);
    EXPECT_EQ!(G.`col 2`, 25);
    fetch G;
    EXPECT!(not G);
end);

The production diff shows that the upsert WHERE is preserved and its backed-column references are rewritten. This runtime case does not independently exercise the predicate’s false branch: the only conflicting key satisfies the predicate, while key 5 is a non-conflicting insert.

How to think about the subsystem

From my chair, schema migration is a compiler inside the compiler. It has its own:

• dependency graph;
• ordering semantics;
• identifier language;
• generated program;
• runtime execution behavior;
• compatibility rules across versions.

That is why the end-to-end upgrade tests matter more than the large golden outputs. They execute the generated migration as a program instead of checking only its text.

3. Global state works better than normal advice predicts

Conventional advice says to avoid global mutable state and pass explicit contexts. CQL mostly keeps each compiler phase’s state in globals.

My approximate current scan found:

• about 214 file-static state declarations;
• about 80 in sem.c;
• about 38 in cg_c.c;
• about 36 in cg_lua.c.

There are real advantages:

• recursive visitors have small signatures;
• current procedure, scope, callbacks, and output buffers are immediately available;
• code for one compiler phase stays together;
• the implementation avoids passing a very large context through thousands of calls.

The details matter: recursion almost never needs all of that state. Most nested contexts save one or two symbol tables, a join scope, or a callback and output buffer. For example:

symtab *saved_locals = locals;
symtab *saved_globals = globals;

locals = globals = NULL;
PUSH_JOIN(expr_scope, jptr);
sem_root_expr(expr, SEM_EXPR_CONTEXT_CONSTRAINT);
POP_JOIN();

locals = saved_locals;
globals = saved_globals;

The generic SQL generator state is a complete three-pointer abstraction with paired save and restore operations:

typedef struct gen_sql_state {
  charbuf *_Nullable gen_output;
  gen_sql_callbacks *_Nullable gen_callbacks;
  symtab *_Nullable used_alias_syms;
} gen_sql_state;

cql_noexport void gen_get_state(gen_sql_state *state) {
  state->gen_output = gen_output;
  state->gen_callbacks = gen_callbacks;
  state->used_alias_syms = used_alias_syms;
}

cql_noexport void gen_set_state(gen_sql_state *state) {
  gen_output = state->gen_output;
  gen_callbacks = state->gen_callbacks;
  used_alias_syms = state->used_alias_syms;
}

That is the normal shape. The shared-fragment case below is exceptional: it re-enters SQL generation while fragment predicates, variable numbering, and fragment buffers are all in flight.

Nested code generation exposed the cost

In March 2025, evaluating a shared-fragment argument containing a nested SELECT re-entered SQL generation and clobbered outer generator state.

The code had already documented exactly what remained to be done:

// todo: if the evaluation has a nested select statement then we will have
// to re-enter all of this.  We can either ban that (which isn't insane
// really) or else we can save the codegen state like callbacks and such so
// that it can re-enter.  That's the desired path.

The save/restore work in the comment had not been implemented. The fix did exactly that:

gen_sql_state state;
gen_get_state(&state);

bytebuf sfs_saved;
memcpy(&sfs_saved, &shared_fragment_strings, sizeof(bytebuf));
memset(&shared_fragment_strings, 0, sizeof(bytebuf));

int cur_fragment_predicate_saved = cur_fragment_predicate;
int max_fragment_predicate_saved = max_fragment_predicate;
int prev_variable_count_saved = prev_variable_count;
int cur_variable_count_saved = cur_variable_count;
bool_t has_conditional_fragments_saved = has_conditional_fragments;
bool_t has_shared_fragments_saved = has_shared_fragments;
bool_t has_variables_saved = has_variables;

CG_PUSH_EVAL(expr, C_EXPR_PRI_ASSIGN);
cg_store(cg_main_output, alias_name, sem_type_var, sem_type_expr, expr_is_null.ptr, expr_value.ptr);
CG_POP_EVAL(expr);

gen_set_state(&state);
memcpy(&shared_fragment_strings, &sfs_saved, sizeof(bytebuf));
cur_fragment_predicate = cur_fragment_predicate_saved;
max_fragment_predicate = max_fragment_predicate_saved;
prev_variable_count = prev_variable_count_saved;
cur_variable_count = cur_variable_count_saved;
has_conditional_fragments = has_conditional_fragments_saved;
has_shared_fragments = has_shared_fragments_saved;
has_variables = has_variables_saved;

Commit: 7dadf83af8d84f3d5d7450769e5520b80efb65cf

The Lua backend received the corresponding state boundary around its actual nested evaluation:

gen_sql_state state;
gen_get_state(&state);
bytebuf sfs_saved;
memcpy(&sfs_saved, &lua_shared_fragment_strings, sizeof(bytebuf));
memset(&lua_shared_fragment_strings, 0, sizeof(bytebuf));

int cur_fragment_predicate_saved = lua_cur_fragment_predicate;
int max_fragment_predicate_saved = lua_max_fragment_predicate;
int prev_variable_count_saved = lua_prev_variable_count;
int cur_variable_count_saved = lua_cur_variable_count;
bool_t has_conditional_fragments_saved = lua_has_conditional_fragments;
bool_t has_shared_fragments_saved = lua_has_shared_fragments;
bool_t has_variables_saved = lua_has_variables;

CG_LUA_PUSH_EVAL(expr, LUA_EXPR_PRI_ASSIGN);
cg_lua_store(cg_main_output, alias_name, sem_type_var, sem_type_expr, expr_value.ptr);
CG_LUA_POP_EVAL(expr);

lua_cur_variable_count = cur_variable_count_saved;
lua_prev_variable_count = prev_variable_count_saved;
lua_cur_fragment_predicate = cur_fragment_predicate_saved;
lua_max_fragment_predicate = max_fragment_predicate_saved;
lua_has_conditional_fragments = has_conditional_fragments_saved;
lua_has_shared_fragments = has_shared_fragments_saved;
lua_has_variables = has_variables_saved;

gen_set_state(&state);
memcpy(&lua_shared_fragment_strings, &sfs_saved, sizeof(bytebuf));

The list is longer here because nested shared-fragment arguments can themselves contain shared fragments. This path has more live recursive context than the usual one or two symbol tables.

A helper dedicated to pushing and popping this shared-fragment generation context would be cleaner, and the manual C/Lua copies are maintenance debt. I never got around to making that helper. Still, this commit is better read as an unfinished, already-documented re-entry requirement than as evidence that all compiler global state needs to be snapshotted. The commit added unrelated Dirname coverage, but no focused regression source for nested shared-fragment argument evaluation.

Repeated compiler invocation has an unusually strong regression test

The amalgam test calls cql_main several times in one process:

const char *cql_success_file = (char *) argv[1];
const char *cql_semantic_error_file = (char *) argv[2];
const char *cql_syntax_error_file = (char *) argv[3];

const char *args_1[5] = {"cql", "--in", cql_success_file, "--sem", "--echo"};
if (cql_main(5, (char **)args_1) != 0) {
  fprintf(stderr, "%s reported an error: this is unexpected\n", cql_success_file);
  exit(1);
}

const char *args_2[5] = {"cql", "--in", cql_semantic_error_file, "--sem", "--echo"};
if (cql_main(5, (char **)args_2) == 0) {
  fprintf(stderr, "%s did not reported an error: this is unexpected\n", cql_semantic_error_file);
  exit(1);
}

const char *args_3[5] = {"cql", "--in", cql_syntax_error_file, "--sem", "--echo"};
if (cql_main(5, (char **)args_3) == 0) {
  fprintf(stderr, "%s did not reported an error: this is unexpected\n", cql_syntax_error_file);
  exit(1);
}

if (cql_main(5, (char **)args_1) != 0) {
  fprintf(stderr, "%s reported an error: this is unexpected\n", cql_success_file);
  exit(1);
}

Source: sources/test/amalgam_test.c

I like this better than a cleanup unit test. It verifies observable reentrancy after both semantic and syntax failure, which directly exercises the risk created by phase globals.

The global-state design is deliberate

Phase globals come with explicit cleanup entry points. For example, C generator initialization calls cleanup before rebuilding dispatch and state, and compiler orchestration cleans every phase before returning.

cql_noexport void cg_c_init(void) {
  cg_c_cleanup(); // reset globals/statics
  cg_common_init();

  Contract(!error_target_used);

  // one table for the whole translation unit
  Contract(!string_literals);
  string_literals = symtab_new_case_sens();

cg_c_cleanup();
sem_cleanup();
ast_cleanup();
gen_cleanup();
rt_cleanup();
parse_cleanup();
cql_cleanup_open_includes();
cql_cleanup_defines();

#ifdef CQL_AMALGAM
  // the variables need to be set back to zero so we can
  // be called again as though we were just loaded
  cql_reset_globals();
#endif

I read the history this way: top-level reset works, ordinary recursion saves little context, and shared-fragment generation is the exceptional wide re-entry case that still wants a focused helper.

4. Ownership helpers and bulk lifetimes are CQL’s strongest C-level design

C usually gives you leaks, use-after-free, double-free, dangling pointers, aliasing errors, buffer overruns, integer overflow, and undefined behavior from alignment or effective-type violations.

CQL removes much of the ordinary ownership burden by dividing memory into a few strong lifetime classes.

AST and durable compiler data use arenas

The minipool is intentionally simple. The exact excerpts below omit the remainder of the introductory commentary and minipool_open, which eagerly initializes the first block:

// This is the stupidest pool allocator ever, it's only useful for cases where
// everything lives until the end.  Currently it's used to hold ast nodes and
// duplicated strings.

cql_noexport void minipool_close(minipool **head) {
  minipool *pool = *head;
  while (pool) {
    minipool *next = pool->next;
    free(pool->bytes);
    free(pool);
    pool = next;
  }
  *head = NULL;
}

// Get needed memory; if the memory is not available then we allocate a new
// block and thread it into the list.  Note that this is different than the
// other CQL helper bytebuf. This allocation never moves bytes or copies bytes.
// Once you get memory it is valid until you close the buffer. Bytebuf cannot
// hold pointers because the structure moves when it grows and it is one
// continuous allocation so it's only for byte streams. Minipool doesn't move
// objects around so pointers remain valid.
cql_noexport void *minipool_alloc(minipool *pool, uint32_t needed) {
  void *result;

  // for alignment, all allocs will be on a 8 byte boundary
  needed += 7;
  needed &= u32_not(7);

  if (needed > pool->available) {
    // Make a copy of the most recent head node and link to it we have to do
    // this because the head of the pool never changes once it's created.  Note
    // that none of the byte buffers are copied, only the minipool struct.  Once
    // we've copied it, we re-initialize the minipool to a size that is at least
    // big enough for the next allocation.
    minipool *old = malloc(sizeof(minipool));
    *old = *pool;
    uint32_t blocksize = MAX(needed, MINIBLOCK);
    pool->bytes = malloc(blocksize);
    pool->current = pool->bytes;
    pool->available = blocksize;
    pool->next = old;
  }

  result = pool->current;
  pool->current += needed;
  pool->available -= needed;
  return result;
}

The key property is in the comment: allocations never move and remain valid until the whole pool closes. Compiler ASTs naturally live for the compilation, so individual free calls would add risk with very little useful memory recovery.

That runs directly against generic “free each allocation promptly” advice. For this workload, I think the bulk lifetime is the better engineering choice.

Reference ownership needed type-specific operations

In November 2024, generated code released a destination reference before evaluating a create call. That was wrong when the old destination value was also an argument to the call:

static void cg_copy_for_create(charbuf *output, CSTR var, sem_t sem_type_var, CSTR value) {
   if (is_text(sem_type_var)) {
     bprintf(cg_main_output, "%s(%s);\n", rt->cql_string_release, var);
   }
   else if (is_blob(sem_type_var)) {
     bprintf(output, "%s(%s);\n", rt->cql_blob_release, var);
   }
   else if (is_object(sem_type_var)) {
     bprintf(output, "%s(%s);\n", rt->cql_object_release, var);
   }
   bprintf(output, "%s = %s;\n", var, value);
}

The fix introduced explicit setters for incoming +1 references:

void cql_set_created_object_ref(
  cql_object_ref _Nullable *_Nonnull target,
  cql_object_ref _Nullable source)
{
  // no upcount, we were given an upcounted ref. We just release what we had and store
  cql_object_release(*target);
  *target = source;
}

The executable aliasing case removed its temporary workaround and passes the destination directly as an argument to the create function:

-let b_temp := my_blob; -- avoid bug in cg_copy_for_create
-my_blob := corrupt_blob_with_invalid_shenanigans(b_temp);
+my_blob := corrupt_blob_with_invalid_shenanigans(my_blob);

The generator now emits the type-specific created-reference operations:

if (is_text(sem_type_var)) {
  bprintf(output, "cql_set_created_string_ref(%s%s, %s);\n", prefix, var, value);
}
else if (is_blob(sem_type_var)) {
  bprintf(output, "cql_set_created_blob_ref(%s%s, %s);\n", prefix, var, value);
}
else if (is_object(sem_type_var)) {
  bprintf(output, "cql_set_created_object_ref(%s%s, %s);\n", prefix, var, value);
}

Selected exact cases from separate locations in cg_test.sql cover object, blob, string, and boxing paths:

-- + cql_set_created_object_ref(&obj_var, obj_func_create());
set obj_var := obj_func_create();

-- + cql_set_created_blob_ref(&blob_var, blob_func_create());
set blob_var := blob_func_create();

-- + cql_set_created_string_ref(&s, cql_cursor_format(&C_dyn));
PROC BigFormat ()
BEGIN
  DECLARE C CURSOR FOR SELECT * FROM big_data;
  LOOP FETCH C
  BEGIN
    LET s := cql_cursor_format(C);
  END;
END;

-- + cql_set_created_object_ref(&C_object_, cql_box_stmt(C_stmt));
-- + cql_set_object_ref(&box, C_object_);
proc call_in_loop_boxed()
begin
  declare i int;
  set i := 0;
  while i < 5
  begin
    set i := i + 1;
    declare C cursor for call simple_select();
    declare box object<C cursor>;
    set box from cursor C;
    declare D cursor for box;
    fetch D;
  end;
end;

Commit: 0001945b26ad1b50b80f439ce57f2b9fbe89f523

I much prefer this to merely reordering the release. The helper name encodes the ownership contract and reduces the legal spellings of a subtle operation.

Net reference balancing is checked for every runtime test

The runtime TEST! macro records outstanding references around each test:

start_refs := get_outstanding_refs();
@id("test_", name!)();
end_refs := get_outstanding_refs();

if start_refs != end_refs then
  printf("Test %s unbalanced refs.", @text(name!));
  printf("  Starting refs %d, ending refs %d.\n", start_refs, end_refs);
  fails := fails + 1;
end if;

Source: sources/test/run_test.sql

This is a powerful general invariant. It catches net leaks regardless of the feature a test was written to exercise. FWIW, the limit is just as clear: balanced counts cannot prove correct ordering, absence of temporary use-after-release, or correct ownership transfer. The created-reference bug survived until the abstraction itself got stronger.

Why this design holds up

CQL’s best C mitigations reduce the number of lifetimes a programmer has to reason about:

• AST objects: compilation arena lifetime.
• Interned strings: durable shared lifetime.
• output buffers: lexical open/close lifetime.
• generated runtime references: retain/release helper protocol.
• created runtime references: distinct set_created protocol.

The remaining ownership bugs tend to appear at a crossing between lifetime classes, or when code spells out the protocol by hand.

5. Representation boundaries expose hidden assumptions

This family has two layers: a longer serializer history and a concentrated late-2026 AI-assisted correctness audit.

I weight the late audit changes differently from observed operational bugs. Several strengthened an invariant that no known production caller violated. They make the code better and expose low-level preconditions that had been implicit. They do not show that buffers were routinely failing in operation.

Two serializers, two starting contracts

There are two binary serializer histories in this codebase and they should not be mashed together.

The original cursor serializer was designed to take arbitrary cursor-shaped data and turn it into a blob for storage or transport. From the beginning we expected version skew: the reader might be older or newer than the writer. We also expected the blob to be attacker controlled. The March 2022 introduction, b573687a , said so directly:

The blob reading code is hardened so that it assumes the content of the blob is hostile. There are tests for this.

That was not an aspiration added years later. It was in the first change. cql_cursor_from_bytes still puts the contract next to the implementation:

// This is the inverse of cql_cursor_to_bytebuf, it takes a byte stream and
// reconstructs a dynamic cursor from it.  The byte stream is assumed to be
// hostile. It could be corrupted in any kind of way and this code is expected
// to handle that.
cql_code cql_cursor_from_bytes(
  cql_dynamic_cursor *_Nonnull dyn_cursor,
  const uint8_t *_Nonnull bytes,
  uint32_t size)

The type signature in the blob provides shape versioning. A newer reader can fill missing trailing nullable fields with NULL. Two days after the initial change, 0b3c88a6 completed the other direction so that an older reader can ignore extra trailing fields from a newer writer. The current comment is wonderfully specific:

// Extra fields do not have to match, the assumption is that this is a
// future version of the type talking to a past version.  The past version
// sees only what it expects to see.  However, we did have to compute the
// nullable_count and bool_count to get the bit vector size correct.

The first few changes already included malformed-input cases, exhaustive truncation, and mutation under sanitizers. The serializer still had bugs. The most informative one appeared when the Lua implementation was required to produce exactly the same bytes as C in d94eded9 . That comparison found two shared-format mistakes:

• nested blob lengths used a fixed native integer instead of a varint;
• a nullable Boolean that was NULL did not advance the Boolean bit index.

The second problem made nullable Booleans effectively unusable. Ordinary C round trips did not reveal either problem because the C encoder and decoder agreed with the same representation. A fixed byte sequence and an independent Lua implementation did.

The bgetkey/bcreatekey/bupdatekey family had a completely different origin. These are SQLite extensions for backed tables. The operation names are configurable because applications are allowed to provide whatever storage format they want.

At Meta we had a hardened implementation that depended on internal infrastructure and could not be open-sourced. I needed enough public implementation to run the compiler tests, so the first open-source version was deliberately a stub. The September 2022 source could hardly have been clearer:

// this is nothing like a generic blob format but it's enough to do some tests
// the blobs that are stored can only be integers of course but since the
// codegen is the same for all types it still makes for a useful test
typedef struct test_blob {
  int64_t type;
  int64_t int1;
  int64_t int2;
  int64_t int3;
} test_blob;

Over time the stub became useful enough that it started becoming real. The 2023 implementation added all the primitive types, sparse values, canonical keys, updates, and a versioned header. The comments openly tracked the unfinished parts:

// NOTE: THIS CODE DOES NOT HANDLE NULL COLUMNS (which are basically the same as missing)
void bcreateval(sqlite3_context *_Nonnull context, int32_t argc, sqlite3_value *_Nonnull *_Nonnull argv) {

// NOTE: THIS CODE DOES NOT HANDLE THE CASE WHERE NEW COLUMNS ARE ADDED
void bupdateval(sqlite3_context *_Nonnull context, int32_t argc, sqlite3_value *_Nonnull *_Nonnull argv) {

Those limitations were subsequently removed, but the input contract was still the old one. A later version said:

// not computed but that is of no import since we are not yet validating all
// the internal offsets (blobs are assumed to be well formed for now)

and its failure path returned SQL NULL with the comment:

// If anything goes wrong we just return a null blob
// We could probably do better than this.

That history explains the 2026 sequence much better than a flat list of bounds bugs. The code’s job changed in stages:

1. exercise the backed-table rewrites;
2. provide a convenient general binary implementation;
3. produce canonical durable keys and sparse values;
4. survive arbitrary persisted input;
5. distinguish an absent nullable value from malformed storage by returning NULL for the former and SQLITE_ERROR for the latter.

CQL now offers the configurable binary UDF implementation and SQLite JSON/JSONB backing as ready-made choices, while custom implementations remain part of the design.

The different histories matter to the conclusions. The cursor serializer started with hostile-input and version-skew requirements, so its later defects show the limits of strong defenses and shared round-trip tests. The backed serializer started as visible test scaffolding. Its repeated upgrades show what happens when useful sample code gradually acquires a production contract.

bytebuf self-append was a prophylactic invariant change

In October 2026, bytebuf_append was changed to support a source range inside the same buffer even when growth moves the allocation:

+ uint32_t offset = 0;
+ bool_t aliases_buffer = false;
+
+ if (buf->ptr) {
+   uintptr_t source = (uintptr_t)bytes;
+   uintptr_t start = (uintptr_t)buf->ptr;
+   if (source >= start && source - start < buf->used) {
+     offset = (uint32_t)(source - start);
+     aliases_buffer = true;
+   }
+ }
+
  void *mem = bytebuf_alloc(buf, count);
- memcpy(mem, bytes, count);
+ if (aliases_buffer) {
+   bytes = buf->ptr + offset;
+ }
+ memmove(mem, bytes, count);

Commit: 25f48ef5985a15b805f2dda66347c7c33bb71ef9

The regression test forces growth and then tests both whole-buffer and interior-slice aliasing:

static bool test_bytebuf_self_append() {
bytebuf buffer;
bytebuf_open(&buffer);

// Duplicating this seed forces growth while the source aliases the buffer.
char seed[BYTEBUF_GROWTH_SIZE + 1];
for (uint32_t i = 0; i < sizeof(seed); i++) {
  seed[i] = (char)(i & 0x7f);
}
bytebuf_append(&buffer, seed, sizeof(seed));
bytebuf_append(&buffer, buffer.ptr, buffer.used);

bool result =
  buffer.used == 2 * sizeof(seed) &&
  !memcmp(buffer.ptr, seed, sizeof(seed)) &&
  !memcmp(buffer.ptr + sizeof(seed), seed, sizeof(seed));

bytebuf_append(&buffer, buffer.ptr + 17, 31);
result &= buffer.used == 2 * sizeof(seed) + 31;
result &= !memcmp(buffer.ptr + 2 * sizeof(seed), seed + 17, 31);

bytebuf_close(&buffer);
return result;
}

It is registered in run_unit_tests with the literal assertion TEST_ASSERT(test_bytebuf_self_append());.

I searched the repository and checked what I know about the call sites. I found no known caller appending an overlapping range. The shown code does not establish that negative-history claim; the test constructs the case deliberately. I would call this an invariant test rather than a regression for an observed compiler failure.

The old helper had an unenforced assumption: source storage would not be invalidated by destination growth. The new helper removes that assumption and defines self-append as supported behavior. That is good prophylactic work.

bprintf format aliasing was prohibited explicitly

The same audit added an always-on contract to vbprintf:

uintptr_t source = (uintptr_t)format;
uintptr_t start = (uintptr_t)b->ptr;
Contract(source < start || source - start >= b->max);

Commit: 14b0eaa75cba729abe3e16bacbbca417990a6c13

I found no call where the dynamic format string lives inside the destination allocation. The snippet itself proves only the new precondition. The contract prevents a future caller from depending on storage that can move during formatting. The commit added no test. Righteous fix, hidden assumption, no observed bug.

Unsigned loop bounds make empty input special

Another audit fix changed:

- for (uint32_t i = 0; i < b->used - 2; i++) {
+ for (uint32_t i = 0; i + 2 < b->used; i++) {

Commit: 89f2879b74b3dff61fd40afccad8f9ccc6a11fd3

The original arithmetic looks natural when read locally, but it underflows for short buffers. The repaired form says what the code actually needs: three positions must fit. This was boundary hardening found by inspection, not evidence that empty comment buffers had caused field failures.

The complete table-driven test starts with a truly empty buffer, then covers one-character, two-character, and ordinary inputs while checking that the logical buffer length is unchanged:

static bool test_comment_marker_scan() {
  CHARBUF_OPEN(temp);

  cg_remove_slash_star_and_star_slash(&temp);
  bool result = temp.used == 1 && !strcmp(temp.ptr, "");

  const char *inputs[] = { "/", "*", "/*", "*/", "/* */", "text/*", "text*/" };
  const char *outputs[] = { "/", "*", "/+", "+/", "/+ +/", "text/+", "text+/" };
  for (uint32_t i = 0; i < sizeof(inputs) / sizeof(inputs[0]); i++) {
    bclear(&temp);
    bprintf(&temp, "%s", inputs[i]);
    uint32_t used = temp.used;
    cg_remove_slash_star_and_star_slash(&temp);
    result &= temp.used == used && !strcmp(temp.ptr, outputs[i]);
  }

  CHARBUF_CLOSE(temp);
  return result;
}

It is registered in run_unit_tests with TEST_ASSERT(test_comment_marker_scan());.

Serialized blobs need whole-shape validation

By 2026 the two serializer histories converged on the same mechanical lesson, even though they arrived there from different starting contracts. The backed-blob and blob-stream fixes form a recognizable family:

• canonical field ordering;
• offset-table off-by-one errors;
• incomplete stored-field validation;
• confusion between physical trailing bytes and logical field lengths;
• arithmetic performed before validating the entire offset table.

The October 2026 blob-stream audit moved toward a better pattern. The complete read path proves the table shape, validates both ends of the selected range, and clears any existing cursor row on rejection:

cql_code cql_cursor_from_blob_stream(
  sqlite3 *_Nonnull db,
  cql_dynamic_cursor *_Nonnull dyn_cursor,
  cql_blob_ref _Nonnull b,
  cql_int32 index)
{
  cql_contract(b);
  cql_bool *has_row = dyn_cursor->cursor_has_row;

  const uint8_t *bytes = (const uint8_t *)cql_get_blob_bytes(b);
  const uint32_t len = (uint32_t)cql_get_blob_size(b);

  if (len < 4) {
    goto error;
  }

  // the first 4 bytes are the count of blobs
  // note that we're assuming little endian here, this could be generalized
  uint32_t count;
  memcpy(&count, bytes, sizeof(count));

  // Prove the entire count/offset table fits before any count or index arithmetic.
  if (count > len / sizeof(uint32_t) - 1) {
    goto error;
  }

  if (index < 0 || (uint32_t)index >= count) {
    goto error;
  }

  // note that we're assuming little endian here, this could be generalized
  uint32_t end;
  memcpy(&end, bytes + ((uint32_t)index + 1) * sizeof(end), sizeof(end));
  if (end > len) {
    goto error;
  }

  // the first blob starts after the offsets and count
  uint32_t table_end = (count + 1) * sizeof(uint32_t);
  uint32_t start = table_end;

  if (index > 0) {
    // Subsequent blobs start at the preceding exclusive end.
    // note that we're assuming little endian here, this could be generalized
    memcpy(&start, bytes + (uint32_t)index * sizeof(start), sizeof(start));
  }

  if (start < table_end || start > end) {
    goto error;
  }

  return cql_cursor_from_bytes(dyn_cursor, bytes + start, end - start);

error:
  *has_row = false;
  cql_clear_references_before_deserialization(dyn_cursor);
  return SQLITE_ERROR;
}