Research · AMQL · November 2026

Entail: statements compiled into weights.

A predicate says what must be the case. An entailment says what must follow from what — so it can be compiled. Every statement in an Entail program becomes a specific group of weights in an ordinary transformer: nothing is learned, sampled or fitted. The result is exact integer logic running inside a model, or beside it, with the model's own prose left untouched.

210/210Exact arithmetic, up from 92/210
767/767Prose predictions unchanged by the graft
~1 MBFour programs beside a model, sparse
8 MBRead exactly in 2,640 bytes of state
The theory

From fitted rules to compiled consequences.

AMQL's bake-predicate showed that a rule can be solved into weights — but what it writes is fitted to examples of a model obeying the rule. Entail is the deterministic counterpart: a program is a dataflow graph of consequences, and the compiler emits the weights that compute it.

A program is a dataflow graph

An Entail program says what each token means, what follows from that, and what token must come next. It has no runtime of its own: compile turns it into a model, and from then on it is just weights. A state is written, then read; writing to it after anything has read it is an error, so a program cannot contain a cycle.

There are no layers in the language. The scheduler places each statement at the earliest layer its inputs allow, and reuses a residual dimension once nothing reads that lane any more — so a large program can compile to a narrow model. for, if, def/call and sum are generators: they expand at compile time and leave no trace in the weights.

Several entail statements on one state add up, so exclusive conditions read as "or" and a value under exclusive conditions as a case analysis. Expressions stay linear — a constant plus lanes times constants — because that is what a residual stream can carry exactly; a product of two states is refused, and gated with when instead.

# the digit after the one before the "?", wrapping 9 to 0 program next_digit state val, known, asked for d in 0 to 9 token "{d}" means val = d, known = 1 end token "?" means asked = 1 state before, there attend back 1 before <- val there <- known end state ask, wrap, answer entail ask when asked, there entail wrap when before >= 9, ask entail answer = before + 1 - 10*wrap when ask state o[10] for d in 0 to 9 entail o[d] when answer == d, ask emit "{d}" when o[d] end # meaning without weights, then weights, then a host amql-cli entail run next_digit.entail.txt --expr " 9?" amql-cli entail compile next_digit.entail.txt --out model amql-cli entail graft next_digit.entail.txt --host qwen --out grafted
You writeYou get
token "7" means val = 7Values in an embedding row — an input fact
state x[8] scale 1/16Residual lanes, for as long as the lane is live
let, const, for, if, def, sumNothing — substituted or expanded at compile time
attend back k / attend back by ptrOne query head sharing a constant key and a value table; the second computes its distance
entail y when E >= cA pair of SwiGLU neurons (four for E == c)
emit "7" when o[7]An output row, or one column of the tied embedding in a host
only / forbidA mask column, negative in the rows ruled out — the host still chooses
prefer / decideA decision weighed against the host's own probability (adapters only)
carry dest from sourceA recurrent register: a fold over text of any length
vote "text" by exprA soft output for describing a model before replacing it — refused by compile
The mathematics

Why a stack of norms and SwiGLUs is integer logic.

Three properties of the emitted weights turn floating-point machinery into exact arithmetic rather than an approximation of it — and two more let a program see further than its heads and read further than its context.

Exact gain: the ballast

RMSNorm divides by the root of the mean of squares. Entail spreads a constant ballast over just enough dimensions that the mean of squares is a power of four, so the norm's divisor is a power of two — exactly representable in binary floating point. Everything else in the lane is too small to survive the rounding back to float, so a lane reads back as precisely the number written to it.

Whole numbers therefore stay whole from layer to layer. Long division, which multiplies its remainder by ten at every step, depends on this. The budget is explicit: the sum of squares of all raw lanes at one position stays near 65536 × ballast dimensions, a threshold's whole number stays below 131072 for 24-bit exactness at gain 128, and every scale is a power of two. A host's embedding is bf16, so token values above 256 arrive in parts and are joined by a let.

Regeneration: thresholds that clean their input

A threshold is compiled as silu(g₁) − silu(g₂): two ramps whose difference is exact at every integer, with ±3/8 of noise margin. A gated value is silu(g) − silu(g − c), flat above its gate, so a fact slightly off its height still gates a value exactly.

The consequence is that a threshold's output is exactly 0 or 1 even when its input is slightly off — thresholds regenerate their input. A chain of them cannot accumulate noise, which is what makes a carry chain of comparisons across eight decimal places at a time possible at all.

Attention as lookup, not blend

Keys are constant and a query is a sum of RoPE phases, so a head's score peaks at exactly one relative offset. entail check proves the margin over the whole context — 580 logits over 32k positions — so the softmax output is one token's value, not a blend of neighbours.

On a host the lookup is rebuilt from the host's own rotary pairs: up to 16 of them, half-split, interleaved, partial or zero-tail, at its own frequencies, including linear, llama3, yarn and longrope scaling, with the score set for its scale and softcap. Where no choice of pairs makes every head a lookup, the graft is refused by name — Gemma 2's softcap of 50 cannot separate neighbouring offsets.

Seeing further than a window: doubling

attend back k sees one fixed place, so a window would cost a head per token. Instead each token combines with what lies 1 back, then 2, 4, 8 and so on: nine heads carry a fact over 512 tokens, one head per doubling step. The JSON program uses this for sums such as nesting depth and for "the latest value written" at each depth.

A marker that clears what came before makes the result start where the text starts — the same parallel-prefix trick that lets a shallow circuit see a long way, priced at log₂(n) heads instead of n.

Reading any length: carry as a fold

carry dest from source starts every token holding another lane's final value from the token before. A program built this way is a fold: each token updates a fixed set of lanes from the ones before it, so it reads 64 bytes or 8 MB in the same memory. The scan program answered a question about records anywhere in an 8 MB text holding 2,640 bytes of state — three rows of 110 lanes, whatever the length. Before carry, a program could see only as far as its heads reached: nine doubling heads for 512 tokens. The fold makes the programmatic context thousands of times larger than any head-limited window, at constant memory.

A fixed-depth transformer cannot do this — a value written at the top layer for one token would have to be read at the bottom layer for the next — so compile and graft refuse a carry, while the interpreter and the sparse adapter run one. In weights the same operation is a recurrent layer: a Gated DeltaNet with decay 1 keeps a value, and its delta rule replaces the value stored under a key. That is the route to running folds inside a model.

The scheduler's cost model

Depth follows from the longest chain of dependencies; attention at a layer runs before that layer's neurons. Cost is counted as neurons plus eight per attention head, and the optimiser's gate accepts a change only if it lowers that cost without losing agreement.

The details matter: a lone threshold compiles directly, while one inside a when list needs a lane of its own and costs an extra layer; the same threshold asked twice shares that lane. Computing a value once at the position that ends the question and reading it back with attend back by took the refined mathematics program from 3,408 cost to 498 at identical answers.

Grafting

Exactness inside a host that never notices.

Overwriting weights in place does not suit this: measured on Qwen 3.5 0.8B, the residual norm of one token type ranges from 1.7 to 12.6, no quiet subspace exists to carry a register, and a single head costs 5 nats to remove. So graft restructures the checkpoint instead, keeping every host weight's value.

The widening algebra

The hidden size widens and the new dimensions are the program's lanes. Host matrices gain zeros there, so no host weight reads or writes a lane; host norm weights are rescaled so the wider mean leaves them where they were. The compiled layers go in front of the host's first layer, read only lanes, and finish by clearing every lane and the ballast exactly — so the host starts from its own embedding.

Each output lane is a column of the tied embedding, non-zero in its own token's row and the only lanes the final norm passes: a set lane raises exactly one logit. The end-of-answer lane lowers the logits of the tokens an answer is made of, so the number stops and the host says what follows. Wherever the program says nothing, the host's computation is unchanged.

Scales must stay powers of two. A host that scales its embedding by √hidden gets a width that is a power of four, so the scale remains a power of two and its own embedding rows move to keep their product, written in f32. In an MoE layer the block's neurons go in the shared expert with routed experts zero, or in every expert alike with down weights divided by the routing weight an even split gives.

amql-cli entail graft mathematics.entail.txt \ --host ./Qwen3.5-0.8B --out ./grafted # judged by the importer's own rules, not by family name 16 zoo hosts: Llama, Qwen3, yarn Qwen2, Phi-3, longrope Phi-3, GLM-4, Gemma 3, OLMo 2/3, SmolLM3, Ministral, Mixtral, OLMoE, Qwen2-MoE, Qwen3-MoE … silent text within 1e-3 of the host's logit scale; the program's token wins by over 100 logits # refused by name, with the property that breaks exactness LayerNorm hosts · parallel blocks · ungated FFN · non-power-of-two multipliers · uneven MoE splits · dynamic rope past its maximum · Gemma 2's softcap

Verified, not assumed

Against the Qwen-only graft it replaces, all 705 tensors and the config are identical. Of the 16 zoo hosts, 14 reload and behave the same under each host's own transformers class; two cannot be reloaded there for reasons in the loader, though AMQL runs both. Silent text moves a little in every graft by one known mechanism: a host norm's epsilon stays the same while the mean it is added to shrinks with the wider stream, acting like an epsilon h′/h times larger.

Bigger than memory is fine

graft and encode stream: the host is read through its memory map and each tensor written as it is made, so neither holds a model in memory and a sharded host grafts like a single file. entail eval switches to disk-backed weights by itself when a graft's f32 weights exceed the budget — exact f32 on the CPU, keeping what fits and reading the rest on every pass.

The price, stated plainly

Exactness needs float32: the fp16 GPU weight path cannot hold the ballast. A grafted mathematics model is 59 layers instead of 24 and hidden 1664 instead of 1024, with the block stored in f32 — larger and slower. Where that trade is wrong, the same program runs as an adapter beside the untouched model, at the host's own precision.

The mathematics of improvement

Deriving laws from a model's own answers.

Trained weights cannot be decompiled directly — what a model knows is spread across many units and superposed with everything else. So the cycle works from evidence: where a subject lives, measured by ablation, and what the model does, measured by probing it across a declared domain. Then it replaces the table with something exact.

Evidence, margin and the gate

Each probed row records the context, the model's answer and its margin: the smallest gap between the top two probabilities anywhere in the answer. A row near 1 was decided; a row near 0 was a coin toss that greedy decoding happened to resolve. About a third of rows are held out and no rule may be fitted to them — only the gate reads them. Negative evidence comes from background.jsonl: a false fire is a position where the program names a different token than the model predicts.

Every change, from any author — mechanical rule or reviewing model — passes the same gate: it must compile, add no false fires, and either raise agreement with decided held-out rows, or keep agreement on decided rows and lower cost (neurons plus eight per head). An elegant change that loses a row is refused.

The gate measures agreement with the model, which is not the same as being right. Rows where a correct law disagrees with a confident model are listed in overruled.jsonl, most decided first — the most interesting output of the cycle, because either the model is wrong there or the law is, and the gate cannot tell which.

amql-cli entail find <container> --subject s.json --top 128 --verify amql-cli entail extract <container> --subject s.json --out dir amql-cli entail optimise dir amql-cli entail review dir --endpoint <url> --model <name> amql-cli entail accept dir --candidate c.txt --rule R5 --why "…" # R4: a law exists only if the coefficients come out whole # and at least 60% of rows satisfy it exactly: value = c0 + c1·a + c2·b + c3·a·b # trimmed least squares on decided, fitted rows. # nothing is searched; a mere trend gives fractions, refused.
RuleChangeAuthor
R1 — keep only what was decidedDelete rows below the confidence threshold. A probability is not a fact.Mechanical
R2 — hoist shared conditionsState repeated literal conditions once as a fact and reuse it. Meaning preserved exactly.Mechanical
R3 — drop what does not matterRemove a slot whose variation never changes the answer; unseen values are then answered too — a generalisation the held-out rows test.Mechanical
R4 — replace a table with a lawTrimmed least squares on the fixed bilinear basis; whole coefficients and ≥60% exact rows, else rejected.Mechanical
R5 — reuse a proven definitionSwap a narrow law for a general one already defined exactly, justified by held-out agreement or coverage.Reviewer
R6 — share work between positionsCompute once where the question ends; later positions read the result with attend back by.Reviewer
R7 — collapse thresholdsFewest thresholds that decide the same whole numbers; a lone threshold costs no extra layer.Reviewer
R8 — name the structureGive a class of rows a shared fact and write one rule per class instead of one row per member.Reviewer
R9 — add a rule to the guideAn improvement no rule covers: make it, cite R9, and say what the new rule should be. If it survives the gate and recurs, every later reviewer inherits it.Reviewer

What the cycle achieved

Two-digit +, − and *, 300 instances, 91 held out: the extracted table agreed with the model on 0 of 70 decided held-out rows at cost 2,236; R4 derived a+b, a−b and a*b from the model's own answers — no arithmetic given to it — reaching 69/70; R6 cut cost to 498 at the same answers.

The refined program disagreed with the model on 47 rows. Checked against true arithmetic afterwards, the program was right on all 47 and the model wrong. Mean margin was 0.79 on the model's right answers and 0.14 on its wrong ones; 46 of the 47 fell below threshold. The one that did not, 51*47 = 2407, is a confident mistake. Grafted back, unseen operand pairs went from 126/150 (84%) to 150/150, with prose top-1 agreement 767/767 and loss 3.55195 → 3.55198.

Training without autograd

The LoRA trainer's forward pass is AMQL's own runtime; the backward pass runs exactly through the output head and the final RMSNorm, which for a LoRA on the last layer's down-projection is the whole chain — so its gradient is exact. A finite-difference check runs at the first step of every run (predicted vs measured ratio 1.0001–1.0002). For earlier layers the gradient follows the direct residual path only, and the same check measures how far off that is rather than assuming.

Layers below the first LoRA layer run once per example; a spike guard undoes any step that raises loss by more than half and halves the rate; unlabelled anchors become extra targets from the model's own replies, so new behaviour is learned without forgetting the old. And entail from-lora reads each rank component back as Entail — showing, without a held-out set, that one training example taught a bias on the first token ("suppress To and I"), not a rule.

Calibrating a decision on the host's evidence

The 0.8B model's first-token probability of <tool_call> already separates requests that need a tool from those that don't (AUC 0.98) — it is simply below prose's, so the model acts only on better-than-even. On real conversations the raw probability weakens to AUC 0.69, because each has its own baseline inclination to call.

The fix is contextual calibration per prompt: re-run the conversation with the latest message blanked and take the ratio. AUC rises to 0.78 on real conversations and stays 0.98 on clean ones. One statement — decide "<tool_call>" at 2 times prior, ten neurons, no word lists — calls when the latest message at least doubles that inclination: clean-set calls 23 → 36 of 36 with no new false call, and on 120 real policy conversations calls 64 → 94 with right tools 46 → 70. The model supplies the evidence; the program supplies the threshold and where to apply it.

Measured results

What compiled statements achieve.

Greedy decoding on the CPU, Qwen 3.5 0.8B before and after, on operands and cases drawn with seeds not used during development. Answers were worked out separately in Python.

Exact answersBaseGrafted / beside
Add, 2 / 3 / 5 / 8 digits15 / 15 / 11 / 4 of 1515 / 15 / 15 / 15
Subtract, 2 / 3 / 5 / 8 digits14 / 6 / 5 / 3 of 1515 / 15 / 15 / 15
Multiply, 2×2 / 3×3 / 4×4 / 6×613 / 3 / 0 / 0 of 1515 / 15 / 15 / 15
Divide, exact / 5 by 2 digits2 / 1 of 1515 / 15
Direct arithmetic, overall92/210 (43.8%)210/210 (100%)
Chat: add 5 digits / multiply 3×3 / subtract 4 digits13 / 1 / 7 of 1515 / 15 / 7 of 15
Word problems, step by step10/1515/15
Dates (weekdays, differences, sums, leap years)54/250250/250
Number bases, 2 to 16, up to 32 bits41/315315/315
Boolean logic: tautology, fallacies, truth tables45/218217/218 one expression over the 28-event limit
JSON verdicts, valid and altered texts217/370370/370
JSON guard: unfinished texts still valid38/6060/60
Physics: laws, prefixed units, sums, refused mismatches12/180 numeric192/196 on new numbers
Chemistry: molar masses, balanced and unbalanced equations104/182182/182, then 70/70 new
Prose: top-1 agreement with base / loss, nats per token3.55195767/767 · 3.55210 graft · identical beside

The graft broke no answer the base model had right. The eight chat failures are one case: asked for a negative difference, the model states a number without writing the expression, so the program has nothing to read. Physics is the algebra of units — a unit is a vector of exponents over kg, m, s, A, K, mol plus a power of ten, so multiplying adds vectors and adding demands they match; chemistry is a sum over atoms, with a balance check on total mass. Neither sets up a problem: each gives the exact answer to what was written.

Programmatic context: from 512 tokens to 8 MB

A model's context is limited by the layers that keep every past token — on Qwen 3.5 0.8B, six full-attention layers out of twenty-four; the other eighteen are recurrent and already read any length in fixed memory. A program's context need not be limited at all. Before carry, an Entail program could see only as far as its heads reached: one head per token of window, or log₂(n) heads by doubling — nine heads for 512 tokens. A carry adds one edge from each token to the next, and the window becomes the whole text.

Measured with scan.entail.txt: the question is read first, then records, and the answer is the largest value filed under the asked key — with one higher value planted at a random place, so answering depends on reading everything. From 66 bytes to 8,388,609 bytes the state held is the same 2,640 bytes, and every answer is right. That is a programmatic context thousands of times larger than the doubling window, and far beyond the host's own 32k-token context, at constant memory.

The strategy matters more than the length: put the question first, fold in windows the model can handle, keep the carried state small and typed. Asked at the end instead, the same scan keeps one register per key — 5,640 bytes — and still answers the 2 MB text correctly. What this shows is the program side reading 8 MB exactly in fixed memory; the model itself reading 2 MB, with the loop writing the notes, is the next experiment.

# one recurrent register: a fold over text of any length carry last_best from best amql-cli entail run scan.entail.txt --file first-8mb.txt # 8,388,609 bytes read, one record decides the answer, # 2,640 bytes of state held — the same as for 66 bytes
TextBytesAnswerState heldTime
first-64b66correct2,640 bytes0.0 s
first-4kb4,098correct2,640 bytes0.1 s
first-2mb2,097,153correct2,640 bytes34 s
first-8mb8,388,609correct2,640 bytes151 s

State is three rows of 110 lanes — the row being built, the one before it, and the first — whatever the length. Times are the interpreter's, in doubles, with no attempt at speed. compile, graft and the dense adapter refuse a carry by name; in weights the same operation is a recurrent layer, which is the route to running folds inside a model.

Beside the model: adapters

An adapter leaves the partner alone: the program runs next to it as a blank model of its own, reads the same tokens, and rules on the next one at the sampler — in doubles, whatever precision the partner is served at. The two share nothing but token ids, so what came from Entail and what came from the model can always be told apart: each token's provenance is Partner, Narrowed or Entail.

Sparse by construction

A compiled program is almost entirely zeros — 0.063% non-zero across the four formal programs, 14.4 GB dense. So the default adapter holds no weights at all: it compiles the program at load and executes its statements directly, a head reading a fixed place in a ring of recent tokens. All four together cost about 1 MB and 5.2 ms a token, and all 1,176 sparse replies match the dense form exactly.

Constrained decoding in weights

only and forbid narrow what the host may say without choosing for it: a JSON guard rules out every token that would break a text as it is written, with no sampler outside the model. Adapters join in order — first Say wins, Only sets intersect — and if rules conflict or leave nothing to choose, the partner goes free rather than the text stopping.

Why programmatic model engineering

The advantages of writing a model's behaviour down.

Fine-tuning reaches conclusions already reachable from human-written text, and fits them statistically. A compiled program is a different instrument: exact, readable, small, and separable from the model it improves. These are the properties that matter in practice — and the limits we state plainly.

Deterministic, not fitted

A law needs no training data and no sampling: the calendar, positional notation, truth tables and a grammar are rule systems with nothing to learn. Exactness is provable before deployment — entail check proves every head's attention margin over the context — and entail run executes the program's meaning with no weights at all, which is what the compiled model must reproduce.

Auditable and reversible

The program is readable source, not a delta whose effect must be guessed. The optimisation journal records every accepted and refused change with its rule and reason; overruled.jsonl lists where the program and the model disagree. An adapter never touches the partner's weights, programs stay separate and switchable per request, and removing one restores the model exactly.

Small where it matters

The whole tool-call strategy is ten neurons and one threshold; the refined arithmetic program costs 498 against the hand-written general mathematics program's larger coverage. The split of labour is the point: the model supplies meaning — which tool is near, where a value lies — and the program supplies strategy: when to call, what takes precedence, what to carry. And context stops being a window: with the question first, a fold reads 8 MB in 2,640 bytes of state — the program's context is the whole text, not the host's window.

Independent of precision

A graft needs f32 to hold its ballast; an adapter does not depend on the host's precision at all, because it runs in doubles beside weights served at fp16 or bf16. Prose loss with adapters on and off is identical, 3.55195 — a graft's widening changes rounding and can flip a near tie, an adapter cannot.

A route beyond fine-tuning

Because a flaw becomes an address, relationships humanity has not described linguistically can be adjusted the same way — and the extract/optimise cycle derives laws from a model's own noisy answers, then overrules its confident mistakes. That reaches things fine-tuning cannot: it is gated at human-level text, while a compiled law is gated at nothing but its own correctness.

The limits, stated plainly

This adds exactness, not reasoning: a model that sets up the wrong sum gets the wrong sum's exact answer. A program acts on notation it can read — Qwen writes its working in LaTeX, so no chat answer improved. Subjects must be expressible as relations over a declared domain; R4's basis is bilinear over two numeric slots; a host must be one the importer can judge. Research code, developed in the open — refusals and all.

From lab to product

This research ships as Ariadne.

Entail is part of AMQL, the model-introspection engine behind the Ariadne AI Platform. See the rest of the research lab, or how it comes together in production.