A predicate says what must be the case. An entailment says what must follow from what — so it can be compiled. Every statement in an Entail program becomes a specific group of weights in an ordinary transformer: nothing is learned, sampled or fitted. The result is exact integer logic running inside a model, or beside it, with the model's own prose left untouched.
AMQL's bake-predicate showed that a rule can be solved into weights — but what it writes is fitted to examples of a model obeying the rule. Entail is the deterministic counterpart: a program is a dataflow graph of consequences, and the compiler emits the weights that compute it.
An Entail program says what each token means, what follows from that, and what token must come next. It has no runtime of its own: compile turns it into a model, and from then on it is just weights. A state is written, then read; writing to it after anything has read it is an error, so a program cannot contain a cycle.
There are no layers in the language. The scheduler places each statement at the earliest layer its inputs allow, and reuses a residual dimension once nothing reads that lane any more — so a large program can compile to a narrow model. for, if, def/call and sum are generators: they expand at compile time and leave no trace in the weights.
Several entail statements on one state add up, so exclusive conditions read as "or" and a value under exclusive conditions as a case analysis. Expressions stay linear — a constant plus lanes times constants — because that is what a residual stream can carry exactly; a product of two states is refused, and gated with when instead.
# the digit after the one before the "?", wrapping 9 to 0
program next_digit
state val, known, asked
for d in 0 to 9
token "{d}" means val = d, known = 1
end
token "?" means asked = 1
state before, there
attend back 1
before <- val
there <- known
end
state ask, wrap, answer
entail ask when asked, there
entail wrap when before >= 9, ask
entail answer = before + 1 - 10*wrap when ask
state o[10]
for d in 0 to 9
entail o[d] when answer == d, ask
emit "{d}" when o[d]
end
# meaning without weights, then weights, then a host
amql-cli entail run next_digit.entail.txt --expr " 9?"
amql-cli entail compile next_digit.entail.txt --out model
amql-cli entail graft next_digit.entail.txt --host qwen --out grafted| You write | You get |
|---|---|
token "7" means val = 7 | Values in an embedding row — an input fact |
state x[8] scale 1/16 | Residual lanes, for as long as the lane is live |
let, const, for, if, def, sum | Nothing — substituted or expanded at compile time |
attend back k / attend back by ptr | One query head sharing a constant key and a value table; the second computes its distance |
entail y when E >= c | A pair of SwiGLU neurons (four for E == c) |
emit "7" when o[7] | An output row, or one column of the tied embedding in a host |
only / forbid | A mask column, negative in the rows ruled out — the host still chooses |
prefer / decide | A decision weighed against the host's own probability (adapters only) |
carry dest from source | A recurrent register: a fold over text of any length |
vote "text" by expr | A soft output for describing a model before replacing it — refused by compile |
Three properties of the emitted weights turn floating-point machinery into exact arithmetic rather than an approximation of it — and two more let a program see further than its heads and read further than its context.
RMSNorm divides by the root of the mean of squares. Entail spreads a constant ballast over just enough dimensions that the mean of squares is a power of four, so the norm's divisor is a power of two — exactly representable in binary floating point. Everything else in the lane is too small to survive the rounding back to float, so a lane reads back as precisely the number written to it.
Whole numbers therefore stay whole from layer to layer. Long division, which multiplies its remainder by ten at every step, depends on this. The budget is explicit: the sum of squares of all raw lanes at one position stays near 65536 × ballast dimensions, a threshold's whole number stays below 131072 for 24-bit exactness at gain 128, and every scale is a power of two. A host's embedding is bf16, so token values above 256 arrive in parts and are joined by a let.
A threshold is compiled as silu(g₁) − silu(g₂): two ramps whose difference is exact at every integer, with ±3/8 of noise margin. A gated value is silu(g) − silu(g − c), flat above its gate, so a fact slightly off its height still gates a value exactly.
The consequence is that a threshold's output is exactly 0 or 1 even when its input is slightly off — thresholds regenerate their input. A chain of them cannot accumulate noise, which is what makes a carry chain of comparisons across eight decimal places at a time possible at all.
Keys are constant and a query is a sum of RoPE phases, so a head's score peaks at exactly one relative offset. entail check proves the margin over the whole context — 580 logits over 32k positions — so the softmax output is one token's value, not a blend of neighbours.
On a host the lookup is rebuilt from the host's own rotary pairs: up to 16 of them, half-split, interleaved, partial or zero-tail, at its own frequencies, including linear, llama3, yarn and longrope scaling, with the score set for its scale and softcap. Where no choice of pairs makes every head a lookup, the graft is refused by name — Gemma 2's softcap of 50 cannot separate neighbouring offsets.
attend back k sees one fixed place, so a window would cost a head per token. Instead each token combines with what lies 1 back, then 2, 4, 8 and so on: nine heads carry a fact over 512 tokens, one head per doubling step. The JSON program uses this for sums such as nesting depth and for "the latest value written" at each depth.
A marker that clears what came before makes the result start where the text starts — the same parallel-prefix trick that lets a shallow circuit see a long way, priced at log₂(n) heads instead of n.
carry as a foldcarry dest from source starts every token holding another lane's final value from the token before. A program built this way is a fold: each token updates a fixed set of lanes from the ones before it, so it reads 64 bytes or 8 MB in the same memory. The scan program answered a question about records anywhere in an 8 MB text holding 2,640 bytes of state — three rows of 110 lanes, whatever the length. Before carry, a program could see only as far as its heads reached: nine doubling heads for 512 tokens. The fold makes the programmatic context thousands of times larger than any head-limited window, at constant memory.
A fixed-depth transformer cannot do this — a value written at the top layer for one token would have to be read at the bottom layer for the next — so compile and graft refuse a carry, while the interpreter and the sparse adapter run one. In weights the same operation is a recurrent layer: a Gated DeltaNet with decay 1 keeps a value, and its delta rule replaces the value stored under a key. That is the route to running folds inside a model.
Depth follows from the longest chain of dependencies; attention at a layer runs before that layer's neurons. Cost is counted as neurons plus eight per attention head, and the optimiser's gate accepts a change only if it lowers that cost without losing agreement.
The details matter: a lone threshold compiles directly, while one inside a when list needs a lane of its own and costs an extra layer; the same threshold asked twice shares that lane. Computing a value once at the position that ends the question and reading it back with attend back by took the refined mathematics program from 3,408 cost to 498 at identical answers.
Overwriting weights in place does not suit this: measured on Qwen 3.5 0.8B, the residual norm of one token type ranges from 1.7 to 12.6, no quiet subspace exists to carry a register, and a single head costs 5 nats to remove. So graft restructures the checkpoint instead, keeping every host weight's value.
The hidden size widens and the new dimensions are the program's lanes. Host matrices gain zeros there, so no host weight reads or writes a lane; host norm weights are rescaled so the wider mean leaves them where they were. The compiled layers go in front of the host's first layer, read only lanes, and finish by clearing every lane and the ballast exactly — so the host starts from its own embedding.
Each output lane is a column of the tied embedding, non-zero in its own token's row and the only lanes the final norm passes: a set lane raises exactly one logit. The end-of-answer lane lowers the logits of the tokens an answer is made of, so the number stops and the host says what follows. Wherever the program says nothing, the host's computation is unchanged.
Scales must stay powers of two. A host that scales its embedding by √hidden gets a width that is a power of four, so the scale remains a power of two and its own embedding rows move to keep their product, written in f32. In an MoE layer the block's neurons go in the shared expert with routed experts zero, or in every expert alike with down weights divided by the routing weight an even split gives.
amql-cli entail graft mathematics.entail.txt \
--host ./Qwen3.5-0.8B --out ./grafted
# judged by the importer's own rules, not by family name
16 zoo hosts: Llama, Qwen3, yarn Qwen2, Phi-3, longrope Phi-3,
GLM-4, Gemma 3, OLMo 2/3, SmolLM3, Ministral, Mixtral,
OLMoE, Qwen2-MoE, Qwen3-MoE …
silent text within 1e-3 of the host's logit scale;
the program's token wins by over 100 logits
# refused by name, with the property that breaks exactness
LayerNorm hosts · parallel blocks · ungated FFN ·
non-power-of-two multipliers · uneven MoE splits ·
dynamic rope past its maximum · Gemma 2's softcapAgainst the Qwen-only graft it replaces, all 705 tensors and the config are identical. Of the 16 zoo hosts, 14 reload and behave the same under each host's own transformers class; two cannot be reloaded there for reasons in the loader, though AMQL runs both. Silent text moves a little in every graft by one known mechanism: a host norm's epsilon stays the same while the mean it is added to shrinks with the wider stream, acting like an epsilon h′/h times larger.
graft and encode stream: the host is read through its memory map and each tensor written as it is made, so neither holds a model in memory and a sharded host grafts like a single file. entail eval switches to disk-backed weights by itself when a graft's f32 weights exceed the budget — exact f32 on the CPU, keeping what fits and reading the rest on every pass.
Exactness needs float32: the fp16 GPU weight path cannot hold the ballast. A grafted mathematics model is 59 layers instead of 24 and hidden 1664 instead of 1024, with the block stored in f32 — larger and slower. Where that trade is wrong, the same program runs as an adapter beside the untouched model, at the host's own precision.
Trained weights cannot be decompiled directly — what a model knows is spread across many units and superposed with everything else. So the cycle works from evidence: where a subject lives, measured by ablation, and what the model does, measured by probing it across a declared domain. Then it replaces the table with something exact.
Each probed row records the context, the model's answer and its margin: the smallest gap between the top two probabilities anywhere in the answer. A row near 1 was decided; a row near 0 was a coin toss that greedy decoding happened to resolve. About a third of rows are held out and no rule may be fitted to them — only the gate reads them. Negative evidence comes from background.jsonl: a false fire is a position where the program names a different token than the model predicts.
Every change, from any author — mechanical rule or reviewing model — passes the same gate: it must compile, add no false fires, and either raise agreement with decided held-out rows, or keep agreement on decided rows and lower cost (neurons plus eight per head). An elegant change that loses a row is refused.
The gate measures agreement with the model, which is not the same as being right. Rows where a correct law disagrees with a confident model are listed in overruled.jsonl, most decided first — the most interesting output of the cycle, because either the model is wrong there or the law is, and the gate cannot tell which.
amql-cli entail find <container> --subject s.json --top 128 --verify
amql-cli entail extract <container> --subject s.json --out dir
amql-cli entail optimise dir
amql-cli entail review dir --endpoint <url> --model <name>
amql-cli entail accept dir --candidate c.txt --rule R5 --why "…"
# R4: a law exists only if the coefficients come out whole
# and at least 60% of rows satisfy it exactly:
value = c0 + c1·a + c2·b + c3·a·b
# trimmed least squares on decided, fitted rows.
# nothing is searched; a mere trend gives fractions, refused.| Rule | Change | Author |
|---|---|---|
| R1 — keep only what was decided | Delete rows below the confidence threshold. A probability is not a fact. | Mechanical |
| R2 — hoist shared conditions | State repeated literal conditions once as a fact and reuse it. Meaning preserved exactly. | Mechanical |
| R3 — drop what does not matter | Remove a slot whose variation never changes the answer; unseen values are then answered too — a generalisation the held-out rows test. | Mechanical |
| R4 — replace a table with a law | Trimmed least squares on the fixed bilinear basis; whole coefficients and ≥60% exact rows, else rejected. | Mechanical |
| R5 — reuse a proven definition | Swap a narrow law for a general one already defined exactly, justified by held-out agreement or coverage. | Reviewer |
| R6 — share work between positions | Compute once where the question ends; later positions read the result with attend back by. | Reviewer |
| R7 — collapse thresholds | Fewest thresholds that decide the same whole numbers; a lone threshold costs no extra layer. | Reviewer |
| R8 — name the structure | Give a class of rows a shared fact and write one rule per class instead of one row per member. | Reviewer |
| R9 — add a rule to the guide | An improvement no rule covers: make it, cite R9, and say what the new rule should be. If it survives the gate and recurs, every later reviewer inherits it. | Reviewer |
Two-digit +, − and *, 300 instances, 91 held out: the extracted table agreed with the model on 0 of 70 decided held-out rows at cost 2,236; R4 derived a+b, a−b and a*b from the model's own answers — no arithmetic given to it — reaching 69/70; R6 cut cost to 498 at the same answers.
The refined program disagreed with the model on 47 rows. Checked against true arithmetic afterwards, the program was right on all 47 and the model wrong. Mean margin was 0.79 on the model's right answers and 0.14 on its wrong ones; 46 of the 47 fell below threshold. The one that did not, 51*47 = 2407, is a confident mistake. Grafted back, unseen operand pairs went from 126/150 (84%) to 150/150, with prose top-1 agreement 767/767 and loss 3.55195 → 3.55198.
The LoRA trainer's forward pass is AMQL's own runtime; the backward pass runs exactly through the output head and the final RMSNorm, which for a LoRA on the last layer's down-projection is the whole chain — so its gradient is exact. A finite-difference check runs at the first step of every run (predicted vs measured ratio 1.0001–1.0002). For earlier layers the gradient follows the direct residual path only, and the same check measures how far off that is rather than assuming.
Layers below the first LoRA layer run once per example; a spike guard undoes any step that raises loss by more than half and halves the rate; unlabelled anchors become extra targets from the model's own replies, so new behaviour is learned without forgetting the old. And entail from-lora reads each rank component back as Entail — showing, without a held-out set, that one training example taught a bias on the first token ("suppress To and I"), not a rule.
The 0.8B model's first-token probability of <tool_call> already separates requests that need a tool from those that don't (AUC 0.98) — it is simply below prose's, so the model acts only on better-than-even. On real conversations the raw probability weakens to AUC 0.69, because each has its own baseline inclination to call.
The fix is contextual calibration per prompt: re-run the conversation with the latest message blanked and take the ratio. AUC rises to 0.78 on real conversations and stays 0.98 on clean ones. One statement — decide "<tool_call>" at 2 times prior, ten neurons, no word lists — calls when the latest message at least doubles that inclination: clean-set calls 23 → 36 of 36 with no new false call, and on 120 real policy conversations calls 64 → 94 with right tools 46 → 70. The model supplies the evidence; the program supplies the threshold and where to apply it.
Greedy decoding on the CPU, Qwen 3.5 0.8B before and after, on operands and cases drawn with seeds not used during development. Answers were worked out separately in Python.
| Exact answers | Base | Grafted / beside |
|---|---|---|
| Add, 2 / 3 / 5 / 8 digits | 15 / 15 / 11 / 4 of 15 | 15 / 15 / 15 / 15 |
| Subtract, 2 / 3 / 5 / 8 digits | 14 / 6 / 5 / 3 of 15 | 15 / 15 / 15 / 15 |
| Multiply, 2×2 / 3×3 / 4×4 / 6×6 | 13 / 3 / 0 / 0 of 15 | 15 / 15 / 15 / 15 |
| Divide, exact / 5 by 2 digits | 2 / 1 of 15 | 15 / 15 |
| Direct arithmetic, overall | 92/210 (43.8%) | 210/210 (100%) |
| Chat: add 5 digits / multiply 3×3 / subtract 4 digits | 13 / 1 / 7 of 15 | 15 / 15 / 7 of 15 |
| Word problems, step by step | 10/15 | 15/15 |
| Dates (weekdays, differences, sums, leap years) | 54/250 | 250/250 |
| Number bases, 2 to 16, up to 32 bits | 41/315 | 315/315 |
| Boolean logic: tautology, fallacies, truth tables | 45/218 | 217/218 one expression over the 28-event limit |
| JSON verdicts, valid and altered texts | 217/370 | 370/370 |
| JSON guard: unfinished texts still valid | 38/60 | 60/60 |
| Physics: laws, prefixed units, sums, refused mismatches | 12/180 numeric | 192/196 on new numbers |
| Chemistry: molar masses, balanced and unbalanced equations | 104/182 | 182/182, then 70/70 new |
| Prose: top-1 agreement with base / loss, nats per token | 3.55195 | 767/767 · 3.55210 graft · identical beside |
The graft broke no answer the base model had right. The eight chat failures are one case: asked for a negative difference, the model states a number without writing the expression, so the program has nothing to read. Physics is the algebra of units — a unit is a vector of exponents over kg, m, s, A, K, mol plus a power of ten, so multiplying adds vectors and adding demands they match; chemistry is a sum over atoms, with a balance check on total mass. Neither sets up a problem: each gives the exact answer to what was written.
A model's context is limited by the layers that keep every past token — on Qwen 3.5 0.8B, six full-attention layers out of twenty-four; the other eighteen are recurrent and already read any length in fixed memory. A program's context need not be limited at all. Before carry, an Entail program could see only as far as its heads reached: one head per token of window, or log₂(n) heads by doubling — nine heads for 512 tokens. A carry adds one edge from each token to the next, and the window becomes the whole text.
Measured with scan.entail.txt: the question is read first, then records, and the answer is the largest value filed under the asked key — with one higher value planted at a random place, so answering depends on reading everything. From 66 bytes to 8,388,609 bytes the state held is the same 2,640 bytes, and every answer is right. That is a programmatic context thousands of times larger than the doubling window, and far beyond the host's own 32k-token context, at constant memory.
The strategy matters more than the length: put the question first, fold in windows the model can handle, keep the carried state small and typed. Asked at the end instead, the same scan keeps one register per key — 5,640 bytes — and still answers the 2 MB text correctly. What this shows is the program side reading 8 MB exactly in fixed memory; the model itself reading 2 MB, with the loop writing the notes, is the next experiment.
# one recurrent register: a fold over text of any length
carry last_best from best
amql-cli entail run scan.entail.txt --file first-8mb.txt
# 8,388,609 bytes read, one record decides the answer,
# 2,640 bytes of state held — the same as for 66 bytes| Text | Bytes | Answer | State held | Time |
|---|---|---|---|---|
| first-64b | 66 | correct | 2,640 bytes | 0.0 s |
| first-4kb | 4,098 | correct | 2,640 bytes | 0.1 s |
| first-2mb | 2,097,153 | correct | 2,640 bytes | 34 s |
| first-8mb | 8,388,609 | correct | 2,640 bytes | 151 s |
State is three rows of 110 lanes — the row being built, the one before it, and the first — whatever the length. Times are the interpreter's, in doubles, with no attempt at speed. compile, graft and the dense adapter refuse a carry by name; in weights the same operation is a recurrent layer, which is the route to running folds inside a model.
An adapter leaves the partner alone: the program runs next to it as a blank model of its own, reads the same tokens, and rules on the next one at the sampler — in doubles, whatever precision the partner is served at. The two share nothing but token ids, so what came from Entail and what came from the model can always be told apart: each token's provenance is Partner, Narrowed or Entail.
A compiled program is almost entirely zeros — 0.063% non-zero across the four formal programs, 14.4 GB dense. So the default adapter holds no weights at all: it compiles the program at load and executes its statements directly, a head reading a fixed place in a ring of recent tokens. All four together cost about 1 MB and 5.2 ms a token, and all 1,176 sparse replies match the dense form exactly.
only and forbid narrow what the host may say without choosing for it: a JSON guard rules out every token that would break a text as it is written, with no sampler outside the model. Adapters join in order — first Say wins, Only sets intersect — and if rules conflict or leave nothing to choose, the partner goes free rather than the text stopping.
Fine-tuning reaches conclusions already reachable from human-written text, and fits them statistically. A compiled program is a different instrument: exact, readable, small, and separable from the model it improves. These are the properties that matter in practice — and the limits we state plainly.
A law needs no training data and no sampling: the calendar, positional notation, truth tables and a grammar are rule systems with nothing to learn. Exactness is provable before deployment — entail check proves every head's attention margin over the context — and entail run executes the program's meaning with no weights at all, which is what the compiled model must reproduce.
The program is readable source, not a delta whose effect must be guessed. The optimisation journal records every accepted and refused change with its rule and reason; overruled.jsonl lists where the program and the model disagree. An adapter never touches the partner's weights, programs stay separate and switchable per request, and removing one restores the model exactly.
The whole tool-call strategy is ten neurons and one threshold; the refined arithmetic program costs 498 against the hand-written general mathematics program's larger coverage. The split of labour is the point: the model supplies meaning — which tool is near, where a value lies — and the program supplies strategy: when to call, what takes precedence, what to carry. And context stops being a window: with the question first, a fold reads 8 MB in 2,640 bytes of state — the program's context is the whole text, not the host's window.
A graft needs f32 to hold its ballast; an adapter does not depend on the host's precision at all, because it runs in doubles beside weights served at fp16 or bf16. Prose loss with adapters on and off is identical, 3.55195 — a graft's widening changes rounding and can flip a near tie, an adapter cannot.
Because a flaw becomes an address, relationships humanity has not described linguistically can be adjusted the same way — and the extract/optimise cycle derives laws from a model's own noisy answers, then overrules its confident mistakes. That reaches things fine-tuning cannot: it is gated at human-level text, while a compiled law is gated at nothing but its own correctness.
This adds exactness, not reasoning: a model that sets up the wrong sum gets the wrong sum's exact answer. A program acts on notation it can read — Qwen writes its working in LaTeX, so no chat answer improved. Subjects must be expressible as relations over a declared domain; R4's basis is bilinear over two numeric slots; a host must be one the importer can judge. Research code, developed in the open — refusals and all.
Entail is part of AMQL, the model-introspection engine behind the Ariadne AI Platform. See the rest of the research lab, or how it comes together in production.