Verified Knowledge Extraction from Language ModelsRelations as operators, a knowledge graph as the gate, and three retracted results

Epistemic status: pilot scale, single model (Qwen2.5-1.5B), six relations, all bars pre-registered before scoring. The architecture claim is solid; the scale claims are not yet made. Every number here is transcribed from a pre-registered measurement report. Nothing was re-framed after seeing results.

Three organs, strictly separated

A knowledge graph is queryable, auditable, and silent about what it does not contain. A language model covers vastly more ground with no index, no provenance, and no way to tell recall from fabrication. The only conventional readout, generation, is exactly the operation that invents.

So the design problem is not extraction. Any method will emit confident triples. The problem is verification under adversarial self-interest, including our own. Geometry may only propose. Symbols must verify. Everything that survives carries how it is known, and everything that does not is reported as absent.

The commit law

One rule governs the system, derived by measurement rather than assumed: confidence is carried by edge-disjoint, provenance-distinct anchor count. In our graph one supporting path yields 88.3% precision; two disjoint paths yield 100.0%. That step is why the gate commits at two.

Independent is the load-bearing word. Two chains through the same hub are one path wearing two hats - enforcing that cut fabrications from 19 to 2 in an earlier experiment. Two samples from the same model are likewise one source: asked to corroborate seven plausible fake entities, two passes of the same model family agreed on the same wrong answer 7 times out of 7, while the symbolic gate abstained on all seven.

For six relations we sampled verified subject-object pairs from a 66.4M-edge graph (2,000 train / 250 blind held-out each, split by subject-name hash), encoded through Qwen2.5-1.5B. From the training pairs we estimated an additive offset and a ridge affine map, then decoded held-out subjects by nearest neighbour against a 16,648-entity bank.

The control that matters is the identity baseline: decode the subject's own state with no operator applied. If proximity alone explained the result, it would score well.

The wall that reproduced across architectures

The bottom two rows are the interesting failure. Relations whose objects are attributes or types form a strong channel. Relations whose objects are specific individuals drawn from a large set do not - the operator adds roughly three points over raw proximity. We had already measured that same ceiling in a different embedding space with different machinery. Finding it again inside a language model's own contextual space suggests it is a property of the representation class:

Geometry carries kinds; it does not carry individuals. Symbols must own identity.

We treat that as a routing rule rather than an obstacle. Attribute questions go to the geometric channel, which is cheap and batched at 399 entities per second. Identity questions go to the graph. A related result supports the same split: decoding operator output through the model's own vocabulary was essentially dead at ≤2%, so extraction is inherently closed-world - the graph supplies the vocabulary of possible answers, the model supplies only the geometry that selects among them.

The gate, tested where it can fail

Corroboration ran with the target relation masked across the entire graph, including its inverse. The gate cannot verify a proposal by looking the answer up, only by assembling independent evidence from other relations.

The pooled failures are not random. They are type-confusions verified by association: proposing "Thai" for a language slot attracts hundreds of converging anchors, because Thai and Thailand are genuinely related. The gate as built verifies relatedness. The missing constraint is that the committed object must be of the type the relation demands. That constraint was available in the graph and deliberately not applied after seeing the results, since fixing a bar post-hoc invalidates it.

Correcting our own headline

A follow-up applied the missing type constraint, and more importantly ran the null arm the original probe had not: what does a constant guess achieve through the same gate?

The type constraint worked as predicted - pooled precision rose from 86.1% to 97.1%, eliminating 13 of 15 wrong commits at a 35% coverage cost. But the null arms corrected us.

A real channel survives underneath. Excluding high-base-rate objects and sweeping 62,000 subjects produced 696 residual extractions with diverse objects, against a wrong-subject null of 32 and a random-direction null of 1 - beating its nulls by roughly 22×. Graded externally, 107 of 400 sampled rows were gradeable at all (73% are tail entities no external source covers) and those scored 90.7%. In the reverse direction, 48 disagreements with the graph were hand-classified: 26 graph-right, 18 entity-linking artifacts, and 2 genuine graph errors the model caught. Against the pre-declared bar this is partial, so the artifact ships exploratory and is not promoted into the graph.

The same discipline that produced our best number is what took it back. A headline that cannot survive a constant-guess baseline was never a measurement of the method.

Result II — a negative result, reported first-class

Sparse autoencoders decompose activations into features. The standing weakness of that literature is that features get labeled by asking a language model what they seem to mean, which is unverified and circular. Our corpus aligns text to verified triples, so for any context we already know which entities and classes are present. That turns feature labeling into a measurement against external ground truth.

We trained a top-k sparse autoencoder (24,576 features, k=32) on 250,000 activation positions. It reconstructed well - 96.5% of variance explained against a 60% floor, 0.07% dead features - and under our first grounding bench it labeled 41 features at ≥90% held-out precision. That number is wrong, and our own controls said so.

The mechanism of the false positive generalises, which is why it is reported in detail. One class was 63% of the evaluation data and nearly linearly separable in raw activations; a threshold test on a dominant, geometrically loud class is easy to pass by accident. Cosine against an unnormalised class centroid mostly measures whether this is an entity mention at all, not which class. And running 24,576 candidates against a 5% false-pass threshold buys roughly 1,200 free passes. Two weak correlational tests conjoined still admit chance.

The corrected bench subtracts the shared mention direction, replaces the significance threshold with a selection-matched null (beat the best of 24,576 random directions), and requires a margin over every competing class. Random directions then ground nothing at all. The pre-declared retry at half the sparsity budget changed nothing: zero grounded, retry delta zero. Diagnosis: roughly eight scoreable activations per feature. Starvation, not refutation.

The measurable claim of this work is not that our extraction works. It is that when it does not work, our instruments say so before we do.

A third path, and a third null

Weights are the cheapest access of all, requiring no forward pass. Feed-forward layers behave as key-value memories, so a value vector can be decoded through the output embedding to ask what a neuron writes. We scanned every such neuron for a target vocabulary of programming idioms, on CPU, in about seven minutes.

Against the pre-declared bar it passed on all 13 targets - and it is still a null. The bar compared against 500 random draws, which scored zero, making every ratio infinite. But the search ranged over roughly 396,000 candidate directions. Running the null as wide as the search collapses the effect to 0.7×-1.8×: real neurons hit idiom tokens at essentially the random rate, and random directions achieve rank 1 on 11 of 13 targets, matching the real ones.

One structural finding survives regardless. The idiom we searched for tokenises as a three-token sequence, so it cannot be stored in one neuron even in principle. The knowledge is a sequence of associations, not an address.

The pattern: calibration is the whole game

Three independent attempts to read knowledge out of a model each produced an apparently strong result, and each was retracted by a null matched to the actual selection pressure.

A bar that a random baseline can also pass is not evidence, however many candidates clear it. The null must search as wide as the claim.

This is the most transferable result in the work. Each of the three would have been publishable as a positive finding under a conventionally-specified control. What distinguishes the surviving positives is not that they were larger - it is that they were put through nulls built to match how hard we had searched. One of those nulls, applied late, correctly demoted our own best number.

What this is for

1. Verified knowledge-base growth and repair. Extraction is batched matrix arithmetic, not autoregressive generation - a 100,000-entity sweep costs about four minutes per template. The output is bidirectional: facts the model holds that the graph lacks, and facts the graph holds that the model contradicts. In an earlier hybrid measurement the graph overrode the model 11 times and was right 10.
2. Abstention before generation. The fabrication test inverts into a guardrail. Confident geometric proposal with zero independent corroboration is the signature of a question that should be refused rather than answered, with a measured zero-fabrication property behind it rather than a heuristic.
3. Knowledge inventory as a model audit. Per model, a map of which relations have a strong channel and which do not - the weak ones being where that model is most likely to confabulate. Model cards report benchmarks; this reports a verified factual inventory.
4. Program synthesis with zero weights, which shows the same complementarity. A sibling system synthesises Python from a task description using no neural weights at all. Under MBPP+ deflation it reaches 136 of 224 covered tasks (60.7%). On the identical pool, identical spec, identical grader, Qwen2.5-Coder-7B scores 161/224 - and that flatters it, since MBPP is in its pretraining. But the union is 186/224 (83.0%), and 25 tasks are solved only by the zero-weight system. Uncorrelated errors plus a verifier that can tell which channel to keep.

Honest limits

• This does not become a language model. It is a knowledge and verification engine: closed-world, auditable, silent where it cannot verify.
• Identity stays walled, reproduced across two embedding spaces and three mechanisms.
• Coverage has a principled floor. An entity with fewer than two independent facts cannot be verified at any depth. Measured: zero recovery on 69 such cases, 59 of which had at most one fact in total. Depth does not manufacture independence.
• Relation linearity is not our discovery. It is established in prior work. The contribution is the gated architecture around it.
• The feature and single-neuron rungs are null at the scales tested.
• Base rates can masquerade as verification. A gate cannot refuse a high-frequency object; constant-guess baselines must be run per relation. Our own strongest figure was demoted by exactly this.
• Scale. One 1.5B model, six relations, one pilot-scale autoencoder. The throughput figures make broader sweeps cheap; they have not yet been run.

Method notes

The protocol is part of the contribution, since the failure mode of this area is not bad ideas but unfalsifiable ones. Bars were written before results, and the pooled 86.1% gate figure is reported as a partial failure rather than re-framed, with the identified fix withheld from the run. Adversarial controls were mandatory, and killed a 41-feature result that would otherwise have been reported as a success. Splits are blind by entity. Corroboration is masked, so the gate cannot verify by lookup. New code received an independent adversarial audit, which found a commit path that could emit code contradicting a user's own stated example; it was fixed before the result was reported. Negative results are ledgered so that dead levers stay dead.

Selected references

Hernandez et al., Linearity of Relation Decoding in Transformer Language Models, ICLR 2024 (arXiv:2308.09124). · Merullo, Eickhoff, Pavlick, Language Models Implement Simple Word2Vec-style Vector Arithmetic, NAACL 2024 (arXiv:2305.16130). · Geva et al., Transformer Feed-Forward Layers Are Key-Value Memories, EMNLP 2021 (arXiv:2012.14913). · Gao et al., Scaling and Evaluating Sparse Autoencoders, 2024 (arXiv:2406.04093). · Meng et al., Locating and Editing Factual Associations in GPT, NeurIPS 2022 (arXiv:2202.05262). · Cunningham et al., Sparse Autoencoders Find Highly Interpretable Features, ICLR 2024 (arXiv:2309.08600). · Park, Choe, Veitch, The Linear Representation Hypothesis, ICML 2024 (arXiv:2311.03658). · Petroni et al., Language Models as Knowledge Bases?, EMNLP 2019 (arXiv:1909.01066).