We are releasing two papers from our retrieval work. The first measures how many facts a single text embedding carries. The second measures how much of a reader’s quantization loss retrieval absorbs. Both PDFs are free to download at the end of this article, and both went through an external review pass that made our claims smaller and better supported.
Paper one: a chunk embedding carries about one fact
Every RAG system you have built compresses each chunk of text into one vector, and you have tuned the chunk size by trial and error. We asked a narrower question: when a chunk holds K independent facts, how many of them can a retrieval system get back out of the vector?
We built synthetic chunks with K facts each, embedded them with nine encoders (Arctic, E5, BGE-M3, Nomic, Qwen3-Embedding, MiniLM, MPNet, one of our production fine-tunes, and a plain BERT with no retrieval training), and trained probes to read the facts back. The answer did not depend on the encoder. A query-conditioned probe recovers about 1/K of the facts. Probes that know which slot to read, and are therefore free of that 1/K bound, decode between 1.0 and 1.6 facts per chunk at every K we tested.
| Facts per chunk (K) | Query-conditioned probe, 9 encoders | One fact plus guessing | Facts decodable with slot probes |
|---|---|---|---|
| 2 | 0.51 to 0.55 | 0.513 | 1.07 to 1.22 |
| 4 | 0.28 to 0.31 | 0.269 | 0.92 to 1.39 |
| 8 | 0.15 to 0.18 | 0.147 | 0.99 to 1.63 |
The BERT control settles where the limit comes from. A mean-pooled BERT with no contrastive training follows the same curve as the retrieval models, so the bottleneck sits in pooling token states into one vector, not in retrieval training. Training decides something else: which fact survives. Every contrastively trained encoder keeps the first fact in the chunk (Arctic finds the first fact of a two-fact chunk 93% of the time and the second 29%), and plain BERT shows no such preference.
On real text the effect shows up where chunks mix topics. In a wide-body aircraft operations manual, questions about facts in the last third of a chunk find that chunk half as often as questions about the first third (0.36 against 0.18 for E5), while a BM25 baseline on the same questions finds it equally often either way. On SciFact abstracts, where every sentence concerns the same study, the penalty disappears. The chunk-count penalty we saw on the manual turned out to be mostly chunk length, and we say so in the paper.
You can act on this today. Embed one claim per vector, put the entity first, and return the parent section to the model for context. The paper ships a probe that measures any encoder’s capacity curve in about four minutes on a laptop, with no labels and no reader.
Paper two: what grounding does to a quantized reader
You normally pick a model’s bit-width by measuring a benchmark the model answers from memory. A reader inside a RAG pipeline answers from the passages you hand it, and quantization damages memory first. We quantized Qwen3-8B to 8, 6, 4, 3 and 2 bits with plain round-to-nearest quantization and measured it two ways: closed-book on PopQA, and grounded on 465 judged questions over two real corpora, with the retrieved passages held fixed so only the reader changed.
| Bits | Closed-book PopQA | Grounded, two real corpora |
|---|---|---|
| 6 | 0.198 (+0.5%) | 0.617 (+0.7%, indistinguishable from 8-bit) |
| 4 | 0.173 (−12.2%) | 0.583 (−4.9%, depends on the corpus) |
| 3 | 0.152 (−22.8%) | 0.538 (−12.3%) |
| 2 | 0.007 | 0.000 |
Read that table in both units. In relative terms the grounded reader loses about half as much as the closed-book reader at 4 and 3 bits. In absolute points it loses as much or more (0.030 and 0.075 against 0.024 and 0.045), because it starts from 0.61 instead of 0.20. The grounded curve has the shape of our reranker’s own quantization curve: 6-bit costs nothing we can measure, 4-bit depends on the corpus, 3-bit breaks, 2-bit destroys the model. On a matched single-fact task, where the gold sentence sits in the context, the closed-book loss is not paid at all: the dividend is 0.017 at 4-bit and 0.050 at 3-bit, with confidence intervals clear of zero.
Three results from our first draft did not survive. We had proposed retrieval recall as a cheap predictor of how safely you can quantize the reader. Conditioning on whether the gold passage was retrieved shows the same bit-loss on hits and misses, so we withdrew it, along with a twenty-question calibration rule that could never have resolved the effect. A small-to-large reader cascade looked like a 2.2-point gain; shuffling the gate signal produces the same gain as often as not, so we report it as noise. The reader-size floor held: a 4B reader ties a 35B on grounded questions, and 1.7B and 0.6B readers stay below it at 8-bit as well as 4-bit.
The practical outcome is a stack where every component sits at its own measured floor: an 8-bit embedder, a 4-bit reranker and a 6-bit 8B reader in about 8 GB of weights, or a 4-bit 4B reader for a total near 3.5 GB resident. Spend the floor on each stage and stop; more memory bought no grounded accuracy in any run we did.
Why the claims shrank
Both papers went through an external review pass before release, and both came out with smaller claims. The capacity paper’s original probe had a structural bias toward 1/K, so we added two probe families that do not share it and reran every encoder; the law held, in corrected form. The quantization report had conflated a relative claim with an absolute one and had built a prediction rule on two corpora. We kept what survived, wrote down what did not, and put the point-by-point responses in each paper’s download bundle. If you read one section of either paper, read the limitations.
Download the papers
Both PDFs are free. The first is a preprint prepared for arXiv in LNCS format; the second is a baa.ai technical report.
One Vector, One Fact: A Capacity Law for Single-Vector Text Embeddings
15 pages, 7 tables. Nine encoders, a four-minute audit, two real corpora.
Download the PDFI. Kennedy and T. Kennedy, October 2026. Preprint, licensed under CC BY-NC-ND 4.0.
Retrieval as a Quantization Multiplier: How Grounding Changes a Reader’s Bit-Robustness
9 pages, 7 tables. Qwen3-8B at five bit-widths, closed-book against grounded.
Download the PDFI. Kennedy and T. Kennedy, October 2026. Technical report.
Where this runs in our products
Paddock indexes small child chunks and returns the parent section to the model, the configuration that won our retrieval bake-off and the one the first paper explains. Watchman grades every compressed build we ship, which is how the second paper’s reader floors were measured. If you run your own pipeline, the capacity audit in the first paper’s supplement runs against any sentence-transformers encoder. Book a technical call if you want either result applied to your corpus.
Corpora: a wide-body aircraft operations manual and a bank’s public customer-support website, plus the public SciFact abstracts. These are corpora we measured on, not customers. Quantization in the second paper is calibration-free round-to-nearest at group size 64 in MLX; calibrated and vector quantizers may move the bit boundaries, and the report says so.
