TL;DR
Memorilla learns to present query-dependent memory tokens to a frozen language model. Its authors report leading five of eight evaluated benchmarks with 16 tokens in supervised experiments, but the approach loses some exact details and requires compatible open model weights. Its agent experiment adds eight memory tokens to a full history rather than replacing that history.
Introduction
An agent’s useful history can grow faster than its context budget. Adding every conversation, document and previous action to every prompt increases cost and can bury relevant information. Retrieval helps, but a useful fact may be distributed across documents or expressed in terms that do not resemble the current query.
The Memorilla research project explores a different point in the design space: represent the collection outside the model and generate a small, learned set of memory tokens for each question. The paper was first submitted in October and revised on November 3. Its results are research measurements, not a guarantee that the same approach will improve an arbitrary production agent. (Source : arXiv — Memorilla: Latent Semantic Memory for LLMs)
How the memory is built
The authors describe a frozen encoder that turns incoming documents into reusable embeddings. A trained memory module reads the question and stored collection, then supplies a fixed number of query-dependent tokens in the language model’s input embedding space. The language model itself stays frozen while the memory module is trained. (Source : Memorilla project — Method)
This is not the same as simply summarizing a user’s history into 16 ordinary words. The learned tokens are model inputs produced by the memory module; their content is not necessarily human-readable. The number of slots is configurable. Sixteen refers to the supervised experiments described by the authors, not a universal limit on all uses.
That distinction changes deployment. A transparent text-based retrieval system can show the passages supplied to the model. A latent representation may compress information more aggressively, but it makes tracing a particular answer back to its source less straightforward. For applications that require citations or auditability, an additional retrieval and verification path may be necessary.
The benchmark gains are real measurements, not universal wins
On FactKG, the paper reports 84.33% accuracy for Memorilla versus 80.43% for full-context inference, with 108 times fewer document tokens. The abstract says the system leads evaluated baselines on five of eight benchmarks. (Source : arXiv — Memorilla abstract)
The project page reports 44.60% accuracy on PersonaMem-v2 versus 27.92% for full context and 30.90% for its strongest evaluated baseline, xRAG. It also discloses additional benchmark-specific fine-tuning for PersonaMem-v2 and PersonalizationV4. That qualification matters: scores reflect the specified training and evaluation setup, not a drop-in memory module trained once for every possible task. (Source : Memorilla project — Results)
The failures are equally informative. On PubMedQA, the project reports 59.00 for Memorilla versus 69.00 for its best baseline, LLMLingua-2. The authors note that reading a short source directly can be better. A system designed to compress long collections does not automatically outperform direct access when the evidence is already compact. (Source : Memorilla project — Results)
Latency depends on the experiment
For 512 documents, the project reports 0.74 seconds for Memorilla versus 19.9 seconds for full context. The disclosed setup uses Qwen3-8B, batch size one and a single H100 with 80 GB of memory; full context uses YaRN. The page also cautions that timings depend on the serving stack. (Source : Memorilla project — Measured serving cost)
This is a compelling comparison for that setup, not a universal service-level agreement. Document encoding, memory-module hosting, hardware utilization, batching, concurrent requests and integration work all affect real costs. A production team should measure end-to-end latency and dollar cost, including ingestion, rather than converting a single GPU result directly into a cloud invoice.
Compression claims also need a denominator. The project’s document-token comparisons exclude the query and shared instructions. They should not be interpreted as a reduction of the entire prompt, total inference cost or storage by the same factor.
The agent test adds memory to history
In TextWorld, the authors use a frozen Qwen3-4B agent and eight learned memory tokens. The project says both compared agents read the same full history, isolating what the extra memory adds. Average success on held-out games rises from 18.6% to 24.2%. (Source : Memorilla project — Memory for agents)
That is evidence that learned memory can help an agent use its history in this particular game suite. It is not evidence that the eight tokens replace the whole action history, nor that the system reliably learns across unrelated environments. The authors note that the experiment is limited to one suite and episodes of at most 50 turns.
Where a production pilot should start
The project says the memory module writes directly into input embeddings, requiring open model weights and a module trained for each model. Its experiments pair Qwen encoders and decoders on text, leaving transfer across model families and modalities untested. Exact names and other fine-grained details can be lost. (Source : Memorilla project — Current limits)
A sensible evaluation would compare full context, retrieval and learned memory on the same held-out tasks, with equal attention to ingestion cost, privacy, missing facts and provenance. A hybrid may be promising, but the project’s combined-answer upper bound is an oracle calculation, not a deployed hybrid result. Keep that distinction visible before claiming that a combined system achieves the reported upper bound.
FAQ
Are the 16 memory tokens a plain-text summary?
No. They are learned inputs in the model’s embedding space. The count is configurable and describes a supervised experimental setup.
Did Memorilla win every benchmark?
No. The authors report the best result on five of eight evaluated benchmarks and disclose weaker performance on others, including PubMedQA.
Can it be added to any hosted model API?
Not as described: writing directly into input embeddings requires compatible open weights and a model-specific memory module.
Did the agent experiment replace the full history?
No. The project says both compared agents received the full history; the experiment measured the effect of adding eight learned memory tokens.
Further Reading
— The Agent Report