What problem it solves

A frozen LLM can only use information that fits in its context window, making it difficult to answer questions from large document collections or long interaction histories. Memorilla addresses this by reading a collection and supplying the LLM with a fixed number of query-dependent memory tokens.

Key features

• Query-dependent memory compression — The system reads a complete document collection and converts it into a fixed-size set of memory tokens tailored to each question. The frozen LLM receives the memory rather than the full collection.

  Source: README

• Training and evaluation recipes — Repository scripts support multi-stage training, benchmark evaluation, and comparisons with closed-book, retrieval, and full-context baselines. Checkpoints are available for several personalization and general-purpose configurations.

  Source: README, configs, scripts

• Personalization benchmark — PersonalizationV4 is a synthetic benchmark for questions that require combining facts about a fictional user across earlier conversations. Its task, data format, scoring approach, and generation pipeline are documented in the repository.

  Source: README, personalizationv4

• Reinforcement-learning support — The reinforcement-learning pipeline trains a memory module with GRPO on TextWorld text adventures and can use reward alone for agent learning. Installation, data preparation, training, and evaluation are documented separately.

  Source: README, rl

• Support for custom datasets — Custom collections can be trained and evaluated without code changes by following a defined parquet row format. Documentation covers file layout, document chunking, embedding generation, and scoring options.

  Source: README, docs

Quick start

Copied verbatim from the project's README.

$ git clone https://github.com/snap-stanford/memorilla.git && cd memorilla
$ uv venv --python 3.11 && source .venv/bin/activate
$ uv pip install -e .
$ uv pip install --no-build-isolation flash-attn==2.7.4.post1

from memorilla.benchmarks import PERSONA_PROMPT
from memorilla.data import ensure_data, load_split
from memorilla.vllm import MemoryVLLM

row = load_split("pmv2", "test", ensure_data("pmv2", ["test"]))[73]  # one user's question and past conversations

model = MemoryVLLM.from_pretrained("memorilla/Memorilla-Qwen3-8B", "pmv2")  # Qwen3-8B, the memory module and the encoder
answer = model.answer(row["question"], row["documents"], system_prompt=PERSONA_PROMPT)
print(f"Question: {row['question']}\nMemorilla: {answer}")

Who it may suit

Inference: this may suit researchers and engineers working with long document collections, conversational personalization, or text-based agents who need a compact memory interface for a frozen LLM.

Topics to discuss

• Which document collections and question types should be prioritized when adapting the memory system to custom datasets?

• How should retrieval baselines, full-context inputs, and query-dependent memory tokens be compared in future evaluations?

• What additional applications could benefit from using memory tokens in personalization and reinforcement-learning agents?

Written by ReadmeX AI from the project's README, source and public GitHub data; it may contain mistakes.

Read the original README https://github.com/snap-stanford/memorilla#readme

ReadmeX opened this community for a trending project; it is not run by the project. Maintainers can claim it from the community page.