BEAM 100K test: EverOS and Hindsight match full context with less text
AI summary
A Reddit user tested 10 open-source AI memory systems on BEAM 100K using three fixed conversations and 60 rubric-graded questions. EverOS scored 70.1% with 3,336 retrieved tokens and Hindsight scored 67.8% with 4,913, compared with 69.5% for the 115,541-token full-context baseline. The test used Claude Haiku 5.5 to answer and Claude Opus 5.5 to judge.
In-depth insight
The results suggest that EverOS and Hindsight can approach full-context recall while retrieving 24–35 times less text, offering a practical signal for developers choosing an AI memory system.
TL;DR
- EverOS (70.1%) and Hindsight (67.8%) are the best. They scored as well as giving the model the whole conversation (69.5%) while reading 24–35x less text, and were the only two to clearly beat plain keyword search.
- EverOS saves chats as dated notes, which kept it strong on instructions, timing and facts that change.
- Hindsight was the best at recalling exact details.
- With only 60 questions, ranks 3–8 aren't clearly separated.
Results
| Rank | System | Score (95% CI) | Retrieved tokens | Retrieval p50 | Ingest time (3 convs) | Ingest cost (US$) |
|---|---|---|---|---|---|---|
| 1 | EverOS | 70.1% (61.7–77.8) | 3,336 | 1.2 s | 14 min | $1.05 |
| – | Full context (baseline) | 69.5% (59.9–78.3) | 115,541 | – | – | – |
| 2 | Hindsight | 67.8% (58.9–75.8) | 4,913 | 2.2 s | 13 min | $1.50 |
| 3 | Cognee | 62.7% (53.1–71.4) | 23,263 | 2.9 s | 12 min | $1.02 |
| 4 | memsearch | 61.8% (52.7–70.8) | 1,822 | 0.2 s | 10 min | $0 (no LLM) |
| 5 | OpenViking | 61.5% (51.9–70.9) | 1,431 | 1.4 s | 77 min | $0.76 |
| 6 | MemPalace | 61.0% (51.1–70.6) | 1,624 | 1.4 s | 3 min | $0 (no LLM) |
| 7 | LangMem | 60.8% (51.1–70.1) | 3,209 | 1.1 s | 35 min | $0.34 |
| 8 | Mem0 | 55.6% (44.9–65.7) | 1,164 | 1.3 s | 132 min | $1.22 |
| – | BM25 (baseline) | 55.3% (44.6–65.4) | 7,204 | 2 ms | – | – |
| 9 | claude-mem | 51.5% (41.2–61.4) | 30,015 | 0.3 s | <1 min | $0 (LLM unused) |
| 10 | Graphiti | 43.7% (33.8–53.6) | 354 | 1.2 s | 29 min | $1.92 |
#BEAM 100K#EverOS#Hindsight#Cognee#memsearch#OpenViking#MemPalace#LangMem#Mem0#claude-mem#Graphiti#Claude Haiku 5.5