Recallr scores 86.7% on LongMemEval, the standard test of long-term memory.
Each question hides one detail in about 48 earlier chats, close to 490 messages, then asks for it back. That puts Recallr in the range of the memory systems AI developers build on.
- The only consumer app on this list. The others are memory systems for developers to build into their own products.
- Tested with a fast, low-cost everyday model. Some of the results here used much larger ones.
86.7%104 of 120 questions
In the range of the memory systems AI developers build on.
Published LongMemEval scores, each at its team's best result, with the kind of AI model that answered and a link to the source.
- 1Mastra Observational MemoryMemory for AI agents (open source)94.9%a small general modelMastra research, Feb 2026 · 84.23% with an older general model
- 2HindsightMemory for AI agents (research)91.4%a larger backbone modelLatimer et al., arXiv, Dec 2025 · 83.6% with an open-source 20B model
- 3RecallrThe consumer app, as it ships86.7%a fast everyday model
- 4EmergenceMemMemory for AI agents86.0%an internal modelEmergence AI blog, June 2025 · Not publicly reproducible; its public method scored 82.4% with an older general model
- 5SupermemoryMemory API for AI apps85.2%a large general modelSupermemory research · 84.6% with another large general model
- 6ZepMemory for AI agents71.2%an older general model
- 7An AI model reading the whole historyNo memory system: the full ~115,000-token history in every prompt60.6%an older general model
Asked in a new chat, weeks or months later.
Four of Recallr's right answers, word for word. The people and their earlier chats are the test's own.
First mentioned about 8 months earlier
How many bikes do I currently own?
Four. In your October 2 message you said you'd just bought a new hybrid bike, so you now have a road bike, mountain bike, commuter bike and that hybrid - up from the three you told me about back on February 15.
First mentioned about 4 months earlier
How many times have I tried making a Negroni at home since my friend Emma showed me how to make it?
Ten times - you told me that on Thursday, October 15, 2026. (Back in June you'd said it was five, so you've doubled it since.)
First mentioned 3 weeks earlier
What is my current highest score in Ticket to Ride?
132 points - you hit that on Oct 4, 2026, when you mentioned it during the Scythe game-night chat. That beat your earlier 124 from Oct 1.
First mentioned 4 weeks earlier
How many days passed between the day I started watering my herb garden and the day I harvested my first batch of fresh herbs?
24 days - you started watering daily on September 23, 2026, and harvested your first batch on October 17, 2026.
The test, and how we ran it.
- The test
- LongMemEval (Wu et al., ICLR 2025) checks whether a chat assistant remembers what you told it across many conversations: facts you mentioned, things that changed later, dates, your preferences, and what it told you. Its "S" set gives every question about 48 earlier chats to search. The paper.
- Recallr
- The current version of Recallr, exactly as it ships, tested October 2026. Each question's earlier chats were loaded into a fresh account as past chats; the question was then asked in a new chat.
- The questions
- 120 of the 500, an even 20 from each of its six kinds, from the cleaned version of the set.
- The AI model
- A fast, low-cost everyday model answered every question; it's one of the AI models Recallr uses. Each answer was graded blind, with the benchmark's own answer-check prompts and grader model.
- Other systems
- Their own published results on the same test, 6 sources, checked October 20, 2026. Read the chart as a range, not a race: every system answered with a different AI model, and they ran all 500 questions.
Good to know.
Still curious? Ask Recallr in the corner, or write to support@recallr-ai.org.
01What is LongMemEval?
The standard benchmark for long-term memory in chat assistants, published at ICLR 2025. Each of its 500 questions comes with dozens of earlier conversations and asks about one detail in them, so an assistant has to find and use what you said, not guess.
02Is Recallr's memory the same in the app?
Yes. The test ran on the current version of Recallr exactly as it ships: the same memory, the same search across your past chats, the same answer checks.
03Why 120 questions and not all 500?
Each question needs its own account with dozens of earlier chats, so we ran an even 20 from each of the test's six kinds. When a new version of Recallr ships, we run it again and this page updates.
Tell it once.
Ask it months later.
Save what matters today. Ask for it the day you need it.
Free to start. No card needed.