Agent Memory Challenge 2026 Cycle 2 opens September 20 across Textual, Coding, and Multimodal Memory.
Long-term Agent Memory is not about keeping more history.
It is about retrieving the right evidence from long-running experience, recognizing when a fact or decision has changed, avoiding stale context, and using that evidence reliably in the next task.
That is the question behind the Agent Memory Challenge, the public evaluation program of the Agent Memory Leaderboard (AML).
Cycle 2 opens on September 20, 2026, at 00:00 UTC+8.
Why memory needs to be evaluated differently
As AI agents move beyond one-off chats toward long-running work, memory becomes a core capability.
An agent may need to retain context across conversations, software repositories, documents, images, and evolving tasks. It may need to remember a user preference expressed weeks ago, recover a debugging path from an earlier development session, or distinguish a superseded project decision from the current one.
But “remembering” is not enough.
Retrieving yesterday’s decision after it has been corrected can be worse than retrieving nothing. Returning a familiar but outdated fact can quietly derail a later task. A memory system needs to preserve evidence while also helping an agent understand what remains current.
This makes public comparisons difficult. Different memory systems are often tested with different datasets, answer models, prompts, scoring rules, and evaluation procedures. A score may reflect not only the memory layer, but also everything surrounding it.
AML creates a clearer boundary.
Participants provide the memory layer through Add and Search APIs. AML standardizes the downstream Answer, Eval, result review, and leaderboard-publication process.
The goal is not to declare one universal “best memory system.” It is to create a more open, reproducible way to compare different approaches under a shared evaluation flow.
What Cycle 1 established
AML published its first leaderboard cycle on August 12, 2026.
The first cycle received 136 team applications, and 67 representative memory systems completed official evaluation. AML’s official website has since received more than 300,000 cumulative visits, while its Hugging Face Space reached #1 on the Spaces trending list during the cycle.
Those numbers reflect participation and community interest—not a claim that any individual system is universally superior outside its submitted version, track, suite, and evaluation conditions.
What they do show is that Agent Memory has become a shared technical question: how should systems store experience, retrieve useful evidence, handle change, and prove that the evidence returned to an agent is actually reliable?
Cycle 2 expands that question across three distinct settings.
Three tracks, one evaluation boundary
Textual Memory
The Textual Memory track evaluates long conversations and cross-session history.
It includes explicit fact recall, multi-hop relations, temporal events, personalization, rule following, and memory governance. The track asks whether a system can retrieve relevant evidence without confusing historical statements with current state.
Cycle 2 also introduces Streaming Memory. Here, Add and Search calls may occur as events unfold. The challenge is not simply to search a completed archive, but to use information that was available at the right point in time.
Coding Memory
The Coding Memory track asks a more practical question: can historical engineering experience help an agent solve a later software task?
Systems may need to retrieve and filter repository history such as implementation context, debugging trails, failure cases, module relationships, testing evidence, and prior development decisions.
The formal Coding evaluation includes 150 software-engineering tasks, each tested under both relevant-history and noisy-history conditions—300 task-condition units in total.
The distinction matters. A useful coding memory system should not only find old information. It should help an agent identify which past experience is relevant, which context is distracting, and which evidence can support the next implementation or debugging decision.
Multimodal Memory
The Multimodal Memory track evaluates how systems write, retrieve, and use text-and-image memory.
Participants may process original images, captions, or both. They must correctly handle ordered multimodal content arrays and clearly disclose which media types their systems actually use.
As agents work increasingly across screenshots, product documents, visual references, and written instructions, memory can no longer be treated as text alone. The question is whether a system can preserve and retrieve the connections between visual and textual evidence.
How AML evaluates memory systems
The evaluation boundary is deliberately simple:
- Add receives historical content that a system should remember.
- Search retrieves relevant memory evidence for a later query and user scope.
- AML then runs the downstream Answer and Eval process under a standardized protocol.
Search is not expected to generate the final answer or return a disguised answer key. Its job is to return useful, traceable memory evidence.
The workflow is:
- Choose a track and division.
- Deploy stable, publicly reachable Add and Search APIs.
- Submit a fixed system version.
- Pass a per-track Smoke test.
- Start a formal Full evaluation.
- Keep the submitted version, endpoints, authentication, and operating conditions stable.
- AML reviews result validity, version consistency, and compliance status before leaderboard publication.
This separation makes a meaningful comparison possible. Different systems can make different architectural choices—raw-history retrieval, summaries, structured facts, graphs, hybrid retrieval, temporal reasoning, or learned memory policies—while being evaluated through the same downstream Answer and Eval flow.
Technical principles that protect comparability
A benchmark is only useful if the evaluation contract is clear.
Cycle 2 therefore requires several core principles:
- Search returns evidence, not a final answer.
- user_id is a strict isolation boundary. Evaluation memory must not be shared across users, tasks, samples, teams, or runs.
- Synchronous Add must persist content before returning success. Newly written memory must be immediately searchable.
- request_id should be idempotent. Network retries should not create duplicate memory.
- Full evaluations use a fixed version and contract.
- Scores and rankings apply only to the corresponding submitted version, track, suite, and rules.
These constraints are not intended to prescribe a single memory architecture. They make it possible to inspect what a system actually did, reproduce its operating conditions, and interpret its result within a defined scope.
Who can participate
Cycle 2 is open globally to universities and research institutions, open-source maintainers, independent research teams, commercial product teams, individual developers, and cross-organization teams.
Each track has two separate divisions:
- Open-source Methods
- Commercial Products
Participants may enter multiple tracks, but each submission selects one division. Rankings are published separately by track and division.
Participation is free. Teams operate their own APIs, databases, bandwidth, and compute resources; AML runs the standardized evaluation orchestration, Answer, Eval, result review, and leaderboard publication process.
Eligible Open-source Methods teams will compete for a total RMB 150,000 prize pool across the three tracks. Commercial Products are ranked in a separate division and are not eligible for the prize pool.
Key dates
- September 20, 2026, 00:00 UTC+8: Cycle 2 opens
- September 20–October 31: Registration, API integration, Smoke tests, Full evaluations, and result review
- October 31, 2026, 23:59 UTC+8: Materials submission deadline
- November 4, 2026, 23:59 UTC+8: Evaluation closes
- Mid-November 2026: Official results planned for release
Teams should submit stable APIs early. Formal evaluation involves resource scheduling, execution, and necessary follow-up review; stable endpoints and operating conditions are essential throughout the process.
From storing history to using experience
Agent Memory is not a single capability.
It includes how a system writes information, organizes it, retrieves it, handles conflict and change, respects isolation boundaries, and supplies evidence that a downstream agent can actually use.
Cycle 2 is an invitation to test these approaches under a common evaluation boundary—across conversations, codebases, and multimodal context.
The benchmark will not settle every question about long-term memory. But it can make the questions sharper, the results more comparable, and technical progress easier to verify.
Agent Memory Challenge 2026 Cycle 2 opens September 20.
Official website · Evaluation entry · Participation guide · API guide · GitHub
Final eligibility, result review, awards, and scheduling are subject to the latest official rules and organizer announcements.