Systematic Diagnosis and Benchmarking of Memory Systems in Autonomous AI Research Agents: A Low-Resource Framework

Autonomous AI research agents can execute long-running scientific workflows, revise code, inspect results, and reuse prior observations. Their reliability depends on memory architectures that retrieve relevant evidence without overwhelming the agent or discarding useful context. This hands-on tutorial presents a low-resource framework for diagnosing, benchmarking, and improving memory systems in autonomous research agents. Participants learn taxonomies of agent memory, trace-based diagnostics for retrieval and utilisation failures, LLM-as-judge evaluation protocols, and standardised benchmarking across heterogeneous memory providers. Exercises use real autonomous-research traces and lightweight open-source tooling that runs on a standard laptop with modest API credits.