Giving an agent a memory hierarchy is not what makes it reason better over long tasks. What helps is scoring each memory and letting unused ones decay out of the prompt.

Agents that keep everything in context lose accuracy to their own stale records. So build a rule that demotes memories the selector keeps skipping, rather than a bigger store.

For a long-horizon agent, a demotion rule buys more than a better retriever: lower a memory's standing each time the selector passes it over.

This paper separates hierarchy from scoring, and only the scored version improves accuracy and prompt cost in every model tested.

The Weighted Memory Tree scores each memory by whether its action succeeded, decays it whenever the memory is eligible but unselected, and is set against linear history and an unscored tree.

– arxiv. org/abs/2608.20631

Title: "Weighted Memory Tree: Remembering What Matters for Long-Horizon LLM Agents"