Agents can accumulate hundreds of skills without becoming proportionally better, which makes skill consolidation and reuse a bigger problem than simply generating more skills.

If you want an agent to improve over repeated work, keeping its context and feedback is already useful; autonomous skill creation is still unreliable except where reusable procedures really matter.

Agents can get better from experience, but this paper finds that explicit skill libraries are not yet consistently better than carrying forward context and feedback.

ContinualSkillBench gives agents 100 connected tasks in each of 5 domains, lets them keep feedback and update reusable skills, and compares that with solving every task from scratch.

Sequential execution improved normalized reward in 14 of 15 model-domain settings, a 16.9% relative gain overall.

But the ablation changes the takeaway: on GPT-5.3-Codex across Law, Finance, and Healthcare, pure in-context learning averaged 0.605 normalized reward versus 0.602 with explicit skill maintenance.

So much of the gain appears to come from carrying forward context and feedback, not from agents reliably abstracting reusable skills.

– arxiv. org/abs/2608.03874

Title: "ContinualSkillBench: Can LLM Agents Truly Evolve Their Capabilities?"