跳到正文
@rohanpaul_ai· @rohanpaul_ai · X·· 25 天前AI 评分54
AI 导读

LinkedIn 的一篇论文测试一个模型继承另一个模型创建的 Agent 记忆会发生什么,发现固定 schema 记忆在 2 个测试模型间几乎不变,而自由形式笔记在一个迁移方向上准确率下降 13.28 分,因为旧模型此前已丢掉有用信息。

正文

New Linkedin paper shows Agent memory is not automatically portable:

fixed-schema memory survived the model swap, free-form notes changed sharply, and mixed embeddings hurt retrieval, so treat upgrades as memory migrations.

The paper tests what happens when 1 model inherits memory created by another.

A new model may inherit the same memory store but remember differently, so agent upgrades should include memory compatibility tests rather than only model benchmarks.

Memory saved in a fixed structure barely changed across the 2 tested models because each model used the same fields and format.

Free-form notes were much less reliable. In 1 direction, accuracy dropped 13.28 points because the old model had already left out useful information.

RAG failed differently. Mixing old and new embeddings in the same index recovered only 4.96 of the 11.90-point gain from fully rebuilding the index.

Once compressed notes had lost a fact, rewriting them could not bring it back.

So test memory whenever you change models, rebuild embedding indexes fully, prefer structured memory where it fits, and keep protected raw history when policy allows.

来源:@rohanpaul_ai · x.com