跳到正文
Rohan Paul· @rohanpaul_ai · X·· 2 小时前AI 评分54
AI 导读

北京大学与三星的论文提出 RealtimeBench,用 6 款游戏共 180 个任务评测推理时间不暂停的智能体,GPT-6.1-sol 在暂停模式下 5/6 游戏胜过更快的 Jev,但实时得分仅 1.51 对 13.51(满分 100)。其方法 TORSI 让慢模型离线分析快模型失败并写成指令,两轮后 Jev 平均分从 13.51 升至 24.19,6 类游戏全部提升。

正文

New Peking University and Samsung paper shows that a slow reasoning model can coach a fast model offline, raising its live game score from 13.51 to 24.19.

shows that the stronger reasoning model in paused games can fall far behind a faster model once the game keeps moving.

Many game evaluations pause the world while the model thinks, so slow answers cost nothing. In real settings, a correct action can arrive too late.

They built RealtimeBench, 180 tasks across 6 games like Mario, Snake, and Tetris that keep running during inference. GPT-6.1-sol beat the faster Jev in 5 of 6 games when paused, but scored 1.51 versus 13.51 out of 100 live.

Their fix, TORSI, has the slow model study the fast model's failures between games and write instructions the fast model follows during play. After 2 rounds, Jev's average score rose from 13.51 to 24.19, improving all 6 game types.

For real-time agents, keep a fast model in the loop and use the slow model offline to write and test its instructions.

来源:Rohan Paul · x.com