NYU与Amazon新论文发现，蒸馏少数持续产生有效训练信号的技能，效果可匹配甚至超越蒸馏大至11倍的技能库。按主题匹配挑选的技能中，不到25%能在3个Qwen模型上产生有效信号。SGUID只保留训练早期和后期都有帮助的技能，6个技能即让3/4模型在数学竞赛测试上追平或超过30至71个技能的完整技能库；第二轮加入3个新技能后，Qwen3-8B从64.3%提升至66.3%。
Good paper for selecting your skill files.
Before distilling a skill bank, log which skills give a steady training signal and drop the rest.
New NYU and Amazon paper finds that distilling a few skills that keep producing a useful training signal matches or beats distilling a skill bank up to 11× larger.
Skills are short written tips, like a rule for counting cases, that a model absorbs by learning from a copy of itself that reads them. Picked by topic match, under 25% of them gave any useful signal across 3 Qwen models.
SGUID keeps only skills that help early in training and still help late. With 6 such skills, 3 of 4 models matched or beat the full bank of 30 to 71 skills on math contest tests. A 2nd round with 3 new skills lifted Qwen3-8B from 64.3% to 66.3%.
来源：Rohan Paul · x.com