跳到正文
@SemiAnalysis_· @SemiAnalysis_ · X·· 24 天前AI 评分37
AI 导读

据称 GPT-6 Astra 采用循环 Transformer,通过多次经过同一批层来增加计算深度,而非扩大参数量。有观点认为,实验室在路线图中不再激进扩展参数规模,这本身就是一个信号。

正文

GPT-6 Astra reportedly gets deeper without getting bigger. The labs already know what that means for scaling.

"It's basically confirmed that GPT-6 Astra uses loop transformers, which means that instead of adding parameter count, it goes through the layers more than once. You add compute depth, but you don't increase the size of the model."

"The labs are probably the people that are best positioned to say which way models are scaling. I think that's a tell that they're not seeing parameter sizes scaling as aggressively in their roadmaps, in what they find in their research."

来源:@SemiAnalysis_ · x.com