AI 导读
其他收益来自：移除每步序列长度同步、捕获元数据胶水图、在 MoE、router 和 KDA 之间添加 PDL 链、将 SwiGLU 融合进 up GEMM、重新调优 KDA 链验证，以及以 bf16 运行 router 和 lm_head。https://t.co/LfNfOvwNSm
正文
Other gains came from removing per step sequence length synchronization, capturing metadata glue graphs, adding PDL chains across MoE, router, and KDA, fusing SwiGLU into the up GEMM, retuning KDA chain verify, and running the router and lm_head in bf16. https://t.co/LfNfOvwNSm
来源：@AntLingAGI · x.com