面壁智能 OpenBMB 开源了 MiniCPM5-2B 背后的 RL 训练栈,包含服务化 RL 训练框架 Meshy 和信用分配方法 JustRL II。
Yesterday, we open-sourced MiniCPM5-2B. Today, we’re opening up the RL stack behind it.
Meshy is a service-based RL training framework that decouples inference, rollout, and training into independent services over a unified data plane.
• Supports on-policy, bounded off-policy, and fully asynchronous RL
• Recipe-driven SPMD placement, built on SGLang + TorchTitan
• Unified data plane for tensors and control signals
• Ready-made recipes for MiniCPM5-1B / 2.6B, including 128K context-parallel training
JustRL II introduces a critic-based approach for more precise token-level credit assignment across long reasoning chains, enabling more stable long-horizon RL for small language model.
Together, Meshy and JustRL II form the RL stack behind MiniCPM5-2B.
🔧 Meshy: https://t.co/jpIvQSz2bc
📝 Meshy blog: https://t.co/JYUR3Xsspo
📝 JustRL II: https://t.co/iFDbZNldRq
🤗 Hugging Face: https://t.co/FZOMTZhBjq
来源:@OpenBMB · x.com