AI 导读
感谢测试并分享这些数据！🙌 “在本地跑一群这样的模型”——这正是我们当初的愿景。2B 参数，Q4 量化仅 1.56GB，在 4090 上 200+ tok/s，还能跟上 4B 模型的水平。本地 AI 智能体就该是这样。 迫不及待想看到你们用它们做出什么 🔥 https://t.co/ajbTlaN6hb
正文
Thanks for testing it out and sharing these numbers! 🙌
"Run a swarm of these locally" — that's exactly the vision we had. 2B params, 1.56GB Q4, 200+ tok/s on a 4090, and still keeping up with 4B models. That's what local AI agents should look like.
Can't wait to see what you build with them 🔥 https://t.co/ajbTlaN6hb
来源：@OpenBMB · x.com