Kimi-K3.1-Pro by Moonshot AI: interesting facts

It's huge, and sparse
• It has 2.8T total parameters, but only 104B are active per token.
• Along with Kimi-K3, it's the largest real model release on the Hub. The next biggest are Qwen3.8-2.4T and DeepSeek-V4-Pro (1.6T).
• It has 896 experts and sends each token to 16 of them. (DeepSeek-V4-Pro has 2 times less and Mistral-Large-4 has 7 times less)

Clever attention for a 1M-token context
• Most of its 93 layers use linear attention (KDA). Every 4th layer uses DeepSeek-style sparse attention, which only looks at the top 2048 tokens.
• This is how it handles a 1M-token context (the same as the other frontier open models)
• It uses a custom activation called SiTU-GLU and has a 401M-parameter vision encoder (MoonViT-V2), so it takes both text and images.

Small on disk for its size
• It was trained to run in 4-bit (MXFP4), so the whole repo is only ~1.56 TB.
• Stored in regular BF16 it would be about 5.6 TB(!)

Benchmarks (from the model card)
• It goes back and forth with Claude Opus 5.5 and GPT-6 Astra. It's top of the card's table on BrowseComp (92.1), HLE with tools (70.0) and SWE Marathon (51.7).
• It's weaker on pure reasoning without tools and on cyber security.
• Moonshot ran most of their own coding evals using the Claude Code harness.

License (a modified MIT)
• It's free to use, modify and redistribute.
• API businesses making over $20M a year need a deal with Moonshot.
• Products with over 100M monthly users or over $20M monthly revenue must display "Kimi K3.1-Pro".

On the Hub
• It went public about 1.5 hours ago and already has 214 likes, about 9× Kimi-K3's first hour. It's #19 on the trending list.
• It isn't gated, but loading it in transformers requires "trust_remote_code"