AI 导读
SemiAnalysis 实验显示,将 Engram 卸载到 DRAM 后,在 H200、B200、B300 乃至 GB300 NVL72 上性能提升最高达 50%。
正文
Experiments have shown that offloading Engram to DRAM results in up to 50% better performance on H200, B200, B300, and even GB300 NVL72. Our technical team has collaborated with 10x engineers from @EmadBarsoumPi & @AnushElangovan to upstream support for @vllm_project ROCm Engram DRAM offloading too, and we are already seeing up to 50% better perf too in PR 57491. As such, in PRs 985, 1002, 1005, 1006 @NVIDIA, @AIatAMD and @SemiAnalysis_ are updating the default recommended recipes to use Engram offloading.
来源:@SemiAnalysis_ · x.com