AI 导读
一个守护进程还能在同一块 GPU 上支持多个引擎。 它们映射同一份物理权重内存，因此磁盘加载和量化只发生一次。 这支持多实例服务，并让低优先级工作负载能在不到一秒内被驱逐和恢复。
正文
One daemon can also support multiple engines on the same GPU.
They map the same physical weight memory, so disk loading and quantization happen only once.
This supports multi-instance serving and lets low-priority workloads be evicted and restored in under a second. https://t.co/xaMvoVFXM3
来源：@AntLingAGI · x.com