ATTN-FFN disaggregation is promising but not yet proven, as Nvidia has not published any real benchmarks of this system.

The key challenge will be ensuring that the connection between the GPU and LPU maintains low enough latency to avoid bottlenecking at high decode speeds. This is because you need a lot of communication to transfer data back and forth for each full-attention layer in the model.