FlashAttention-3：FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision

arXiv · Colfax Research, Meta, NVIDIA, Princeton

通过 Warp 协作异步流水线掩盖 Softmax 延迟，并结合块量化与非相干变换释放 Hopper FP8 算力，实现最高 2 倍加速与 1.2 PFLOPs 吞吐。