Flash Attention 2: Faster Attention with Better Parallelism and Work Partitioning

[@HeyFaheem]

Thanks for your Excellent Efforts Sir. Never seen a guy who explains a recent nlp mechanism eloquently... Once again thank you sir. Because of you i got deeper intuition about FA and understood completely.

[@mraarone]

Does the end normalization in FA2 only stay stable with double precision or fewer tokens?

[@chaitanyap1000]

Thankyou for the detailed video . can this be combined with paged attention ?