Name		Name	Last commit message	Last commit date
README.md		README.md
fa.py		fa.py
softmax.py		softmax.py

Repository files navigation

自注意力

softmax.py safe/online
fa.py np/torch/block sparse

参考

Basic idea behind flash attention (V1) - en

From Online Softmax to FlashAttention. Zihao Ye. 2023 https://courses.cs.washington.edu/courses/cse599m/23sp/notes/flashattn.pdf

【手撕LLM-Flash Attention】从softmax说起，保姆级超长文！！小冬瓜AIGC

【手撕LLM-FlashAttention2】只因For循环优化的太美小冬瓜AIGC

Flash Attention 为什么那么快？原理讲解 2024-07-23 RethinkFun

图解大模型计算加速系列：FlashAttention V1，从硬件到计算逻辑猛猿

About

No description, website, or topics provided.

0 stars

0 watching

Report repository

Releases

Packages

Contributors

Languages