yujunjun/LLMs-from-scratch
1
0
Fork 0
mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2025-08-29 19:10:19 +00:00
Code Issues Packages Projects Releases Wiki Activity
LLMs-from-scratch/ch03/02_bonus_efficient-multihead-attention
History
Sebastian Raschka faadd13871 Update mha plot (#560)
2025-03-06 20:29:04 -06:00
..
mha-implementations.ipynb
Update mha plot (#560)
2025-03-06 20:29:04 -06:00
README.md
Einsum multi-head attention (#345)
2024-09-05 18:24:33 +02:00

README.md

More Efficient Multi-Head Attention Implementations

Summary

The figures below summarize the performance benchmarks (lower is better).

 

Forward pass only

 

Forward and backward pass

 

Forward and backward pass after compilation