mirror of
https://github.com/rasbt/LLMs-from-scratch.git
synced 2025-08-21 15:12:41 +00:00
2ee3df622e
* fixed typo * removed remaining nbviewer links * Update mha-implementations.ipynb --------- Co-authored-by: Sebastian Raschka <mail@sebastianraschka.com>
More Efficient Multi-Head Attention Implementations
- mha-implementations.ipynb contains and compares different implementations of multi-head attention
Summary
The figures below summarize the performance benchmarks (lower is better).
Forward pass only
Forward and backward pass
Forward and backward pass after compilation
