mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2025-12-26 06:32:10 +00:00

History

Daniel Kleine cba4f89514 updates for PyTorch 2.5 (#408 )

* updated Dockerfile

* updated MHA implementations for PT 2.5

* fixed typo

* update installation instruction

* Update setup/03_optional-docker-environment/.devcontainer/Dockerfile

---------

Co-authored-by: rasbt <mail@sebastianraschka.com>

2024-10-22 20:23:31 -05:00

mha-implementations.ipynb

updates for PyTorch 2.5 (#408 )

2024-10-22 20:23:31 -05:00

README.md

Einsum multi-head attention (#345 )

2024-09-05 18:24:33 +02:00

README.md

More Efficient Multi-Head Attention Implementations

mha-implementations.ipynb contains and compares different implementations of multi-head attention

Summary

The figures below summarize the performance benchmarks (lower is better).

README.md

More Efficient Multi-Head Attention Implementations

Summary

Forward pass only

Forward and backward pass

Forward and backward pass after compilation