mirror of
https://github.com/rasbt/LLMs-from-scratch.git
synced 2025-08-29 19:10:19 +00:00
22 lines
759 B
Markdown
22 lines
759 B
Markdown
# Chapter 3: Coding Attention Mechanisms
|
|
|
|
|
|
## Main Chapter Code
|
|
|
|
- [01_main-chapter-code](01_main-chapter-code) contains the main chapter code.
|
|
|
|
|
|
## Bonus Materials
|
|
|
|
- [02_bonus_efficient-multihead-attention](02_bonus_efficient-multihead-attention) implements and compares different implementation variants of multihead-attention
|
|
- [03_understanding-buffers](03_understanding-buffers) explains the idea behind PyTorch buffers, which are used to implement the causal attention mechanism in chapter 3
|
|
|
|
|
|
|
|
In the video below, I provide a code-along session that covers some of the chapter contents as supplementary material.
|
|
|
|
<br>
|
|
<br>
|
|
|
|
[](https://www.youtube.com/watch?v=-Ll8DtpNtvk)
|