LLMs-from-scratch/ch03/README.md

# Chapter 3: Coding Attention Mechanisms

&nbsp;
## Main Chapter Code

- [01_main-chapter-code](01_main-chapter-code) contains the main chapter code.

&nbsp;
## Bonus Materials

- [02_bonus_efficient-multihead-attention](02_bonus_efficient-multihead-attention) implements and compares different implementation variants of multihead-attention
- [03_understanding-buffers](03_understanding-buffers) explains the idea behind PyTorch buffers, which are used to implement the causal attention mechanism in chapter 3


In the video below, I provide a code-along session that covers some of the chapter contents as supplementary material.

<br>
<br>

[![Link to the video](https://img.youtube.com/vi/-Ll8DtpNtvk/0.jpg)](https://www.youtube.com/watch?v=Ll8DtpNtvk)
add and update readme files 2024-02-05 06:51:58 -06:00			`# Chapter 3: Coding Attention Mechanisms`
add ch03 and TOC 2023-12-09 17:13:56 -06:00
Update bonus section formatting (#400) 2024-10-12 10:26:08 -05:00			` `
distinguish better between main chapter code and bonus materials 2024-06-11 21:07:42 -05:00			`## Main Chapter Code`

mha variants 2024-03-06 08:30:32 -06:00			`- [01_main-chapter-code](01_main-chapter-code) contains the main chapter code.`
distinguish better between main chapter code and bonus materials 2024-06-11 21:07:42 -05:00
Update bonus section formatting (#400) 2024-10-12 10:26:08 -05:00			` `
distinguish better between main chapter code and bonus materials 2024-06-11 21:07:42 -05:00			`## Bonus Materials`

Understanding PyTorch Buffers (#288) 2024-07-26 08:45:36 -05:00			`- [02_bonus_efficient-multihead-attention](02_bonus_efficient-multihead-attention) implements and compares different implementation variants of multihead-attention`
Add chapter 3 coding along video link (#572) 2025-03-16 16:07:14 -05:00			`- [03_understanding-buffers](03_understanding-buffers) explains the idea behind PyTorch buffers, which are used to implement the causal attention mechanism in chapter 3`



			`In the video below, I provide a code-along session that covers some of the chapter contents as supplementary material.`

			`<br>`
			`<br>`

			`[![Link to the video](https://img.youtube.com/vi/-Ll8DtpNtvk/0.jpg)](https://www.youtube.com/watch?v=Ll8DtpNtvk)`