yujunjun/LLMs-from-scratch
1
0
Fork 0
mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2025-09-01 20:38:11 +00:00
Code Issues Packages Projects Releases Wiki Activity

Update bonus section formatting (#400)

Browse Source
This commit is contained in:
Sebastian Raschka 2024-10-12 10:26:08 -05:00 committed by GitHub
parent 35ecca0feb
commit 6a9bedc2ec
7 changed files with 23 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
ch01/README.md
View File

@ -1,8 +1,15 @@
# Chapter 1: Understanding Large Language Models # Chapter 1: Understanding Large Language Models
 
## Main Chapter Code
There is no code in this chapter. There is no code in this chapter.
<br>
&nbsp;
## Bonus Materials
As optional bonus material, below is a video tutorial where I explain the LLM development lifecycle covered in this book: As optional bonus material, below is a video tutorial where I explain the LLM development lifecycle covered in this book:
<br> <br>

3
ch02/README.md
View File

@ -1,10 +1,11 @@
# Chapter 2: Working with Text Data # Chapter 2: Working with Text Data
&nbsp;
## Main Chapter Code ## Main Chapter Code
- [01_main-chapter-code](01_main-chapter-code) contains the main chapter code and exercise solutions - [01_main-chapter-code](01_main-chapter-code) contains the main chapter code and exercise solutions
&nbsp;
## Bonus Materials ## Bonus Materials
- [02_bonus_bytepair-encoder](02_bonus_bytepair-encoder) contains optional code to benchmark different byte pair encoder implementations - [02_bonus_bytepair-encoder](02_bonus_bytepair-encoder) contains optional code to benchmark different byte pair encoder implementations

2
ch03/README.md
View File

@ -1,9 +1,11 @@
# Chapter 3: Coding Attention Mechanisms # Chapter 3: Coding Attention Mechanisms
&nbsp;
## Main Chapter Code ## Main Chapter Code
- [01_main-chapter-code](01_main-chapter-code) contains the main chapter code. - [01_main-chapter-code](01_main-chapter-code) contains the main chapter code.
&nbsp;
## Bonus Materials ## Bonus Materials
- [02_bonus_efficient-multihead-attention](02_bonus_efficient-multihead-attention) implements and compares different implementation variants of multihead-attention - [02_bonus_efficient-multihead-attention](02_bonus_efficient-multihead-attention) implements and compares different implementation variants of multihead-attention

7
ch04/README.md
View File

@ -1,10 +1,13 @@
# Chapter 4: Implementing a GPT Model from Scratch to Generate Text # Chapter 4: Implementing a GPT Model from Scratch to Generate Text
&nbsp;
## Main Chapter Code ## Main Chapter Code
- [01_main-chapter-code](01_main-chapter-code) contains the main chapter code. - [01_main-chapter-code](01_main-chapter-code) contains the main chapter code.
## Optional Code &nbsp;
## Bonus Materials
- [02_performance-analysis](02_performance-analysis) contains optional code analyzing the performance of the GPT model(s) implemented in the main chapter. - [02_performance-analysis](02_performance-analysis) contains optional code analyzing the performance of the GPT model(s) implemented in the main chapter
- [ch05/07_gpt_to_llama](../ch05/07_gpt_to_llama) contains a step-by-step guide for converting a GPT architecture implementation to Llama 3.2 and loads pretrained weights from Meta AI (it might be interesting to look at alternative architectures after completing chapter 4, but you can also save that for after reading chapter 5)

2
ch05/README.md
View File

@ -1,9 +1,11 @@
# Chapter 5: Pretraining on Unlabeled Data # Chapter 5: Pretraining on Unlabeled Data
&nbsp;
## Main Chapter Code ## Main Chapter Code
- [01_main-chapter-code](01_main-chapter-code) contains the main chapter code - [01_main-chapter-code](01_main-chapter-code) contains the main chapter code
&nbsp;
## Bonus Materials ## Bonus Materials
- [02_alternative_weight_loading](02_alternative_weight_loading) contains code to load the GPT model weights from alternative places in case the model weights become unavailable from OpenAI - [02_alternative_weight_loading](02_alternative_weight_loading) contains code to load the GPT model weights from alternative places in case the model weights become unavailable from OpenAI

3
ch06/README.md
View File

@ -1,10 +1,11 @@
# Chapter 6: Finetuning for Classification # Chapter 6: Finetuning for Classification
&nbsp;
## Main Chapter Code ## Main Chapter Code
- [01_main-chapter-code](01_main-chapter-code) contains the main chapter code - [01_main-chapter-code](01_main-chapter-code) contains the main chapter code
&nbsp;
## Bonus Materials ## Bonus Materials
- [02_bonus_additional-experiments](02_bonus_additional-experiments) includes additional experiments (e.g., training the last vs first token, extending the input length, etc.) - [02_bonus_additional-experiments](02_bonus_additional-experiments) includes additional experiments (e.g., training the last vs first token, extending the input length, etc.)

2
ch07/README.md
View File

@ -1,9 +1,11 @@
# Chapter 7: Finetuning to Follow Instructions # Chapter 7: Finetuning to Follow Instructions
&nbsp;
## Main Chapter Code ## Main Chapter Code
- [01_main-chapter-code](01_main-chapter-code) contains the main chapter code and exercise solutions - [01_main-chapter-code](01_main-chapter-code) contains the main chapter code and exercise solutions
&nbsp;
## Bonus Materials ## Bonus Materials
- [02_dataset-utilities](02_dataset-utilities) contains utility code that can be used for preparing an instruction dataset - [02_dataset-utilities](02_dataset-utilities) contains utility code that can be used for preparing an instruction dataset