LLMs-from-scratch/ch05/01_main-chapter-code/README.md

# Chapter 5: Pretraining on Unlabeled Data

- [ch05.ipynb](ch05.ipynb) contains all the code as it appears in the chapter
- [previous_chapters.py](previous_chapters.py) is a Python module that contains the `MultiHeadAttention` module and `GPTModel` class from the previous chapters, which we import in [ch05.ipynb](ch05.ipynb) to pretrain the GPT model
- [gpt_train.py](gpt_train.py) is a standalone Python script file with the code that we implemented in [ch05.ipynb](ch05.ipynb) to train the GPT model (you can think of it as a code file summarizing this chapter)
- [gpt_generate.py](gpt_generate.py) is a standalone Python script file with the code that we implemented in [ch05.ipynb](ch05.ipynb) to load and use the pretrained model weights from OpenAI
- [gpt_download.py](gpt_download.py) contains the utility functions for downloading the pretrained GPT model weights
- [exercise-solutions.ipynb](exercise-solutions.ipynb) contains the exercise solutions for this chapter
Add alternative weight loading strategy as backup (#82) 2024-03-20 08:43:18 -05:00			`# Chapter 5: Pretraining on Unlabeled Data`

			`- [ch05.ipynb](ch05.ipynb) contains all the code as it appears in the chapter`
add readme 2024-05-13 08:50:55 -05:00			- [previous_chapters.py](previous_chapters.py) is a Python module that contains the `MultiHeadAttention` module and `GPTModel` class from the previous chapters, which we import in [ch05.ipynb](ch05.ipynb) to pretrain the GPT model
			`- [gpt_train.py](gpt_train.py) is a standalone Python script file with the code that we implemented in [ch05.ipynb](ch05.ipynb) to train the GPT model (you can think of it as a code file summarizing this chapter)`
Organized setup instructions (#115) * Organized setup instructions * update tets * link checker action * raise error upon broken link * fix links * fix links * delete duplicated paragraph 2024-04-10 22:09:46 -04:00			`- [gpt_generate.py](gpt_generate.py) is a standalone Python script file with the code that we implemented in [ch05.ipynb](ch05.ipynb) to load and use the pretrained model weights from OpenAI`
add readme 2024-05-13 08:50:55 -05:00			`- [gpt_download.py](gpt_download.py) contains the utility functions for downloading the pretrained GPT model weights`
			`- [exercise-solutions.ipynb](exercise-solutions.ipynb) contains the exercise solutions for this chapter`
Add alternative weight loading strategy as backup (#82) 2024-03-20 08:43:18 -05:00