mirror of
https://github.com/rasbt/LLMs-from-scratch.git
synced 2025-07-24 01:13:44 +00:00
e757091301
* Organized setup instructions * update tets * link checker action * raise error upon broken link * fix links * fix links * delete duplicated paragraph
Chapter 5: Pretraining on Unlabeled Data
- ch05.ipynb contains all the code as it appears in the chapter
- previous_chapters.py is a Python module that contains the
MultiHeadAttentionmodule from the previous chapter, which we import in ch05.ipynb to pretrain the GPT model - gpt_train.py is a standalone Python script file with the code that we implemented in ch05.ipynb to train the GPT model
- gpt_generate.py is a standalone Python script file with the code that we implemented in ch05.ipynb to load and use the pretrained model weights from OpenAI