LLMs-from-scratch/ch05/01_main-chapter-code
2024-03-31 08:48:19 -05:00
..
2024-03-31 08:48:19 -05:00
2024-03-31 08:48:19 -05:00
2024-03-18 08:16:17 -05:00
2024-03-29 09:03:36 -05:00

Chapter 5: Pretraining on Unlabeled Data

  • ch05.ipynb contains all the code as it appears in the chapter
  • previous_chapters.py is a Python module that contains the MultiHeadAttention module from the previous chapter, which we import in ch05.ipynb to pretrain the GPT model
  • train.py is a standalone Python script file with the code that we implemented in ch05.ipynb to train the GPT model
  • generate.py is a standalone Python script file with the code that we implemented in ch05.ipynb to load and use the pretrained model weights from OpenAI