yujunjun/LLMs-from-scratch
1
0
Fork 0
mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2025-08-31 03:50:23 +00:00
Code Issues Packages Projects Releases Wiki Activity
LLMs-from-scratch/ch05/01_main-chapter-code/README.md
Sebastian Raschka 4582995ced Add alternative weight loading strategy as backup (#82)
2024-03-20 08:43:18 -05:00

658 B
Raw Blame History

Chapter 5: Pretraining on Unlabeled Data

  • ch05.ipynb contains all the code as it appears in the chapter
  • previous_chapters.py is a Python module that contains the MultiHeadAttention module from the previous chapter, which we import in ch05.ipynb to pretrain the GPT model
  • train.py is a standalone Python script file with the code that we implemented in ch05.ipynb to train the GPT model
  • generate.py is a standalone Python script file with the code that we implemented in ch05.ipynb to load and use the pretrained model weights from OpenAI