yujunjun/LLMs-from-scratch
1
0
Fork 0
mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2025-10-31 01:41:26 +00:00
Code Issues Packages Projects Releases Wiki Activity
LLMs-from-scratch/ch05/01_main-chapter-code
History
rasbt 9cc9c4244e simplify
2024-03-26 07:52:36 -05:00
..
ch05.ipynb
simplify
2024-03-26 07:52:36 -05:00
generate.py
Add alternative weight loading strategy as backup (#82)
2024-03-20 08:43:18 -05:00
previous_chapters.py
Update pep8 (#78)
2024-03-18 08:16:17 -05:00
README.md
Add alternative weight loading strategy as backup (#82)
2024-03-20 08:43:18 -05:00
tests.py
Ch05 supplementary code (#81)
2024-03-19 09:26:26 -05:00
train.py
simplify .view code
2024-03-25 08:09:31 -05:00

README.md

Chapter 5: Pretraining on Unlabeled Data

  • ch05.ipynb contains all the code as it appears in the chapter
  • previous_chapters.py is a Python module that contains the MultiHeadAttention module from the previous chapter, which we import in ch05.ipynb to pretrain the GPT model
  • train.py is a standalone Python script file with the code that we implemented in ch05.ipynb to train the GPT model
  • generate.py is a standalone Python script file with the code that we implemented in ch05.ipynb to load and use the pretrained model weights from OpenAI