mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2025-08-18 21:52:23 +00:00

History

* Use instance tokenizer

* consistency updates

---------

Co-authored-by: Sebastian Raschka <mail@sebastianraschka.com>

2024-04-10 21:16:19 -04:00

ch05.ipynb

2024-04-07 08:41:09 -05:00

exercise-solutions.ipynb

renumber exercises

2024-04-07 06:03:41 -05:00

gpt_download.py

2024-04-07 08:41:09 -05:00

gpt_generate.py

2024-04-07 08:41:09 -05:00

gpt_train.py

2024-04-05 07:24:46 -05:00

previous_chapters.py

2024-04-10 21:16:19 -04:00

README.md

2024-03-20 08:43:18 -05:00

tests.py

2024-04-05 07:24:46 -05:00

Chapter 5: Pretraining on Unlabeled Data

ch05.ipynb contains all the code as it appears in the chapter
previous_chapters.py is a Python module that contains the MultiHeadAttention module from the previous chapter, which we import in ch05.ipynb to pretrain the GPT model
train.py is a standalone Python script file with the code that we implemented in ch05.ipynb to train the GPT model
generate.py is a standalone Python script file with the code that we implemented in ch05.ipynb to load and use the pretrained model weights from OpenAI