mirror of
https://github.com/rasbt/LLMs-from-scratch.git
synced 2025-08-30 19:41:51 +00:00
8 lines
658 B
Markdown
8 lines
658 B
Markdown
|
# Chapter 5: Pretraining on Unlabeled Data
|
||
|
|
|
||
|
|
- [ch05.ipynb](ch05.ipynb) contains all the code as it appears in the chapter
|
||
|
|
- [previous_chapters.py](previous_chapters.py) is a Python module that contains the `MultiHeadAttention` module from the previous chapter, which we import in [ch05.ipynb](ch05.ipynb) to pretrain the GPT model
|
||
|
|
- [train.py](train.py) is a standalone Python script file with the code that we implemented in [ch05.ipynb](ch05.ipynb) to train the GPT model
|
||
|
|
- [generate.py](generate.py) is a standalone Python script file with the code that we implemented in [ch05.ipynb](ch05.ipynb) to load and use the pretrained model weights from OpenAI
|
||
|
|
|