yujunjun/LLMs-from-scratch
1
0
Fork 0
mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2025-08-10 17:51:43 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'main' of https://github.com/rasbt/LLMs-from-scratch

Browse Source
This commit is contained in:
rasbt 2024-05-10 06:45:14 -05:00
commit ca84d797a7
2 changed files with 4 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
ch06/01_main-chapter-code/ch06.ipynb
View File

@ -2043,7 +2043,7 @@
"metadata": {},
"source": [
"- We can see that the training and test set performances are practically identical\n",
"- However, based on the slightly lower test set performance, we can see that the model overfits the training data to a very small degree\n",
"- However, based on the slightly lower test set performance, we can see that the model overfits the training data to a very small degree, as well as the validation data that has been used for tweaking some of the hyperparameters, such as the learning rate\n",
"- This is normal, however, and this gap could potentially be further reduced by increasing the model's dropout rate (`drop_rate`) or the `weight_decay` in the optimizer setting"
]
},
@ -2265,7 +2265,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.12"
"version": "3.10.6"
}
},
"nbformat": 4,

4
setup/README.md
View File

@ -58,7 +58,7 @@ cd LLMs-from-scratch
pip install -r requirements.txt
```
(In contrast to Google Colab, these only need to be executed once since the Lightning AI Studion environments are persistent).
(In contrast to Google Colab, these only need to be executed once since the Lightning AI Studio environments are persistent, even if you switch between CPU and GPU machines.)
Then, navigate to the Python script or Jupyter Notebook you want to run. Optionally, you can also easily connect a GPU to accelerate the code's runtime, for example, when you are pretraining the LLM in chapter 5 or finetuning it in chapters 6 and 7.
@ -87,4 +87,4 @@ You can optionally run the code on a GPU by changing the *Runtime* as illustrate
## Questions?
If you have any questions, please don't hesitate to reach out via the [Discussions](https://github.com/rasbt/LLMs-from-scratch/discussions) forum in this GitHub repository.
If you have any questions, please don't hesitate to reach out via the [Discussions](https://github.com/rasbt/LLMs-from-scratch/discussions) forum in this GitHub repository.