LLMs-from-scratch/ch05/05_bonus_hparam_tuning/README.md

# Optimizing Hyperparameters for Pretraining

The [hparam_search.py](hparam_search.py) script, based on the extended training function in [Appendix D: Adding Bells and Whistles to the Training Loop](../../appendix-D/01_main-chapter-code/appendix-D.ipynb), is designed to find optimal hyperparameters via grid search.

>[!NOTE]
This script will take a long time to run. You may want to reduce the number of hyperparameter configurations explored in the `HPARAM_GRID` dictionary at the top.
Add and link bonus material (#84) 2024-03-23 07:27:43 -05:00			`# Optimizing Hyperparameters for Pretraining`

Organized setup instructions (#115) * Organized setup instructions * update tets * link checker action * raise error upon broken link * fix links * fix links * delete duplicated paragraph 2024-04-10 22:09:46 -04:00			`The [hparam_search.py](hparam_search.py) script, based on the extended training function in [Appendix D: Adding Bells and Whistles to the Training Loop](../../appendix-D/01_main-chapter-code/appendix-D.ipynb), is designed to find optimal hyperparameters via grid search.`
Add and link bonus material (#84) 2024-03-23 07:27:43 -05:00
			`>[!NOTE]`
			This script will take a long time to run. You may want to reduce the number of hyperparameter configurations explored in the `HPARAM_GRID` dictionary at the top.