yujunjun/LLMs-from-scratch
1
0
Fork 0
mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2025-11-02 10:50:30 +00:00
Code Issues Packages Projects Releases Wiki Activity
LLMs-from-scratch/ch05/05_bonus_hparam_tuning
History
James Holcombe 0b866c133f Use instance tokenizer (#116) 
* Use instance tokenizer

* consistency updates

---------

Co-authored-by: Sebastian Raschka <mail@sebastianraschka.com>
2024-04-10 21:16:19 -04:00
..
hparam_search.py
Rename variable to context_length to make it easier on readers (#106)
2024-04-04 07:27:41 -05:00
previous_chapters.py
Use instance tokenizer (#116)
2024-04-10 21:16:19 -04:00
README.md
Add and link bonus material (#84)
2024-03-23 07:27:43 -05:00
the-verdict.txt
Add and link bonus material (#84)
2024-03-23 07:27:43 -05:00

README.md

Optimizing Hyperparameters for Pretraining

The hparam_search.py is script based on the extended training function in Appendix D: Adding Bells and Whistles to the Training Loop to find optimal hyperparameters via grid search

The hparam_search.py script, based on the extended training function in Appendix D: Adding Bells and Whistles to the Training Loop, is designed to find optimal hyperparameters via grid search.

Note

This script will take a long time to run. You may want to reduce the number of hyperparameter configurations explored in the HPARAM_GRID dictionary at the top.