TITC
|
d16527ddf2
|
total training iters may equal to warmup_iters (#301)
total_training_iters=20, warmup_iters=20= len(train_loader) 4 multiply n_epochs 5, then ZeroDivisionError occurred.
```shell
Traceback (most recent call last):
File "LLMs-from-scratch/ch05/05_bonus_hparam_tuning/hparam_search.py", line 191, in <module>
train_loss, val_loss = train_model(
^^^^^^^^^^^^
File "/mnt/raid1/docker/ai/LLMs-from-scratch/ch05/05_bonus_hparam_tuning/hparam_search.py", line 90, in train_model
progress = (global_step - warmup_iters) / (total_training_iters - warmup_iters)
~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ZeroDivisionError: division by zero
```
|
2024-08-06 07:10:05 -05:00 |
|
Daniel Kleine
|
bbb2a0c3d5
|
fixed num_workers (#229)
* fixed num_workers
* ch06 & ch07: added num_workers to create_dataloader_v1
|
2024-06-19 17:36:46 -05:00 |
|
Daniel Kleine
|
dcbdc1d2e5
|
fixes for code (#206)
* updated .gitignore
* removed unused GELU import
* fixed model_configs, fixed all tensors on same device
* removed unused tiktoken
* update
* update hparam search
* remove redundant tokenizer argument
---------
Co-authored-by: rasbt <mail@sebastianraschka.com>
|
2024-06-11 20:59:48 -05:00 |
|
Sebastian Raschka
|
72a073bbbf
|
Remove leftover instances of self.tokenizer (#201)
* Remove leftover instances of self.tokenizer
* add endoftext token
|
2024-06-08 14:57:34 -05:00 |
|
rasbt
|
6f0a5c320b
|
fix learning rate scheduler
|
2024-06-03 07:06:42 -05:00 |
|
rasbt
|
b40c260859
|
update how to retrieve learning rate
|
2024-05-23 17:19:01 -05:00 |
|
Sebastian Raschka
|
97ed38116a
|
Rename drop_resid to drop_shortcut (#136)
|
2024-04-28 14:31:27 -05:00 |
|
Sebastian Raschka
|
c70ddff558
|
Return nan if val loader is empty (#124)
|
2024-04-20 08:02:30 -05:00 |
|
Sebastian Raschka
|
dd51d4ad83
|
Make datesets and loaders compatible with multiprocessing (#118)
|
2024-04-13 13:57:56 -05:00 |
|
Sebastian Raschka
|
e757091301
|
Organized setup instructions (#115)
* Organized setup instructions
* update tets
* link checker action
* raise error upon broken link
* fix links
* fix links
* delete duplicated paragraph
|
2024-04-10 22:09:46 -04:00 |
|
James Holcombe
|
05718c6b94
|
Use instance tokenizer (#116)
* Use instance tokenizer
* consistency updates
---------
Co-authored-by: Sebastian Raschka <mail@sebastianraschka.com>
|
2024-04-10 21:16:19 -04:00 |
|
Sebastian Raschka
|
2de60d1bfb
|
Rename variable to context_length to make it easier on readers (#106)
* rename to context length
* fix spacing
|
2024-04-04 07:27:41 -05:00 |
|
Sebastian Raschka
|
3829ccdb34
|
Remove reundant dropout in MLP module (#105)
|
2024-04-03 20:19:08 -05:00 |
|
rasbt
|
88b2dd780a
|
make batch loss calculatution more efficient
|
2024-03-27 07:11:56 -05:00 |
|
rasbt
|
3cb5a52a1b
|
simplify calc_loss_loader
|
2024-03-26 20:34:50 -05:00 |
|
Sebastian Raschka
|
cf39abac04
|
Add and link bonus material (#84)
|
2024-03-23 07:27:43 -05:00 |
|