yujunjun/LLMs-from-scratch
1
0
Fork 0
mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2025-08-28 18:40:01 +00:00
Code Issues Packages Projects Releases Wiki Activity
826 Commits 8 Branches 0 Tags
Commit Graph

33 Commits

Author SHA1 Message Date
Sebastian Raschka
d387696f93 A few cosmetic updates (#504) 2025-01-23 09:38:55 -06:00
casinca
2fd07e2cfd potential little fixes appendix-D4 .ipynb (#427) 
* Update appendix-D.ipynb

- lr missing argument for passing peak_lr to the optimizer
- filling 1 step gap for gradient clipping

* adjustments

---------

Co-authored-by: rasbt <mail@sebastianraschka.com>
 2024-11-03 12:12:58 -06:00
rasbt
50500e94b5 Note about warm-up steps 2024-11-01 16:47:12 -05:00
Sebastian Raschka
11e2f56af5 Note about MPS devices (#329) 2024-08-19 20:58:45 -05:00
Daniel Kleine
73be1c592f fixed num_workers (#229) 
* fixed num_workers

* ch06 & ch07: added num_workers to create_dataloader_v1
 2024-06-19 17:36:46 -05:00
Sebastian Raschka
40ba3a4068 Remove leftover instances of self.tokenizer (#201) 
* Remove leftover instances of self.tokenizer

* add endoftext token
 2024-06-08 14:57:34 -05:00
rasbt
089dfb756a restore file 2024-06-03 07:17:56 -05:00
rasbt
d51099a9e7 add number of workers to data loader 2024-06-03 07:12:47 -05:00
rasbt
5a1e0eecce fix learning rate scheduler 2024-06-03 07:06:42 -05:00
rasbt
fe8bb9291e update formatting 2024-05-24 07:20:37 -05:00
rasbt
aa084656e0 update how to retrieve learning rate 2024-05-23 17:19:01 -05:00
DrCesar
d2410cb0c6 fix move model to device before calculating loss 2024-05-14 22:28:00 -07:00
rasbt
13e4282567 tests and exercises 2024-05-13 07:45:59 -05:00
Sebastian Raschka
a5b353667d Rename drop_resid to drop_shortcut (#136) 2024-04-28 14:31:27 -05:00
Sebastian Raschka
4557d5830e Return nan if val loader is empty (#124) 2024-04-20 08:02:30 -05:00
Sebastian Raschka
49f01d06d0 Calculate warmup steps as a fraction (#121) 2024-04-17 20:30:42 -05:00
Sebastian Raschka
bae4b0fb08 Make datesets and loaders compatible with multiprocessing (#118) 2024-04-13 13:57:56 -05:00
James Holcombe
0b866c133f Use instance tokenizer (#116) 
* Use instance tokenizer

* consistency updates

---------

Co-authored-by: Sebastian Raschka <mail@sebastianraschka.com>
 2024-04-10 21:16:19 -04:00
rasbt
c8cffefb6f cleanup 2024-04-04 07:58:41 -05:00
Sebastian Raschka
ccd7cebbb3 Rename variable to context_length to make it easier on readers (#106) 
* rename to context length

* fix spacing
 2024-04-04 07:27:41 -05:00
Sebastian Raschka
5beff4e25a Remove reundant dropout in MLP module (#105) 2024-04-03 20:19:08 -05:00
rasbt
776a517d18 figure scaling 2024-04-01 08:05:01 -05:00
rasbt
005835bfce make figures for appendix d 2024-03-31 21:24:41 -05:00
rasbt
ac2bdb02bd make figures for appendix d 2024-03-31 21:22:49 -05:00
rasbt
88b2dd780a make batch loss calculatution more efficient 2024-03-27 07:11:56 -05:00
rasbt
3cb5a52a1b simplify calc_loss_loader 2024-03-26 20:34:50 -05:00
rasbt
de576296de simplify .view code 2024-03-25 08:09:31 -05:00
Sebastian Raschka
cf39abac04 Add and link bonus material (#84) 2024-03-23 07:27:43 -05:00
Sebastian Raschka
a2cd8436cb Ch05 supplementary code (#81) 2024-03-19 09:26:26 -05:00
Sebastian Raschka
9d6da22ebb Update pep8 (#78) 
* simplify requirements file

* style

* apply linter
 2024-03-18 08:16:17 -05:00
rasbt
ff8657ac92 fix ipywidgets formatting issue 2024-03-16 08:35:43 -05:00
rasbt
a155879d71 update formatting 2024-03-16 08:10:58 -05:00
rasbt
6a585e08bc Add appendix D 2024-03-11 07:07:36 -05:00