Sebastian Raschka
701090815e
Add backup URL for gpt2 weights ( #469 )
...
* Add backup URL for gpt2 weights
* newline
2025-01-05 11:28:09 -06:00
Daniel Kleine
e95c898545
Fixed command for row 16 additional experiment ( #439 )
...
* fixed command for row 16 experiment
* Update README.md
---------
Co-authored-by: Sebastian Raschka <mail@sebastianraschka.com>
2024-11-17 06:50:00 +09:00
Sebastian Raschka
ccade77bf4
Add flexible padding bonus experiment ( #438 )
...
* Add flexible padding bonus experiment
* fix links
2024-11-15 08:51:01 +09:00
Sebastian Raschka
38969864e6
Add mean pooling experiment to classifier bonus experiments ( #406 )
...
* Add mean pooling experiment to classifier bonus experiments
* formatting
* add average embeddings option
* pep8
2024-10-20 11:04:18 -05:00
Sebastian Raschka
564362044a
add BERT experiment results ( #333 )
...
* add BERT experiment results
* cleanup
* formatting
2024-08-23 08:40:40 -05:00
Sebastian Raschka
ed257789a4
grammar fix
2024-08-21 19:16:21 -05:00
rasbt
e7cb2ebd8d
update lora experiments
2024-08-16 08:57:46 -05:00
TITC
d2befdf00e
typo fix ( #323 )
2024-08-16 06:57:30 -05:00
TITC
f6526f7766
typo fix ( #321 )
...
* typo fix
experiment 5 is the same size as experiment 1, both are 124 Million. and experiment 6 is 355M ~3x 124M.
* typo fix
all layer FT vs all layer FT, "slightly worse by 1.3% " indicates it's exp 5 vs exp 9
* exp 5 vs 9
* Update ch06/02_bonus_additional-experiments/README.md
---------
Co-authored-by: Sebastian Raschka <mail@sebastianraschka.com>
2024-08-15 08:45:34 -05:00
Sebastian Raschka
4ea526908e
Update README.md
2024-08-15 08:22:47 -05:00
Sebastian Raschka
c5eaae11b1
Revert accidental edit
2024-08-06 19:54:34 -05:00
rasbt
06151a809e
note about logistic sigmoid
2024-08-06 19:48:30 -05:00
Sebastian Raschka
70e5714556
improve gradient accumulation ( #300 )
2024-08-05 18:27:20 -05:00
Sebastian Raschka
8d02cb1cee
Add download help message ( #274 )
2024-07-19 08:29:29 -05:00
Jeroen Van Goey
48bd72c890
fix typos, add codespell pre-commit hook ( #264 )
...
* fix typos, add codespell pre-commit hook
* Update .pre-commit-config.yaml
---------
Co-authored-by: Sebastian Raschka <mail@sebastianraschka.com>
2024-07-16 07:07:04 -05:00
Daniel Kleine
90b25ece3d
fixed spelling typos ( #258 )
2024-07-03 07:47:33 -05:00
Daniel Kleine
1e69c8e0b5
fixed minor issues ( #252 )
...
* fixed typo
* fixed var name in md text
2024-06-29 06:38:25 -05:00
Daniel Kleine
bbb2a0c3d5
fixed num_workers ( #229 )
...
* fixed num_workers
* ch06 & ch07: added num_workers to create_dataloader_v1
2024-06-19 17:36:46 -05:00
Daniel Kleine
ef40f2f9ad
minor bug fixes ( #207 )
...
* fixed path arg for create_dataset_csvs()
* updated assign_check() to remove user warning
2024-06-12 06:27:56 -05:00
Daniel Kleine
dcbdc1d2e5
fixes for code ( #206 )
...
* updated .gitignore
* removed unused GELU import
* fixed model_configs, fixed all tensors on same device
* removed unused tiktoken
* update
* update hparam search
* remove redundant tokenizer argument
---------
Co-authored-by: rasbt <mail@sebastianraschka.com>
2024-06-11 20:59:48 -05:00
rasbt
1d278c65da
fix gradient comment
2024-06-09 20:23:18 -05:00
Sebastian Raschka
72a073bbbf
Remove leftover instances of self.tokenizer ( #201 )
...
* Remove leftover instances of self.tokenizer
* add endoftext token
2024-06-08 14:57:34 -05:00
Daniel Kleine
ba1c1e74aa
updated Dockerfile and Additional Classification Finetuning Experiments ( #195 )
...
* accuracy to .2f
* added curl
2024-06-05 20:17:49 -05:00
Sebastian Raschka
49b8e2e767
Update README.md
2024-05-25 10:45:29 -05:00
rasbt
b43a7a8820
trainable token -> trainable token position
2024-05-23 11:43:20 -05:00
rasbt
18e729643d
add assertion about data set length
2024-05-23 06:50:43 -05:00
Daniel Kleine
195849cf8d
fixed last_two_blocks
2024-05-22 02:02:43 +00:00
rasbt
caf3725001
fix table alignment
2024-05-21 19:51:22 -05:00
rasbt
725bed56f7
experiment with last two blocks
2024-05-21 19:49:34 -05:00
Daniel Kleine
f39087e573
improved readability of Additional Experiments table
2024-05-21 19:26:25 +00:00
rasbt
c2028871e4
update lora init
2024-05-19 20:11:56 -05:00
rasbt
a5593f9860
change defaults to 0 temp
2024-05-19 09:04:49 -05:00
rasbt
1463b2ae47
use default value for temperature
2024-05-19 08:48:10 -05:00
rasbt
1b340c9eb6
add ignore index experiment
2024-05-19 07:24:49 -05:00
rasbt
5ef4edf2b5
new experiment w/o causal mask
2024-05-18 17:03:36 -05:00
Sebastian Raschka
57634f2045
fix row number typo
2024-05-18 15:54:13 -05:00
rasbt
4851d5a0fa
add eos_id option for ch07
2024-05-18 12:35:40 -05:00
rasbt
42cb0cbd59
Add experiment with gradient accumulation
2024-05-17 21:31:22 -05:00
rasbt
fc88fefd9c
fix no padding option
2024-05-17 21:06:51 -05:00
rasbt
5cfc64d038
fix indent
2024-05-17 07:58:01 -05:00
rasbt
04b9540938
Add new experiment without padding
2024-05-17 07:55:51 -05:00
Sebastian Raschka
5094eb7567
val before test acc
2024-05-13 07:36:18 -05:00
Sebastian Raschka
6fe8d1a10e
Update README.md
2024-05-11 06:42:05 -05:00
Sebastian Raschka
a3e1fa35f5
Move GPU column so that test accuracy is always visible
2024-05-11 06:41:25 -05:00
Sebastian Raschka
41288a3d3a
Add LoRA experiments ( #151 )
...
* Add LoRA experiments
* Update ch06/02_bonus_additional-experiments/additional-experiments.py
2024-05-10 07:26:41 -05:00
rasbt
756ff780de
experiments with largest model
2024-05-09 07:40:09 -05:00
rasbt
6f486460bc
ouput -> output
2024-05-05 12:21:10 -05:00
rasbt
0ac19a1e50
use training set len
2024-04-29 21:50:07 -05:00
Sebastian Raschka
97ed38116a
Rename drop_resid to drop_shortcut ( #136 )
2024-04-28 14:31:27 -05:00
Sebastian Raschka
e1d094b655
Update README.md
2024-04-27 07:59:42 -05:00
