Sebastian Raschka
092b5b5429
topk comment
2024-08-20 20:44:15 -05:00
rasbt
f4e45a3f40
add note about duplicated cell
2024-08-19 21:04:18 -05:00
Sebastian Raschka
11e2f56af5
Note about MPS devices ( #329 )
2024-08-19 20:58:45 -05:00
Sebastian Raschka
a82169290e
Note about MPS in ch06 and ch07 ( #325 )
2024-08-19 08:11:33 -05:00
Sebastian Raschka
0991c1ff24
Note about ch05 mps support ( #324 )
2024-08-19 07:40:24 -05:00
rasbt
1a962f3983
update lora experiments
2024-08-16 08:57:46 -05:00
rasbt
742525cb28
remove redundant indentation
2024-08-16 07:54:02 -05:00
TITC
70a7cddf9c
typo fix ( #323 )
2024-08-16 06:57:30 -05:00
TITC
e1072bbcd6
typo fix ( #321 )
...
* typo fix
experiment 5 is the same size as experiment 1, both are 124 Million. and experiment 6 is 355M ~3x 124M.
* typo fix
all layer FT vs all layer FT, "slightly worse by 1.3% " indicates it's exp 5 vs exp 9
* exp 5 vs 9
* Update ch06/02_bonus_additional-experiments/README.md
---------
Co-authored-by: Sebastian Raschka <mail@sebastianraschka.com>
2024-08-15 08:45:34 -05:00
Sebastian Raschka
e4cfcc5b66
Update README.md
2024-08-15 08:22:47 -05:00
Daniel Kleine
bf1d8687f4
added std error bars ( #320 )
...
* added std error bars
* fixed changes
* Update on A100
---------
Co-authored-by: Sebastian Raschka <mail@sebastianraschka.com>
2024-08-13 20:57:41 -05:00
Sebastian Raschka
bb73efb5cf
Consistency update in README.md
2024-08-13 07:33:10 -05:00
TITC
0b998dff97
track tokens seen in chapter5, track examples seen in chapter6 ( #319 )
2024-08-13 07:09:05 -05:00
rasbt
6533ce63c1
fix code cell ordering
2024-08-12 19:04:05 -05:00
Jeroen Van Goey
75abb61f13
Small typo fix ( #313 )
...
* typo fix
* Update ch03/02_bonus_efficient-multihead-attention/mha-implementations.ipynb
* Update ch03/02_bonus_efficient-multihead-attention/mha-implementations.ipynb
* Update ch03/02_bonus_efficient-multihead-attention/mha-implementations.ipynb
---------
Co-authored-by: Sebastian Raschka <mail@sebastianraschka.com>
2024-08-12 07:54:12 -05:00
Eric Thomson
b3d550bfd5
Adds .vscode folder to .gitignore ( #314 )
...
* added .vscode folder to .gitignore
* Update .gitignore
---------
Co-authored-by: Sebastian Raschka <mail@sebastianraschka.com>
2024-08-12 07:49:11 -05:00
Sebastian Raschka
f58c2b969b
update attention benchmarks ( #307 )
2024-08-10 09:44:11 -05:00
Sebastian Raschka
8d79fb13b0
Update README.md
2024-08-10 07:54:51 -05:00
Daniel Kleine
c91999b9f4
fixed bash command ( #305 )
...
Co-authored-by: Sebastian Raschka <mail@sebastianraschka.com>
2024-08-09 21:29:04 -05:00
TITC
3067ed83dc
remove all non-English texts and notice ( #304 )
...
* remove all non-English texts and notice
1. almost 18GB txt left after `is_english` filtered.
2. remove notice use gutenberg's strip_headers
3. after re-run get_data.py, seems all data are under `gutenberg/data/.mirror` folder.
* some improvements
* update readme
---------
Co-authored-by: rasbt <mail@sebastianraschka.com>
2024-08-09 17:09:14 -05:00
Sebastian Raschka
de277ebfe6
Update README.md
2024-08-08 07:50:45 -05:00
Sebastian Raschka
4601219c32
Update README.md
2024-08-08 07:47:31 -05:00
Sebastian Raschka
895512ebee
Revert accidental edit
2024-08-06 19:54:34 -05:00
rasbt
7a5771932b
note about logistic sigmoid
2024-08-06 19:48:30 -05:00
rasbt
9ef132bb64
note about logistic sigmoid
2024-08-06 19:48:06 -05:00
rasbt
2245f8d9c1
extend equation description
2024-08-06 19:46:50 -05:00
rasbt
a65e06ff99
add more explanations
2024-08-06 19:45:11 -05:00
Sebastian Raschka
b1a420c8ca
Update README.md
2024-08-06 08:02:01 -05:00
TITC
7374d617b4
total training iters may equal to warmup_iters ( #301 )
...
total_training_iters=20, warmup_iters=20= len(train_loader) 4 multiply n_epochs 5, then ZeroDivisionError occurred.
```shell
Traceback (most recent call last):
File "LLMs-from-scratch/ch05/05_bonus_hparam_tuning/hparam_search.py", line 191, in <module>
train_loss, val_loss = train_model(
^^^^^^^^^^^^
File "/mnt/raid1/docker/ai/LLMs-from-scratch/ch05/05_bonus_hparam_tuning/hparam_search.py", line 90, in train_model
progress = (global_step - warmup_iters) / (total_training_iters - warmup_iters)
~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ZeroDivisionError: division by zero
```
2024-08-06 07:10:05 -05:00
Sebastian Raschka
192bdc3501
improve gradient accumulation ( #300 )
2024-08-05 18:27:20 -05:00
rasbt
089901db26
small figure update
2024-08-05 17:57:16 -05:00
Sebastian Raschka
b39234fc25
Update README.md
2024-08-05 17:47:06 -05:00
Daniel Kleine
dcdf04e3bd
minor DPO fixes ( #298 )
...
* fixed issues, updated .gitignore
* added closing paren
* fixed CEL spelling
* fixed more minor issues
* Update ch07/01_main-chapter-code/ch07.ipynb
* Update ch07/04_preference-tuning-with-dpo/dpo-from-scratch.ipynb
* Update ch07/04_preference-tuning-with-dpo/dpo-from-scratch.ipynb
* Update ch07/04_preference-tuning-with-dpo/dpo-from-scratch.ipynb
---------
Co-authored-by: Sebastian Raschka <mail@sebastianraschka.com>
2024-08-05 08:40:46 -05:00
rasbt
6030071e3f
update model path
2024-08-05 07:36:08 -05:00
SSebo
22681878a8
Update ch05.ipynb ( #297 )
...
typo
2024-08-05 07:12:27 -05:00
Sebastian Raschka
e4ed2c91cc
Update README.md
2024-08-04 16:06:38 -05:00
rasbt
f302f5e8d5
improve latex rendering in dpo notebook
2024-08-04 09:19:59 -05:00
Sebastian Raschka
2c6cdb497f
Update matplotlib tests on Windows ( #295 )
2024-08-04 09:18:19 -05:00
Sebastian Raschka
09dc080cf3
Direct Preference Optimization from scratch ( #294 )
2024-08-04 08:57:36 -05:00
Sebastian Raschka
3ea0798d44
Update README.md
2024-08-01 18:17:42 -05:00
rasbt
eabeab04d1
restructure into local and cloud setup
2024-07-31 06:59:04 -05:00
Sebastian Raschka
b4b2ac668c
Add video tutorial
2024-07-30 06:57:46 -05:00
Sebastian Raschka
6c2d34b03d
Update README.md
2024-07-30 06:55:41 -05:00
rasbt
358717870b
add state_dict example
2024-07-28 14:15:32 -05:00
rasbt
ce33e706ba
Fix 8-billion-parameter spelling
2024-07-28 10:48:56 -05:00
Daniel Kleine
60752e3b3a
fixed typos and formatting ( #291 )
2024-07-28 10:04:33 -05:00
Sebastian Raschka
75a24c4897
Update README.md
2024-07-28 09:28:11 -05:00
Sebastian Raschka
011ee83ea7
Update README.md
2024-07-28 08:21:38 -05:00
Sebastian Raschka
1b86a61a2e
buffer tutorial
2024-07-27 17:06:16 -05:00
Sebastian Raschka
f66c089f0b
Test with PyTorch 2.0 and 2.4 ( #290 )
...
* Test with PyTorch 2.0 and 2.4
* Update basic-tests-old-pytorch.yml
* skip version cell
2024-07-27 15:09:02 -05:00
