Sebastian Raschka
a0f5326a25
Update pixi ( #661 )
...
* Llama3 from scratch improvements
* Update HF hub version in pixi.toml
* Update README.md
2025-06-13 10:50:17 -05:00
Daniel Kleine
479b0e2aa9
fixed gqa qkv code comments ( #660 )
2025-06-13 08:21:28 -05:00
Greg Gandenberger
2af3cf070c
Update ch07.ipynb ( #643 )
...
Correct function name
2025-06-13 08:17:10 -05:00
Shimpei Kojio
1446cfd824
fixed video link ( #646 )
2025-06-13 08:16:18 -05:00
Pratyush Subhadarshi
d142741ef4
Correcting the wrong reference ( #649 )
...
Co-authored-by: Sebastian Raschka <mail@sebastianraschka.com>
2025-06-12 16:35:51 -05:00
Sebastian Raschka
a3c4c33347
Reduce Llama 3 RoPE memory requirements ( #658 )
...
* Llama3 from scratch improvements
* Fix Llama 3 expensive RoPE memory issue
* updates
* update package
* benchmark
* remove unused rescale_theta
2025-06-12 11:08:02 -05:00
Sebastian Raschka
55e2a0978a
DeBERTa-v3 baseline ( #630 )
...
* Llama3 from scratch improvements
* deberta-baseline
* restore
2025-04-19 21:16:17 -05:00
Sebastian Raschka
02ca4ac42d
BPE cosmetics ( #629 )
...
* Llama3 from scratch improvements
* Cosmetic BPE improvements
* restore
* Update ch02/05_bpe-from-scratch/bpe-from-scratch.ipynb
* Update ch02/05_bpe-from-scratch/bpe-from-scratch.ipynb
* endoftext whitespace
2025-04-18 18:57:09 -05:00
Sebastian Raschka
ec062e1099
Dpo vocab size clarification ( #628 )
...
* Llama3 from scratch improvements
* vocab size should be 50257 not 50256
* restore
2025-04-18 17:20:56 -05:00
Sebastian Raschka
3eca919a52
Llama3 from scratch improvements ( #621 )
...
* Llama3 from scratch improvements
* restore
2025-04-16 18:08:26 -05:00
casinca
1cbdcd86c3
Minor DPO fixes ( #617 )
...
* minor dpo fixes
* Update dpo-from-scratch.ipynb
metadata diff
2025-04-16 12:56:49 -05:00
Daniel Kleine
6ec8fb3dfe
fixed <|endoftext|> token ( #620 )
2025-04-16 12:15:59 -05:00
Henry Shi
88250d953d
updated exercise 5.3 ( #615 )
...
* updated exercise 5.3
temperature can be set to 0 to regardless of top_k setting to force deterministic behavior
* fix notebook json
---------
Co-authored-by: Sebastian Raschka <mail@sebastianraschka.com>
2025-04-13 13:06:57 -05:00
Sebastian Raschka
48e98abc8e
add special token handling to bpe from scratch code ( #616 )
2025-04-13 12:38:22 -05:00
Sebastian Raschka
d5eaa36416
Ch06 and Ch07 videos ( #613 )
...
* Ch06 and Ch07 videos
* exclude google scholar from link checking
2025-04-12 14:51:02 -05:00
PRASHANTH REDDY NIMMAKAYALA
cff41c1fbc
fix: typo in ch07.ipynb ( #612 )
2025-04-12 10:29:53 -05:00
Sebastian Raschka
b662ec9ada
Improve ModernBERT comments ( #606 )
...
* Improve modernbert comments
* bash code formatting
2025-04-06 18:29:22 -05:00
Sebastian Raschka
9e08fff657
align formulas in notes with code ( #605 )
2025-04-06 16:46:53 -05:00
Sebastian Raschka
97a199e40b
Disable mask saving as weight in Llama 3 model ( #604 )
...
* Disable mask saving as weight
* update pixi
* update pixi
2025-04-06 09:33:36 -05:00
Sebastian Raschka
c43d7ef663
reformat nbs ( #602 )
2025-04-05 16:18:27 -05:00
Sebastian Raschka
ab17357474
Correct BERT experiments ( #600 )
2025-04-05 10:05:15 -05:00
Sebastian Raschka
14f976e024
Add ModernBERT ( #598 )
2025-04-05 09:13:30 -05:00
Sebastian Raschka
396e96ab07
Fix Llama language typo in bonus materials ( #597 )
2025-04-02 21:41:36 -05:00
Sebastian Raschka
f61baf86f2
Fix link ( #596 )
2025-04-02 09:47:07 -05:00
Sebastian Raschka
2dc2df593a
Llama3Fast ( #593 )
...
* Llama3Fast
* Update pkg/llms_from_scratch/tests/test_llama3.py
2025-04-01 12:56:11 -05:00
Sebastian Raschka
4128a91c1d
Add Llama 3.2 to pkg ( #591 )
...
* Add Llama 3.2 to pkg
* remove redundant attributes
* update tests
* updates
* updates
* updates
* fix link
* fix link
2025-03-31 18:59:47 -05:00
casinca
d7c316533a
removing unused RoPE parameters ( #590 )
...
* removing unused RoPE parameters
* remove redundant context_length in GQA
---------
Co-authored-by: Sebastian Raschka <mail@sebastianraschka.com>
2025-03-31 17:10:39 -05:00
Sebastian Raschka
d75f74bd0c
Fix data download if UCI is temporarily down ( #592 )
2025-03-31 16:25:53 -05:00
Sebastian Raschka
0bdcce4e40
Clarify dataset length in chapter 2 ( #589 )
2025-03-30 16:01:37 -05:00
Sebastian Raschka
4e3b752e5e
Memory optimized Llama ( #588 )
...
* Memory optimized Llama
* re-ad login
2025-03-30 15:18:12 -05:00
Sebastian Raschka
e55e3e88e1
Alt weight loading code via PyTorch ( #585 )
...
* Alt weight loading code via PyTorch
* commit additional files
2025-03-27 20:10:23 -05:00
Sebastian Raschka
e07a7abdd5
Add GPTModelFast ( #584 )
...
* Add GPTModelFast
* update
2025-03-27 14:00:25 -05:00
Sebastian Raschka
c9271ac427
Adjust comment to save compiled model ( #583 )
2025-03-27 10:43:45 -05:00
Daniel Kleine
d4d420361c
updated .gitignore ( #581 )
2025-03-26 13:21:14 -05:00
Sebastian Raschka
857acfcc12
Vocab padding clarification ( #582 )
...
* vocab padding clarification
* Update ch05/10_llm-training-speed/README.md
2025-03-26 13:19:55 -05:00
Sebastian Raschka
fee7d4bb05
More explicit torchrun usage doc ( #578 )
2025-03-24 12:01:03 -05:00
Sebastian Raschka
cf6fb73553
Add readme ( #577 )
2025-03-23 19:35:12 -05:00
Sebastian Raschka
7114ccd10d
Add PyPI package ( #576 )
...
* Add PyPI package
* fixes
* fixes
2025-03-23 19:28:49 -05:00
Sebastian Raschka
85f2bc0a58
Speed comparison figure ( #575 )
2025-03-21 11:29:49 -05:00
Sebastian Raschka
84591abd5f
add ch04 code along video ( #573 )
2025-03-17 11:20:55 -05:00
Sebastian Raschka
d21d821412
Add chapter 3 coding along video link ( #572 )
2025-03-16 16:07:14 -05:00
Sebastian Raschka
384b9ce959
Better instruction eva prompt ( #571 )
2025-03-15 17:13:15 -05:00
Greg Gandenberger
1ec5631c70
Fix minor printing issue and note inconsistency across platforms ( #563 )
...
* Fix printing issue and note inconsistency
* Rerun notebook
2025-03-14 15:12:09 -05:00
Sebastian Raschka
67244aba8a
Clarification about launching jupyter lab ( #565 )
...
* Clarification about launching jupyter lab
* fix juputer typo
2025-03-14 14:37:22 -05:00
Sebastian Raschka
634a531223
Cosmetic improvements to the BPE code ( #562 )
2025-03-09 10:49:40 -05:00
Sebastian Raschka
6aec412421
Fix BPE bonus materials ( #561 )
...
* Fix BPE bonus materials
* fix bpe implementation
* update
* Add 'Hello, world. Is this-- a test?' test case
* update link to test file
* update path handling
* update path handling
* fix pytest paths
2025-03-08 17:21:30 -06:00
Sebastian Raschka
faadd13871
Update mha plot ( #560 )
2025-03-06 20:29:04 -06:00
Sebastian Raschka
4fb0ea9d1f
Specify UTF-8 encoding in the json load command explicitely ( #557 )
2025-03-05 11:46:21 -06:00
Sebastian Raschka
de60da9a6b
Add a note about "zsh: illegal hardware instruction python" error ( #555 )
2025-03-02 15:18:24 -06:00
Sebastian Raschka
5be0e3cbbd
add link to supplementary ch02 video ( #553 )
2025-03-02 13:17:42 -06:00
