Sebastian Raschka
2af686d70b
Add KV cache ( #671 )
2025-06-15 09:58:08 -05:00
Sebastian Raschka
78bbcb3643
Remove redundant model = ( #663 )
2025-06-13 11:03:55 -05:00
Sebastian Raschka
3dfd7e5f06
Update pixi ( #661 )
...
* Llama3 from scratch improvements
* Update HF hub version in pixi.toml
* Update README.md
2025-06-13 10:50:17 -05:00
Daniel Kleine
c2cfb47b1a
fixed gqa qkv code comments ( #660 )
2025-06-13 08:21:28 -05:00
Greg Gandenberger
7632eb018b
Update ch07.ipynb ( #643 )
...
Correct function name
2025-06-13 08:17:10 -05:00
Shimpei Kojio
baaa6c9283
fixed video link ( #646 )
2025-06-13 08:16:18 -05:00
Pratyush Subhadarshi
d56417c34c
Correcting the wrong reference ( #649 )
...
Co-authored-by: Sebastian Raschka <mail@sebastianraschka.com>
2025-06-12 16:35:51 -05:00
Sebastian Raschka
c4cde1c21b
Reduce Llama 3 RoPE memory requirements ( #658 )
...
* Llama3 from scratch improvements
* Fix Llama 3 expensive RoPE memory issue
* updates
* update package
* benchmark
* remove unused rescale_theta
2025-06-12 11:08:02 -05:00
Sebastian Raschka
c278745aff
DeBERTa-v3 baseline ( #630 )
...
* Llama3 from scratch improvements
* deberta-baseline
* restore
2025-04-19 21:16:17 -05:00
Sebastian Raschka
4ff743051e
BPE cosmetics ( #629 )
...
* Llama3 from scratch improvements
* Cosmetic BPE improvements
* restore
* Update ch02/05_bpe-from-scratch/bpe-from-scratch.ipynb
* Update ch02/05_bpe-from-scratch/bpe-from-scratch.ipynb
* endoftext whitespace
2025-04-18 18:57:09 -05:00
Sebastian Raschka
adaf4faaae
Dpo vocab size clarification ( #628 )
...
* Llama3 from scratch improvements
* vocab size should be 50257 not 50256
* restore
2025-04-18 17:20:56 -05:00
Sebastian Raschka
47c036058d
Llama3 from scratch improvements ( #621 )
...
* Llama3 from scratch improvements
* restore
2025-04-16 18:08:26 -05:00
casinca
1b242d01a5
Minor DPO fixes ( #617 )
...
* minor dpo fixes
* Update dpo-from-scratch.ipynb
metadata diff
2025-04-16 12:56:49 -05:00
Daniel Kleine
f3d1566c2e
fixed <|endoftext|> token ( #620 )
2025-04-16 12:15:59 -05:00
Henry Shi
02779f5e35
updated exercise 5.3 ( #615 )
...
* updated exercise 5.3
temperature can be set to 0 to regardless of top_k setting to force deterministic behavior
* fix notebook json
---------
Co-authored-by: Sebastian Raschka <mail@sebastianraschka.com>
2025-04-13 13:06:57 -05:00
Sebastian Raschka
72efebd7f8
add special token handling to bpe from scratch code ( #616 )
2025-04-13 12:38:22 -05:00
Sebastian Raschka
92b308e512
Ch06 and Ch07 videos ( #613 )
...
* Ch06 and Ch07 videos
* exclude google scholar from link checking
2025-04-12 14:51:02 -05:00
PRASHANTH REDDY NIMMAKAYALA
74b9f1fcde
fix: typo in ch07.ipynb ( #612 )
2025-04-12 10:29:53 -05:00
Sebastian Raschka
9df572fdf4
Improve ModernBERT comments ( #606 )
...
* Improve modernbert comments
* bash code formatting
2025-04-06 18:29:22 -05:00
Sebastian Raschka
3654571184
align formulas in notes with code ( #605 )
2025-04-06 16:46:53 -05:00
Sebastian Raschka
67e0680210
Disable mask saving as weight in Llama 3 model ( #604 )
...
* Disable mask saving as weight
* update pixi
* update pixi
2025-04-06 09:33:36 -05:00
Sebastian Raschka
f1434652f2
reformat nbs ( #602 )
2025-04-05 16:18:27 -05:00
Sebastian Raschka
371ab9e8ff
Correct BERT experiments ( #600 )
2025-04-05 10:05:15 -05:00
Sebastian Raschka
4a9654173c
Add ModernBERT ( #598 )
2025-04-05 09:13:30 -05:00
Sebastian Raschka
d4c8d8f2c9
Fix Llama language typo in bonus materials ( #597 )
2025-04-02 21:41:36 -05:00
Sebastian Raschka
49330d0990
Fix link ( #596 )
2025-04-02 09:47:07 -05:00
Sebastian Raschka
43e25a5165
Llama3Fast ( #593 )
...
* Llama3Fast
* Update pkg/llms_from_scratch/tests/test_llama3.py
2025-04-01 12:56:11 -05:00
Sebastian Raschka
aedad7efc3
Add Llama 3.2 to pkg ( #591 )
...
* Add Llama 3.2 to pkg
* remove redundant attributes
* update tests
* updates
* updates
* updates
* fix link
* fix link
2025-03-31 18:59:47 -05:00
casinca
152a087a37
removing unused RoPE parameters ( #590 )
...
* removing unused RoPE parameters
* remove redundant context_length in GQA
---------
Co-authored-by: Sebastian Raschka <mail@sebastianraschka.com>
2025-03-31 17:10:39 -05:00
Sebastian Raschka
222803737d
Fix data download if UCI is temporarily down ( #592 )
2025-03-31 16:25:53 -05:00
Sebastian Raschka
6ea4dd3ae7
Clarify dataset length in chapter 2 ( #589 )
2025-03-30 16:01:37 -05:00
Sebastian Raschka
0f6894f41e
Memory optimized Llama ( #588 )
...
* Memory optimized Llama
* re-ad login
2025-03-30 15:18:12 -05:00
Sebastian Raschka
3f93d73d6d
Alt weight loading code via PyTorch ( #585 )
...
* Alt weight loading code via PyTorch
* commit additional files
2025-03-27 20:10:23 -05:00
Sebastian Raschka
ffd4035144
Add GPTModelFast ( #584 )
...
* Add GPTModelFast
* update
2025-03-27 14:00:25 -05:00
Sebastian Raschka
2e143f17b8
Adjust comment to save compiled model ( #583 )
2025-03-27 10:43:45 -05:00
Daniel Kleine
f01e163aad
updated .gitignore ( #581 )
2025-03-26 13:21:14 -05:00
Sebastian Raschka
92f1313e00
Vocab padding clarification ( #582 )
...
* vocab padding clarification
* Update ch05/10_llm-training-speed/README.md
2025-03-26 13:19:55 -05:00
Sebastian Raschka
b7893457da
More explicit torchrun usage doc ( #578 )
2025-03-24 12:01:03 -05:00
Sebastian Raschka
feb1e9a83d
Add readme ( #577 )
2025-03-23 19:35:12 -05:00
Sebastian Raschka
c21bfe4a23
Add PyPI package ( #576 )
...
* Add PyPI package
* fixes
* fixes
2025-03-23 19:28:49 -05:00
Sebastian Raschka
7757c3d308
Speed comparison figure ( #575 )
2025-03-21 11:29:49 -05:00
Sebastian Raschka
73f4342664
add ch04 code along video ( #573 )
2025-03-17 11:20:55 -05:00
Sebastian Raschka
4db0e826b7
Add chapter 3 coding along video link ( #572 )
2025-03-16 16:07:14 -05:00
Sebastian Raschka
54474fb452
Better instruction eva prompt ( #571 )
2025-03-15 17:13:15 -05:00
Greg Gandenberger
c1611d4ea8
Fix minor printing issue and note inconsistency across platforms ( #563 )
...
* Fix printing issue and note inconsistency
* Rerun notebook
2025-03-14 15:12:09 -05:00
Sebastian Raschka
8d01d3b0c1
Clarification about launching jupyter lab ( #565 )
...
* Clarification about launching jupyter lab
* fix juputer typo
2025-03-14 14:37:22 -05:00
Sebastian Raschka
2f41429cf4
Cosmetic improvements to the BPE code ( #562 )
2025-03-09 10:49:40 -05:00
Sebastian Raschka
f63f04d8d5
Fix BPE bonus materials ( #561 )
...
* Fix BPE bonus materials
* fix bpe implementation
* update
* Add 'Hello, world. Is this-- a test?' test case
* update link to test file
* update path handling
* update path handling
* fix pytest paths
2025-03-08 17:21:30 -06:00
Sebastian Raschka
96ca2fcb2f
Update mha plot ( #560 )
2025-03-06 20:29:04 -06:00
Sebastian Raschka
86b714a5e0
Specify UTF-8 encoding in the json load command explicitely ( #557 )
2025-03-05 11:46:21 -06:00
