Sebastian Raschka
218221ab62
Mixture-of-Experts intro ( #888 )
2025-10-19 22:17:59 -05:00
Sebastian Raschka
27b6dfab9e
Make it easier to toggle between thinking and instruct variants ( #887 )
2025-10-16 20:37:31 -05:00
Sebastian Raschka
b969b3ef7a
Use figure numbers in ch05-7 ( #881 )
2025-10-13 16:26:35 -05:00
rasbt
b8e12e1dd1
Use inference_device
2025-10-09 10:59:17 -05:00
Sebastian Raschka
7bd263144e
Switch from urllib to requests to improve reliability ( #867 )
...
* Switch from urllib to requests to improve reliability
* Keep ruff linter-specific
* update
* update
* update
2025-10-07 15:22:59 -05:00
casinca
00c240ff87
some typo fixes ( #858 )
...
* fix(typo): correct scaling
* fix(typo): correct comment for `instruct`
2025-09-30 11:18:02 -05:00
Sebastian Raschka
47867bc1cb
Update generate script ( #847 )
...
* Custom python 3.13 entry in pyproject.toml
* amend
* Update generate script
* update
* Update pyproject.toml
2025-09-27 08:03:54 -05:00
Sebastian Raschka
9bc827ea7e
Numerically stable generate on mps ( #849 )
...
* Numerically stable generate on mps
* add file
2025-09-26 22:42:44 -05:00
Sebastian Raschka
322000d833
Windows compile ( #845 )
...
* Custom python 3.13 entry in pyproject.toml
* amend
* Note about compile on Windows
* update
2025-09-26 12:01:19 -05:00
Sebastian Raschka
e742d8af2c
Improve MoE implementation ( #841 )
2025-09-22 15:21:06 -05:00
casinca
42c130623b
Qwen3Tokenizer fix for Qwen3 Base models and generation mismatch with HF (#828 )
...
* prevent `self.apply_chat_template` being applied for base Qwen models
* - added no chat template comparison in `test_chat_wrap_and_equivalence`
- removed duplicate comparison
* Revert "- added no chat template comparison in `test_chat_wrap_and_equivalence`"
This reverts commit 3a5ee8cfa19aa7e4874cd5f35171098be760b05f.
* Revert "prevent `self.apply_chat_template` being applied for base Qwen models"
This reverts commit df504397a8957886c6d6d808615545e37ceffcad.
* copied `download_file` in `utils` from https://github.com/rasbt/reasoning-from-scratch/blob/main/reasoning_from_scratch/utils.py
* added copy of test `def test_tokenizer_equivalence()` from `reasoning-from-scratch` in `test_qwen3.py`
* removed duplicate code fragment in`test_chat_wrap_and_equivalence`
* use apply_chat_template
* add toggle for instruct model
* Update tokenizer usage
---------
Co-authored-by: rasbt <mail@sebastianraschka.com>
2025-09-17 08:14:11 -05:00
Sebastian Raschka
b6cd0a312f
More efficient angles computation in RoPE ( #830 )
2025-09-16 03:23:33 +00:00
Sebastian Raschka
8add26cbe9
Improve weight tying handling ( #826 )
...
* Improve weight tying handling
* fix
2025-09-14 15:46:48 -05:00
Sebastian Raschka
32965e0edd
remove redundant next_cache ( #817 )
2025-09-11 15:16:08 -05:00
Sebastian Raschka
c7a4362ca4
Add defensive context trimming for multiturn ( #815 )
...
* Add defensive context trimming for multiturn
* add all mods
2025-09-09 20:19:00 -05:00
Sebastian Raschka
215abdbcdd
Improve multiturn stopping condition ( #814 )
...
* Improve multiturn stopping condition
* improve
2025-09-09 19:37:15 -05:00
Sebastian Raschka
4b0021416a
Clarify Qwen3 notebook purpose ( #812 )
...
* Clarify Qwen3 notebook purpose
* Update README.md
* Update README.md
2025-09-06 15:31:35 -05:00
Sebastian Raschka
efad18bd0b
Fix accidental indentation
2025-09-04 14:41:08 -05:00
Sebastian Raschka
9bfa92fb3e
Update README.md
2025-09-03 12:41:32 -05:00
rasbt
65e67a9681
fix typo
2025-09-02 10:17:51 -05:00
Hayato Hongo
2d8d6224ed
added brief explanations about 2 different ways of RoPE implementations ( #802 )
...
* added brief explanations about 2 different ways of RoPE implementations
* improve comment
---------
Co-authored-by: rasbt <mail@sebastianraschka.com>
2025-09-02 10:14:36 -05:00
rasbt
9ea2c57c5f
simplify
2025-09-01 22:15:47 -05:00
rasbt
643f800a94
remove local config files
2025-09-01 20:52:40 -05:00
Sebastian Raschka
9eee9296d9
Interactive qwen3 chat interface ( #801 )
...
* Interactive qwen3 chat interface
* update
* update
* update url
2025-09-01 20:50:25 -05:00
Sebastian Raschka
d87d91b23c
Add KVCache variant of Qwen3 notebook ( #800 )
...
* Add KVCache variant of Qwen3 notebook
* cleanup
2025-08-31 11:11:12 -05:00
casinca
670f7a4dd0
- added (missing) Gemma3 bullet point in parent folder's readme.md ( #788 )
...
- typo in nbs
2025-08-22 15:03:47 -05:00
Sebastian Raschka
4a84cfccf9
Minor cosmetic fixes in Gemma 3 nbs ( #780 )
2025-08-19 21:08:29 -05:00
Sebastian Raschka
f571b5e493
Add Gemma3 KV cache variant ( #776 )
...
* Add Gemma3 KV cache variant
* update
2025-08-19 12:37:49 -05:00
Sebastian Raschka
80d4732456
add HF equivalency tests for standalone nbs ( #774 )
...
* add HF equivalency tests for standalone nbs
* update
* update
* update
* update
2025-08-18 18:58:46 -05:00
Sebastian Raschka
a6b883c9f9
Gemma 3 270M From Scratch ( #771 )
...
* Gemma 3 270M From Scratch
* fix path
* update readme
2025-08-17 08:23:05 -05:00
Sebastian Raschka
e9c1c1da38
Fix qk_norm comment ( #769 )
2025-08-15 08:38:48 -05:00
Sebastian Raschka
5febcf8a1b
MoE Nb readability improvements ( #761 )
2025-08-01 19:58:18 -05:00
Sebastian Raschka
f92b40e4ab
Qwen3 Coder Flash & MoE from Scratch ( #760 )
...
* Qwen3 Coder Flash & MoE from Scratch
* update
* refinements
* updates
* update
* update
* update
2025-08-01 19:13:17 -05:00
casinca
145322ded8
[Minor] Qwen3 typo & optim ( #758 )
...
* typo
* remove weight dict after loading
2025-07-28 17:29:44 -05:00
Sebastian Raschka
b12dbf6c68
Interleaved Q and K for RoPE in Llama 2 ( #750 )
2025-07-23 08:02:02 -05:00
Sebastian Raschka
13f049f6a4
Minor typo: pply -> Apply ( #749 )
2025-07-22 08:19:25 -05:00
Sebastian Raschka
3233ddc475
get rid of redundant memory profiler import ( #744 )
2025-07-16 07:36:51 -05:00
Sebastian Raschka
a354555049
Batched KV Cache Inference for Qwen3 ( #735 )
2025-07-10 08:09:35 -05:00
Sebastian Raschka
21c41721cc
Add more sophisticated Qwen3 tokenizer ( #729 )
2025-07-09 13:16:26 -05:00
Matthew Hernandez
83c76891fc
Fix issue 724: unused args ( #726 )
...
* Fix issue 724: unused args
* Update 02_opt_multi_gpu_ddp.py
2025-07-08 06:37:39 -05:00
Sebastian Raschka
c4ec55edac
Support different Qwen3 sizes in pkg ( #714 )
2025-06-28 08:00:23 -05:00
Sebastian Raschka
8b3e4b24b0
Remove unused params for hparam script ( #710 )
2025-06-25 12:50:32 -05:00
Sebastian Raschka
190c66b3b0
Add Qwen3 1.7, 4B, 8B, and 32B support to from-scratch nb ( #709 )
2025-06-25 08:53:09 -05:00
Sebastian Raschka
3bdf18a599
Update Llama 3 table for consistency with Qwen3
2025-06-23 18:33:04 -05:00
Sebastian Raschka
81eda38d3b
Improve KV cache code for torch.compile ( #705 )
...
* Improve KV cache code for torch.compile
* cleanup
* cleanup
2025-06-23 18:08:49 -05:00
Sebastian Raschka
37b26c2e04
CPU compile performance for Qwen3 models ( #704 )
...
* Ch06 classifier function asserts
* Qwen3 cpu compilation perf
2025-06-23 11:06:10 -05:00
Sebastian Raschka
d37ddb668a
Fix code comment: embed_dim -> d_out ( #698 )
2025-06-22 16:36:39 -05:00
Sebastian Raschka
01be5a42e4
Use more recent sentencepiece tokenizer API ( #696 )
2025-06-22 13:52:30 -05:00
Sebastian Raschka
bcfdbd7008
Fix some wording issues in the notes ( #695 )
2025-06-22 13:46:16 -05:00
Sebastian Raschka
0a2e8c39c4
Qwen3 KV cache ( #688 )
2025-06-21 17:34:39 -05:00
