9 Commits

Author SHA1 Message Date
Sebastian Raschka
e742d8af2c
Improve MoE implementation (#841) 2025-09-22 15:21:06 -05:00
casinca
42c130623b
Qwen3Tokenizer fix for Qwen3 Base models and generation mismatch with HF (#828)
* prevent `self.apply_chat_template` being applied for base Qwen models

* - added no chat template comparison in `test_chat_wrap_and_equivalence`
- removed duplicate comparison

* Revert "- added no chat template comparison in `test_chat_wrap_and_equivalence`"

This reverts commit 3a5ee8cfa19aa7e4874cd5f35171098be760b05f.

* Revert "prevent `self.apply_chat_template` being applied for base Qwen models"

This reverts commit df504397a8957886c6d6d808615545e37ceffcad.

* copied `download_file` in `utils` from https://github.com/rasbt/reasoning-from-scratch/blob/main/reasoning_from_scratch/utils.py

* added copy of test `def test_tokenizer_equivalence()` from `reasoning-from-scratch` in `test_qwen3.py`

* removed duplicate code fragment in`test_chat_wrap_and_equivalence`

* use apply_chat_template

* add toggle for instruct model

* Update tokenizer usage

---------

Co-authored-by: rasbt <mail@sebastianraschka.com>
2025-09-17 08:14:11 -05:00
Sebastian Raschka
b6cd0a312f
More efficient angles computation in RoPE (#830) 2025-09-16 03:23:33 +00:00
Sebastian Raschka
8add26cbe9
Improve weight tying handling (#826)
* Improve weight tying handling

* fix
2025-09-14 15:46:48 -05:00
Sebastian Raschka
32965e0edd
remove redundant next_cache (#817) 2025-09-11 15:16:08 -05:00
Sebastian Raschka
efad18bd0b
Fix accidental indentation 2025-09-04 14:41:08 -05:00
Sebastian Raschka
9eee9296d9
Interactive qwen3 chat interface (#801)
* Interactive qwen3 chat interface

* update

* update

* update url
2025-09-01 20:50:25 -05:00
Sebastian Raschka
5febcf8a1b
MoE Nb readability improvements (#761) 2025-08-01 19:58:18 -05:00
Sebastian Raschka
f92b40e4ab
Qwen3 Coder Flash & MoE from Scratch (#760)
* Qwen3 Coder Flash & MoE from Scratch

* update

* refinements

* updates

* update

* update

* update
2025-08-01 19:13:17 -05:00