3 Commits

Author SHA1 Message Date
Sebastian Raschka
7bd263144e
Switch from urllib to requests to improve reliability (#867)
* Switch from urllib to requests to improve reliability

* Keep ruff linter-specific

* update

* update

* update
2025-10-07 15:22:59 -05:00
casinca
42c130623b
Qwen3Tokenizer fix for Qwen3 Base models and generation mismatch with HF (#828)
* prevent `self.apply_chat_template` being applied for base Qwen models

* - added no chat template comparison in `test_chat_wrap_and_equivalence`
- removed duplicate comparison

* Revert "- added no chat template comparison in `test_chat_wrap_and_equivalence`"

This reverts commit 3a5ee8cfa19aa7e4874cd5f35171098be760b05f.

* Revert "prevent `self.apply_chat_template` being applied for base Qwen models"

This reverts commit df504397a8957886c6d6d808615545e37ceffcad.

* copied `download_file` in `utils` from https://github.com/rasbt/reasoning-from-scratch/blob/main/reasoning_from_scratch/utils.py

* added copy of test `def test_tokenizer_equivalence()` from `reasoning-from-scratch` in `test_qwen3.py`

* removed duplicate code fragment in`test_chat_wrap_and_equivalence`

* use apply_chat_template

* add toggle for instruct model

* Update tokenizer usage

---------

Co-authored-by: rasbt <mail@sebastianraschka.com>
2025-09-17 08:14:11 -05:00
Sebastian Raschka
80d4732456
add HF equivalency tests for standalone nbs (#774)
* add HF equivalency tests for standalone nbs

* update

* update

* update

* update
2025-08-18 18:58:46 -05:00