mirror of
https://github.com/rasbt/LLMs-from-scratch.git
synced 2025-11-17 10:34:50 +00:00
42c130623b
* prevent `self.apply_chat_template` being applied for base Qwen models * - added no chat template comparison in `test_chat_wrap_and_equivalence` - removed duplicate comparison * Revert "- added no chat template comparison in `test_chat_wrap_and_equivalence`" This reverts commit 3a5ee8cfa19aa7e4874cd5f35171098be760b05f. * Revert "prevent `self.apply_chat_template` being applied for base Qwen models" This reverts commit df504397a8957886c6d6d808615545e37ceffcad. * copied `download_file` in `utils` from https://github.com/rasbt/reasoning-from-scratch/blob/main/reasoning_from_scratch/utils.py * added copy of test `def test_tokenizer_equivalence()` from `reasoning-from-scratch` in `test_qwen3.py` * removed duplicate code fragment in`test_chat_wrap_and_equivalence` * use apply_chat_template * add toggle for instruct model * Update tokenizer usage --------- Co-authored-by: rasbt <mail@sebastianraschka.com>