yujunjun/LLMs-from-scratch
1
0
Fork 0
mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2025-08-30 03:20:51 +00:00
Code Issues Packages Projects Releases Wiki Activity
LLMs-from-scratch/ch02/05_bpe-from-scratch
History
Matthew Hernandez 6f12edb0cc
Fix issue: 731 by resolving semantic error (#738) 
* fix issue 731

* update test path

---------

Co-authored-by: rasbt <mail@sebastianraschka.com>
2025-07-10 16:30:15 -05:00
..
bpe-from-scratch.ipynb
Fix issue: 731 by resolving semantic error (#738)
2025-07-10 16:30:15 -05:00
README.md
Bonus material: extending tokenizers (#496)
2025-01-22 09:26:54 -06:00
tests.py
Fix issue: 731 by resolving semantic error (#738)
2025-07-10 16:30:15 -05:00

README.md

Byte Pair Encoding (BPE) Tokenizer From Scratch

  • bpe-from-scratch.ipynb contains optional (bonus) code that explains and shows how the BPE tokenizer works under the hood.