Sebastian Raschka dcaac28b92 Bonus material: extending tokenizers (#496)
* Bonus material: extending tokenizers

* small wording update
2025-01-22 09:26:54 -06:00

4 lines
201 B
Markdown

# Byte Pair Encoding (BPE) Tokenizer From Scratch
- [bpe-from-scratch.ipynb](bpe-from-scratch.ipynb) contains optional (bonus) code that explains and shows how the BPE tokenizer works under the hood.