yujunjun/LLMs-from-scratch
1
0
Fork 0
mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2025-10-29 00:39:58 +00:00
Code Issues Packages Projects Releases Wiki Activity
LLMs-from-scratch/ch02/02_bonus_bytepair-encoder
History
Daniel Kleine 9175590ea4 add GPT2TokenizerFast to BPE comparison (#498) 
* added HF BPE Fast

* update benchmarks

* add note about performance

* revert accidental changes

---------

Co-authored-by: rasbt <mail@sebastianraschka.com>
2025-01-22 09:26:44 -06:00
..
gpt2_model
Set up basic test gh worklows (#79)
2024-03-18 11:58:37 -05:00
bpe_openai_gpt2.py
Ch05 supplementary code (#81)
2024-03-19 09:26:26 -05:00
compare-bpe-tiktoken.ipynb
add GPT2TokenizerFast to BPE comparison (#498)
2025-01-22 09:26:44 -06:00
README.md
add readme files
2023-10-25 18:46:40 -05:00
requirements-extra.txt
add requirements file for bonus notebook
2024-03-02 16:54:24 -06:00

README.md

Chapter 2: Working with Text Data