Vladimir Blagojevic
|
66f3f42a46
|
fix: Replace multiprocessing tokenization with batched fast tokenization (#3089)
* Replace multiprocessing tokenization with batched fast tokenization
* Replace deprecated tokenization method invocations
|
2022-08-31 07:33:39 -04:00 |
|
Sara Zan
|
4e45062a00
|
Simplify language_modeling.py and tokenization.py (#2703)
* Simplification of language_model.py and tokenization.py to remove code duplication
Co-authored-by: vblagoje <dovlex@gmail.com>
|
2022-07-22 16:29:30 +02:00 |
|
Sara Zan
|
ff4303c51b
|
[CI refactoring] Categorize tests into folders (#2554)
* Categorize tests into folders
* Fix linux_ci.yml and an import
* Wrong path
|
2022-05-17 09:55:53 +01:00 |
|