rasbt
f03f545a17
Note about warm-up steps
2024-11-01 16:47:12 -05:00
Daniel Kleine
81eed9afe2
updated RoPE statement ( #423 )
...
* updated RoPE statement
* updated .gitignore
* Update ch05/07_gpt_to_llama/converting-gpt-to-llama2.ipynb
---------
Co-authored-by: Sebastian Raschka <mail@sebastianraschka.com>
2024-10-30 08:00:08 -05:00
Sebastian Raschka
b5f2aa3500
Update README.md
2024-10-29 20:20:48 -05:00
ROHAN WINSOR
cd24a27161
Fix argument name in LlamaTokenizer constructor ( #421 )
...
This PR addresses an oversight in the LlamaTokenizer class by changing the constructor argument from filepath to tokenizer_file.
2024-10-29 18:01:36 -05:00
Daniel Kleine
e8c2f962e9
minor fixes: Llama 3.2 standalone ( #420 )
...
* minor fixes
* reformat rope base as float
---------
Co-authored-by: rasbt <mail@sebastianraschka.com>
2024-10-25 21:08:06 -05:00
Sebastian Raschka
1516de54a5
RoPE theta rescaling ( #419 )
...
* rope fixes
* update
* update
* cleanup
2024-10-25 15:27:23 -05:00
Daniel Kleine
5ff72c2850
fixed typos ( #414 )
...
* fixed typos
* fixed formatting
* Update ch03/02_bonus_efficient-multihead-attention/mha-implementations.ipynb
* del weights after load into model
---------
Co-authored-by: Sebastian Raschka <mail@sebastianraschka.com>
2024-10-24 18:23:53 -05:00
Daniel Kleine
d38083c401
Updated Llama 2 to 3 paths ( #413 )
...
* llama 2 and 3 path fixes
* updated llama 3, 3.1 and 3.2 paths
* updated .gitignore
* Typo fix
---------
Co-authored-by: Sebastian Raschka <mail@sebastianraschka.com>
2024-10-24 07:40:08 -05:00
Sebastian Raschka
e1dfd2cb7a
Update test-requirements-extra.txt
2024-10-23 19:19:58 -05:00
Sebastian Raschka
7cd6a670ed
RoPE updates ( #412 )
...
* RoPE updates
* Apply suggestions from code review
* updates
* updates
* updates
2024-10-23 18:07:49 -05:00
Sebastian Raschka
4f9c9fb703
Update tests.py
2024-10-23 07:48:33 -05:00
Daniel Kleine
ef4018181e
updates for PyTorch 2.5 ( #408 )
...
* updated Dockerfile
* updated MHA implementations for PT 2.5
* fixed typo
* update installation instruction
* Update setup/03_optional-docker-environment/.devcontainer/Dockerfile
---------
Co-authored-by: rasbt <mail@sebastianraschka.com>
2024-10-22 20:23:31 -05:00
Sebastian Raschka
534a704364
RoPE increase ( #407 )
2024-10-21 19:58:38 -05:00
Sebastian Raschka
75133605c5
Set sampler in DDP example
2024-10-21 09:26:01 -05:00
Sebastian Raschka
38969864e6
Add mean pooling experiment to classifier bonus experiments ( #406 )
...
* Add mean pooling experiment to classifier bonus experiments
* formatting
* add average embeddings option
* pep8
2024-10-20 11:04:18 -05:00
Sebastian Raschka
467197bbf5
Test PyTorch 2.5 ( #405 )
2024-10-20 10:23:31 -05:00
Sebastian Raschka
1f61aeb7c4
Note about SSL certificates ( #404 )
2024-10-19 16:27:19 -05:00
rasbt
cd2753a36d
update mmap section
2024-10-14 14:27:19 -05:00
rasbt
08362fd290
add mmap=True comparison
2024-10-14 11:09:55 -05:00
Sebastian Raschka
05b04f2a5a
Memory efficient weight loading ( #401 )
...
* memory efficient weight loading
* remove unused code
2024-10-14 10:30:25 -05:00
rasbt
a20ce1b817
remove redundant code line
2024-10-13 15:58:11 -05:00
Sebastian Raschka
b6c4b2f9f1
Update bonus section formatting ( #400 )
2024-10-12 10:26:08 -05:00
Sebastian Raschka
233a3b0c8b
Update check-links.yml
2024-10-11 12:20:49 -05:00
rasbt
93d9dae95f
update card
2024-10-11 12:15:01 -05:00
rasbt
1f4fca9f8e
update reference numbers
2024-10-11 12:13:10 -05:00
Sebastian Raschka
6d0f59a49c
Add MFU formula as reference material ( #395 )
...
* add MFU formula as reference material
* Update previous_chapters.py
2024-10-10 19:42:53 -05:00
Sebastian Raschka
1a8d2929dd
Update check-links.yml
2024-10-08 08:38:48 -05:00
Sebastian Raschka
ec18b6a8a3
Add Llama 3.2 RoPE to CI ( #391 )
...
* add Llama 3.2 RoPE to CI
* update
2024-10-08 08:28:34 -05:00
Sebastian Raschka
1eb0b3810a
Introduce buffers to improve Llama 3.2 efficiency ( #389 )
...
* Introduce buffers to improve Llama 3.2 efficiency
* update
* update
2024-10-06 12:49:04 -05:00
Daniel Kleine
a0c0c765a8
fixed Llama 2 to 3.2 NBs ( #388 )
...
* updated requirements
* fixes llama2 to llama3
* fixed llama 3.2 standalone
* fixed typo
* fixed rope formula
* Update requirements-extra.txt
* Update ch05/07_gpt_to_llama/converting-llama2-to-llama3.ipynb
* Update ch05/07_gpt_to_llama/converting-llama2-to-llama3.ipynb
* Update ch05/07_gpt_to_llama/standalone-llama32.ipynb
---------
Co-authored-by: Sebastian Raschka <mail@sebastianraschka.com>
2024-10-06 09:56:55 -05:00
Sebastian Raschka
0972ded530
Add a note about weight tying in Llama 3.2 ( #386 )
2024-10-05 09:20:54 -05:00
Sebastian Raschka
8a448a4410
Llama 3 ( #384 )
...
* Implement Llama 3.2
* Add Llama 3.2 files
* exclude IMDB link because stanford website seems down
2024-10-05 07:52:15 -05:00
Sebastian Raschka
8553644440
Llama 3.2 requirements file
2024-10-05 07:32:43 -05:00
Sebastian Raschka
b44096acef
Implement Llama 3.2 ( #383 )
2024-10-05 07:30:47 -05:00
Sebastian Raschka
a5405c255d
Cos-sin fix in Llama 2 bonus notebook ( #381 )
2024-10-03 20:45:40 -05:00
Sebastian Raschka
b993c2b25b
Improve rope settings for llama3 ( #380 )
2024-10-03 08:29:54 -05:00
rasbt
278a50a348
add section numbers
2024-09-30 08:42:22 -05:00
Sebastian Raschka
4caafddb93
Improve DDP on Windows ( #376 )
...
* Update DDP-script.py for Windows
* Windows handling
---------
Co-authored-by: Nathan Brown <nathan@nkbrown.us>
2024-09-29 16:53:48 -05:00
rasbt
bfa4215774
llama note
2024-09-26 07:41:11 -05:00
Sebastian Raschka
7ef5129e18
Fix truncation issue in classify_review function ( #373 )
2024-09-25 19:54:36 -05:00
Sebastian Raschka
b56d0b2942
Add llama2 unit tests ( #372 )
...
* add llama2 unit tests
* update
* updates
* updates
* update file path
* update requirements file
* rmsnorm test
* update
2024-09-25 19:40:36 -05:00
rasbt
a6d8e93da3
improve formatting
2024-09-24 18:49:17 -05:00
Daniel Kleine
ff31b345b0
ch05/07 gpt_to_llama text improvements ( #369 )
...
* fixed typo
* fixed RMSnorm formula
* fixed SwiGLU formula
* temperature=0 for untrained model for reproducibility
* added extra info hf token
2024-09-24 18:45:49 -05:00
rasbt
d144bd5b7a
add json import
2024-09-23 09:12:35 -05:00
rasbt
6bc3de165c
move access token to config.json
2024-09-23 08:56:16 -05:00
rasbt
58df945ed4
add llama3 comparison
2024-09-23 08:17:10 -05:00
Sebastian Raschka
0467c8289b
GPT to Llama ( #368 )
...
* GPT to Llama
* fix urls
2024-09-23 07:34:06 -05:00
rasbt
dc1b1a05b0
note about random numbers
2024-09-22 12:02:03 -05:00
Daniel Kleine
5f36d2af4c
removed unnecessary imports ( #367 )
2024-09-22 11:59:37 -05:00
Sebastian Raschka
76e9a9ec02
Add user interface to ch06 and ch07 ( #366 )
...
* Add user interface to ch06 and ch07
* pep8
* fix url
2024-09-21 20:33:00 -05:00
