Sebastian Raschka
2dc46bedc6
Fix typo in Ch02 comments ( #516 )
2025-02-04 20:16:07 -06:00
Sebastian Raschka
8cfa52bf1d
More pythonic way to find the longest sequence ( #512 )
...
* More pythonic way to find the longest sequence
* pep8 fix
2025-02-01 10:22:47 -06:00
Sebastian Raschka
0e14c76dee
Test PyTorch nightly releases ( #509 )
2025-01-30 12:45:48 -06:00
Sebastian Raschka
25ea71e713
Alternative weight loading via .safetensors ( #507 )
2025-01-29 08:15:29 -06:00
Sebastian Raschka
9daa7e7511
Fix default argument in ex 7.2 ( #506 )
2025-01-25 10:46:48 -06:00
Sebastian Raschka
fd8d77a79d
A few cosmetic updates ( #504 )
2025-01-23 09:38:55 -06:00
Sebastian Raschka
0911e71497
Test for PyTorch 2.6 release candidate ( #500 )
...
* Test for PyTorch 2.6 release candidate
* update
* update
* remove extra added file
2025-01-22 18:37:48 -06:00
Sebastian Raschka
a22d612be6
Bonus material: extending tokenizers ( #496 )
...
* Bonus material: extending tokenizers
* small wording update
2025-01-22 09:26:54 -06:00
Daniel Kleine
dce46038da
add GPT2TokenizerFast to BPE comparison ( #498 )
...
* added HF BPE Fast
* update benchmarks
* add note about performance
* revert accidental changes
---------
Co-authored-by: rasbt <mail@sebastianraschka.com>
2025-01-22 09:26:44 -06:00
Austin Welch
0f35e370ed
fix: preserve newline tokens in BPE encoder ( #495 )
...
* fix: preserve newline tokens in BPE encoder
* further fixes
* more fixes
---------
Co-authored-by: rasbt <mail@sebastianraschka.com>
2025-01-21 12:47:15 -06:00
Daniel Kleine
60acb94894
BPE: fixed typo ( #492 )
...
* fixed typo
* use rel path if exists
* mod gitignore and use existing vocab files
---------
Co-authored-by: rasbt <mail@sebastianraschka.com>
2025-01-20 20:49:53 -06:00
Sebastian Raschka
0d4967eda6
Implementingthe BPE Tokenizer from Scratch ( #487 )
2025-01-17 12:22:00 -06:00
rvaneijk
2fef2116a6
04_optional-aws-sagemaker-notebook ( #451 )
...
* 04_optional-aws-sagemaker-notebook
* Update setup/04_optional-aws-sagemaker-notebook/cloudformation-template.yml
* Update README.md
---------
Co-authored-by: Sebastian Raschka <mail@sebastianraschka.com>
2025-01-17 10:07:10 -06:00
Sebastian Raschka
126adb7663
Include mathematical breakdown for exercise solution 4.1 ( #483 )
2025-01-14 19:23:00 -06:00
Henry Shi
b3150eebd8
Print out embeddings for more illustrative learning ( #481 )
...
* print out embeddings for illustrative learning
* suggestion print embeddingcontents
---------
Co-authored-by: rasbt <mail@sebastianraschka.com>
2025-01-13 14:44:06 -06:00
rasbt
b524afe3da
fix reward margins plot label in dpo nb
2025-01-12 14:04:05 -06:00
Sebastian Raschka
4bfbcd069d
Auto download DPO dataset if not already available in path ( #479 )
...
* Auto download DPO dataset if not already available in path
* update tests to account for latest HF transformers release in unit tests
* pep 8
2025-01-12 12:27:28 -06:00
Sebastian Raschka
a48f9c7fe2
adds no-grad context for reference model to DPO ( #473 )
2025-01-07 20:49:01 -06:00
Sebastian Raschka
2d7ca7ee4b
fix ch07 unit test ( #470 )
2025-01-05 17:40:57 -06:00
Sebastian Raschka
701090815e
Add backup URL for gpt2 weights ( #469 )
...
* Add backup URL for gpt2 weights
* newline
2025-01-05 11:28:09 -06:00
QS
9b95557ba2
typo fixed ( #468 )
...
* typo fixed
* only update plot
---------
Co-authored-by: rasbt <mail@sebastianraschka.com>
2025-01-05 09:17:13 -06:00
Tao Qian
cec445f146
Minor readability improvement in dataloader.ipynb ( #461 )
...
* Minor readability improvement in dataloader.ipynb
- The tokenizer and encoded_text variables at the root level are unused.
- The default params for create_dataloader_v1 are confusing, especially for the default batch_size 4, which happens to be the same as the max_length.
* readability improvements
---------
Co-authored-by: rasbt <mail@sebastianraschka.com>
2025-01-04 11:26:10 -06:00
Sebastian Raschka
1b635f760e
fix misplaced parenthesis and update license ( #466 )
2025-01-04 11:14:08 -06:00
casinca
bb31de8999
[minor] typo & comments ( #441 )
...
* typo & comment
- safe -> save
- commenting code: batch_size, seq_len = in_idx.shape
* comment
- adding # NEW for assert num_heads % num_kv_groups == 0
* update memory wording
---------
Co-authored-by: rasbt <mail@sebastianraschka.com>
2024-11-18 19:52:42 +09:00
Daniel Kleine
e95c898545
Fixed command for row 16 additional experiment ( #439 )
...
* fixed command for row 16 experiment
* Update README.md
---------
Co-authored-by: Sebastian Raschka <mail@sebastianraschka.com>
2024-11-17 06:50:00 +09:00
Sebastian Raschka
ccade77bf4
Add flexible padding bonus experiment ( #438 )
...
* Add flexible padding bonus experiment
* fix links
2024-11-15 08:51:01 +09:00
Sebastian Raschka
f6281ab91b
Add utility to prevent double execution of certain cells ( #437 )
2024-11-14 19:56:49 +09:00
Sebastian Raschka
f61c008c5d
Add missing device transfer in gpt_generate.py ( #436 )
2024-11-14 19:12:53 +09:00
Sebastian Raschka
27a6a7e64a
Add chapter names
2024-11-08 08:39:34 -06:00
Sebastian Raschka
f4ed263847
Add "What's next" section ( #432 )
...
* Add What's next section
* Delete appendix-D/01_main-chapter-code/appendix-D-Copy2.ipynb
* Delete ch03/01_main-chapter-code/ch03-Copy1.ipynb
* Delete appendix-D/01_main-chapter-code/appendix-D-Copy1.ipynb
* Update ch07.ipynb
* Update ch07.ipynb
2024-11-07 20:12:59 -06:00
rasbt
1183fd7837
add dropout scaling note
2024-11-06 05:52:47 -06:00
casinca
9ce0be333b
potential little fixes appendix-D4 .ipynb ( #427 )
...
* Update appendix-D.ipynb
- lr missing argument for passing peak_lr to the optimizer
- filling 1 step gap for gradient clipping
* adjustments
---------
Co-authored-by: rasbt <mail@sebastianraschka.com>
2024-11-03 12:12:58 -06:00
Sebastian Raschka
ba3137fa2c
Update CITATION.cff
2024-11-01 21:32:17 -05:00
Sebastian Raschka
734f36aac1
Update CITATION.cff
2024-11-01 21:29:22 -05:00
rasbt
7553e87af0
Add citation file
2024-11-01 21:21:25 -05:00
rasbt
f03f545a17
Note about warm-up steps
2024-11-01 16:47:12 -05:00
Daniel Kleine
81eed9afe2
updated RoPE statement ( #423 )
...
* updated RoPE statement
* updated .gitignore
* Update ch05/07_gpt_to_llama/converting-gpt-to-llama2.ipynb
---------
Co-authored-by: Sebastian Raschka <mail@sebastianraschka.com>
2024-10-30 08:00:08 -05:00
Sebastian Raschka
b5f2aa3500
Update README.md
2024-10-29 20:20:48 -05:00
ROHAN WINSOR
cd24a27161
Fix argument name in LlamaTokenizer constructor ( #421 )
...
This PR addresses an oversight in the LlamaTokenizer class by changing the constructor argument from filepath to tokenizer_file.
2024-10-29 18:01:36 -05:00
Daniel Kleine
e8c2f962e9
minor fixes: Llama 3.2 standalone ( #420 )
...
* minor fixes
* reformat rope base as float
---------
Co-authored-by: rasbt <mail@sebastianraschka.com>
2024-10-25 21:08:06 -05:00
Sebastian Raschka
1516de54a5
RoPE theta rescaling ( #419 )
...
* rope fixes
* update
* update
* cleanup
2024-10-25 15:27:23 -05:00
Daniel Kleine
5ff72c2850
fixed typos ( #414 )
...
* fixed typos
* fixed formatting
* Update ch03/02_bonus_efficient-multihead-attention/mha-implementations.ipynb
* del weights after load into model
---------
Co-authored-by: Sebastian Raschka <mail@sebastianraschka.com>
2024-10-24 18:23:53 -05:00
Daniel Kleine
d38083c401
Updated Llama 2 to 3 paths ( #413 )
...
* llama 2 and 3 path fixes
* updated llama 3, 3.1 and 3.2 paths
* updated .gitignore
* Typo fix
---------
Co-authored-by: Sebastian Raschka <mail@sebastianraschka.com>
2024-10-24 07:40:08 -05:00
Sebastian Raschka
e1dfd2cb7a
Update test-requirements-extra.txt
2024-10-23 19:19:58 -05:00
Sebastian Raschka
7cd6a670ed
RoPE updates ( #412 )
...
* RoPE updates
* Apply suggestions from code review
* updates
* updates
* updates
2024-10-23 18:07:49 -05:00
Sebastian Raschka
4f9c9fb703
Update tests.py
2024-10-23 07:48:33 -05:00
Daniel Kleine
ef4018181e
updates for PyTorch 2.5 ( #408 )
...
* updated Dockerfile
* updated MHA implementations for PT 2.5
* fixed typo
* update installation instruction
* Update setup/03_optional-docker-environment/.devcontainer/Dockerfile
---------
Co-authored-by: rasbt <mail@sebastianraschka.com>
2024-10-22 20:23:31 -05:00
Sebastian Raschka
534a704364
RoPE increase ( #407 )
2024-10-21 19:58:38 -05:00
Sebastian Raschka
75133605c5
Set sampler in DDP example
2024-10-21 09:26:01 -05:00
Sebastian Raschka
38969864e6
Add mean pooling experiment to classifier bonus experiments ( #406 )
...
* Add mean pooling experiment to classifier bonus experiments
* formatting
* add average embeddings option
* pep8
2024-10-20 11:04:18 -05:00
