yujunjun/LLMs-from-scratch
1
0
Fork 0
mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2025-08-29 11:00:55 +00:00
Code Issues Packages Projects Releases Wiki Activity
617 Commits 8 Branches 0 Tags
Commit Graph

109 Commits

Author SHA1 Message Date
Sebastian Raschka
6dd8666d9c Test code in pytorch 2.4 (#285) 
* test code in pytorch 2.4

* update
 2024-07-24 21:53:41 -05:00
Sebastian Raschka
d0f3b034d8 Add download help message (#274) 2024-07-19 08:29:29 -05:00
Jeroen Van Goey
70cfced899 fix typos, add codespell pre-commit hook (#264) 
* fix typos, add codespell pre-commit hook

* Update .pre-commit-config.yaml

---------

Co-authored-by: Sebastian Raschka <mail@sebastianraschka.com>
 2024-07-16 07:07:04 -05:00
Sebastian Raschka
2ce1d16de0 show how to use the finetuned model 2024-07-09 06:43:26 -07:00
Daniel Kleine
87f47a281a fixed spelling typos (#258) 2024-07-03 07:47:33 -05:00
Sebastian Raschka
2d8eacb0fa Fix links in summary sections (#254) 2024-06-29 07:51:31 -05:00
rasbt
5e24a042c1 add links to summary sections 2024-06-29 07:33:26 -05:00
Daniel Kleine
fb4e37ae15 fixed minor issues (#252) 
* fixed typo

* fixed var name in md text
 2024-06-29 06:38:25 -05:00
Daniel Kleine
e387742b77 minor markdown fixes (#236) 2024-06-21 13:55:34 -05:00
Sebastian Raschka
87deec0f5f Add standalone finetuning and evaluation scripts for chapter 7 (#234) 
* add finetuning and eval scripts

* update link

* update links

* fix link
 2024-06-21 05:23:24 -05:00
rasbt
c1f9361428 add main and optional sections 2024-06-19 17:48:25 -05:00
Daniel Kleine
73be1c592f fixed num_workers (#229) 
* fixed num_workers

* ch06 & ch07: added num_workers to create_dataloader_v1
 2024-06-19 17:36:46 -05:00
Jinge Wang
8e2c8d0987 Fixed some typos in ch06.ipynb (#219) 2024-06-18 05:54:01 -05:00
rasbt
c8c0fd4fb5 fix spelling 2024-06-18 05:50:40 -05:00
rasbt
88ad21490c replace figure 2024-06-18 05:46:36 -05:00
Daniel Kleine
e5c3c5ce99 minor bug fixes (#207) 
* fixed path arg for create_dataset_csvs()

* updated assign_check() to remove user warning
 2024-06-12 06:27:56 -05:00
rasbt
b2ff989174 distinguish better between main chapter code and bonus materials 2024-06-11 21:07:42 -05:00
Daniel Kleine
79210eb393 fixes for code (#206) 
* updated .gitignore

* removed unused GELU import

* fixed model_configs, fixed all tensors on same device

* removed unused tiktoken

* update

* update hparam search

* remove redundant tokenizer argument

---------

Co-authored-by: rasbt <mail@sebastianraschka.com>
 2024-06-11 20:59:48 -05:00
rasbt
b9ed5811c3 fix gradient comment 2024-06-09 20:23:18 -05:00
rasbt
f0e4c99bc3 fix typo in comment 2024-06-09 06:14:02 -05:00
Sebastian Raschka
40ba3a4068 Remove leftover instances of self.tokenizer (#201) 
* Remove leftover instances of self.tokenizer

* add endoftext token
 2024-06-08 14:57:34 -05:00
Daniel Kleine
e637393056 updated Dockerfile and Additional Classification Finetuning Experiments (#195) 
* accuracy to .2f

* added curl
 2024-06-05 20:17:49 -05:00
Sebastian Raschka
605b4b03d7 Update README.md 2024-05-25 10:45:29 -05:00
rasbt
fe8bb9291e update formatting 2024-05-24 07:20:37 -05:00
rasbt
30ba6a3f4b trainable token -> trainable token position 2024-05-23 11:43:20 -05:00
rasbt
c35cf65dbf add assertion about data set length 2024-05-23 06:50:43 -05:00
rasbt
c4cd48475c Fix device setting 2024-05-22 17:51:51 -05:00
Daniel Kleine
4b0fdab1de removed empty line 2024-05-22 16:15:13 +00:00
Daniel Kleine
a81ba9bd8b fixed last_two_blocks 2024-05-22 02:02:43 +00:00
rasbt
80d857c605 fix table alignment 2024-05-21 19:51:22 -05:00
rasbt
8a27baf7c2 experiment with last two blocks 2024-05-21 19:49:34 -05:00
Daniel Kleine
130a69ce27 improved readability of Additional Experiments table 2024-05-21 19:26:25 +00:00
rasbt
7b9b53c9f2 update lora init 2024-05-19 20:11:56 -05:00
rasbt
3b72e55c26 remove duplicated text 2024-05-19 11:34:47 -05:00
rasbt
bc5cbbf1bd change defaults to 0 temp 2024-05-19 09:04:49 -05:00
rasbt
59f5ed8d68 use default value for temperature 2024-05-19 08:48:10 -05:00
rasbt
faffebae4b add ignore index experiment 2024-05-19 07:24:49 -05:00
rasbt
5541f7c8fe add test mode for dataset download 2024-05-18 17:38:19 -05:00
rasbt
bdea15f6c6 new experiment w/o causal mask 2024-05-18 17:03:36 -05:00
Sebastian Raschka
00a466f0b9 fix row number typo 2024-05-18 15:54:13 -05:00
rasbt
9d84935b69 add eos_id option for ch07 2024-05-18 12:35:40 -05:00
rasbt
10ebc47720 Add experiment with gradient accumulation 2024-05-17 21:31:22 -05:00
rasbt
623bc19665 fix no padding option 2024-05-17 21:06:51 -05:00
rasbt
05738f8be6 fix link 2024-05-17 08:20:35 -05:00
rasbt
f1db50fe9a fix indent 2024-05-17 07:58:01 -05:00
rasbt
2653c36957 Add new experiment without padding 2024-05-17 07:55:51 -05:00
Sebastian Raschka
47b3ff15ec improve bonus code in chapter 06 2024-05-14 20:35:50 -04:00
Sebastian Raschka
30010c7a91 Merge branch 'main' into main 2024-05-14 08:28:02 -05:00
rasbt
6aff47ba60 fix file path name 2024-05-14 08:27:46 -05:00
Sebastian Raschka
2f1e1a3d4b Merge branch 'main' into main 2024-05-14 08:12:19 -05:00