yujunjun/LLMs-from-scratch
1
0
Fork 0
mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2025-09-03 13:28:18 +00:00
Code Issues Packages Projects Releases Wiki Activity

update

Browse Source
This commit is contained in:
rasbt 2024-06-21 06:31:31 -05:00
parent 87deec0f5f
commit 7b67302da3
2 changed files with 4 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
ch07/01_main-chapter-code/README.md
View File

@ -30,6 +30,7 @@ Validation set length: 55
Test set length: 110 Test set length: 110
-------------------------------------------------- --------------------------------------------------
Device: cpu Device: cpu
--------------------------------------------------
File already exists and is up-to-date: gpt2/355M/checkpoint File already exists and is up-to-date: gpt2/355M/checkpoint
File already exists and is up-to-date: gpt2/355M/encoder.json File already exists and is up-to-date: gpt2/355M/encoder.json
File already exists and is up-to-date: gpt2/355M/hparams.json File already exists and is up-to-date: gpt2/355M/hparams.json
@ -50,7 +51,7 @@ Training completed in 15.66 minutes.
Plot saved as loss-plot-standalone.pdf Plot saved as loss-plot-standalone.pdf
-------------------------------------------------- --------------------------------------------------
Generating responses Generating responses
100%|██████████████████████████████████████████████████████████████████████████| 110/110 [06:57<00:00, 3.80s/it] 100%|█████████████████████████████████████████████████████████| 110/110 [06:57<00:00, 3.80s/it]
Responses saved as instruction-data-with-response-standalone.json Responses saved as instruction-data-with-response-standalone.json
Model saved as gpt2-medium355M-sft-standalone.pth Model saved as gpt2-medium355M-sft-standalone.pth
``` ```

2
ch07/01_main-chapter-code/gpt_instruction_finetuning.py
View File

@ -185,6 +185,8 @@ def main():
tokenizer = tiktoken.get_encoding("gpt2") tokenizer = tiktoken.get_encoding("gpt2")
device = torch.device("cuda" if torch.cuda.is_available() else "cpu") device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
print("Device:", device) print("Device:", device)
print(50*"-")
customized_collate_fn = partial(custom_collate_fn, device=device, allowed_max_length=1024) customized_collate_fn = partial(custom_collate_fn, device=device, allowed_max_length=1024)
num_workers = 0 num_workers = 0