From d391796ec2f8347313599c2095a6c2125b0b0a04 Mon Sep 17 00:00:00 2001 From: Sebastian Raschka Date: Thu, 29 Aug 2024 09:09:10 +0200 Subject: [PATCH] use nbviewer links (#339) --- README.md | 52 ++++++++++++++++++++++++++-------------------------- 1 file changed, 26 insertions(+), 26 deletions(-) diff --git a/README.md b/README.md index 0b357ea..53cd1f0 100644 --- a/README.md +++ b/README.md @@ -65,17 +65,17 @@ You can alternatively view this and other files on GitHub at [https://github.com |------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------|-------------------------------| | [Setup recommendations](setup) | - | - | | Ch 1: Understanding Large Language Models | No code | - | -| Ch 2: Working with Text Data | - [ch02.ipynb](ch02/01_main-chapter-code/ch02.ipynb)
- [dataloader.ipynb](ch02/01_main-chapter-code/dataloader.ipynb) (summary)
- [exercise-solutions.ipynb](ch02/01_main-chapter-code/exercise-solutions.ipynb) | [./ch02](./ch02) | -| Ch 3: Coding Attention Mechanisms | - [ch03.ipynb](ch03/01_main-chapter-code/ch03.ipynb)
- [multihead-attention.ipynb](ch03/01_main-chapter-code/multihead-attention.ipynb) (summary)
- [exercise-solutions.ipynb](ch03/01_main-chapter-code/exercise-solutions.ipynb)| [./ch03](./ch03) | -| Ch 4: Implementing a GPT Model from Scratch | - [ch04.ipynb](ch04/01_main-chapter-code/ch04.ipynb)
- [gpt.py](ch04/01_main-chapter-code/gpt.py) (summary)
- [exercise-solutions.ipynb](ch04/01_main-chapter-code/exercise-solutions.ipynb) | [./ch04](./ch04) | -| Ch 5: Pretraining on Unlabeled Data | - [ch05.ipynb](ch05/01_main-chapter-code/ch05.ipynb)
- [gpt_train.py](ch05/01_main-chapter-code/gpt_train.py) (summary)
- [gpt_generate.py](ch05/01_main-chapter-code/gpt_generate.py) (summary)
- [exercise-solutions.ipynb](ch05/01_main-chapter-code/exercise-solutions.ipynb) | [./ch05](./ch05) | -| Ch 6: Finetuning for Text Classification | - [ch06.ipynb](ch06/01_main-chapter-code/ch06.ipynb)
- [gpt_class_finetune.py](ch06/01_main-chapter-code/gpt_class_finetune.py)
- [exercise-solutions.ipynb](ch06/01_main-chapter-code/exercise-solutions.ipynb) | [./ch06](./ch06) | -| Ch 7: Finetuning to Follow Instructions | - [ch07.ipynb](ch07/01_main-chapter-code/ch07.ipynb)
- [gpt_instruction_finetuning.py](ch07/01_main-chapter-code/gpt_instruction_finetuning.py) (summary)
- [ollama_evaluate.py](ch07/01_main-chapter-code/ollama_evaluate.py) (summary)
- [exercise-solutions.ipynb](ch07/01_main-chapter-code/exercise-solutions.ipynb) | [./ch07](./ch07) | -| Appendix A: Introduction to PyTorch | - [code-part1.ipynb](appendix-A/01_main-chapter-code/code-part1.ipynb)
- [code-part2.ipynb](appendix-A/01_main-chapter-code/code-part2.ipynb)
- [DDP-script.py](appendix-A/01_main-chapter-code/DDP-script.py)
- [exercise-solutions.ipynb](appendix-A/01_main-chapter-code/exercise-solutions.ipynb) | [./appendix-A](./appendix-A) | +| Ch 2: Working with Text Data | - [ch02.ipynb](https://nbviewer.org/github/rasbt/LLMs-from-scratch/blob/main/ch02/01_main-chapter-code/ch02.ipynb)
- [dataloader.ipynb](https://nbviewer.org/github/rasbt/LLMs-from-scratch/blob/main/ch02/01_main-chapter-code/dataloader.ipynb) (summary)
- [exercise-solutions.ipynb](https://nbviewer.org/github/rasbt/LLMs-from-scratch/blob/main/ch02/01_main-chapter-code/exercise-solutions.ipynb) | [./ch02](./ch02) | +| Ch 3: Coding Attention Mechanisms | - [ch03.ipynb](https://nbviewer.org/github/rasbt/LLMs-from-scratch/blob/main/ch03/01_main-chapter-code/ch03.ipynb)
- [multihead-attention.ipynb](https://nbviewer.org/github/rasbt/LLMs-from-scratch/blob/main/ch03/01_main-chapter-code/multihead-attention.ipynb) (summary)
- [exercise-solutions.ipynb](https://nbviewer.org/github/rasbt/LLMs-from-scratch/blob/main/ch03/01_main-chapter-code/exercise-solutions.ipynb)| [./ch03](./ch03) | +| Ch 4: Implementing a GPT Model from Scratch | - [ch04.ipynb](https://nbviewer.org/github/rasbt/LLMs-from-scratch/blob/main/ch04/01_main-chapter-code/ch04.ipynb)
- [gpt.py](https://nbviewer.org/github/rasbt/LLMs-from-scratch/blob/main/ch04/01_main-chapter-code/gpt.py) (summary)
- [exercise-solutions.ipynb](https://nbviewer.org/github/rasbt/LLMs-from-scratch/blob/main/ch04/01_main-chapter-code/exercise-solutions.ipynb) | [./ch04](./ch04) | +| Ch 5: Pretraining on Unlabeled Data | - [ch05.ipynb](https://nbviewer.org/github/rasbt/LLMs-from-scratch/blob/main/ch05/01_main-chapter-code/ch05.ipynb)
- [gpt_train.py](https://nbviewer.org/github/rasbt/LLMs-from-scratch/blob/main/ch05/01_main-chapter-code/gpt_train.py) (summary)
- [gpt_generate.py](https://nbviewer.org/github/rasbt/LLMs-from-scratch/blob/main/ch05/01_main-chapter-code/gpt_generate.py) (summary)
- [exercise-solutions.ipynb](https://nbviewer.org/github/rasbt/LLMs-from-scratch/blob/main/ch05/01_main-chapter-code/exercise-solutions.ipynb) | [./ch05](./ch05) | +| Ch 6: Finetuning for Text Classification | - [ch06.ipynb](https://nbviewer.org/github/rasbt/LLMs-from-scratch/blob/main/ch06/01_main-chapter-code/ch06.ipynb)
- [gpt_class_finetune.py](https://nbviewer.org/github/rasbt/LLMs-from-scratch/blob/main/ch06/01_main-chapter-code/gpt_class_finetune.py)
- [exercise-solutions.ipynb](https://nbviewer.org/github/rasbt/LLMs-from-scratch/blob/main/ch06/01_main-chapter-code/exercise-solutions.ipynb) | [./ch06](./ch06) | +| Ch 7: Finetuning to Follow Instructions | - [ch07.ipynb](https://nbviewer.org/github/rasbt/LLMs-from-scratch/blob/main/ch07/01_main-chapter-code/ch07.ipynb)
- [gpt_instruction_finetuning.py](https://nbviewer.org/github/rasbt/LLMs-from-scratch/blob/main/ch07/01_main-chapter-code/gpt_instruction_finetuning.py) (summary)
- [ollama_evaluate.py](https://nbviewer.org/github/rasbt/LLMs-from-scratch/blob/main/ch07/01_main-chapter-code/ollama_evaluate.py) (summary)
- [exercise-solutions.ipynb](https://nbviewer.org/github/rasbt/LLMs-from-scratch/blob/main/ch07/01_main-chapter-code/exercise-solutions.ipynb) | [./ch07](./ch07) | +| Appendix A: Introduction to PyTorch | - [code-part1.ipynb](https://nbviewer.org/github/rasbt/LLMs-from-scratch/blob/main/appendix-A/01_main-chapter-code/code-part1.ipynb)
- [code-part2.ipynb](https://nbviewer.org/github/rasbt/LLMs-from-scratch/blob/main/appendix-A/01_main-chapter-code/code-part2.ipynb)
- [DDP-script.py](https://nbviewer.org/github/rasbt/LLMs-from-scratch/blob/main/appendix-A/01_main-chapter-code/DDP-script.py)
- [exercise-solutions.ipynb](https://nbviewer.org/github/rasbt/LLMs-from-scratch/blob/main/appendix-A/01_main-chapter-code/exercise-solutions.ipynb) | [./appendix-A](./appendix-A) | | Appendix B: References and Further Reading | No code | - | | Appendix C: Exercise Solutions | No code | - | -| Appendix D: Adding Bells and Whistles to the Training Loop | - [appendix-D.ipynb](appendix-D/01_main-chapter-code/appendix-D.ipynb) | [./appendix-D](./appendix-D) | -| Appendix E: Parameter-efficient Finetuning with LoRA | - [appendix-E.ipynb](appendix-E/01_main-chapter-code/appendix-E.ipynb) | [./appendix-E](./appendix-E) | +| Appendix D: Adding Bells and Whistles to the Training Loop | - [appendix-D.ipynb](https://nbviewer.org/github/rasbt/LLMs-from-scratch/blob/main/appendix-D/01_main-chapter-code/appendix-D.ipynb) | [./appendix-D](./appendix-D) | +| Appendix E: Parameter-efficient Finetuning with LoRA | - [appendix-E.ipynb](https://nbviewer.org/github/rasbt/LLMs-from-scratch/blob/main/appendix-E/01_main-chapter-code/appendix-E.ipynb) | [./appendix-E](./appendix-E) |
@@ -102,28 +102,28 @@ Several folders contain optional materials as a bonus for interested readers: - [Installing Python Packages and Libraries Used In This Book](setup/02_installing-python-libraries) - [Docker Environment Setup Guide](setup/03_optional-docker-environment) - **Chapter 2:** - - [Comparing Various Byte Pair Encoding (BPE) Implementations](ch02/02_bonus_bytepair-encoder) - - [Understanding the Difference Between Embedding Layers and Linear Layers](ch02/03_bonus_embedding-vs-matmul) - - [Dataloader Intuition with Simple Numbers](ch02/04_bonus_dataloader-intuition) + - [Comparing Various Byte Pair Encoding (BPE) Implementations](https://nbviewer.org/github/rasbt/LLMs-from-scratch/blob/main/ch02/02_bonus_bytepair-encoder) + - [Understanding the Difference Between Embedding Layers and Linear Layers](https://nbviewer.org/github/rasbt/LLMs-from-scratch/blob/main/ch02/03_bonus_embedding-vs-matmul) + - [Dataloader Intuition with Simple Numbers](https://nbviewer.org/github/rasbt/LLMs-from-scratch/blob/main/ch02/04_bonus_dataloader-intuition) - **Chapter 3:** - - [Comparing Efficient Multi-Head Attention Implementations](ch03/02_bonus_efficient-multihead-attention/mha-implementations.ipynb) - - [Understanding PyTorch Buffers](ch03/03_understanding-buffers/understanding-buffers.ipynb) + - [Comparing Efficient Multi-Head Attention Implementations](https://nbviewer.org/github/rasbt/LLMs-from-scratch/blob/main/ch03/02_bonus_efficient-multihead-attention/mha-implementations.ipynb) + - [Understanding PyTorch Buffers](https://nbviewer.org/github/rasbt/LLMs-from-scratch/blob/main/ch03/03_understanding-buffers/understanding-buffers.ipynb) - **Chapter 4:** - - [FLOPS Analysis](ch04/02_performance-analysis/flops-analysis.ipynb) + - [FLOPS Analysis](https://nbviewer.org/github/rasbt/LLMs-from-scratch/blob/main/ch04/02_performance-analysis/flops-analysis.ipynb) - **Chapter 5:** - - [Alternative Weight Loading from Hugging Face Model Hub using Transformers](ch05/02_alternative_weight_loading/weight-loading-hf-transformers.ipynb) - - [Pretraining GPT on the Project Gutenberg Dataset](ch05/03_bonus_pretraining_on_gutenberg) - - [Adding Bells and Whistles to the Training Loop](ch05/04_learning_rate_schedulers) - - [Optimizing Hyperparameters for Pretraining](ch05/05_bonus_hparam_tuning) + - [Alternative Weight Loading from Hugging Face Model Hub using Transformers](https://nbviewer.org/github/rasbt/LLMs-from-scratch/blob/main/ch05/02_alternative_weight_loading/weight-loading-hf-transformers.ipynb) + - [Pretraining GPT on the Project Gutenberg Dataset](https://nbviewer.org/github/rasbt/LLMs-from-scratch/blob/main/ch05/03_bonus_pretraining_on_gutenberg) + - [Adding Bells and Whistles to the Training Loop](https://nbviewer.org/github/rasbt/LLMs-from-scratch/blob/main/ch05/04_learning_rate_schedulers) + - [Optimizing Hyperparameters for Pretraining](https://nbviewer.org/github/rasbt/LLMs-from-scratch/blob/main/ch05/05_bonus_hparam_tuning) - **Chapter 6:** - - [Additional experiments finetuning different layers and using larger models](ch06/02_bonus_additional-experiments) - - [Finetuning different models on 50k IMDB movie review dataset](ch06/03_bonus_imdb-classification) + - [Additional experiments finetuning different layers and using larger models](https://nbviewer.org/github/rasbt/LLMs-from-scratch/blob/main/ch06/02_bonus_additional-experiments) + - [Finetuning different models on 50k IMDB movie review dataset](https://nbviewer.org/github/rasbt/LLMs-from-scratch/blob/main/ch06/03_bonus_imdb-classification) - **Chapter 7:** - - [Dataset Utilities for Finding Near Duplicates and Creating Passive Voice Entries](ch07/02_dataset-utilities) - - [Evaluating Instruction Responses Using the OpenAI API and Ollama](ch07/03_model-evaluation) - - [Generating a Dataset for Instruction Finetuning](ch07/05_dataset-generation) - - [Generating a Preference Dataset with Llama 3.1 70B and Ollama](ch07/04_preference-tuning-with-dpo/create-preference-data-ollama.ipynb) - - [Direct Preference Optimization (DPO) for LLM Alignment](ch07/04_preference-tuning-with-dpo/dpo-from-scratch.ipynb) + - [Dataset Utilities for Finding Near Duplicates and Creating Passive Voice Entries](https://nbviewer.org/github/rasbt/LLMs-from-scratch/blob/main/ch07/02_dataset-utilities) + - [Evaluating Instruction Responses Using the OpenAI API and Ollama](https://nbviewer.org/github/rasbt/LLMs-from-scratch/blob/main/ch07/03_model-evaluation) + - [Generating a Dataset for Instruction Finetuning](https://nbviewer.org/github/rasbt/LLMs-from-scratch/blob/main/ch07/05_dataset-generation) + - [Generating a Preference Dataset with Llama 3.1 70B and Ollama](https://nbviewer.org/github/rasbt/LLMs-from-scratch/blob/main/ch07/04_preference-tuning-with-dpo/create-preference-data-ollama.ipynb) + - [Direct Preference Optimization (DPO) for LLM Alignment](https://nbviewer.org/github/rasbt/LLMs-from-scratch/blob/main/ch07/04_preference-tuning-with-dpo/dpo-from-scratch.ipynb)