mirror of
				https://github.com/rasbt/LLMs-from-scratch.git
				synced 2025-11-03 19:30:26 +00:00 
			
		
		
		
	Add readme (#577)
This commit is contained in:
		
							parent
							
								
									7114ccd10d
								
							
						
					
					
						commit
						cf6fb73553
					
				@ -16,4 +16,11 @@
 | 
			
		||||
- [07_gpt_to_llama](07_gpt_to_llama) contains a step-by-step guide for converting a GPT architecture implementation to Llama 3.2 and loads pretrained weights from Meta AI
 | 
			
		||||
- [08_memory_efficient_weight_loading](08_memory_efficient_weight_loading) contains a bonus notebook showing how to load model weights via PyTorch's `load_state_dict` method more efficiently
 | 
			
		||||
- [09_extending-tokenizers](09_extending-tokenizers) contains a from-scratch implementation of the GPT-2 BPE tokenizer
 | 
			
		||||
- [10_llm-training-speed](10_llm-training-speed) shows PyTorch performance tips to improve the LLM training speed
 | 
			
		||||
- [10_llm-training-speed](10_llm-training-speed) shows PyTorch performance tips to improve the LLM training speed
 | 
			
		||||
 | 
			
		||||
 | 
			
		||||
 | 
			
		||||
<br>
 | 
			
		||||
<br>
 | 
			
		||||
 | 
			
		||||
[](https://www.youtube.com/watch?v=Zar2TJv-sE0)
 | 
			
		||||
							
								
								
									
										110
									
								
								pkg/llms_from_scratch/README.md
									
									
									
									
									
										Normal file
									
								
							
							
						
						
									
										110
									
								
								pkg/llms_from_scratch/README.md
									
									
									
									
									
										Normal file
									
								
							@ -0,0 +1,110 @@
 | 
			
		||||
# `llms-from-scratch` PyPI Package
 | 
			
		||||
 | 
			
		||||
This optional PyPI package lets you conveniently import code from various chapters of the *Build a Large Language Model From Scratch* book.
 | 
			
		||||
 | 
			
		||||
 
 | 
			
		||||
## Installation
 | 
			
		||||
 | 
			
		||||
 
 | 
			
		||||
### From PyPI
 | 
			
		||||
 | 
			
		||||
Install the `llms-from-scratch` package from the official [Python Package Index](https://pypi.org/project/llms-from-scratch/) (PyPI):
 | 
			
		||||
 | 
			
		||||
```bash
 | 
			
		||||
pip install llms-from-scratch
 | 
			
		||||
```
 | 
			
		||||
 | 
			
		||||
> **Note:** If you're using [`uv`](https://github.com/astral-sh/uv), replace `pip` with `uv pip` or use `uv add`:
 | 
			
		||||
 | 
			
		||||
```bash
 | 
			
		||||
uv add llms-from-scratch
 | 
			
		||||
```
 | 
			
		||||
 | 
			
		||||
 | 
			
		||||
 | 
			
		||||
 
 | 
			
		||||
### Editable Install from GitHub
 | 
			
		||||
 | 
			
		||||
If you'd like to modify the code and have those changes reflected during development:
 | 
			
		||||
 | 
			
		||||
```bash
 | 
			
		||||
git clone https://github.com/rasbt/LLMs-from-scratch.git
 | 
			
		||||
cd LLMs-from-scratch
 | 
			
		||||
pip install -e .
 | 
			
		||||
```
 | 
			
		||||
 | 
			
		||||
> **Note:** With `uv`, use:
 | 
			
		||||
 | 
			
		||||
```bash
 | 
			
		||||
uv add --editable . --dev
 | 
			
		||||
```
 | 
			
		||||
 | 
			
		||||
 | 
			
		||||
 | 
			
		||||
 
 | 
			
		||||
## Using the Package
 | 
			
		||||
 | 
			
		||||
Once installed, you can import code from any chapter using:
 | 
			
		||||
 | 
			
		||||
```python
 | 
			
		||||
from llms_from_scratch.ch02 import GPTDatasetV1, create_dataloader_v1
 | 
			
		||||
 | 
			
		||||
from llms_from_scratch.ch03 import (
 | 
			
		||||
    MultiHeadAttention,
 | 
			
		||||
    SelfAttention_v1,
 | 
			
		||||
    SelfAttention_v2,
 | 
			
		||||
    CausalAttention,
 | 
			
		||||
    MultiHeadAttentionWrapper
 | 
			
		||||
)
 | 
			
		||||
 | 
			
		||||
from llms_from_scratch.ch04 import (
 | 
			
		||||
    LayerNorm,
 | 
			
		||||
    GELU,
 | 
			
		||||
    FeedForward,
 | 
			
		||||
    TransformerBlock,
 | 
			
		||||
    GPTModel,
 | 
			
		||||
    generate_text_simple
 | 
			
		||||
)
 | 
			
		||||
 | 
			
		||||
from llms_from_scratch.ch05 import (
 | 
			
		||||
    generate,
 | 
			
		||||
    train_model_simple,
 | 
			
		||||
    evaluate_model,
 | 
			
		||||
    generate_and_print_sample,
 | 
			
		||||
    assign,
 | 
			
		||||
    load_weights_into_gpt,
 | 
			
		||||
    text_to_token_ids,
 | 
			
		||||
    token_ids_to_text,
 | 
			
		||||
    calc_loss_batch,
 | 
			
		||||
    calc_loss_loader,
 | 
			
		||||
    plot_losses
 | 
			
		||||
)
 | 
			
		||||
 | 
			
		||||
from llms_from_scratch.ch06 import (
 | 
			
		||||
    download_and_unzip_spam_data,
 | 
			
		||||
    create_balanced_dataset,
 | 
			
		||||
    random_split,
 | 
			
		||||
    SpamDataset,
 | 
			
		||||
    calc_accuracy_loader,
 | 
			
		||||
    evaluate_model,
 | 
			
		||||
    train_classifier_simple,
 | 
			
		||||
    plot_values,
 | 
			
		||||
    classify_review
 | 
			
		||||
)
 | 
			
		||||
 | 
			
		||||
from llms_from_scratch.ch07 import (
 | 
			
		||||
    download_and_load_file,
 | 
			
		||||
    format_input,
 | 
			
		||||
    InstructionDataset,
 | 
			
		||||
    custom_collate_fn,
 | 
			
		||||
    check_if_running,
 | 
			
		||||
    query_model,
 | 
			
		||||
    generate_model_scores
 | 
			
		||||
)
 | 
			
		||||
 | 
			
		||||
	
 | 
			
		||||
from llms_from_scratch.appendix_a import NeuralNetwork, ToyDataset
 | 
			
		||||
 | 
			
		||||
from llms_from_scratch.appendix_d import find_highest_gradient, train_model
 | 
			
		||||
```
 | 
			
		||||
 | 
			
		||||
		Loading…
	
	
			
			x
			
			
		
	
		Reference in New Issue
	
	Block a user