Native uv docs (#530)

* Replace pip by more modern uv

* uv tests

* Native uv docs

* resolve merge conflicts

* resolve merge conflicts
This commit is contained in:
Sebastian Raschka 2025-02-15 20:35:23 -06:00 committed by GitHub
parent fac61004b0
commit aa60bb3cd5
4 changed files with 215 additions and 4 deletions

2
.gitignore vendored
View File

@ -244,6 +244,8 @@ celerybeat.pid
# Environments
.env
.venv
.python-version
uv.lock
env/
venv/
ENV/

29
pyproject.toml Normal file
View File

@ -0,0 +1,29 @@
[project]
name = "llms-from-scratch"
version = "0.1.0"
description = "mplement a ChatGPT-like LLM in PyTorch from scratch, step by step"
readme = "README.md"
requires-python = ">=3.10"
dependencies = [
"torch>=2.3.0",
"jupyterlab>=4.0",
"tiktoken>=0.5.1",
"matplotlib>=3.7.1",
"tensorflow>=2.18.0",
"tqdm>=4.66.1",
"numpy>=1.26,<2.1",
"pandas>=2.2.1",
"psutil>=5.9.5",
"packaging>=24.2",
]
[tool.setuptools.packages]
find = {}
[tool.uv.sources]
llms-from-scratch = { workspace = true }
[dependency-groups]
dev = [
"llms-from-scratch",
]

View File

@ -8,20 +8,29 @@ I have been a long-time user of [Conda](https://anaconda.org/anaconda/conda) and
I recommend starting with *Option 1: Using uv* as it is the more modern approach in 2025. If you encounter problems with *Option 1*, consider *Option 2: Using Conda*.
In this tutorial, I am using a computer running macOS, but this workflow is similar for Linux machines and may work for other operating systems as well.
&nbsp;
# Option 1: Using uv
This section guides you through the Python setup and package installation procedure using `uv` via its `uv pip` interface. The `uv pip` interface may feel more familiar to most Python users who have used pip before than the native `uv` commands.
&nbsp;
This section guides you through the Python setup and package installation procedure using `uv`.
In this tutorial, I am using a computer running macOS, but this workflow is similar for Linux machines and may work for other operating systems as well.
> [!NOTE]
> There are alternative ways to install Python and use `uv`. For example, you can install Python directly via `uv` and use `uv add` instead of `uv pip install` for faster package management.
>
> If you prefer the native `uv` commands, refer to the [./native-uv.md tutorial](./native-uv.md). I also recommend checking the official [`uv` documentation](https://docs.astral.sh/uv/).
>
> While `uv add` offers speed advantages, I find `uv pip` slightly more user-friendly, making it a good starting point for beginners. However, if you're new to Python package management, the native `uv` interface is also a great way to learn.
&nbsp;
## 1. Install Python (if not installed)
First, check if you have a modern version of Python installed (I recommend 3.10 or newer) by executing the following code in the terminal:
```bash
@ -29,6 +38,7 @@ python --version
```
If it returns 3.10 or newer, no further action is required.
&nbsp;
> [!NOTE]
> I recommend installing a Python version that is at least 2 versions older than the most recent release to ensure PyTorch compatibility. For example, if the most recent version is Python 3.13, I recommend installing version 3.10 or 3.11.

View File

@ -0,0 +1,170 @@
# Native uv Python and package management
This tutorial is an alternative to *Option 1: Using uv* in the [README.md](./README.md) document for those who prefer `uv`'s native commands over the `uv pip` interface. While `uv pip` is faster than pure `pip`, `uv`'s native interface is even faster than `uv pip` as it has less overhead and doesn't have to handle legacy support for PyPy package dependency management.
Otherwise, similar to *Option 1: Using uv* in the [README.md](./README.md) , this section guides you through the Python setup and package installation procedure using `uv`.
In this tutorial, I am using a computer running macOS, but this workflow is similar for Linux machines and may work for other operating systems as well.
&nbsp;
## 1. Install uv
Uv can be installed as follows, depending on your operating system.
&nbsp;
**macOS and Linux**
```bash
curl -LsSf https://astral.sh/uv/install.sh | sh
```
or
```bash
wget -qO- https://astral.sh/uv/install.sh | sh
```
&nbsp;
**Windows**
```bash
powershell -c "irm https://astral.sh/uv/install.ps1 | more"
```
&nbsp;
> [!NOTE]
> For more installation options, please refer to the official [uv documentation](https://docs.astral.sh/uv/getting-started/installation/#standalone-installer).
&nbsp;
## 2. Install Python
You can install Python using uv:
```bash
uv python install 3.10
```
&nbsp;
> [!NOTE]
> I recommend installing a Python version that is at least 2 versions older than the most recent release to ensure PyTorch compatibility. For example, if the most recent version is Python 3.13, I recommend installing version 3.10 or 3.11. You can find out the most recent Python version by visiting [python.org](https://www.python.org/downloads/).
&nbsp;
## 3. Install Python packages and dependencies
To install all required packages from a `pyproject.toml` file (such as the one located at the top level of this GitHub repository), run the following command, assuming the file is in the same directory as your terminal session:
```bash
uv add . --dev
```
<img src="https://sebastianraschka.com/images/LLMs-from-scratch-images/setup/uv-setup/uv-add.png?1" width="700" height="auto" alt="Uv install">
Note that the `uv add` command above will create a separate virtual environment via the `.venv` subfolder.
You can install new packages, that are not specified in the `pyproject.toml` via `uv add`, for example:
```bash
uv add packaging
```
&nbsp;
## Optional: Manage virtual environments manually
Alternatively, you can still install the dependencies directly from the repository using `uv pip install`. Note that this requires creating and activating the virtual environment manually:
&nbsp;
**1. Create a new virtual environment**
Run the following command to manually create a new virtual environment, which will be saved via a new `.venv` subfolder:
```bash
uv venv --python=python3.10
```
&nbsp;
**2. Activate virtual environment**
Next, we need to activate this new virtual environment.
On macOS/Linux:
```bash
source .venv/bin/activate
```
On Windows (PowerShell):
```bash
.venv\Scripts\activate
```
&nbsp;
**3. Install dependencies**
Finally, we can install dependencies from a remote location using the `uv pip` interface:
```bash
uv pip install -U -r https://raw.githubusercontent.com/rasbt/LLMs-from-scratch/refs/heads/main/requirements.txt
```
&nbsp;
## 4. Run Python code
&nbsp;
**Finalizing the setup**
Your environment should now be ready to run the code in the repository.
Optionally, you can run an environment check by executing the `python_environment_check.py` script in this repository:
```bash
uv run python setup/02_installing-python-libraries/python_environment_check.py
```
<img src="https://sebastianraschka.com/images/LLMs-from-scratch-images/setup/uv-setup/uv-run-check.png?1" width="700" height="auto" alt="Uv install">
Or, if you don't want to type `uv run python` ever time you execute code, manually activate the virtual environment first.
On macOS/Linux:
```bash
source .venv/bin/activate
```
On Windows (PowerShell):
```bash
.venv\Scripts\activate
```
Then, run:
```bash
python setup/02_installing-python-libraries/python_environment_check.py
```
&nbsp;
**Launching JupyterLab**
You can launch a JupyterLab instance via:
```bash
uv run jupyter lab
```
Or, if you manually activated the environment as described earlier, you can drop the `uv run` prefix.
&nbsp;
---
Any questions? Please feel free to reach out in the [Discussion Forum](https://github.com/rasbt/LLMs-from-scratch/discussions).