diff --git a/README.md b/README.md index 67011d5..8604f1b 100644 --- a/README.md +++ b/README.md @@ -33,9 +33,13 @@ Alternatively, you can view this and other files on GitHub at [https://github.co | ----------------------------------------- | ------------------------------------------------------------ | ---------------------------- | | Ch 1: Understanding Large Language Models | No code | No code | | Ch 2: Working with Text Data | - [ch02.ipynb](ch02/01_main-chapter-code/ch02.ipynb)
- [dataloader.ipynb](ch02/01_main-chapter-code/dataloader.ipynb) (summary)
- [exercise-solutions.ipynb](ch02/01_main-chapter-code/exercise-solutions.ipynb) | [./ch02](./ch02) | -| Ch 3: Coding Attention Mechanisms | - [ch03.ipynb](ch03/01_main-chapter-code/ch03.ipynb)
- [multihead-attention.ipynb](ch03/01_main-chapter-code/multihead-attention.ipynb) (summary) | [./ch03](./ch03) | +| Ch 3: Coding Attention Mechanisms | - [ch03.ipynb](ch03/01_main-chapter-code/ch03.ipynb)
- [multihead-attention.ipynb](ch03/01_main-chapter-code/multihead-attention.ipynb) (summary) | [./ch03](./ch03) | | ... | ... | ... | -| Appendix A: Introduction to PyTorch | - [code-part1.ipynb](appendix-A/03_main-chapter-code/code-part1.ipynb)
- [code-part2.ipynb](appendix-A/03_main-chapter-code/code-part2.ipynb)
- [DDP-script.py](appendix-A/03_main-chapter-code/DDP-script.py)
- [exercise-solutions.ipynb](appendix-A/03_main-chapter-code/exercise-solutions.ipynb) | [./appendix-A](./appendix-A) | +| Appendix A: Introduction to PyTorch* | - [code-part1.ipynb](appendix-A/03_main-chapter-code/code-part1.ipynb)
- [code-part2.ipynb](appendix-A/03_main-chapter-code/code-part2.ipynb)
- [DDP-script.py](appendix-A/03_main-chapter-code/DDP-script.py)
- [exercise-solutions.ipynb](appendix-A/03_main-chapter-code/exercise-solutions.ipynb) | [./appendix-A](./appendix-A) | + +(* Please see [this](01_optional-python-setup-preferences) and [this](02_installing-python-libraries) folder if you need more guidance on installing Python and Python packages.) + +

diff --git a/appendix-A/01_optional-python-setup-preferences/README.md b/appendix-A/01_optional-python-setup-preferences/README.md index 96a63b8..edd7627 100644 --- a/appendix-A/01_optional-python-setup-preferences/README.md +++ b/appendix-A/01_optional-python-setup-preferences/README.md @@ -28,6 +28,8 @@ where `Desktop/` is the folder where the Miniforge installer was downloaded to. Next, step through the download instructions, confirming with "Enter". + + ## 2. Create a new virtual environment After the installation was successfully completed, I recommend creating a new virtual environment called `dl-fundamentals`, which you can do by executing @@ -38,6 +40,8 @@ conda create -n LLMs python=3.10 new-env +> Many scientific computing libraries do not immediately support the newest version of Python. Therefore, when installing PyTorch, it's advisable to use a version of Python that is one or two releases older. For instance, if the latest version of Python is 3.13, using Python 3.10 or 3.11 is recommended. + Next, activate your new virtual environment (you have to do it every time you open a new terminal window or tab): ```bash @@ -52,7 +56,7 @@ If you want to style your terminal similar to mine so that you can see which vir -# 3. Install new Python libraries +## 3. Install new Python libraries @@ -70,6 +74,24 @@ You can also still use `pip` to install libraries. By default, `pip` should be l check-pip + + +## 4. Install PyTorch + +PyTorch can be installed just like any other Python library or package using pip. For example: + +```bash +pip install torch==2.0.1 +``` + +However, since PyTorch is a comprehensive library featuring CPU- and GPU-compatible codes, the installation may require additional settings and explanation (see the *A.1.3 Installing PyTorch in the book for more information*). + +It's also highly recommended to consult the installation guide menu on the official PyTorch website at [https://pytorch.org](https://pytorch.org). + + + + + --- diff --git a/appendix-A/01_optional-python-setup-preferences/figures/pytorch-installer.jpg b/appendix-A/01_optional-python-setup-preferences/figures/pytorch-installer.jpg new file mode 100644 index 0000000..979200b Binary files /dev/null and b/appendix-A/01_optional-python-setup-preferences/figures/pytorch-installer.jpg differ diff --git a/appendix-A/02_installing-python-libraries/README.md b/appendix-A/02_installing-python-libraries/README.md index 207968e..b69a41a 100644 --- a/appendix-A/02_installing-python-libraries/README.md +++ b/appendix-A/02_installing-python-libraries/README.md @@ -1,20 +1,17 @@ -# Libraries Used In This Workshop +# Libraries Used In This Book -We will be using the following libraries in this workshop, and I highly recommend installing them before attending the event: +This document provides more information on double-checking your installed Python version and packages. (Please see the [../01_optional-python-setup-preferences](../01_optional-python-setup-preferences) folder for more information on installing Python and Python packages.) -- numpy >= 1.24.3 (The fundamental package for scientific computing with Python) -- scipy >= 1.10.1 (Additional functions for NumPy) -- pandas >= 2.0.2 (A data frame library) -- matplotlib >= 3.7.1 (A plotting library) -- jupyterlab >= 4.0 (An application for running Jupyter notebooks) -- ipywidgets >= 8.0.6 (Fixes progress bar issues in Jupyter Lab) -- scikit-learn >= 1.2.2 (A general machine learning library) -- watermark >= 2.4.2 (An IPython/Jupyter extension for printing package information) -- torch >= 2.0.1 (The PyTorch deep learning library) -- torchvision >= 0.15.2 (PyTorch utilities for computer vision) -- torchmetrics >= 0.11.4 (Metrics for PyTorch) -- transformers >= 4.30.2 (Language transformers and LLMs for PyTorch) -- lightning >= 2.0.3 (A library for advanced PyTorch features: multi-GPU, mixed-precision etc.) +I used the following main libraries in this book. Newer versions of these libraries are likely compatible as well. However, if you experience any problems with the code, you can try these library versions as a fallback: + +- numpy 1.24.3 +- scipy 1.10.1 +- pandas 2.0.2 +- matplotlib 3.7.1 +- jupyterlab 4.0 +- watermark 2.4.2 +- torch 2.0.1 +- tiktoken 0.5.1 To install these requirements most conveniently, you can use the `requirements.txt` file: @@ -30,17 +27,46 @@ Then, after completing the installation, please check if all the packages are in python_environment_check.py ``` -![check_1](figures/check_1.png) +![check_1](figures/check_1.jpg) -It's also recommended to check the versions in JupyterLab by running the `jupyter_environment_check.ipynb` in this directory. Ideally, it should look like as follows: +It's also recommended to check the versions in JupyterLab by running the `jupyter_environment_check.ipynb` in this directory, which should ideally give you the same results as above. -![check_1](figures/check_2.png) +![check_1](figures/check_2.jpg) If you see the following issues, it's likely that your JupyterLab instance is connected to wrong conda environment: -![jupyter-issues](figures/jupyter-issues.png) +![jupyter-issues](figures/jupyter-issues.jpg) In this case, you may want to use `watermark` to check if you opened the JupyterLab instance in the right conda environment using the `--conda` flag: -![watermark](figures/watermark.png) +![watermark](figures/watermark.jpg) + + + +
+
+ + +## Installing PyTorch + +PyTorch can be installed just like any other Python library or package using pip. For example: + +```bash +pip install torch==2.0.1 +``` + +However, since PyTorch is a comprehensive library featuring CPU- and GPU-compatible codes, the installation may require additional settings and explanation (see the *A.1.3 Installing PyTorch in the book for more information*). + +It's also highly recommended to consult the installation guide menu on the official PyTorch website at [https://pytorch.org](https://pytorch.org). + + + + + +--- + + + + +Any questions? Please feel free to reach out in the [Discussion Forum](https://github.com/rasbt/LLMs-from-scratch/discussions). diff --git a/appendix-A/02_installing-python-libraries/figures/check_1.jpg b/appendix-A/02_installing-python-libraries/figures/check_1.jpg new file mode 100644 index 0000000..98e68e1 Binary files /dev/null and b/appendix-A/02_installing-python-libraries/figures/check_1.jpg differ diff --git a/appendix-A/02_installing-python-libraries/figures/check_1.png b/appendix-A/02_installing-python-libraries/figures/check_1.png deleted file mode 100644 index d3873d7..0000000 Binary files a/appendix-A/02_installing-python-libraries/figures/check_1.png and /dev/null differ diff --git a/appendix-A/02_installing-python-libraries/figures/check_2.jpg b/appendix-A/02_installing-python-libraries/figures/check_2.jpg new file mode 100644 index 0000000..9b96746 Binary files /dev/null and b/appendix-A/02_installing-python-libraries/figures/check_2.jpg differ diff --git a/appendix-A/02_installing-python-libraries/figures/check_2.png b/appendix-A/02_installing-python-libraries/figures/check_2.png deleted file mode 100644 index 88eb7c7..0000000 Binary files a/appendix-A/02_installing-python-libraries/figures/check_2.png and /dev/null differ diff --git a/appendix-A/02_installing-python-libraries/figures/install-requirements.png b/appendix-A/02_installing-python-libraries/figures/install-requirements.png deleted file mode 100644 index 4f079df..0000000 Binary files a/appendix-A/02_installing-python-libraries/figures/install-requirements.png and /dev/null differ diff --git a/appendix-A/02_installing-python-libraries/figures/jupyter-issues.jpg b/appendix-A/02_installing-python-libraries/figures/jupyter-issues.jpg new file mode 100644 index 0000000..5266a6a Binary files /dev/null and b/appendix-A/02_installing-python-libraries/figures/jupyter-issues.jpg differ diff --git a/appendix-A/02_installing-python-libraries/figures/jupyter-issues.png b/appendix-A/02_installing-python-libraries/figures/jupyter-issues.png deleted file mode 100644 index 66d3865..0000000 Binary files a/appendix-A/02_installing-python-libraries/figures/jupyter-issues.png and /dev/null differ diff --git a/appendix-A/02_installing-python-libraries/figures/pytorch-installer.jpg b/appendix-A/02_installing-python-libraries/figures/pytorch-installer.jpg new file mode 100644 index 0000000..979200b Binary files /dev/null and b/appendix-A/02_installing-python-libraries/figures/pytorch-installer.jpg differ diff --git a/appendix-A/02_installing-python-libraries/figures/watermark.jpg b/appendix-A/02_installing-python-libraries/figures/watermark.jpg new file mode 100644 index 0000000..f110ad0 Binary files /dev/null and b/appendix-A/02_installing-python-libraries/figures/watermark.jpg differ diff --git a/appendix-A/02_installing-python-libraries/figures/watermark.png b/appendix-A/02_installing-python-libraries/figures/watermark.png deleted file mode 100644 index 2035f45..0000000 Binary files a/appendix-A/02_installing-python-libraries/figures/watermark.png and /dev/null differ diff --git a/appendix-A/02_installing-python-libraries/jupyter_environment_check.ipynb b/appendix-A/02_installing-python-libraries/jupyter_environment_check.ipynb deleted file mode 100644 index 90b3d79..0000000 --- a/appendix-A/02_installing-python-libraries/jupyter_environment_check.ipynb +++ /dev/null @@ -1,123 +0,0 @@ -{ - "cells": [ - { - "cell_type": "code", - "execution_count": 1, - "id": "18d54544-92d0-412c-8e28-f9083b2bab6f", - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "[OK] Your Python version is 3.10.12\n" - ] - } - ], - "source": [ - "from python_environment_check import check_packages, get_requirements_dict" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "id": "60e03297-4337-4181-b8eb-f483f406954a", - "metadata": {}, - "outputs": [], - "source": [ - "d = get_requirements_dict()" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "id": "d982ddf9-c167-4ed2-9fce-e271f2b1e1de", - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "[OK] numpy 1.25.1\n", - "[OK] scipy 1.11.1\n", - "[OK] pandas 2.0.3\n", - "[OK] matplotlib 3.7.2\n", - "[OK] jupyterlab 4.0.3\n", - "[OK] ipywidgets 8.0.7\n", - "[OK] watermark 2.4.3\n", - "[OK] torch 2.0.1\n" - ] - } - ], - "source": [ - "check_packages(d)" - ] - }, - { - "cell_type": "markdown", - "id": "e0bdd547-333c-42a9-92f3-4e552f206cf3", - "metadata": {}, - "source": [ - "Same checks as above but using watermark:" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "id": "9d696044-9272-4b96-8305-34602807bb94", - "metadata": {}, - "outputs": [], - "source": [ - "%load_ext watermark" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "id": "ce321731-a15a-4579-b33b-035730371eb3", - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "numpy : 1.25.1\n", - "scipy : 1.11.1\n", - "pandas : 2.0.3\n", - "matplotlib: 3.7.2\n", - "sklearn : 1.3.0\n", - "watermark : 2.4.3\n", - "torch : 2.0.1\n", - "\n", - "conda environment: LLMs\n", - "\n" - ] - } - ], - "source": [ - "%watermark --conda -p numpy,scipy,pandas,matplotlib,sklearn,watermark,torch" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3 (ipykernel)", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.10.12" - } - }, - "nbformat": 4, - "nbformat_minor": 5 -} diff --git a/appendix-A/02_installing-python-libraries/python_environment_check.ipynb b/appendix-A/02_installing-python-libraries/python_environment_check.ipynb new file mode 100644 index 0000000..3965b7e --- /dev/null +++ b/appendix-A/02_installing-python-libraries/python_environment_check.ipynb @@ -0,0 +1,62 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": 1, + "id": "67f6f7ed-b67d-465b-bf6f-a99b0d996930", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[OK] Your Python version is 3.11.4\n", + "[OK] numpy 1.25.2\n", + "[OK] scipy 1.11.1\n", + "[OK] pandas 2.0.3\n", + "[OK] matplotlib 3.7.2\n", + "[OK] jupyterlab 4.0.4\n", + "[OK] watermark 2.4.3\n", + "[OK] torch 2.0.1\n", + "[OK] tiktoken 0.5.1\n" + ] + } + ], + "source": [ + "from python_environment_check import check_packages, get_requirements_dict\n", + "\n", + "d = get_requirements_dict()\n", + "check_packages(d)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d5ca05fc-98e0-4bba-a95e-350e1764a12c", + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.4" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/appendix-A/02_installing-python-libraries/python_environment_check.py b/appendix-A/02_installing-python-libraries/python_environment_check.py index 6f681d5..bd8ce71 100644 --- a/appendix-A/02_installing-python-libraries/python_environment_check.py +++ b/appendix-A/02_installing-python-libraries/python_environment_check.py @@ -1,4 +1,4 @@ -# Sebastian Raschka, 2023 +# Sebastian Raschka, 2024 from os.path import dirname, join, realpath from packaging.version import parse as version_parse @@ -25,8 +25,15 @@ def get_packages(pkgs): except AttributeError: try: versions.append(imported.version_info) - except AttributeError: - versions.append('0.0') + except: + try: + import importlib, importlib_metadata + imported = importlib.import_module(p) + version = importlib_metadata.version(p) + versions.append(version) + except ImportError: + version = "not installed" + versions.append('0.0') except ImportError: print(f'[FAIL]: {p} is not installed and/or cannot be imported.') versions.append('N/A') diff --git a/appendix-A/02_installing-python-libraries/requirements.txt b/appendix-A/02_installing-python-libraries/requirements.txt index 41f6fa6..09fc464 100644 --- a/appendix-A/02_installing-python-libraries/requirements.txt +++ b/appendix-A/02_installing-python-libraries/requirements.txt @@ -3,7 +3,6 @@ scipy >= 1.10.1 pandas >= 2.0.2 matplotlib >= 3.7.1 jupyterlab >= 4.0 -ipywidgets >= 8.0.6 watermark >= 2.4.2 torch >= 2.0.1 -tiktoken >= 0.5.1 +tiktoken >= 0.5.1 \ No newline at end of file