yujunjun/LLMs-from-scratch
1
0
Fork 0
mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2025-08-16 12:41:42 +00:00
Code Issues Packages Projects Releases Wiki Activity

add additional lora figure

Browse Source
This commit is contained in:
rasbt 2024-05-07 07:04:35 -05:00
parent c93c90eb1e
commit d18f92fa34
1 changed files with 5 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
appendix-E/01_main-chapter-code/appendix-E.ipynb
View File

@ -615,7 +615,9 @@
"id": "4D21Jk7Vw3nG" "id": "4D21Jk7Vw3nG"
}, },
"source": [ "source": [
"- To try LoRA on the GPT model we defined earlier, we define a `replace_linear_with_lora` function to replace all `Linear` layers in the model with the new `LinearWithLoRA` layers" "- To try LoRA on the GPT model we defined earlier, we define a `replace_linear_with_lora` function to replace all `Linear` layers in the model with the new `LinearWithLoRA` layers\n",
"\n",
"<img src=\"https://sebastianraschka.com/images/LLMs-from-scratch-images/appendix-e_compressed/lora-4.webp\" width=\"400px\">"
] ]
}, },
{ {
@ -642,7 +644,8 @@
"id": "8c172164-cdde-4489-b7d7-aaed9cc2f5f2", "id": "8c172164-cdde-4489-b7d7-aaed9cc2f5f2",
"metadata": {}, "metadata": {},
"source": [ "source": [
"- We then freeze the original model parameter and use the `replace_linear_with_lora` to replace the said `Linear` layers below" "- We then freeze the original model parameter and use the `replace_linear_with_lora` to replace the said `Linear` layers using the code below\n",
"- This will replace the `Linear` layers in the "
] ]
}, },
{ {