yujunjun/LLMs-from-scratch
1
0
Fork 0
mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2025-12-25 06:02:07 +00:00
Code Issues Packages Projects Releases Wiki Activity

formatting

Browse Source
This commit is contained in:
rasbt 2024-07-27 09:51:24 -05:00
parent 2b7bd46a93
commit 4a0a8cd9e4
1 changed files with 2 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
ch07/04_preference-tuning-with-dpo/create-preference-data-ollama.ipynb
View File

@ -40,9 +40,7 @@
" 1. We use the instruction-finetuned LLM to generate multiple responses and have humans rank them based on their preference and/or given preference criteria\n",
" 2. We use the instruction-finetuned LLM to generate multiple responses and have LLMs rank them based on given preference criteria\n",
" 3. We use an LLM to generate preferred and dispreferred responses given certain preference criteria\n",
"- In this notebook, we take approach 3\n",
"\n",
"\n",
"- In this notebook, we consider approach 3\n",
"- This notebook uses a 70 billion parameter Llama 3.1-Instruct model through ollama to generate preference labels for an instruction dataset\n",
"- The expected format of the instruction dataset is as follows:\n",
"\n",
@ -589,7 +587,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.4"
"version": "3.10.6"
}
},
"nbformat": 4,