yujunjun/LLMs-from-scratch
1
0
Fork 0
mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2025-08-29 19:10:19 +00:00
Code Issues Packages Projects Releases Wiki Activity

47,678-->48,725 (#281)

Browse Source
This commit is contained in:
TITC 2024-07-23 10:24:57 +08:00 committed by GitHub
parent 5d8a0c22ab
commit bce3a708f9
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
ch05/01_main-chapter-code/ch05.ipynb
View File

@ -743,7 +743,7 @@
"id": "71ae26dd-d77e-41fd-b924-6bd103dd4ee7", "id": "71ae26dd-d77e-41fd-b924-6bd103dd4ee7",
"metadata": {}, "metadata": {},
"source": [ "source": [
"- The perplexity is often considered more interpretable because it can be understood as the effective vocabulary size that the model is uncertain about at each step (in the example above, that'd be 47,678 words or tokens)\n", "- The perplexity is often considered more interpretable because it can be understood as the effective vocabulary size that the model is uncertain about at each step (in the example above, that'd be 48,725 words or tokens)\n",
"- In other words, perplexity provides a measure of how well the probability distribution predicted by the model matches the actual distribution of the words in the dataset\n", "- In other words, perplexity provides a measure of how well the probability distribution predicted by the model matches the actual distribution of the words in the dataset\n",
"- Similar to the loss, a lower perplexity indicates that the model predictions are closer to the actual distribution" "- Similar to the loss, a lower perplexity indicates that the model predictions are closer to the actual distribution"
] ]