LLMs-from-scratch/appendix-D/01_main-chapter-code/appendix-D.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "af53bcb1-ff9d-49c7-a0bc-5b8d32ff975b",
   "metadata": {},
   "source": [
    "## Appendix D: Adding Bells and Whistles to the Training Loop"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4f58c142-9434-49af-b33a-356b80a45b86",
   "metadata": {},
   "source": [
    "- In this appendix, we add a few more advanced features to the training function, which are used in typical pretraining and finetuning; finetuning is covered in chapters 6 and 7\n",
    "- The next three sections below discuss learning rate warmup, cosine decay, and gradient clipping\n",
    "- The final section adds these techniques to the training function"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "744def4f-c03f-42ee-97bb-5d7d5b89b723",
   "metadata": {},
   "source": [
    "- We start by initializing a model reusing the code from chapter 5:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "8755bd5e-bc06-4e6e-9e63-c7c82b816cbe",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch version: 2.2.1\n",
      "tiktoken version: 0.5.1\n"
     ]
    }
   ],
   "source": [
    "from importlib.metadata import version\n",
    "import torch\n",
    "import tiktoken\n",
    "\n",
    "print(\"torch version:\", version(\"torch\"))\n",
    "print(\"tiktoken version:\", version(\"tiktoken\"))\n",
    "\n",
    "\n",
    "from previous_chapters import GPTModel\n",
    "\n",
    "GPT_CONFIG_124M = {\n",
    "    \"vocab_size\": 50257,  # Vocabulary size\n",
    "    \"ctx_len\": 256,       # Shortened context length (orig: 1024)\n",
    "    \"emb_dim\": 768,       # Embedding dimension\n",
    "    \"n_heads\": 12,        # Number of attention heads\n",
    "    \"n_layers\": 12,       # Number of layers\n",
    "    \"drop_rate\": 0.1,     # Dropout rate\n",
    "    \"qkv_bias\": False     # Query-key-value bias\n",
    "}\n",
    "\n",
    "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
    "\n",
    "torch.manual_seed(123)\n",
    "model = GPTModel(GPT_CONFIG_124M)\n",
    "model.eval();  # Disable dropout during inference"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "51574e57-a098-412c-83e8-66dafa5a0b99",
   "metadata": {},
   "source": [
    "- Next, using the same code we used in chapter 5, we initialize the data loaders:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "386ca110-2bb4-42f1-bd54-8836df80acaa",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import urllib.request\n",
    "\n",
    "file_path = \"the-verdict.txt\"\n",
    "url = \"https://raw.githubusercontent.com/rasbt/LLMs-from-scratch/main/ch02/01_main-chapter-code/the-verdict.txt\"\n",
    "\n",
    "if not os.path.exists(file_path):\n",
    "    with urllib.request.urlopen(url) as response:\n",
    "        text_data = response.read().decode('utf-8')\n",
    "    with open(file_path, \"w\", encoding=\"utf-8\") as file:\n",
    "        file.write(text_data)\n",
    "else:\n",
    "    with open(file_path, \"r\", encoding=\"utf-8\") as file:\n",
    "        text_data = file.read()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "ae96992b-536a-4684-a924-658b9ffb7e9c",
   "metadata": {},
   "outputs": [],
   "source": [
    "from previous_chapters import create_dataloader_v1\n",
    "\n",
    "# Train/validation ratio\n",
    "train_ratio = 0.90\n",
    "split_idx = int(train_ratio * len(text_data))\n",
    "\n",
    "\n",
    "torch.manual_seed(123)\n",
    "\n",
    "train_loader = create_dataloader_v1(\n",
    "    text_data[:split_idx],\n",
    "    batch_size=2,\n",
    "    max_length=GPT_CONFIG_124M[\"ctx_len\"],\n",
    "    stride=GPT_CONFIG_124M[\"ctx_len\"],\n",
    "    drop_last=True,\n",
    "    shuffle=True\n",
    ")\n",
    "\n",
    "val_loader = create_dataloader_v1(\n",
    "    text_data[split_idx:],\n",
    "    batch_size=2,\n",
    "    max_length=GPT_CONFIG_124M[\"ctx_len\"],\n",
    "    stride=GPT_CONFIG_124M[\"ctx_len\"],\n",
    "    drop_last=False,\n",
    "    shuffle=False\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "939c08d8-257a-41c6-b842-019f7897ac74",
   "metadata": {},
   "source": [
    "## D.1 Learning rate warmup"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7fafcd30-ddf7-4a9f-bcf4-b13c052b3133",
   "metadata": {},
   "source": [
    "- When training complex models like LLMs, implementing learning rate warmup can help stabilize the training\n",
    "- In learning rate warmup, we gradually increase the learning rate from a very low value (`initial_lr`) to a user-specified maximum (`peak_lr`)\n",
    "- This way, the model will start the training with small weight updates, which helps decrease the risk of large destabilizing updates during the training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "2bb4790b-b8b6-4e9e-adf4-704a04b31ddf",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "135\n"
     ]
    }
   ],
   "source": [
    "n_epochs = 15\n",
    "peak_lr = 0.01\n",
    "initial_lr = 0.0001\n",
    "\n",
    "optimizer = torch.optim.AdamW(model.parameters(), lr=peak_lr, weight_decay=0.1)\n",
    "total_training_steps = len(train_loader) * n_epochs\n",
    "\n",
    "print(total_training_steps)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5bf3a8da-abc4-4b80-a5d8-f1cc1c7cc5f3",
   "metadata": {},
   "source": [
    "- Typically, the number of warmup steps is between 10% and 20% of the total number of steps\n",
    "- We can compute the increment as the difference between the `peak_lr` and `initial_lr` divided by the number of warmup steps"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "e075f80e-a398-4809-be1d-8019e1d31c90",
   "metadata": {},
   "outputs": [],
   "source": [
    "warmup_steps = 20\n",
    "lr_increment = (peak_lr - initial_lr) / warmup_steps\n",
    "\n",
    "global_step = -1\n",
    "track_lrs = []\n",
    "\n",
    "for epoch in range(n_epochs):\n",
    "    for input_batch, target_batch in train_loader:\n",
    "        optimizer.zero_grad()\n",
    "        global_step += 1\n",
    "    \n",
    "        if global_step < warmup_steps:\n",
    "            lr = initial_lr + global_step * lr_increment\n",
    "        else:\n",
    "            lr = peak_lr\n",
    "        \n",
    "        # Apply the calculated learning rate to the optimizer\n",
    "        for param_group in optimizer.param_groups:\n",
    "            param_group[\"lr\"] = lr\n",
    "        track_lrs.append(optimizer.param_groups[0][\"lr\"])\n",
    "    \n",
    "        # Calculate loss and update weights"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "cb6da121-eeed-4023-bdd8-3666c594b4ed",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlMAAAGwCAYAAACNeeBZAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA9s0lEQVR4nO3dfVyV9eH/8fdB5cYb8G6BCCoVm5UoCYqY+7omRWUryjV0ftPM6dqsNCynplCtxrIsZ1nkfn2zfZfTuZWV32I5rKxJqIgKmTctCxUP3g1QSu7O9fvDnSNnonHkHK5z83o+HjyM63zO4XN9VvLedb3P51gMwzAEAACAixJk9gQAAAB8GWEKAACgDQhTAAAAbUCYAgAAaAPCFAAAQBsQpgAAANqAMAUAANAGHc2egK+y2WyqqKhQt27dZLFYzJ4OAABoBcMwdPLkSUVHRysoyD3XlAhTF6miokKxsbFmTwMAAFyEAwcOKCYmxi2vRZi6SN26dZN05n+M8PBwk2cDAABao6amRrGxsY7f4+5AmLpI9lt74eHhhCkAAHyMOys6FNABAADagDAFAADQBoQpAACANiBMAQAAtAFhCgAAoA0IUwAAAG1AmAIAAGgDwhQAAEAbEKYAAADagDAFAADQBqaHqWXLlmnAgAEKDQ1VSkqKNm/efMHxa9as0cCBAxUaGqqEhAS98847To+//vrruv7669WrVy9ZLBZt3779nNc4ffq0ZsyYoV69eqlr164aN26cKisr3XlaAAAgQJgaplavXq2srCzl5ORo27ZtGjJkiNLT03XkyJEWx2/atEkTJkzQ1KlTVVJSooyMDGVkZKisrMwxpra2VqNGjdKTTz553p/7wAMP6O2339aaNWv04YcfqqKiQrfffrvbzw8AAPg/i2EYhlk/PCUlRcOGDdPzzz8vSbLZbIqNjdV9992nuXPnnjM+MzNTtbW1WrdunePYiBEjlJiYqLy8PKexX375peLi4lRSUqLExETH8erqan3nO9/RypUr9eMf/1iStHv3bl1xxRUqLCzUiBEjWjX3mpoaRUREqLq6mg869qDqrxt0sq7B7GkAAEzQu2uIQjt1cOtreuL3d0e3vMpFqK+vV3FxsebNm+c4FhQUpLS0NBUWFrb4nMLCQmVlZTkdS09P19q1a1v9c4uLi9XQ0KC0tDTHsYEDB6pfv34XDFN1dXWqq6tzfF9TU9Pqn4mLs+mfx/Tf/69INtPiPgDATH+4e7j+67vfMXsa38q0MHXs2DE1NTUpMjLS6XhkZKR2797d4nOsVmuL461Wa6t/rtVqVXBwsLp37+7S6+Tm5urRRx9t9c9B2/191xHZDKlDkEUdgyxmTwcA0M6CLL7xd79pYcrXzJs3z+mqWE1NjWJjY02ckf8rO1QtSVo0brDGJcWYPBsAAFpmWpjq3bu3OnTocM676CorKxUVFdXic6Kiolwaf77XqK+vV1VVldPVqW97nZCQEIWEhLT656BtmmyGPq04E6YSYiJMng0AAOdn2rv5goODlZSUpIKCAscxm82mgoICpaamtvic1NRUp/GStH79+vOOb0lSUpI6derk9Dp79uxReXm5S68Dz9p/7JRq65sU1qmDLvtOV7OnAwDAeZl6my8rK0uTJ09WcnKyhg8friVLlqi2tlZTpkyRJE2aNEl9+/ZVbm6uJGnmzJkaPXq0Fi9erLFjx2rVqlXaunWrli9f7njNEydOqLy8XBUVFZLOBCXpzBWpqKgoRUREaOrUqcrKylLPnj0VHh6u++67T6mpqa1+Jx88r/Tft/iujA5XB/pSAAAvZmqYyszM1NGjR5WdnS2r1arExETl5+c7Subl5eUKCjp78WzkyJFauXKlFixYoPnz5ys+Pl5r167VoEGDHGPeeustRxiTpPHjx0uScnJy9Mgjj0iSnn32WQUFBWncuHGqq6tTenq6XnjhhXY4Y7RW6cEz75ZM6MstPgCAdzN1nylfxj5TnnVH3iZt+fJfWnzHEMrnAAC38cTvb9M/Tgb4T2fK5/++MkX5HADg5QhT8Dr7j53S15TPAQA+gjAFr2Mvn19F+RwA4AMIU/A6Ow+eCVODKJ8DAHwAYQpex77zOe/kAwD4AsIUvErz8vlgyucAAB9AmIJX+eLomfJ55+AOupTyOQDABxCm4FUcO5/3oXwOAPANhCl4FXuYonwOAPAVhCl4FXv5nL4UAMBXEKbgNZpshsoO8Zl8AADfQpiC1/ji6Cl900D5HADgWwhT8BrsfA4A8EWEKXgNyucAAF9EmILXKD3IzucAAN9DmIJXaL7zOWEKAOBLCFPwCpTPAQC+ijAFr7DzIOVzAIBvIkzBK1A+BwD4KsIUvIJ953P6UgAAX0OYgumal8/5GBkAgK8hTMF0/2xWPo/rTfkcAOBbCFMwXSnlcwCADyNMwXSljr5Ud3MnAgDARSBMwXSO8nlMuMkzAQDAdYQpmIqdzwEAvo4wBVNRPgcA+DrCFExlL58Pio6gfA4A8EmEKZiKnc8BAL6OMAVTlVI+BwD4OMIUTNNkM7SL8jkAwMcRpmAae/m8C+VzAIAPI0zBNDsdO59TPgcA+C7CFExTRvkcAOAHCFMwjb18PjiGMAUA8F2EKZiiscmmTyu4MgUA8H2EKZjin0drdbrBpi7BHXRp7y5mTwcAgItGmIIp7Lf4roqOUBDlcwCADyNMwRRljs06ucUHAPBthCmYYufBKkls1gkA8H2EKbS7xiabdh0+s/M55XMAgK8jTKHdUT4HAPgTwhTanaN83pfyOQDA9xGm0O5K6UsBAPwIYQrtzn5lijAFAPAHhCm0q+blc7ZFAAD4A8IU2tXnR0/pdINNXUM6Kq4X5XMAgO8jTKFdlR48c4vvyuhwyucAAL9AmEK7KqMvBQDwM4QptCt7+XwwfSkAgJ8gTKHdsPM5AMAfEabQbiifAwD8EWEK7cZePr+K8jkAwI8QptBuKJ8DAPwRYQrtZqc9TFE+BwD4EcIU2kVjk02fUT4HAPghwhTaBeVzAIC/Mj1MLVu2TAMGDFBoaKhSUlK0efPmC45fs2aNBg4cqNDQUCUkJOidd95xetwwDGVnZ6tPnz4KCwtTWlqa9u3b5zRm7969uvXWW9W7d2+Fh4dr1KhRev/9991+bjhrJ+VzAICfMjVMrV69WllZWcrJydG2bds0ZMgQpaen68iRIy2O37RpkyZMmKCpU6eqpKREGRkZysjIUFlZmWPMokWLtHTpUuXl5amoqEhdunRRenq6Tp8+7Rhz8803q7GxURs2bFBxcbGGDBmim2++WVar1ePnHKgonwMA/JXFMAzDrB+ekpKiYcOG6fnnn5ck2Ww2xcbG6r777tPcuXPPGZ+Zmana2lqtW7fOcWzEiBFKTExUXl6eDMNQdHS0Zs+erQcffFCSVF1drcjISK1YsULjx4/XsWPH9J3vfEcbN27U97//fUnSyZMnFR4ervXr1ystLa1Vc6+pqVFERISqq6sVHh7e1qXwe7e98A+VlFfpd+MTdWtiX7OnAwAIUJ74/W3alan6+noVFxc7hZegoCClpaWpsLCwxecUFhaeE3bS09Md4/fv3y+r1eo0JiIiQikpKY4xvXr10ve+9z394Q9/UG1trRobG/XSSy/pkksuUVJS0nnnW1dXp5qaGqcvtE5jk027Ks6sF1emAAD+xrQwdezYMTU1NSkyMtLpeGRk5Hlvt1mt1guOt/95oTEWi0V///vfVVJSom7duik0NFTPPPOM8vPz1aNHj/PONzc3VxEREY6v2NhY1044gO07ckp1jWfK5wMonwMA/IzpBfT2ZhiGZsyYoUsuuUQfffSRNm/erIyMDP3oRz/S4cOHz/u8efPmqbq62vF14MCBdpy1b7N/uDHlcwCAPzItTPXu3VsdOnRQZWWl0/HKykpFRUW1+JyoqKgLjrf/eaExGzZs0Lp167Rq1Spdc801Gjp0qF544QWFhYXp1VdfPe98Q0JCFB4e7vSF1rGXzwezWScAwA+ZFqaCg4OVlJSkgoICxzGbzaaCggKlpqa2+JzU1FSn8ZK0fv16x/i4uDhFRUU5jampqVFRUZFjzNdffy3pTD+ruaCgINlstrafGM5h3xaBzToBAP6oo5k/PCsrS5MnT1ZycrKGDx+uJUuWqLa2VlOmTJEkTZo0SX379lVubq4kaebMmRo9erQWL16ssWPHatWqVdq6dauWL18u6UwfatasWXr88ccVHx+vuLg4LVy4UNHR0crIyJB0JpD16NFDkydPVnZ2tsLCwvT73/9e+/fv19ixY01ZB3/WfOdzyucAAH9kapjKzMzU0aNHlZ2dLavVqsTEROXn5zsK5OXl5U5XkEaOHKmVK1dqwYIFmj9/vuLj47V27VoNGjTIMWbOnDmqra3V9OnTVVVVpVGjRik/P1+hoaGSztxezM/P18MPP6wf/vCHamho0FVXXaU333xTQ4YMad8FCACUzwEA/s7UfaZ8GftMtc6ftx7QnL/s1IhLe2rV9JZv3wIA0F78ap8pBIbSg+x8DgDwb4QpeJR9WwTK5wAAf0WYgsc0L58Pjulu7mQAAPAQwhQ8xl4+7xbSUf17djZ7OgAAeARhCh5j70td1ZedzwEA/oswBY+x96UonwMA/BlhCh7jCFP0pQAAfowwBY9oaLJpFzufAwACAGEKHrGv8pTqKZ8DAAIAYQoeUXaI8jkAIDAQpuAR9r4U+0sBAPwdYQoesZOdzwEAAYIwBbdraLbzOeVzAIC/I0zB7Rzl81DK5wAA/0eYgtvZy+eDoiMonwMA/B5hCm6381CVJCkhhlt8AAD/R5iC25UeOtOXonwOAAgEhCm4VfPy+WDCFAAgABCm4FZ7K0+eLZ/3onwOAPB/hCm4VfPyucVC+RwA4P8IU3Ar+87nlM8BAIGCMAW3spfP2awTABAoCFNwG3Y+BwAEIsIU3IbyOQAgEBGm4Db28nlCX8rnAIDAQZiC2+w8eDZMAQAQKAhTcBvHtgiEKQBAACFMwS0ammz6zHpSElemAACBhTAFt6B8DgAIVIQpuEXpQcrnAIDARJiCW5QeonwOAAhMhCm4RRkfIwMACFCEKbRZfSPlcwBA4CJMoc3s5fPw0I7q15PyOQAgsBCm0GbN95eifA4ACDSEKbRZKX0pAEAAI0yhzXgnHwAgkBGm0Cb1jTbtPkz5HAAQuAhTaJO9lSdV30T5HAAQuAhTaJPm+0tRPgcABCLCFNpkZ7N38gEAEIgIU2iTMsrnAIAAR5jCRWtePh/ct7u5kwEAwCSEKVw0e/k8IqyTYnuGmT0dAABMQZjCRSt19KXCKZ8DAAIWYQoXrZTyOQAAhClcPHv5nL4UACCQEaZwUdj5HACAMy4qTDU2Nurvf/+7XnrpJZ08eeYXakVFhU6dOuXWycF7UT4HAOCMjq4+4auvvtINN9yg8vJy1dXV6brrrlO3bt305JNPqq6uTnl5eZ6YJ7wM5XMAAM5w+crUzJkzlZycrH/9618KCzt7ReK2225TQUGBWycH71Xq2Kyzu7kTAQDAZC5fmfroo4+0adMmBQcHOx0fMGCADh065LaJwbuVHmTncwAApIu4MmWz2dTU1HTO8YMHD6pbt25umRS8W32jTXuslM8BAJAuIkxdf/31WrJkieN7i8WiU6dOKScnRzfddJM75wYvRfkcAICzXL7Nt3jxYqWnp+vKK6/U6dOn9dOf/lT79u1T79699ac//ckTc4SX2dnsFh/lcwBAoHM5TMXExGjHjh1avXq1duzYoVOnTmnq1KmaOHGiUyEd/oudzwEAOMvlMLVx40aNHDlSEydO1MSJEx3HGxsbtXHjRv3Xf/2XWycI71N2iPI5AAB2Lnemrr32Wp04ceKc49XV1br22mtdnsCyZcs0YMAAhYaGKiUlRZs3b77g+DVr1mjgwIEKDQ1VQkKC3nnnHafHDcNQdna2+vTpo7CwMKWlpWnfvn3nvM7//d//KSUlRWFhYerRo4cyMjJcnnsgqmts0m5rjSRpcAxhCgAAl8OUYRgt9mSOHz+uLl26uPRaq1evVlZWlnJycrRt2zYNGTJE6enpOnLkSIvjN23apAkTJmjq1KkqKSlRRkaGMjIyVFZW5hizaNEiLV26VHl5eSoqKlKXLl2Unp6u06dPO8b89a9/1Z133qkpU6Zox44d+sc//qGf/vSnLs09UO21nlJDk6GIsE6K6cFtXQAALIZhGK0ZePvtt0uS3nzzTd1www0KCQlxPNbU1KSdO3fqe9/7nvLz81v9w1NSUjRs2DA9//zzks5suxAbG6v77rtPc+fOPWd8ZmamamtrtW7dOsexESNGKDExUXl5eTIMQ9HR0Zo9e7YefPBBSWeumEVGRmrFihUaP368GhsbNWDAAD366KOaOnVqq+f6n2pqahQREaHq6mqFh4df9Ov4mpVF5Zr/RqlGXd5bf/xZitnTAQDAJZ74/d3qK1MRERGKiIiQYRjq1q2b4/uIiAhFRUVp+vTp+uMf/9jqH1xfX6/i4mKlpaWdnUxQkNLS0lRYWNjicwoLC53GS1J6erpj/P79+2W1Wp3GREREKCUlxTFm27ZtOnTokIKCgnT11VerT58+uvHGG52ubrWkrq5ONTU1Tl+ByLHzObf4AACQ5EIB/ZVXXpF0ZqfzBx980OVbev/p2LFjampqUmRkpNPxyMhI7d69u8XnWK3WFsdbrVbH4/Zj5xvzxRdfSJIeeeQRPfPMMxowYIAWL16sH/zgB9q7d6969uzZ4s/Ozc3Vo48+6uJZ+h/K5wAAOHO5M5WTk9PmIGUmm80mSXr44Yc1btw4JSUl6ZVXXpHFYtGaNWvO+7x58+apurra8XXgwIH2mrLXaF4+J0wBAHCGy1sjSNJf/vIX/fnPf1Z5ebnq6+udHtu2bVurXqN3797q0KGDKisrnY5XVlYqKiqqxedERUVdcLz9z8rKSvXp08dpTGJioiQ5jl955ZWOx0NCQnTppZeqvLz8vPMNCQlx6okFIsrnAACcy+UrU0uXLtWUKVMUGRmpkpISDR8+XL169dIXX3yhG2+8sdWvExwcrKSkJBUUFDiO2Ww2FRQUKDU1tcXnpKamOo2XpPXr1zvGx8XFKSoqymlMTU2NioqKHGOSkpIUEhKiPXv2OMY0NDToyy+/VP/+/Vs9/0Bk70sNjmHncwAA7Fy+MvXCCy9o+fLlmjBhglasWKE5c+bo0ksvVXZ2dov7T11IVlaWJk+erOTkZA0fPlxLlixRbW2tpkyZIkmaNGmS+vbtq9zcXEnSzJkzNXr0aC1evFhjx47VqlWrtHXrVi1fvlzSmc8JnDVrlh5//HHFx8crLi5OCxcuVHR0tGMfqfDwcN1zzz3KyclRbGys+vfvr6eeekqSdMcdd7i6HAGl9FCVJHY+BwCgOZfDVHl5uUaOHClJCgsL08mTJyVJd955p0aMGOHY5qA1MjMzdfToUWVnZ8tqtSoxMVH5+fmOAnl5ebmCgs5ePBs5cqRWrlypBQsWaP78+YqPj9fatWs1aNAgx5g5c+aotrZW06dPV1VVlUaNGqX8/HyFhoY6xjz11FPq2LGj7rzzTn3zzTdKSUnRhg0b1KNHD1eXI6CUUj4HAOAcrd5nyu7SSy/VX//6V1199dVKTk7WtGnT9POf/1zvvfeexo8f7/LVKV8VaPtM1TU2aVDO39TQZOijOdcqtmdns6cEAIDLTN1nyu6HP/yh3nrrLUnSlClT9MADD+i6665TZmambrvtNrdMCt7HXj7v3pnyOQAAzbl8m2/58uWO7QVmzJihXr16adOmTbrlllv085//3O0ThHfY+e++VEJfyucAADTnUphqbGzUb37zG919992KiYmRJI0fP17jx4/3yOTgPeybdVI+BwDAmUu3+Tp27KhFixapsbHRU/OBl3Jsi0CYAgDAicudqTFjxujDDz/0xFzgpeoam7THeuZdm1yZAgDAmcudqRtvvFFz585VaWmpkpKSzvlomVtuucVtk4N32GM9SfkcAIDzcDlM/fKXv5QkPfPMM+c8ZrFY1NTU1PZZwas031+K8jkAAM5cDlP2d/IhcJSxWScAAOflcmcKgWfnQcIUAADnQ5jCBdU1NmlvJeVzAADOhzCFC6J8DgDAhRGmcEGUzwEAuDDCFC6olL4UAAAX5PK7+Wpqalo8brFYFBISouDg4DZPCt6jlHfyAQBwQS6Hqe7du1/wdk9MTIzuuusu5eTkKCiIC1++7HTD2fJ5QgxhCgCAlrgcplasWKGHH35Yd911l4YPHy5J2rx5s1599VUtWLBAR48e1dNPP62QkBDNnz/f7RNG+7GXz3t07qS+3SmfAwDQEpfD1KuvvqrFixfrJz/5iePYj370IyUkJOill15SQUGB+vXrpyeeeIIw5ePst/gGUT4HAOC8XL4Pt2nTJl199dXnHL/66qtVWFgoSRo1apTKy8vbPjuYip3PAQD4di6HqdjYWL388svnHH/55ZcVGxsrSTp+/Lh69OjR9tnBVPYrU4PpSwEAcF4u3+Z7+umndccdd+jdd9/VsGHDJElbt27V7t279Ze//EWStGXLFmVmZrp3pmhXpxuatMfKzucAAHwbl8PULbfcot27d+ull17S3r17JUk33nij1q5dqwEDBkiSfvGLX7h1kmh/e6wn1WijfA4AwLdxOUxJUlxcnH7729+6ey7wIpTPAQBonYsKU1VVVdq8ebOOHDkim83m9NikSZPcMjGYq4y+FAAAreJymHr77bc1ceJEnTp1SuHh4U5XLSwWC2HKT+zkY2QAAGgVl9/NN3v2bN199906deqUqqqq9K9//cvxdeLECU/MEe2s+c7nlM8BALgwl8PUoUOHdP/996tz586emA+8gL183rNLMOVzAAC+hcthKj09XVu3bvXEXOAldlI+BwCg1VzuTI0dO1YPPfSQdu3apYSEBHXq1Mnp8VtuucVtk4M5yhx9qXCTZwIAgPdzOUxNmzZNkvTYY4+d85jFYlFTU1PbZwVTlfIxMgAAtJrLYeo/t0KAf2lePk+I6W7uZAAA8AEud6bg33Y3K59HR4SaPR0AALxeq65MLV26VNOnT1doaKiWLl16wbH333+/WyYGc7DzOQAArmlVmHr22Wc1ceJEhYaG6tlnnz3vOIvFQpjycfby+WD6UgAAtEqrwtT+/ftb/Gf4n+ZXpgAAwLejMwUH5/I5YQoAgNZw+d18TU1NWrFihQoKClr8oOMNGza4bXJoX5TPAQBwncthaubMmVqxYoXGjh2rQYMGUVL2I833l+J/VwAAWsflMLVq1Sr9+c9/1k033eSJ+cBEpQerJLFZJwAArnC5MxUcHKzLL7/cE3OByUoP1UiifA4AgCtcDlOzZ8/W7373OxmG4Yn5wCSnG5q0j/I5AAAuc/k238cff6z3339f7777rq666qpzPuj49ddfd9vk0H7s5fNelM8BAHCJy2Gqe/fuuu222zwxF5jI3pdi53MAAFzjUphqbGzUtddeq+uvv15RUVGemhNM0PydfAAAoPVc6kx17NhR99xzj+rq6jw1H5jEXj6nLwUAgGtcLqAPHz5cJSUlnpgLTOK08zlXpgAAcInLnalf/vKXmj17tg4ePKikpCR16dLF6fHBgwe7bXJoH58drlHTv8vnfSifAwDgEpfD1Pjx4yVJ999/v+OYxWKRYRiyWCxqampy3+zQLsqafbgx5XMAAFzjcpjav3+/J+YBE9nL54PpSwEA4DKXw1T//v09MQ+YaOfBs1emAACAa1wOU3a7du1SeXm56uvrnY7fcsstbZ4U2s/phibtO3JKEuVzAAAuhsth6osvvtBtt92m0tJSR1dKkqNrQ2fKt1A+BwCgbVzeGmHmzJmKi4vTkSNH1LlzZ3366afauHGjkpOT9cEHH3hgivAke/k8IYbyOQAAF8PlK1OFhYXasGGDevfuraCgIAUFBWnUqFHKzc3V/fffzx5UPsbel+IWHwAAF8flK1NNTU3q1q2bJKl3796qqKiQdKaYvmfPHvfODh5XeojyOQAAbeHylalBgwZpx44diouLU0pKihYtWqTg4GAtX75cl156qSfmCA9pXj5nWwQAAC6Oy2FqwYIFqq2tlSQ99thjuvnmm/X9739fvXr10urVq90+QXjOrn+Xz3t3DVZUOOVzAAAuhsthKj093fHPl19+uXbv3q0TJ06oR48eFJh9DDufAwDQdi53puw+//xz/e1vf9M333yjnj17tmkSy5Yt04ABAxQaGqqUlBRt3rz5guPXrFmjgQMHKjQ0VAkJCXrnnXecHjcMQ9nZ2erTp4/CwsKUlpamffv2tfhadXV1SkxMlMVi0fbt29t0Hr6mlPI5AABt5nKYOn78uMaMGaPvfve7uummm3T48GFJ0tSpUzV79myXJ7B69WplZWUpJydH27Zt05AhQ5Senq4jR460OH7Tpk2aMGGCpk6dqpKSEmVkZCgjI0NlZWWOMYsWLdLSpUuVl5enoqIidenSRenp6Tp9+vQ5rzdnzhxFR0e7PG9/YC+fE6YAALh4LoepBx54QJ06dVJ5ebk6d+7sOJ6Zman8/HyXJ/DMM89o2rRpmjJliq688krl5eWpc+fO+p//+Z8Wx//ud7/TDTfcoIceekhXXHGFfv3rX2vo0KF6/vnnJZ25KrVkyRItWLBAt956qwYPHqw//OEPqqio0Nq1a51e691339V7772np59+2uV5+zqnnc8pnwMAcNFcDlPvvfeennzyScXExDgdj4+P11dffeXSa9XX16u4uFhpaWlnJxQUpLS0NBUWFrb4nMLCQqfx0pkel338/v37ZbVancZEREQoJSXF6TUrKys1bdo0/e///q9TKDyfuro61dTUOH35MsrnAAC4h8thqra2tsXwceLECYWEhLj0WseOHVNTU5MiIyOdjkdGRspqtbb4HKvVesHx9j8vNMYwDN1111265557lJyc3Kq55ubmKiIiwvEVGxvbqud5q7Jmt/gonwMAcPFcDlPf//739Yc//MHxvcVikc1m06JFi3Tttde6dXKe8txzz+nkyZOaN29eq58zb948VVdXO74OHDjgwRl6HuVzAADcw+WtERYtWqQxY8Zo69atqq+v15w5c/Tpp5/qxIkT+sc//uHSa/Xu3VsdOnRQZWWl0/HKykpFRUW1+JyoqKgLjrf/WVlZqT59+jiNSUxMlCRt2LBBhYWF51xJS05O1sSJE/Xqq6+e83NDQkJcvvLmzdj5HAAA93D5ytSgQYO0d+9ejRo1Srfeeqtqa2t1++23q6SkRJdddplLrxUcHKykpCQVFBQ4jtlsNhUUFCg1NbXF56SmpjqNl6T169c7xsfFxSkqKsppTE1NjYqKihxjli5dqh07dmj79u3avn27Y2uF1atX64knnnDpHHwR5XMAANzH5StT0plC98MPP+x07ODBg5o+fbqWL1/u0mtlZWVp8uTJSk5O1vDhw7VkyRLV1tZqypQpkqRJkyapb9++ys3NlSTNnDlTo0eP1uLFizV27FitWrVKW7dudfxci8WiWbNm6fHHH1d8fLzi4uK0cOFCRUdHKyMjQ5LUr18/pzl07dpVknTZZZedU6z3R2fL5yGUzwEAaKOLClMtOX78uF5++WWXw1RmZqaOHj2q7OxsWa1WJSYmKj8/31EgLy8vV1DQ2QtoI0eO1MqVK7VgwQLNnz9f8fHxWrt2rQYNGuQYM2fOHNXW1mr69OmqqqrSqFGjlJ+fr9BQgoPUvC8VTvkcAIA2shiGYbjjhXbs2KGhQ4eqqanJHS/n9WpqahQREaHq6mqFh4ebPR2XPLhmh/5SfFD3//ByZV3/PbOnAwBAu/HE7++L/jgZ+K4yyucAALgNYSrAfFN/tnw+OKa7uZMBAMAPtLozdfvtt1/w8aqqqrbOBe2gefk8Mtx/tnoAAMAsrQ5TEREXviUUERGhSZMmtXlC8KyzO59TPgcAwB1aHaZeeeUVT84D7cS+WWcCt/gAAHALOlMBho+RAQDAvQhTAeRM+fykJMIUAADuQpgKILsO18hmiPI5AABuRJgKIPby+eCYCMrnAAC4CWEqgOw8yGadAAC4G2EqgJzdFoEwBQCAuxCmAgTlcwAAPIMwFSDs5fPvdKN8DgCAOxGmAkTpwSpJZ65KUT4HAMB9CFMBovRQjSTK5wAAuBthKkA4tkUgTAEA4FaEqQDwdX3j2fJ5DGEKAAB3IkwFgM+cyuehZk8HAAC/QpgKAHy4MQAAnkOYCgD28jlhCgAA9yNMBYDSQ1WSCFMAAHgCYcrPfV3fqM+PnJJE+RwAAE8gTPk5e/n8EsrnAAB4BGHKz1E+BwDAswhTfm7nvzfrZOdzAAA8gzDl5+w7n3NlCgAAzyBM+THK5wAAeB5hyo/tqqB8DgCApxGm/Fgpt/gAAPA4wpQfK6V8DgCAxxGm/Ji9fD6YvhQAAB5DmPJTTuVzrkwBAOAxhCk/1bx8fgnlcwAAPIYw5adKucUHAEC7IEz5KfvHyFA+BwDAswhTfoptEQAAaB+EKT/0dX2j/nmU8jkAAO2BMOWH7OXzyHDK5wAAeBphyg/tPMgtPgAA2gthyg+VsfM5AADthjDlhyifAwDQfghTfqa2jvI5AADtiTDlZ3YdpnwOAEB7Ikz5mVLK5wAAtCvClJ8pc/Slups7EQAAAgRhys/stIepmHCTZwIAQGAgTPmR5uVztkUAAKB9EKb8yK7DNTLs5fNulM8BAGgPhCk/crZ83t3ciQAAEEAIU36EzToBAGh/hCk/Ukr5HACAdkeY8hOUzwEAMAdhyk/Yy+dR4aGUzwEAaEeEKT+x89/lc65KAQDQvghTfqKM8jkAAKYgTPkJe/l8cAxhCgCA9kSY8gOnKJ8DAGAarwhTy5Yt04ABAxQaGqqUlBRt3rz5guPXrFmjgQMHKjQ0VAkJCXrnnXecHjcMQ9nZ2erTp4/CwsKUlpamffv2OR7/8ssvNXXqVMXFxSksLEyXXXaZcnJyVF9f75Hz87RdFWfL59/pFmL2dAAACCimh6nVq1crKytLOTk52rZtm4YMGaL09HQdOXKkxfGbNm3ShAkTNHXqVJWUlCgjI0MZGRkqKytzjFm0aJGWLl2qvLw8FRUVqUuXLkpPT9fp06clSbt375bNZtNLL72kTz/9VM8++6zy8vI0f/78djlnd7Pf4uOqFAAA7c9iGIZh5gRSUlI0bNgwPf/885Ikm82m2NhY3XfffZo7d+454zMzM1VbW6t169Y5jo0YMUKJiYnKy8uTYRiKjo7W7Nmz9eCDD0qSqqurFRkZqRUrVmj8+PEtzuOpp57Siy++qC+++KLFx+vq6lRXV+f4vqamRrGxsaqurlZ4uLmbZD6werveKDmkrOu+q/vHxJs6FwAAvFlNTY0iIiLc+vvb1CtT9fX1Ki4uVlpamuNYUFCQ0tLSVFhY2OJzCgsLncZLUnp6umP8/v37ZbVancZEREQoJSXlvK8pnQlcPXv2PO/jubm5ioiIcHzFxsa26hzbw86DVZJ4Jx8AAGYwNUwdO3ZMTU1NioyMdDoeGRkpq9Xa4nOsVusFx9v/dOU1P//8cz333HP6+c9/ft65zps3T9XV1Y6vAwcOXPjk2smpukZ9caxWErf5AAAwQ0ezJ2C2Q4cO6YYbbtAdd9yhadOmnXdcSEiIQkK8r9xtL5/3iaB8DgCAGUy9MtW7d2916NBBlZWVTscrKysVFRXV4nOioqIuON7+Z2tes6KiQtdee61Gjhyp5cuXt+lczGK/xcdVKQAAzGFqmAoODlZSUpIKCgocx2w2mwoKCpSamtric1JTU53GS9L69esd4+Pi4hQVFeU0pqamRkVFRU6veejQIf3gBz9QUlKSXnnlFQUFmf7GxovCzucAAJjL9Nt8WVlZmjx5spKTkzV8+HAtWbJEtbW1mjJliiRp0qRJ6tu3r3JzcyVJM2fO1OjRo7V48WKNHTtWq1at0tatWx1XliwWi2bNmqXHH39c8fHxiouL08KFCxUdHa2MjAxJZ4NU//799fTTT+vo0aOO+Zzvipi3KiVMAQBgKtPDVGZmpo4ePars7GxZrVYlJiYqPz/fUSAvLy93umo0cuRIrVy5UgsWLND8+fMVHx+vtWvXatCgQY4xc+bMUW1traZPn66qqiqNGjVK+fn5Cg0NlXTmStbnn3+uzz//XDExMU7zMXmnCJdQPgcAwHym7zPlqzyxT4Wrir44rszln6hPRKgK540xZQ4AAPgSv9tnCm3DzucAAJiPMOXDKJ8DAGA+wpQPc5TPYwhTAACYhTDlo5qXz7kyBQCAeQhTPurTQ9WOnc97d2XncwAAzEKY8lHsLwUAgHcgTPkowhQAAN6BMOWjHNsiUD4HAMBUhCkfdKquUfspnwMA4BUIUz7IXj6PpnwOAIDpCFM+iJ3PAQDwHoQpH0T5HAAA70GY8kGUzwEA8B6EKR9z8nSDvjhK+RwAAG9BmPIxn1bUSKJ8DgCAtyBM+ZgyyucAAHgVwpSPsfelBtOXAgDAKxCmfEzpQa5MAQDgTQhTPuTk6QZ9wc7nAAB4FcKUD2lePu9F+RwAAK9AmPIh9vJ5An0pAAC8BmHKh+w8yM7nAAB4G8KUD2FbBAAAvA9hykdQPgcAwDsRpnxE2aEz5fO+3cMonwMA4EUIUz7i7C2+cJNnAgAAmiNM+Qj7zufc4gMAwLsQpnzE2W0Rups7EQAA4IQw5QNqKJ8DAOC1CFM+4NNm5fOeXYJNng0AAGiOMOUDKJ8DAOC9CFM+wF4+H0xfCgAAr0OY8gGl7HwOAIDXIkx5uZrTDdpP+RwAAK9FmPJylM8BAPBuhCkvV3qoShJXpQAA8FaEKS9X+u8rUwkxhCkAALwRYcrLlVE+BwDAqxGmvBjlcwAAvB9hyovZr0pRPgcAwHsRpryY48ONuSoFAIDXIkx5McrnAAB4P8KUFys9WCWJK1MAAHgzwpSXqjndoC+Pfy2JMAUAgDcjTHmp5uXzHpTPAQDwWoQpL2UPU4PpSwEA4NUIU15q50E26wQAwBcQprwU2yIAAOAbCFNeiPI5AAC+gzDlhexXpWJ6UD4HAMDbEaa8UOlBbvEBAOArCFNeqPQQ5XMAAHwFYcoLsS0CAAC+gzDlZaq/OVs+HxRNmAIAwNsRprzMp5TPAQDwKYQpL1PK/lIAAPgUwpSXcYQp+lIAAPgErwhTy5Yt04ABAxQaGqqUlBRt3rz5guPXrFmjgQMHKjQ0VAkJCXrnnXecHjcMQ9nZ2erTp4/CwsKUlpamffv2OY05ceKEJk6cqPDwcHXv3l1Tp07VqVOn3H5urjrd0KQOQRauTAEA4CNMD1OrV69WVlaWcnJytG3bNg0ZMkTp6ek6cuRIi+M3bdqkCRMmaOrUqSopKVFGRoYyMjJUVlbmGLNo0SItXbpUeXl5KioqUpcuXZSenq7Tp087xkycOFGffvqp1q9fr3Xr1mnjxo2aPn26x8/32/y/ycNU9ki6UuJ6mT0VAADQChbDMAwzJ5CSkqJhw4bp+eeflyTZbDbFxsbqvvvu09y5c88Zn5mZqdraWq1bt85xbMSIEUpMTFReXp4Mw1B0dLRmz56tBx98UJJUXV2tyMhIrVixQuPHj9dnn32mK6+8Ulu2bFFycrIkKT8/XzfddJMOHjyo6Ojoc35uXV2d6urqHN/X1NQoNjZW1dXVCg8Pd+uaAAAAz6ipqVFERIRbf3+bemWqvr5excXFSktLcxwLCgpSWlqaCgsLW3xOYWGh03hJSk9Pd4zfv3+/rFar05iIiAilpKQ4xhQWFqp79+6OICVJaWlpCgoKUlFRUYs/Nzc3VxEREY6v2NjYiztpAADgV0wNU8eOHVNTU5MiIyOdjkdGRspqtbb4HKvVesHx9j+/bcwll1zi9HjHjh3Vs2fP8/7cefPmqbq62vF14MCBVp4lAADwZx3NnoCvCAkJUUhIiNnTAAAAXsbUK1O9e/dWhw4dVFlZ6XS8srJSUVFRLT4nKirqguPtf37bmP8suDc2NurEiRPn/bkAAAAtMTVMBQcHKykpSQUFBY5jNptNBQUFSk1NbfE5qampTuMlaf369Y7xcXFxioqKchpTU1OjoqIix5jU1FRVVVWpuLjYMWbDhg2y2WxKSUlx2/kBAAD/Z/ptvqysLE2ePFnJyckaPny4lixZotraWk2ZMkWSNGnSJPXt21e5ubmSpJkzZ2r06NFavHixxo4dq1WrVmnr1q1avny5JMlisWjWrFl6/PHHFR8fr7i4OC1cuFDR0dHKyMiQJF1xxRW64YYbNG3aNOXl5amhoUH33nuvxo8f3+I7+QAAAM7H9DCVmZmpo0ePKjs7W1arVYmJicrPz3cUyMvLyxUUdPYC2siRI7Vy5UotWLBA8+fPV3x8vNauXatBgwY5xsyZM0e1tbWaPn26qqqqNGrUKOXn5ys0NNQx5rXXXtO9996rMWPGKCgoSOPGjdPSpUvb78QBAIBfMH2fKV/liX0qAACAZ/ndPlMAAAC+jjAFAADQBoQpAACANiBMAQAAtAFhCgAAoA1M3xrBV9nfBFlTU2PyTAAAQGvZf2+7czMDwtRFOnnypCQpNjbW5JkAAABXnTx5UhEREW55LfaZukg2m00VFRXq1q2bLBaL2163pqZGsbGxOnDgAPtXifVojrU4i7VwxnqcxVqcxVqc1XwtunXrppMnTyo6OtppU/C24MrURQoKClJMTIzHXj88PDzg/+VvjvU4i7U4i7VwxnqcxVqcxVqcZV8Ld12RsqOADgAA0AaEKQAAgDYgTHmZkJAQ5eTkKCQkxOypeAXW4yzW4izWwhnrcRZrcRZrcZan14ICOgAAQBtwZQoAAKANCFMAAABtQJgCAABoA8IUAABAGxCmvMyyZcs0YMAAhYaGKiUlRZs3bzZ7Sh6Xm5urYcOGqVu3brrkkkuUkZGhPXv2OI05ffq0ZsyYoV69eqlr164aN26cKisrTZpx+/ntb38ri8WiWbNmOY4F0locOnRI//3f/61evXopLCxMCQkJ2rp1q+NxwzCUnZ2tPn36KCwsTGlpadq3b5+JM/acpqYmLVy4UHFxcQoLC9Nll12mX//6106fL+av67Fx40b96Ec/UnR0tCwWi9auXev0eGvO+8SJE5o4caLCw8PVvXt3TZ06VadOnWrHs3CPC61FQ0ODfvWrXykhIUFdunRRdHS0Jk2apIqKCqfX8Je1kL79343m7rnnHlksFi1ZssTpuDvWgzDlRVavXq2srCzl5ORo27ZtGjJkiNLT03XkyBGzp+ZRH374oWbMmKFPPvlE69evV0NDg66//nrV1tY6xjzwwAN6++23tWbNGn344YeqqKjQ7bffbuKsPW/Lli166aWXNHjwYKfjgbIW//rXv3TNNdeoU6dOevfdd7Vr1y4tXrxYPXr0cIxZtGiRli5dqry8PBUVFalLly5KT0/X6dOnTZy5Zzz55JN68cUX9fzzz+uzzz7Tk08+qUWLFum5555zjPHX9aitrdWQIUO0bNmyFh9vzXlPnDhRn376qdavX69169Zp48aNmj59enudgttcaC2+/vprbdu2TQsXLtS2bdv0+uuva8+ePbrlllucxvnLWkjf/u+G3RtvvKFPPvlE0dHR5zzmlvUw4DWGDx9uzJgxw/F9U1OTER0dbeTm5po4q/Z35MgRQ5Lx4YcfGoZhGFVVVUanTp2MNWvWOMZ89tlnhiSjsLDQrGl61MmTJ434+Hhj/fr1xujRo42ZM2cahhFYa/GrX/3KGDVq1Hkft9lsRlRUlPHUU085jlVVVRkhISHGn/70p/aYYrsaO3ascffddzsdu/32242JEycahhE46yHJeOONNxzft+a8d+3aZUgytmzZ4hjz7rvvGhaLxTh06FC7zd3d/nMtWrJ582ZDkvHVV18ZhuG/a2EY51+PgwcPGn379jXKysqM/v37G88++6zjMXetB1emvER9fb2Ki4uVlpbmOBYUFKS0tDQVFhaaOLP2V11dLUnq2bOnJKm4uFgNDQ1OazNw4ED169fPb9dmxowZGjt2rNM5S4G1Fm+99ZaSk5N1xx136JJLLtHVV1+t3//+947H9+/fL6vV6rQWERERSklJ8bu1kKSRI0eqoKBAe/fulSTt2LFDH3/8sW688UZJgbcedq0578LCQnXv3l3JycmOMWlpaQoKClJRUVG7z7k9VVdXy2KxqHv37pICby1sNpvuvPNOPfTQQ7rqqqvOedxd68EHHXuJY8eOqampSZGRkU7HIyMjtXv3bpNm1f5sNptmzZqla665RoMGDZIkWa1WBQcHO/4ysIuMjJTVajVhlp61atUqbdu2TVu2bDnnsUBaiy+++EIvvviisrKyNH/+fG3ZskX333+/goODNXnyZMf5tvTfjL+thSTNnTtXNTU1GjhwoDp06KCmpiY98cQTmjhxoiQF3HrYtea8rVarLrnkEqfHO3bsqJ49e/r12pw+fVq/+tWvNGHCBMcHHQfaWjz55JPq2LGj7r///hYfd9d6EKbgVWbMmKGysjJ9/PHHZk/FFAcOHNDMmTO1fv16hYaGmj0dU9lsNiUnJ+s3v/mNJOnqq69WWVmZ8vLyNHnyZJNn1/7+/Oc/67XXXtPKlSt11VVXafv27Zo1a5aio6MDcj1wYQ0NDfrJT34iwzD04osvmj0dUxQXF+t3v/udtm3bJovF4tGfxW0+L9G7d2916NDhnHdlVVZWKioqyqRZta97771X69at0/vvv6+YmBjH8aioKNXX16uqqsppvD+uTXFxsY4cOaKhQ4eqY8eO6tixoz788EMtXbpUHTt2VGRkZMCsRZ8+fXTllVc6HbviiitUXl4uSY7zDZT/Zh566CHNnTtX48ePV0JCgu6880498MADys3NlRR462HXmvOOioo65408jY2NOnHihF+ujT1IffXVV1q/fr3jqpQUWGvx0Ucf6ciRI+rXr5/j79OvvvpKs2fP1oABAyS5bz0IU14iODhYSUlJKigocByz2WwqKChQamqqiTPzPMMwdO+99+qNN97Qhg0bFBcX5/R4UlKSOnXq5LQ2e/bsUXl5ud+tzZgxY1RaWqrt27c7vpKTkzVx4kTHPwfKWlxzzTXnbJGxd+9e9e/fX5IUFxenqKgop7WoqalRUVGR362FdOadWkFBzn9ld+jQQTabTVLgrYdda847NTVVVVVVKi4udozZsGGDbDabUlJS2n3OnmQPUvv27dPf//539erVy+nxQFqLO++8Uzt37nT6+zQ6OloPPfSQ/va3v0ly43pcfG8e7rZq1SojJCTEWLFihbFr1y5j+vTpRvfu3Q2r1Wr21DzqF7/4hREREWF88MEHxuHDhx1fX3/9tWPMPffcY/Tr18/YsGGDsXXrViM1NdVITU01cdbtp/m7+QwjcNZi8+bNRseOHY0nnnjC2Ldvn/Haa68ZnTt3Nv74xz86xvz2t781unfvbrz55pvGzp07jVtvvdWIi4szvvnmGxNn7hmTJ082+vbta6xbt87Yv3+/8frrrxu9e/c25syZ4xjjr+tx8uRJo6SkxCgpKTEkGc8884xRUlLieIdaa877hhtuMK6++mqjqKjI+Pjjj434+HhjwoQJZp3SRbvQWtTX1xu33HKLERMTY2zfvt3p79O6ujrHa/jLWhjGt/+78Z/+8918huGe9SBMeZnnnnvO6NevnxEcHGwMHz7c+OSTT8yeksdJavHrlVdecYz55ptvjF/+8pdGjx49jM6dOxu33XabcfjwYfMm3Y7+M0wF0lq8/fbbxqBBg4yQkBBj4MCBxvLly50et9lsxsKFC43IyEgjJCTEGDNmjLFnzx6TZutZNTU1xsyZM41+/foZoaGhxqWXXmo8/PDDTr8k/XU93n///Rb/jpg8ebJhGK077+PHjxsTJkwwunbtaoSHhxtTpkwxTp48acLZtM2F1mL//v3n/fv0/fffd7yGv6yFYXz7vxv/qaUw5Y71sBhGs+1zAQAA4BI6UwAAAG1AmAIAAGgDwhQAAEAbEKYAAADagDAFAADQBoQpAACANiBMAQAAtAFhCgAAoA0IUwAAAG1AmALg144ePapf/OIX6tevn0JCQhQVFaX09HT94x//kCRZLBatXbvW3EkC8GkdzZ4AAHjSuHHjVF9fr1dffVWXXnqpKisrVVBQoOPHj5s9NQB+gs/mA+C3qqqq1KNHD33wwQcaPXr0OY8PGDBAX331leP7/v3768svv5Qkvfnmm3r00Ue1a9cuRUdHa/LkyXr44YfVseOZ/w9qsVj0wgsv6K233tIHH3ygPn36aNGiRfrxj3/cLucGwHtwmw+A3+ratau6du2qtWvXqq6u7pzHt2zZIkl65ZVXdPjwYcf3H330kSZNmqSZM2dq165deumll7RixQo98cQTTs9fuHChxo0bpx07dmjixIkaP368PvvsM8+fGACvwpUpAH7tr3/9q6ZNm6ZvvvlGQ4cO1ejRozV+/HgNHjxY0pkrTG+88YYyMjIcz0lLS9OYMWM0b948x7E//vGPmjNnjioqKhzPu+eee/Tiiy86xowYMUJDhw7VCy+80D4nB8ArcGUKgF8bN26cKioq9NZbb+mGG27QBx98oKFDh2rFihXnfc6OHTv02GOPOa5sde3aVdOmTdPhw4f19ddfO8alpqY6PS81NZUrU0AAooAOwO+Fhobquuuu03XXXaeFCxfqZz/7mXJycnTXXXe1OP7UqVN69NFHdfvtt7f4WgDQHFemAAScK6+8UrW1tZKkTp06qampyenxoUOHas+ePbr88svP+QoKOvvX5ieffOL0vE8++URXXHGF508AgFfhyhQAv3X8+HHdcccduvvuuzV48GB169ZNW7du1aJFi3TrrbdKOvOOvoKCAl1zzTUKCQlRjx49lJ2drZtvvln9+vXTj3/8YwUFBWnHjh0qKyvT448/7nj9NWvWKDk5WaNGjdJrr72mzZs36+WXXzbrdAGYhAI6AL9VV1enRx55RO+9957++c9/qqGhQbGxsbrjjjs0f/58hYWF6e2331ZWVpa+/PJL9e3b17E1wt/+9jc99thjKikpUadOnTRw4ED97Gc/07Rp0ySdKaAvW7ZMa9eu1caNG9WnTx89+eST+slPfmLiGQMwA2EKAC5CS+8CBBCY6EwBAAC0AWEKAACgDSigA8BFoCEBwI4rUwAAAG1AmAIAAGgDwhQAAEAbEKYAAADagDAFAADQBoQpAACANiBMAQAAtAFhCgAAoA3+P8MbL070/c55AAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "plt.ylabel(\"Learning rate\")\n",
    "plt.xlabel(\"Step\")\n",
    "plt.plot(range(total_training_steps), track_lrs);"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7b3996b6-3f7a-420a-8584-c5760249f3d8",
   "metadata": {},
   "source": [
    "## D.2 Cosine decay"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c5216214-de79-40cf-a733-b1049a73023c",
   "metadata": {},
   "source": [
    "- Another popular technique for training complex deep neural networks is cosine decay, which also adjusts the learning rate across training epochs\n",
    "- In cosine decay, the learning rate follows a cosine curve, decreasing from its initial value to near zero following a half-cosine cycle\n",
    "- This gradual reduction is designed to slow the pace of learning as the model begins to improve its weights; it reduces the risk of overshooting minima as the training progresses,  which is crucial for stabilizing the training in its later stages\n",
    "- Cosine decay is often preferred over linear decay for its smoother transition in learning rate adjustments, but linear decay is also used in practice (for example, [OLMo: Accelerating the Science of Language Models](https://arxiv.org/abs/2402.00838))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "4e8d2068-a057-4abf-b478-f02cc37191f6",
   "metadata": {},
   "outputs": [],
   "source": [
    "import math\n",
    "\n",
    "min_lr = 0.1 * initial_lr\n",
    "track_lrs = []\n",
    "\n",
    "lr_increment = (peak_lr - initial_lr) / warmup_steps\n",
    "global_step = -1\n",
    "\n",
    "for epoch in range(n_epochs):\n",
    "    for input_batch, target_batch in train_loader:\n",
    "        optimizer.zero_grad()\n",
    "        global_step += 1\n",
    "    \n",
    "        # Adjust the learning rate based on the current phase (warmup or cosine annealing)\n",
    "        if global_step < warmup_steps:\n",
    "            # Linear warmup\n",
    "            lr = initial_lr + global_step * lr_increment  \n",
    "        else:\n",
    "            # Cosine annealing after warmup\n",
    "            progress = ((global_step - warmup_steps) / \n",
    "                        (total_training_steps - warmup_steps))\n",
    "            lr = min_lr + (peak_lr - min_lr) * 0.5 * (1 + math.cos(math.pi * progress))\n",
    "        \n",
    "        # Apply the calculated learning rate to the optimizer\n",
    "        for param_group in optimizer.param_groups:\n",
    "            param_group[\"lr\"] = lr\n",
    "        track_lrs.append(optimizer.param_groups[0][\"lr\"])\n",
    "    \n",
    "        # Calculate loss and update weights"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "0e779e33-8a44-4984-bb23-be0603dc4158",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlMAAAGwCAYAAACNeeBZAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAABiTUlEQVR4nO3deVhUZf8G8HsWZoZ12GRHwaVcAFFQxDRbKDJbSDM1SzPTMtewLM20fO1nWpaZmlqWVppmb5mZWYaVG6IIIoi4JIqCwyoM+zJzfn8go7yigSxnlvtzXXP5cuaZ4TvnNbh9zvc8j0QQBAFEREREdFukYhdAREREZMoYpoiIiIiagWGKiIiIqBkYpoiIiIiagWGKiIiIqBkYpoiIiIiagWGKiIiIqBnkYhdgqvR6PbKysmBvbw+JRCJ2OURERNQIgiCguLgYXl5ekEpbZk6JYeo2ZWVlwdfXV+wyiIiI6DZcvHgRPj4+LfJeDFO3yd7eHkDt/xkODg4iV0NERESNodVq4evra/g93hIYpm5T3aU9BwcHhikiIiIT05ItOmxAJyIiImoGhikiIiKiZmCYIiIiImoGhikiIiKiZmCYIiIiImoGhikiIiKiZmCYIiIiImoGhikiIiKiZmCYIiIiImoGhikiIiKiZhA9TK1cuRJ+fn5QqVQICwvD4cOHbzl+69at6Nq1K1QqFQIDA7Fz5856z//www948MEH4eLiAolEgmPHjt3wHhUVFZg8eTJcXFxgZ2eHYcOGITs7uyU/FhEREVkIUcPUli1bEB0djfnz5yMhIQE9e/ZEZGQkcnJyGhx/8OBBjBo1CuPHj0diYiKioqIQFRWFlJQUw5jS0lIMGDAAixcvvun3feWVV/Dzzz9j69at+Pvvv5GVlYWhQ4e2+OcjIiIi8ycRBEEQ65uHhYWhT58+WLFiBQBAr9fD19cXU6dOxRtvvHHD+BEjRqC0tBQ7duwwHOvXrx+Cg4OxevXqemPPnz8Pf39/JCYmIjg42HC8qKgI7dq1w6ZNm/Dkk08CANLS0tCtWzfExsaiX79+japdq9VCrVajqKiIGx23oqKyapRV10AulcJKJoHKSgalXNqiG1QSEZHlaI3f3/IWeZfbUFVVhaNHj2L27NmGY1KpFBEREYiNjW3wNbGxsYiOjq53LDIyEtu2bWv09z169Ciqq6sRERFhONa1a1e0b9/+lmGqsrISlZWVhq+1Wm2jvyfdnoP/5OGZz+Og/5+4byWTwE4ph4O1FVztlGhnp0Q7eyW8HK3h62yN9s426OBiC7W1lTiFExGRRREtTOXl5UGn08Hd3b3ecXd3d6SlpTX4Go1G0+B4jUbT6O+r0WigUCjg6OjYpPdZtGgR3nnnnUZ/H2q+P1JzoBcAiQS4fv60WifgSlk1rpRV40J+2U1f72avxB3u9rjD3R4B3g4I8lHD39UOMilntYiIqOWIFqZMzezZs+vNimm1Wvj6+opYkflLziwEAHzwZE8M7e2NGr2AimodSiprUFxRg6LyauQVVyK3pBLZ2gpkXinHxSvluFhQhpziSsNj/9k8w3vaKmQIbu+Ivn4u6OPvhN7tnaCykon0CYmIyByIFqZcXV0hk8luuIsuOzsbHh4eDb7Gw8OjSeNv9h5VVVUoLCysNzv1b++jVCqhVCob/X2oeXR6ASmZtZdSg3zUkEgksJJJYCWTwl5lBU/1rV9fXFGNszklOJNdgpMaLZIvFeFElhalVTocOJuPA2fzAQAKuRR9/Zxx9x2uuPuOdrjT3Z79WERE1CSihSmFQoGQkBDExMQgKioKQG0DekxMDKZMmdLga8LDwxETE4MZM2YYju3evRvh4eGN/r4hISGwsrJCTEwMhg0bBgA4deoUMjIymvQ+1LrO5ZagvFoHG4UMHdvZNfn19ior9GrvhF7tnQzHanR6nMkpQfyFKziSXoDD6QXQaCuw/2we9p/Nw//tTIOPkzUe7O6BB3u4o4+fMy8JEhHRvxL1Ml90dDTGjh2L0NBQ9O3bF8uWLUNpaSnGjRsHABgzZgy8vb2xaNEiAMD06dMxaNAgLF26FEOGDMHmzZsRHx+PtWvXGt6zoKAAGRkZyMrKAlAblIDaGSkPDw+o1WqMHz8e0dHRcHZ2hoODA6ZOnYrw8PBG38lHre/4pSIAQA8vhxYLNHKZFN08HdDN0wHP9usAQRBwLq8Uf5/Kxd4zuTh0Lh+XrpTjiwPp+OJAOlztFHgkyAuP9vRC7/aOnLEiIqIGiRqmRowYgdzcXMybNw8ajQbBwcHYtWuXock8IyMDUum1pbD69++PTZs2Ye7cuZgzZw66dOmCbdu2ISAgwDBm+/bthjAGACNHjgQAzJ8/H2+//TYA4KOPPoJUKsWwYcNQWVmJyMhIrFq1qg0+MTVWcmZtmAr0dmy17yGRSNCpnR06tbPD8wP8UV6lw94zufj9RDZi0rKRV1KF9QfPY/3B8/B1tsaTvX3xZKgPvB2tW60mIiIyPaKuM2XKuM5U6xq66gASMgqxbEQwonp5t/n3r9bpsf9MHrYnZeG3ExqUVekA1N5ZOLBLOzwT1h73d3PnZUAiIhNjVutMEd1MjU6P1Mu1zeeBPv/Sad5KrGRS3NvVDfd2dUN5lQ67TlzGd0cuIfZcPvaezsXe07no4GKD5/r7YXioL+yU/E+JiMhScWbqNnFmqvWkabR4aNk+2CnlOD7/QUiNaPbnQn4pvj18Ed8ezkBReTUAwF4px1N9fPFcfz/4OtuIXCEREd1Ka/z+Fn2jY6L/dX3zuTEFKQDo4GKLNwZ3Rezs+7AwKgAd29miuLIG6/anY9D7f+Klr48i5Wq/FxERWQZemyCjk3w1TAWJdImvMWwUcjzTrwOe7tsef5/JxRf707HvTB52ndBg1wkNInu445UH7kBXD85aEhGZO4YpMjqGO/l8HMUtpBGkUgnuvdMN997phtPZxVj151n8lJSF305k47cT2RgS5IlXIrqgs5u92KUSEVEr4WU+MirV1zefexvvzFRD7nC3x7KRvfD7jLsxJMgTAPDL8ct44KO9mLE5EedyS0SukIiIWgPDFBmV09nFqKrRw14lRwcTbebu4m6PlU/3xq/TByKyhzsEAdh2LAsPfrQX7/x8AkVl1WKXSERELYhhioxKimGxTrXRNZ83VTdPB6x5NhQ7pg7AfV3dUKMX8OWB87jngz/xzaEL0Ol5Iy0RkTlgmCKjUncnn1jrS7WGAG81vniuD74e3xdd3Oxwpawac7elYMjyfTj4T57Y5RERUTMxTJFRSb5uZsrcDOzSDr9OH4h3HusBtbUV0jTFePqzOLz09VFcLCgTuzwiIrpNDFNkNKpq9Ei7XAwACGrFPfnEJJdJMba/H/569R6MCe8AqQTYdUKDBz76G5/vO8dLf0REJohhiozG6exiVOn0UFtbwdfZvDcTdrJVYMHjAfh1+t0I83dGRbUeC385iaGrDuDk1bsZiYjINDBMkdEw9Et5qyGRmHbzeWPd6WGPzRP7YdHQQNir5Ei6VIRHP9mPpb+fQmWNTuzyiIioERimyGgkZxYCMK/m88aQSCQY1bc9/ogehAe7u6NGL+CTPWfx8Mf7EH++QOzyiIjoXzBMkdGom5kKMsPm88Zwd1BhzbMh+HR0b7jaKfFPbimGr4nFgp9TUVHNWSoiImPFMEVGoaJah9PZtc3nljYzdT2JRILBgZ6IiR6Ep0J9IAjAFwfSEbXyAE5pisUuj4iIGsAwRUbhlKYY1ToBTjZW8HY07+bzxlDbWGHJkz2xbmwoXGwVSNMU49EV+/HlgXQIAu/4IyIyJgxTZBSOX7e5saU0nzfG/d3csWvG3bj3znaoqtHjnZ9TMfbLI8jRVohdGhERXcUwRUYh+VIhAMvtl7qVdvZKfPFcHyx4vAeUcin2ns7FQx/vw+8nNGKXRkREYJgiI5GcWbu2kiX3S92KRCLBmHA/7Jg6AN08HVBQWoWJXx/Fgp9TUa3Ti10eEZFFY5gi0dVrPufM1C11cbfHtsn9MWGgP4Da5vSRaw9BU8TLfkREYmGYItGlXtZCpxfgaqeAp1oldjlGTymX4c0h3bH22RDYq+Q4euFK7abJZ7lpMhGRGBimSHQpmZa38nlLeLCHh+GyX35pFZ5ZF4eVf56Fnvv7ERG1KYYpEp1hGxkfR3ELMUEdXGzx48v9MTzEB3oBeP+3U5jwVTyKyqrFLo2IyGIwTJHokq/bk4+aTmUlw/vDe2LxsEAo5FLEpOXg0RX7DX1oRETUuhimSFRlVTU4k1P7Sz+Id/I1y4g+7fHDpP7wdbZGRkEZhq46iD1p2WKXRURk9himSFQnL2uhFwA3eyXcHdh83lwB3mr8NHkAwvydUVJZg/Eb4vH5vnNcNZ2IqBUxTJGojvMSX4tztlXg6/FhGNXXF4IALPzlJF7/73FU1XA9KiKi1sAwRaIy9EvxEl+LUsil+L8nAjH/0e6QSoDv4i/hmc/jkF9SKXZpRERmh2GKRFW3Jx/7pVqeRCLBuLv88eW4vrBXyXH4fAEeX3kApzRsTCciakkMUySa0soa/JNbAqC214dax6A72uHHl++Cn4sNLl0px5OfHsTBf7jAJxFRS2GYItGcyNJCEAAPBxXc7Nl83po6u9lh2+S70NffGcWVNXjuiyPYnpQldllERGaBYYpEc/xSIQD2S7UVRxsFvnq+L4YEeqJKp8e0bxPx+b5zYpdFRGTyGKZINHXbyATxEl+bUVnJ8MmoXhh3lx+A2jv9Fvycyi1oiIiagWGKRFPXfM6ZqbYllUow75HuePPhbgCALw6kY+rmRFRU60SujIjINDFMkSiKK6pxLrcUANeYEoNEIsGEuzvi45HBsJJJ8MvxyxjzxWEUlXNPPyKipmKYIlGkZGoBAN6O1nCxU4pcjeV6PNgbG8b1hb1SjsPpBRi59hDyuBYVEVGTMEyRKOr6pTgrJb7+nV2x5cVwuNopcfKyFk+tiUVWYbnYZRERmQyGKRIF+6WMS3cvB3z3Yj94qVU4l1uK4atjcT6vVOyyiIhMAsMUiSK5blkEzkwZjY7t7LB1Un/4u9ois7Acw9fEcrV0IqJGYJiiNldUXo3z+WUAGKaMjbejNb57MRxdPeyRW1yJEWtjkXSxUOyyiIiMGsMUtbkTVy/x+Tpbw8lWIXI19L/a2SuxZWI4gn0dUVhWjdGfxyHuXL7YZRERGS2GKWpzx9l8bvTUNlb45oUwhHd0QUllDcZ8cRj7zuSKXRYRkVFimKI2l3ypLkw5ilsI3ZKdUo4vx/XB/V3dUFmjxwsb4rH3NAMVEdH/YpiiNnc8sxAAEMQ7+YyeykqGT58JQUS3q4HqKwYqIqL/xTBFbaqwrAoXC2rXMArwYpgyBQq5FKtGhyCimzuqrgaqvxmoiIgMGKaoTSVf7Zfq4GIDtY2VyNVQY9UGqt54oHttoJrwVTz+OpUjdllEREaBYYra1PFLbD43VQq5FCuf7o0HrwaqiV8fZaAiIgLDFLWxum1k2C9lmhRyKVZcH6i+Ooo/GaiIyMIxTFGbOs47+UxeXaCK7OGOKp0eL359FAfO5oldFhGRaBimqM3kl1Qi8+oGuj28HUSuhpqjLlDV9VC9sCEe8ecLxC6LiEgUDFPUZuqazzu62sJBxeZzU2clk2LF070wsIsryqt1GPflERy/uuciEZElYZiiNlPXLxXIfimzoZTLsPbZUPT1d0bx1ZXSuTkyEVkahilqM7yTzzxZK2T44rk+6HndXn7nckvELouIqM0wTFGbSeaefGbLTinHhnF90NXDHnkllRj9eRwuFpSJXRYRUZtgmKI2kVtcictFFZBIgB4MU2bJ0UaBb14IQ6d2trhcVIHRn8dBU1QhdllERK2OYYraRF2/VKd2drBTykWuhlqLq50SG1/oB19na2QUlOHZdXEoLKsSuywiolYlephauXIl/Pz8oFKpEBYWhsOHD99y/NatW9G1a1eoVCoEBgZi586d9Z4XBAHz5s2Dp6cnrK2tERERgTNnztQbc/r0aTz++ONwdXWFg4MDBgwYgD///LPFPxtdw34py+GhVmHTC/3g7qDEmZwSPL/+CMqqasQui4io1YgaprZs2YLo6GjMnz8fCQkJ6NmzJyIjI5GT0/CKygcPHsSoUaMwfvx4JCYmIioqClFRUUhJSTGMWbJkCZYvX47Vq1cjLi4Otra2iIyMREXFtcsNjzzyCGpqarBnzx4cPXoUPXv2xCOPPAKNRtPqn9lSJWcWAmCYshS+zjb46vkwOKjkSMgoxMsbE1Ct04tdFhFRq5AIgiCI9c3DwsLQp08frFixAgCg1+vh6+uLqVOn4o033rhh/IgRI1BaWoodO3YYjvXr1w/BwcFYvXo1BEGAl5cXZs6ciVdffRUAUFRUBHd3d6xfvx4jR45EXl4e2rVrh71792LgwIEAgOLiYjg4OGD37t2IiIhosNbKykpUVlYavtZqtfD19UVRUREcHLgA5b8J+78/kK2txPcvhSPUz1nscqiNxJ8vwDPr4lBRrccTvbyxdHhPSKUSscsiIgum1WqhVqtb9Pe3aDNTVVVVOHr0aL3wIpVKERERgdjY2AZfExsbe0PYiYyMNIxPT0+HRqOpN0atViMsLMwwxsXFBXfeeSe++uorlJaWoqamBmvWrIGbmxtCQkJuWu+iRYugVqsND19f39v+7JYmW1uBbG0lpBKguxeDpyUJ9XPGqtG9IZNK8GNiJhb+chIi/vuNiKhViBam8vLyoNPp4O7uXu+4u7v7TS+3aTSaW46v+/NWYyQSCf744w8kJibC3t4eKpUKH374IXbt2gUnJ6eb1jt79mwUFRUZHhcvXmzaB7ZgyVf7pTq72cFGweZzS3NfV3e8/2QQAOCLA+lY9dc/IldERNSyRG9Ab2uCIGDy5Mlwc3PDvn37cPjwYURFReHRRx/F5cuXb/o6pVIJBweHeg9qnOOZ3NzY0g3t7YO5Q7oBAN7/7RQ2H84QuSIiopYjWphydXWFTCZDdnZ2vePZ2dnw8PBo8DUeHh63HF/3563G7NmzBzt27MDmzZtx1113oXfv3li1ahWsra2xYcOGFvlsVF/dsghB3EbGor0wsCMm3dMJADDnx2T8doI3fBCReRAtTCkUCoSEhCAmJsZwTK/XIyYmBuHh4Q2+Jjw8vN54ANi9e7dhvL+/Pzw8POqN0Wq1iIuLM4wpK6tdlVkqrf/RpVIp9HrebdTSBEG4tiwCw5TFmxV5J0aE+kIvANO+TURCxhWxSyIiajZRL/NFR0fjs88+w4YNG3Dy5ElMmjQJpaWlGDduHABgzJgxmD17tmH89OnTsWvXLixduhRpaWl4++23ER8fjylTpgCo7YeaMWMGFi5ciO3btyM5ORljxoyBl5cXoqKiANQGMicnJ4wdOxZJSUk4ffo0XnvtNaSnp2PIkCFtfg7MnUZbgbySSsikEnT35KVRSyeRSPDuEwG49852qKzR44UN8UjPKxW7LCKiZhE1TI0YMQIffPAB5s2bh+DgYBw7dgy7du0yNJBnZGTU62Pq378/Nm3ahLVr16Jnz574/vvvsW3bNgQEBBjGzJo1C1OnTsXEiRPRp08flJSUYNeuXVCpVABqLy/u2rULJSUluO+++xAaGor9+/fjp59+Qs+ePdv2BFiAulmpLm52UFnJRK6GjIFcJsWKp3sj0FuNgtIqPPflYeSVVP77C4mIjJSo60yZstZYp8IcLf39FD7ZcxZPhfpgyZMMq3RNbnElhn56ABcLytHT1xGbJ/SDtYKBm4hal1mtM0WW4Vq/lKO4hZDRaWevxPpxfeFoY4Wki4WY+m0idHr+246ITA/DFLUaQRCQnMk9+ejmOrWzw+djQqGQS/HHyWy8vf0EF/UkIpPDMEWtJquoAgWlVZBLJejqYS92OWSkQv2c8fGIYEgkwNeHLmDN3nNil0RE1CQMU9Rqki8VAgDu9LBn8znd0uBAT7w1pDsA4L1f07DjeJbIFRERNR7DFLWaun4pLtZJjfH8AH88f5c/ACD6uySuQUVEJoNhilpNXb9UAPulqJHeHNINEd3cUFWjx8Sv4nGxoEzskoiI/hXDFLWK65vPg7gnHzWSTCrBxyN7obunA/JKqjB+wxFoK6rFLouI6JYYpqhVXLpSjsKyaihkUtzhYSd2OWRCbJVyrHsuFG72SpzOLsGUTYmo0XGrJyIyXgxT1Crq+qXu9LCHUs7mc2oaT7U11o3tA2srGfaezsXbP3PJBCIyXgxT1CqOZxYC4ObGdPsCfdRYNrJ2yYRvDmXgywPnxS6JiKhBDFPUKlIM/VIMU3T7Int4YM7gbgCA//ySij9Ss0WuiIjoRgxT1OIEQbhuGxmGKWqeFwb6Y1Tf9hAEYPrmRJzSFItdEhFRPQxT1OIu5JehuKIGCrkUd7hz5XNqHolEggWP90D/Ti4ordLhha+O4EppldhlEREZMExRizt+9RJfN08HWMn4V4yaz0omxcqne6O9sw0uFpTj5Y0JqOYdfkRkJPibjloc+6WoNTjZKvD52FDYKmSIPZeP/+xIFbskIiIADFPUCo5f3ZOP/VLU0u5wt8fHI3tBIgG+ir2AjXEXxC6JiIhhilqWXi8gJVMLAAjkzBS1goju7nj1wTsBAPN/OoG4c/kiV0RElo5hilrU+fxSlFTWQCmXoosbVz6n1vHyPZ3waE8v1OgFTNqYwD38iEhUDFPUour24+vh5QA5m8+plUgkEiwZFoRAbzUKSqsw4at4lFbWiF0WEVko/rajFlW3vlSQj6O4hZDZs1bIsHZMCFztlEjTFCP6u2PQ67nlDBG1PYYpalHJV8NUAPulqA14qq2x5tkQKGRS/HYiGx/HnBG7JCKyQAxT1GJ0egEnsupmphimqG2EdHDCu08EAAA+jjmDncmXRa6IiCwNwxS1mPS8EpRW6WBtJUOndmw+p7YzPNQX4wf4AwBmfpdkCPVERG2BYYpaTF2/VA8vB8ikEpGrIUsze3BXDOziivJqHSZ+dRT5JZVil0REFoJhiloMNzcmMcllUqwY1Rv+rrbILCzHtM2JqOGWM0TUBhimqMUYtpFhmCKRqG2ssPqZENgoZDhwNh/v/35K7JKIyAIwTFGLqNHpcSKrbuVzR3GLIYt2p4c9ljwZBABY8/c5/HKcDelE1LoYpqhF/JNbivJqHWwVMnR0tRW7HLJwjwR54cW7OwIAXvs+Caezi0WuiIjMGcMUtQjDyufeakjZfE5G4LXIO9G/kwvKqnR48euj0FZUi10SEZkphilqEcmXCgEAQVysk4yEXCbFJ6N6wdvRGul5pYjewhXSiah1MExRizieyTv5yPi42Cnx6TO9oZBL8cfJHKz486zYJRGRGWKYomar0emRamg+Z5gi4xLk44iFUbUrpH/0x2n8mZYjckVEZG4YpqjZzuSUoLJGD3ulHH4ubD4n4/NUqC9Gh7WHIADTNyfifF6p2CURkRlhmKJmu35zYzafk7Ga/2gP9GrvCG1FDV765ijKqmrELomIzATDFDXb8cxCAFysk4ybQi7Fp6ND4GqnRJqmGG/8NxmCwIZ0Imo+hilqtutnpoiMmYdahVWje0MulWB7UhbW7U8XuyQiMgMMU9QsVTV6nNTULojImSkyBX39nTF3SDcAwKJf0xB3Ll/kiojI1DFMUbOczi5GVY0eDio52jvbiF0OUaOM7e+HqGAv6PQCpnybiJziCrFLIiITxjBFzZJ83fpSEgmbz8k0SCQS/N/QQHRxs0NucSWmfZuIGp1e7LKIyEQxTFGzHL/aL8XNjcnU2Cjk+PSZENgoZDh0rgAf7j4tdklEZKIYpqhZUq7OTLFfikxRZzc7LB4WBABY9dc/+CM1W+SKiMgUMUzRbaus0SFNw5XPybQ92tMLz/X3AwBEf3cMFwvKxC2IiEwOwxTdtlOaYlTrBDjaWMHHyVrscohu25yHuyHYt3ZBz0kbj6KiWid2SURkQhim6LYZms+92XxOpk0hl2Ll6N5wsrFCSqYWC3akil0SEZkQhim6bXWLdbJfisyBt6M1lo3sBYkE2BSXgR8SLoldEhGZCIYpum28k4/MzaA72mHafV0AAHN+TDb0BBIR3QrDFN2WimodTmfXrnweyJkpMiPT7u+CgV1cUVGtx8vfJKC4olrskojIyDFM0W1J0xSjRi/AxVYBL7VK7HKIWoxMKsGyEcHwVKtwLq+UGyIT0b+6rTBVU1ODP/74A2vWrEFxce3sRFZWFkpKSlq0ODJeyZcKAXDlczJPLnZKrHi6dkPkX5Iv48sD58UuiYiMWJPD1IULFxAYGIjHH38ckydPRm5uLgBg8eLFePXVV1u8QDJOdf1SQVxfisxUSAcnvHl1Q+T/23kSCRlXRK6IiIxVk8PU9OnTERoaiitXrsDa+traQk888QRiYmJatDgyXnXLIgQwTJEZe66/H4YEeqJGL2DqpkQUllWJXRIRGaEmh6l9+/Zh7ty5UCgU9Y77+fkhMzOzxQoj41VepcOZnNpLukE+juIWQ9SKJBIJFg0LRAcXG2QWluPVrcfZP0VEN2hymNLr9dDpblwd+NKlS7C3t2+Rosi4pV7WQqcX0M5eCXcHpdjlELUqB5UVVj7dGwqZFH+czMa6/elil0RERqbJYerBBx/EsmXLDF9LJBKUlJRg/vz5ePjhh1uyNjJShuZzrnxOFiLAW423Hqntn3rv1zQksn+KiK7T5DC1dOlSHDhwAN27d0dFRQWefvppwyW+xYsXt0aNZGSSM7m5MVmeZ/p1wMOBHqjRC5iyKRFFZVx/iohqyZv6Ah8fHyQlJWHLli1ISkpCSUkJxo8fj9GjR9drSCfzlZxZCIDbyJBlkUgkeG9YEFIytcgoKMOr3ydh7bMhnJ0loqbPTO3duxcAMHr0aCxZsgSrVq3CCy+8ACsrK8NzTbFy5Ur4+flBpVIhLCwMhw8fvuX4rVu3omvXrlCpVAgMDMTOnTvrPS8IAubNmwdPT09YW1sjIiICZ86cueF9fvnlF4SFhcHa2hpOTk6Iiopqcu2WqLSyBmevNp9zZooszfX9U7tTs/EF158iItxGmLr33ntRUFBww/GioiLce++9TXqvLVu2IDo6GvPnz0dCQgJ69uyJyMhI5OTkNDj+4MGDGDVqFMaPH4/ExERERUUhKioKKSkphjFLlizB8uXLsXr1asTFxcHW1haRkZGoqKgwjPnvf/+LZ599FuPGjUNSUhIOHDiAp59+ukm1W6rUy1roBcDdQQk3B658TpYn0EeNuYb+qZM4drFQ3IKISHQSoYn3+UqlUmRnZ6Ndu3b1jp8+fRqhoaHQahu/MWhYWBj69OmDFStWAKi9U9DX1xdTp07FG2+8ccP4ESNGoLS0FDt27DAc69evH4KDg7F69WoIggAvLy/MnDnTsIBoUVER3N3dsX79eowcORI1NTXw8/PDO++8g/Hjxze61srKSlRWVhq+1mq18PX1RVFRERwcHBr9Pqbui/3pWLAjFRHd3PH52FCxyyEShSAImLwpATuTNfBxssYvUwdCbWMldllE1AharRZqtbpFf383emZq6NChGDp0KCQSCZ577jnD10OHDsXjjz+OyMhI9O/fv9HfuKqqCkePHkVERMS1YqRSREREIDY2tsHXxMbG1hsPAJGRkYbx6enp0Gg09cao1WqEhYUZxiQkJCAzMxNSqRS9evWCp6cnBg8eXG92qyGLFi2CWq02PHx9fRv9Wc1J3WKd7JciS1bXP9Xe2QaXrpTjte+TuP4UkQVrdJiqCxGCIMDe3r5esPDw8MDEiRPxzTffNPob5+XlQafTwd3dvd5xd3d3aDSaBl+j0WhuOb7uz1uNOXfuHADg7bffxty5c7Fjxw44OTnhnnvuafDyZZ3Zs2ejqKjI8Lh48WKjP6s5OX7dnnxEluz6/qnfU7O5fx+RBWv03XxffvklgNqVzl999VXY2tq2WlGtSa/XAwDefPNNDBs2DEDtZ/Px8cHWrVvx4osvNvg6pVIJpdKyF6gsqazBubxSAGw+JwJq/1Hx5pBumL/9BBb9ehIhHZzQ09dR7LKIqI01uQF9/vz5LRKkXF1dIZPJkJ2dXe94dnY2PDw8GnyNh4fHLcfX/XmrMZ6engCA7t27G55XKpXo2LEjMjIymvGJzN+JzCIIAuClVsHVzrKDJVGdMeEdMDjAA9W62j6qonKuP0VkaZocpgDg+++/x1NPPYV+/fqhd+/e9R6NpVAoEBISUm9zZL1ej5iYGISHhzf4mvDw8Bs2U969e7dhvL+/Pzw8POqN0Wq1iIuLM4wJCQmBUqnEqVOnDGOqq6tx/vx5dOjQodH1W6K6file4iO6RiKRYPGTQfB1tsalK+WYxf4pIovT5DC1fPlyjBs3Du7u7khMTETfvn3h4uKCc+fOYfDgwU16r+joaHz22WfYsGEDTp48iUmTJqG0tBTjxo0DAIwZMwazZ882jJ8+fTp27dqFpUuXIi0tDW+//Tbi4+MxZcoUALU/1GbMmIGFCxdi+/btSE5OxpgxY+Dl5WVYR8rBwQEvvfQS5s+fj99//x2nTp3CpEmTAADDhw9v6umwKMcv1TWfO4pbCJGRqeufspJJ8NuJbKw/eF7skoioDTV5BfRVq1Zh7dq1GDVqFNavX49Zs2ahY8eOmDdv3i0buBsyYsQI5ObmYt68edBoNAgODsauXbsMDeQZGRmQSq/lvf79+2PTpk2YO3cu5syZgy5dumDbtm0ICAgwjJk1axZKS0sxceJEFBYWYsCAAdi1axdUqmtrIr3//vuQy+V49tlnUV5ejrCwMOzZswdOTk5NPR0WpW5mKoD9UkQ3CPJxxJsPd8PbP6fi/3aeRO/27J8ishRNXmfKxsYGJ0+eRIcOHeDm5obdu3ejZ8+eOHPmDPr164f8/PzWqtWotMY6FcZMW1GNoLd/BwAkvPUAnG0VIldEZHwEQcCkbxKw64QGvs7W2DF1INTWXH+KyJiIus5UHQ8PD8MMVPv27XHo0CEAtWs8sU/AfKVcnZXycbJmkCK6iev7py4WlOON/x7nz0UiC9DkMHXfffdh+/btAIBx48bhlVdewQMPPIARI0bgiSeeaPECyTgkX+JinUSNoba2wopRtf1Tv6ZosDGOdwkTmbsm90ytXbvWsFbT5MmT4eLigoMHD+Kxxx676RpNZPrYL0XUeD19HfH6Q12x8JeTWLAjFSEdnNDN0/zbAYgsVZNmpmpqarBw4cJ6K5SPHDkSy5cvx9SpU6FQ8PKPuTJsI+PtKG4hRCZi/AB/3NfVDVU1ekzZlICyqhqxSyKiVtKkMCWXy7FkyRLU1PCHgiUpKqvGhfwyAFz5nKixJBIJ3n8yCO4OSvyTW4q3t58QuyQiaiVN7pm6//778ffff7dGLWSk6mal2jvbQG3DO5OIGsvFTollI3pBKgG+i7+En45lil0SEbWCJvdMDR48GG+88QaSk5MREhJyw9Yyjz32WIsVR8aBK58T3b7wTi6Yel8XfBxzBnN+SEZPH0f4uZrm3qZE1LAmh6mXX34ZAPDhhx/e8JxEIoFOp2t+VWRUkjMLAQBBvMRHdFum3tcZsefycTi9AFO+TcB/J/WHUi4TuywiaiFNvsyn1+tv+mCQMk9128hwZoro9shlUnw8MhiONlZIydRi8a+n/v1FRGQybmujY7IcV0qrcOlKOQAui0DUHJ5qa3zwZE8AwBcH0vFHarbIFRFRS2GYoluq65fyd7WFg4rN50TNEdHdHc/f5Q8AeO37JFwuKhe5IiJqCQxTdEuG5nPOShG1iNcH34kAbwdcKavG9M3HUKPTi10SETUTwxTd0vFLhQC4jQxRS1HKZfhkVG/YKmQ4nF6AT/acFbskImomhim6pZRMLQD2SxG1JH9XW/zf0EAAwCd7ziD2n3yRKyKi5mhymNJqtQ0+iouLUVVV1Ro1kkjySiqRWVgOiQTo4cV9xYha0uPB3hge4gO9AMzYkoiCUv78JDJVTQ5Tjo6OcHJyuuHh6OgIa2trdOjQAfPnzzdshkymq65fqqOrLezZfE7U4t55vAc6tbNFtrYSr25NgiAIYpdERLehyWFq/fr18PLywpw5c7Bt2zZs27YNc+bMgbe3Nz799FNMnDgRy5cvx3vvvdca9VIbSr66vlSQj6O4hRCZKRuFHCue7g2FXIo9aTlYtz9d7JKI6DY0eQX0DRs2YOnSpXjqqacMxx599FEEBgZizZo1iImJQfv27fHuu+9izpw5LVosta26mSn2SxG1nm6eDnjrke54a1sKFu9KQ19/Z/4DhsjENHlm6uDBg+jVq9cNx3v16oXY2FgAwIABA5CRkdH86khU12amGKaIWtMzYe3xUA8PVOsETP02EcUV1WKXRERN0OQw5evri3Xr1t1wfN26dfD19QUA5Ofnw8nJqfnVkWhytBXQaCsglQDdPdl8TtSaJBIJFg8LgrejNS7kl2HOjynsnyIyIU2+zPfBBx9g+PDh+PXXX9GnTx8AQHx8PNLS0vD9998DAI4cOYIRI0a0bKXUpuou8XVqZwdbZZP/mhBRE6ltrLB8VC88tSYWPydlYWBnVzzVx1fssoioEZo8M/XYY48hLS0NgwcPRkFBAQoKCjB48GCkpaXhkUceAQBMmjQJH374YYsXS23HsPI5L/ERtZmQDk6Y+eAdAIB521NwJrtY5IqIqDFua8rB39+fd+uZOUO/FJvPidrUS3d3Quw/+dh3Jg9TNiXipyl3QWUlE7ssIrqF2wpThYWFOHz4MHJycm5YT2rMmDEtUhiJRxAEHDfMTDmKWwyRhZFKJfjwqWAM/ngfTmUX4z87UvHuE4Fil0VEt9DkMPXzzz9j9OjRKCkpgYODAyQSieE5iUTCMGUGsrWVyC2uZPM5kUja2Svx0YieeHbdYWyMy0D/Tq4YEuQpdllEdBNN7pmaOXMmnn/+eZSUlKCwsBBXrlwxPAoKClqjRmpjdf1Sd7jbw1rBywtEYhjYpR0m3dMJAPDGD8dxsaBM5IqI6GaaHKYyMzMxbdo02NjYtEY9ZASSLxUCAALZL0UkqugH7kDv9o4orqjB1G8TUa3jNl1ExqjJYSoyMhLx8fGtUQsZibp+KS7WSSQuK5kUH4/sBQeVHMcuFuKD30+JXRIRNaDJPVNDhgzBa6+9htTUVAQGBsLKqv4GuI899liLFUdtTxAEpHAbGSKj4etsgyVPBuGlbxKw5u9zCO/ognvudBO7LCK6jkRo4jK7UunNJ7MkEgl0Ol2zizIFWq0WarUaRUVFcHAwnybtrMJy9H9vD+RSCVLeieQt2URG4q1tKfj60AW42Crw6/SBcHNQiV0SkUlqjd/fTb7Mp9frb/qwlCBlzo5futZ8ziBFZDzeHNINXT3skV9ahRlbjkGn53YzRMaiyWGKzFtyZiEA9ksRGRuVlQwrnu4NaysZDv6Tj1V/nhW7JCK6qlE9U8uXL8fEiROhUqmwfPnyW46dNm1aixRG4kjO1AJgvxSRMersZof/RAXg1a1J+OiP0wjr6IK+/s5il0Vk8RrVM+Xv74/4+Hi4uLjA39//5m8mkeDcuXMtWqCxMseeKUEQ0Ps/u3GlrBrbp9yFIK5+TmSUorccww+JmfBUq7Bz2kA42SrELonIZLTG7+9GzUylp6c3+L/JvFy6Uo4rZdWwkklwp4e92OUQ0U0siApA4sVCpOeV4rXvj+OzMSH1dqMgorbFnikyqFv5/E4PeyjlbD4nMlZ2SjlWPN0LCpkUf5zMxpcHzotdEpFFa/I6UzqdDuvXr0dMTEyDGx3v2bOnxYqjtlUXpgK9HcUthIj+VQ8vNd4c0g3zt5/Aol9Poo+fMwJ54wiRKJocpqZPn47169djyJAhCAgI4NSyGUm+xJXPiUzJmPAOOHA2D7+nZmPqtwnYMW0g7JRN/rFORM3U5P/qNm/ejO+++w4PP/xwa9RDIhEE4bqZKYYpIlMgkUiw5MkgnFi+H+fzy/Dmj8lYNiKY/8glamNN7plSKBTo3Llza9RCIrpYUI6i8mooZFLc4c7mcyJT4WijwPJRwZBJJfjpWBa2Hr0kdklEFqfJYWrmzJn4+OOP0cRdaMjIHb+6WGc3T3so5LwvgciUhHRwRvQDdwAA5v90AmdzikWuiMiyNPky3/79+/Hnn3/i119/RY8ePW7Y6PiHH35oseKo7dT1S7GBlcg0TRrUCbH/5GP/2TxM3piIn6bcxS2hiNpIk6cgHB0d8cQTT2DQoEFwdXWFWq2u9yDTVLcnXxDv5CMySVKpBB+O6AlXOwVOZRdjwY5UsUsishhNmpmqqanBvffeiwcffBAeHh6tVRO1Mb1eQEpWbZjiNjJEpsvNXoWPRgRjzBeHsSkuA3d1csWQIE+xyyIye02amZLL5XjppZdQWVnZWvWQCC4UlKG4ogZKuRRd3O3ELoeImmFgl3aYNKgTAOCN/x7HxYIykSsiMn9NvszXt29fJCYmtkYtJJLjlwoBAN29HGAlY/M5kal75YE7ENLBCcWVNZjybSKqavT//iIium1NbkB/+eWXMXPmTFy6dAkhISGwtbWt93xQUFCLFUdtw7BYJy/xEZkFK5kUH48MxsMf70PSxUJ88PspzHm4m9hlEZmtJoepkSNHAgCmTZtmOCaRSCAIAiQSCXQ6XctVR22ibrFO9ksRmQ8fJxu8P7wnXvz6KNbuPYfwTi649043scsiMktNDlPp6emtUQeJRK8XkJJZt42Mo7jFEFGLiuzhgbHhHbAh9gJmfpeEX6cPhLuDSuyyiMxOk8NUhw4dWqMOEsm5vFKUVulgbSVDp3a2//4CIjIpsx/uhiPnryD1shYzNh/DNy+EQSbldjNELem2d8RMTU1FRkYGqqqq6h1/7LHHml0UtZ26WanuXg6Qs/mcyOyorGRY8XQvPPLJfsSey8fKP89i2v1dxC6LyKw0OUydO3cOTzzxBJKTkw29UgAMG2uyZ8q01C3Wyc2NicxXx3Z2WBgVgOjvkrDsj9MI83dGWEcXscsiMhtNnoqYPn06/P39kZOTAxsbG5w4cQJ79+5FaGgo/vrrr1YokVpT8tU9+YK4jQyRWRva2wfDevtALwDTNx9DQWnVv7+IiBqlyWEqNjYWCxYsgKurK6RSKaRSKQYMGIBFixbVu8OPjJ9OLyAlUwuAYYrIEix4vAc6trOFRluB17YmccN6ohbS5DCl0+lgb28PAHB1dUVWVhaA2sb0U6dOtWx11KrO5ZagvFoHG4UM/q5c+ZzI3Nkq5VgxqjcUcili0nLwxYHzYpdEZBaaHKYCAgKQlJQEAAgLC8OSJUtw4MABLFiwAB07dmzxAqn11PVLBXipeXcPkYXo7uWAt4bULuD53q8ncexiobgFEZmBJoepuXPnQq+v3ZpgwYIFSE9Px8CBA7Fz504sX778topYuXIl/Pz8oFKpEBYWhsOHD99y/NatW9G1a1eoVCoEBgZi586d9Z4XBAHz5s2Dp6cnrK2tERERgTNnzjT4XpWVlQgODoZEIsGxY8duq35TVbdYZyAv8RFZlGf6dcDDgR6o1gmYvDEBRWXVYpdEZNKaHKYiIyMxdOhQAEDnzp2RlpaGvLw85OTk4L777mtyAVu2bEF0dDTmz5+PhIQE9OzZE5GRkcjJyWlw/MGDBzFq1CiMHz8eiYmJiIqKQlRUFFJSUgxjlixZguXLl2P16tWIi4uDra0tIiMjUVFRccP7zZo1C15eXk2u2xzU7cnHfikiyyKRSPDesCB0cLFBZmE5Xv2e/VNEzXHbCwudPXsWv/32G8rLy+Hs7HzbBXz44YeYMGECxo0bh+7du2P16tWwsbHBF1980eD4jz/+GA899BBee+01dOvWDf/5z3/Qu3dvrFixAkDtrNSyZcswd+5cPP744wgKCsJXX32FrKwsbNu2rd57/frrr/j999/xwQcf3Hb9pqpGp0fq5drmc24jQ2R5HFRWWPl0byhkUuxOzca6/dzdguh2NTlM5efn4/7778cdd9yBhx9+GJcvXwYAjB8/HjNnzmzSe1VVVeHo0aOIiIi4VpBUioiICMTGxjb4mtjY2HrjgdrZsrrx6enp0Gg09cao1WqEhYXVe8/s7GxMmDABX3/9NWxsbP611srKSmi12noPU3Y2twQV1XrYKeXwd+HK50SWKMBbjbce7Q4AeO/XNCRkXBG5IiLT1OQw9corr8DKygoZGRn1QsiIESOwa9euJr1XXl4edDod3N3d6x13d3eHRqNp8DUajeaW4+v+vNUYQRDw3HPP4aWXXkJoaGijal20aBHUarXh4evr26jXGStD87m3A6RsPieyWM+EtccjQZ6o0QuYsjEBV7j+FFGTNTlM/f7771i8eDF8fHzqHe/SpQsuXLjQYoW1pk8++QTFxcWYPXt2o18ze/ZsFBUVGR4XL15sxQpbX/Ilbm5MRLX9U4uGBsLf1RZZRRWYuTUJej37p4iaoslhqrS0tMHLYgUFBVAqlU16L1dXV8hkMmRnZ9c7np2dDQ8PjwZf4+HhccvxdX/easyePXsQGxsLpVIJuVyOzp07AwBCQ0MxduzYBr+vUqmEg4NDvYcpq7uTj/1SRGSvssKKp3tBIZdiT1oO1u47J3ZJRCalyWFq4MCB+OqrrwxfSyQS6PV6LFmyBPfee2+T3kuhUCAkJAQxMTGGY3q9HjExMQgPD2/wNeHh4fXGA8Du3bsN4/39/eHh4VFvjFarRVxcnGHM8uXLkZSUhGPHjuHYsWOGpRW2bNmCd999t0mfwRRVX9d8HsQwRUQAenip8fajPQAA7/92CkfOF4hcEZHpaPJGx0uWLMH999+P+Ph4VFVVYdasWThx4gQKCgpw4MCBJhcQHR2NsWPHIjQ0FH379sWyZctQWlqKcePGAQDGjBkDb29vLFq0CEDt3oCDBg3C0qVLMWTIEGzevBnx8fFYu3YtgNpwN2PGDCxcuBBdunSBv78/3nrrLXh5eSEqKgoA0L59+3o12NnVrv7dqVOnGy5fmqPT2cWoqtHDXiVHB5d/b74nIsswqq8v4tLz8dOxLEzdlIhfpg2Ai13TrjgQWaImh6mAgACcPn0aK1asgL29PUpKSjB06FBMnjwZnp6eTS5gxIgRyM3Nxbx586DRaBAcHIxdu3YZGsgzMjIglV6bQOvfvz82bdqEuXPnYs6cOejSpQu2bduGgIAAw5hZs2ahtLQUEydORGFhIQYMGIBdu3ZBpVI1uT5zlFK3WKe3GhIJm8+JqJZEIsH/PRGI5MwinMstxSvfJWH9c314kwrRv5AILbRS26VLl7BgwQLDDJG502q1UKvVKCoqMrn+qTd/TMbGuAy8OKgjZg/uJnY5RGRk0jRaRK08gIpqPV6LvBOT7+0sdklELaY1fn/f9qKd/ys/Px/r1q1rqbejVlTXfB7k7ShuIURklLp6OGDBY7Wz/Ut/P4VD5/JFrojIuLVYmCLTUFWjR9rlYgDcRoaIbm54qA+G9vaGXgCmfZuI3OJKsUsiMloMUxbmdHYxqnR6qK2t4ONkLXY5RGSkJBIJFkYFoIubHXKKK/HKlmPQcf0pogYxTFmY44bFOtl8TkS3ZqOQY9Xo3rC2kmH/2Tys2HNW7JKIjFKj7+YbOnToLZ8vLCxsbi3UBpIzCwHU3slHRPRvurjbY2FUAGZuTcKymNPo4+eE/p1dxS6LyKg0Okyp1bf+5atWqzFmzJhmF0St6/qZKSKixhgW4oO49Hx8F38J0zYfw87pA+Bmz6VmiOo0Okx9+eWXrVkHtYGKah1OZ9c2n3MbGSJqinceC0DSxSKcyi7G1E2J2PhCGOQydooQAeyZsiinNMWo1glwtlXA25HN50TUeNYKGVY90xu2Chni0gvw/m+nxC6JyGgwTFmQ41z5nIiaoVM7O7w/vCcAYM3ec9iVclnkioiMA8OUBUlhvxQRNdPDgZ54YYA/AODVrcdxLrdE5IqIxMcwZUHqZqbYL0VEzfH64K7o6+eMksoaTPomAWVVNWKXRCQqhikLcX3zOWemiKg5rGRSrHi6F1ztlDiVXYw3f0xBC23zSmSSGKYsROplLXR6Aa52Sng48JZmImoeNwcVVj7dCzKpBD8mZuKbuAyxSyISDcOUhUjJ5MrnRNSywjq64PWH7gQALPj5BI5dLBS3ICKRMExZiLrFOtkvRUQtacLAjniohweqdQJe/uYoCkqrxC6JqM0xTFmI5Lo7+RimiKgFSSQSvD88CP6utsgqqsD0zYncEJksDsOUBSirqsGZnNrm80A2nxNRC7NXWWH1MyGwtpJh35k8fPzHabFLImpTDFMW4ORlLfQC4GavhDubz4moFdzpYY9FQwMBAMv3nMWetGyRKyJqOwxTFoCbGxNRW4jq5Y0x4R0AAK9sScLFgjKRKyJqGwxTFqCuXyrQ21HcQojI7L05pBuCfR1RVF6NF78+ivIqndglEbU6hikLcDyTM1NE1DaUchlWje4NVzsFUi9r8fp/j3NBTzJ7DFNmrrSyBv9c3TuLyyIQUVvwcrTGyqd7Qy6VYHtSFj7fly52SUStimHKzJ3I0kIQAE+1Cu3slWKXQ0QWIqyjC956pDsAYNGvJ7H/TJ7IFRG1HoYpM3f8UiEAIJCzUkTUxsaEd8Cw3j7QC8DUbxPYkE5mi2HKzKWwX4qIRCKRSPDuEwEI8lHjShkb0sl8MUyZubrmc/ZLEZEYVFYyrH4mBC62tQ3pb/zAhnQyPwxTZqy4ohrncksB8DIfEYnHy9EaK0fXNqT/dCwL6/azIZ3MC8OUGUvJ1AIAvB2t4WLH5nMiEk+/ji6YO6QbAOD/dp7EgbNsSCfzwTBlxtgvRUTGZGx/P0ND+pRNbEgn88EwZcbYL0VExqSuIT3Qmw3pZF4YpsxY8tVlETgzRUTGQmUlw5pn2ZBO5oVhykwVlVfjfH7tFDqbz4nImNQ1pMuuNqRzhXQydQxTZurE1Ut8vs7WcLRRiFwNEVF91zekL/r1JP48lSNyRUS3j2HKTBk2N/Z2FLcQIqKbeK6/H0aE+kIvANM2JeJsTonYJRHdFoYpM5V8qTZMBbJfioiMlEQiwX+iAtDHzwnFlTV4YcMRFJZViV0WUZMxTJmpZMPMFMMUERkvhVyKT58JgbejNc7nl2HKpkTU6PRil0XUJAxTZqiwrAoZV9dv6cEwRURGztVOic/GhMJGIcP+s3lY+MtJsUsiahKGKTNUNyvl52IDtbWVyNUQEf277l4O+PCpYADA+oPnsSkuQ9yCiJqAYcoMHTf0SzmKWwgRURM8FOCBmQ/cAQCY91MKDp3LF7kiosZhmDJDKeyXIiITNeW+zni0pxdq9AImfXMUGfnccoaMH8OUGaqbmeI2MkRkaiQSCd5/MghBPrVbzjy/4QiKyqvFLovolhimzEx+SSUyC8sBAAHeDiJXQ0TUdCorGdY+GwoPBxXO5pRg8sYEVPMOPzJiDFNmpq75vGM7W9ir2HxORKbJQ63C52Ov3eE3f/sJ7uFHRothysywX4qIzEWAtxrLR/aCRAJsisvAuv3cw4+ME8OUmWG/FBGZk4ju7pg7pDsA4N2dJ/H7CY3IFRHdiGHKzBhWPueyCERkJp6/yw/P9GsPQQCmbz5mmIEnMhYMU2Ykt7gSl4sqIJEAPbzYfE5E5kEikeDtR3tgYBdXlFfrMH7DEVwuKhe7LCIDhikzUvevtU7t7GCrlItcDRFRy5HLpFg5ujfucLdDtrYS49fHo6SyRuyyiAAwTJmVun4pNp8TkTlyUFlh3dg+cLVTIPWylksmkNFgmDIjyZmFAIBAH4YpIjJPvs42WDe2D6ytZPj7dC7e/DGZSyaQ6BimzMi15nOGKSIyXz19HbHi6V6QSoDv4i9hecxZsUsiC8cwZSaytRXI1lZCKgG6ezJMEZF5u7+bO/4TFQAA+OiP0/gu/qLIFZElY5gyE8lX+6W6uNnDWiETuRoiotY3OqwDXr6nEwBg9g/J+Pt0rsgVkaVimDITx69e4mO/FBFZktci70RUsBd0egEvf3OUa1CRKBimzEQK+6WIyAJJJBIsebIn+ndyQWmVDuPWH8GlK2Vil0UWhmHKDAiCwG1kiMhiKeRSrH42BHe62yO3uBJjvziMgtIqscsiC2IUYWrlypXw8/ODSqVCWFgYDh8+fMvxW7duRdeuXaFSqRAYGIidO3fWe14QBMybNw+enp6wtrZGREQEzpw5Y3j+/PnzGD9+PPz9/WFtbY1OnTph/vz5qKoyzf/4NNoK5JVUQiaVoLsnVz4nIsvjoLLCl+P6wFOtwj+5pXh+/RGUclFPaiOih6ktW7YgOjoa8+fPR0JCAnr27InIyEjk5OQ0OP7gwYMYNWoUxo8fj8TERERFRSEqKgopKSmGMUuWLMHy5cuxevVqxMXFwdbWFpGRkaioqAAApKWlQa/XY82aNThx4gQ++ugjrF69GnPmzGmTz9zS6mal7nC3h8qKzedEZJm8HK3x1fN94WhjhWMXC/HSN0dRVcNFPan1SQSRVzsLCwtDnz59sGLFCgCAXq+Hr68vpk6dijfeeOOG8SNGjEBpaSl27NhhONavXz8EBwdj9erVEAQBXl5emDlzJl599VUAQFFREdzd3bF+/XqMHDmywTref/99fPrppzh37lyj6tZqtVCr1SgqKoKDg7izQUt/P4VP9pzFiFBfLH4ySNRaiIjElpBxBaM/i0N5tQ6P9vTCxyOCIZVKxC6LjERr/P4WdWaqqqoKR48eRUREhOGYVCpFREQEYmNjG3xNbGxsvfEAEBkZaRifnp4OjUZTb4xarUZYWNhN3xOoDVzOzs43fb6yshJarbbew1gY+qXYfE5EhN7tnbD62RDIpRL8nJSFd34+wVXSqVWJGqby8vKg0+ng7u5e77i7uzs0Gk2Dr9FoNLccX/dnU97z7Nmz+OSTT/Diiy/etNZFixZBrVYbHr6+vrf+cG1EEIRrK5+z+ZyICAAw6I52WPpUTwDAhtgLWLGHq6RT6xG9Z0psmZmZeOihhzB8+HBMmDDhpuNmz56NoqIiw+PiReNYbTerqAIFpVWwkknQ1dNe7HKIiIzG48HeePvR7gCApbtPY2PcBZErInMlaphydXWFTCZDdnZ2vePZ2dnw8PBo8DUeHh63HF/3Z2PeMysrC/feey/69++PtWvX3rJWpVIJBweHeg9jkHypEEBt87lSzuZzIqLrPXeXP6bd1xkAMHdbCn45flnkisgciRqmFAoFQkJCEBMTYzim1+sRExOD8PDwBl8THh5ebzwA7N692zDe398fHh4e9cZotVrExcXVe8/MzEzcc889CAkJwZdffgmp1DQn6er6pbhYJxFRw1554A6MDmsPQQCmb05EzMnsf38RUROIniCio6Px2WefYcOGDTh58iQmTZqE0tJSjBs3DgAwZswYzJ492zB++vTp2LVrF5YuXYq0tDS8/fbbiI+Px5QpUwDUroY7Y8YMLFy4ENu3b0dycjLGjBkDLy8vREVFAbgWpNq3b48PPvgAubm50Gg0N+2pMmZ1/VKB3o7iFkJEZKQkEgkWPB6Ax4O9UKMXMGljAg6czRO7LDIjcrELGDFiBHJzczFv3jxoNBoEBwdj165dhgbyjIyMerNG/fv3x6ZNmzB37lzMmTMHXbp0wbZt2xAQEGAYM2vWLJSWlmLixIkoLCzEgAEDsGvXLqhUKgC1M1lnz57F2bNn4ePjU68eU7rjo17zOWemiIhuSiaV4IPhPVFepcPvqdl4YUM8vh7fF6F+N7+Lm6ixRF9nylQZwzpTFwvKMHDJn1DIpEh5JxIKuegTjURERq2yRoeJXx3F36dzYa+UY+OEMAT5OIpdFrUhs1tnipqnrl+qq6c9gxQRUSMo5TKsfiYEYf7OKK6swZgvDiNNYzzrBpJp4m9gE3Y8sxAAEMj1pYiIGs1aIcO65/og2NcRhWXVeObzwziXWyJ2WWTCGKZMWAr7pYiIboudUo4N4/qiu6cD8koqMfrzOGTkl4ldFpkohikTJQjCtW1kODNFRNRkahsrfD2+Lzq72eFyUQVGro1loKLbwjBloi7kl6G4ogYKuRR3uHPlcyKi2+Fip8SmF8LQsZ0tsq4Gqgv5pWKXRSaGYcpEHb96ia+7pwOsZPy/kYjodrk5qLB5Qj90MgSqQwxU1CT8LWyi2C9FRNRy3BxU+HZibaC6fDVQnc9joKLGYZgyUcev7snHfikiopbhZl8/UI36jIGKGodhygTp9QJSMmvXReHMFBFRy6kLVNea0hmo6N8xTJmg8/mlKKmsgcpKis7t7MQuh4jIrLjZq/DthNpApdFWYMTaWJzNKRa7LDJiDFMmKPm65nM5m8+JiFpcO3slvp3QD13c7JCtrcRTaw4ZelWJ/hd/E5uguvWluJ8UEVHraWevxJYXwxHorUZBaRVGfXYI8ecLxC6LjBDDlAlKvhqmuI0MEVHrcrZVYOOEMPTxc0JxRQ2eXXcY+87kil0WGRmGKROj0ws4kcVlEYiI2oqDygpfPR+GQXe0Q3m1DuPXx+O3ExqxyyIjwjBlYtLzSlBapYONQoaObD4nImoT1goZPhsTisEBHqjS6fHyxgT8mHhJ7LLISDBMmZi6fqkeXg6QSSUiV0NEZDkUcik+GdULT4b4QKcXEP1dEr6KPS92WWQEGKZMzHFDv5SjuIUQEVkguUyKJcOC8Fx/PwgCMO+nE3j/tzQIgiB2aSQihikTw21kiIjEJZVKMP/R7ngl4g4AwMo//8HMrUmo1ulFrozEwjBlQmp0epzIql35nNvIEBGJRyKRYHpEFywZFgSZVIIfEjLx/PojKKmsEbs0EgHDlAn5J7cU5dU62Cpk6OhqK3Y5REQW76k+vvh8bCisrWTYdyYPI9bEIqe4QuyyqI0xTJmQupXPA7zVkLL5nIjIKNx7pxs2T+wHF1sFTmRpMXTVQfyTWyJ2WdSGGKZMSPKlQgDslyIiMjY9fR3xw8v94edig0tXyvHkpwdxhKulWwyGKRNy/LqZKSIiMi4dXGzx30n90dPXEVfKqvH0Z4ewNf6i2GVRG2CYMhE1Oj1Srzafc08+IiLj5GKnxLcTwjA4wAPVOgGvfX8c/7fzJHR6Lp1gzhimTMSZnBJU1uhhr5Kjg7ON2OUQEdFN2CjkWPl0b0y7rzMAYO3ec5j4VTyKK6pFroxaC8OUibh+c2M2nxMRGTepVILoB+/E8lG9oJRLEZOWg2GfHsTFgjKxS6NWwDBlIo5nFgKoDVNERGQaHuvphS0vhsPNXonT2SV4fOUBHE5nY7q5YZgyEYaZKd7JR0RkUoJ9HbF9ygAEeqtRUFqF0Z8fwoaD57kFjRlhmDIBVTV6nNQUAwCCuCcfEZHJ8VCr8N2L4XgkyBPVOgHzt5/AtM3HUMoV080Cw5QJOJ1djKoaPdTWVvB1tha7HCIiug3WChk+GdULbz3SHXKpBD8nZeHxlQdwNqdY7NKomRimTEDdyueB3mpIJGw+JyIyVRKJBOMH+GPzxH5wd1DibE4JHltxAD8nZYldGjUDw5QJMIQp9ksREZmFUD9n7Jg6EOEdXVBWpcPUbxPx9vYTqKrRi10a3QaGKRNQ13wexDv5iIjMRjt7Jb4e3xcv39MJALD+4Hk8tSYWF/JLRa6MmophyshV1uiQpqld+ZzbyBARmRe5TIpZD3XFZ2NC4aCS49jFQjz88T5sjb/Iu/1MCMOUkTulKUa1ToCTjRV8nNh8TkRkjh7o7o6d0weir78zSqt0eO3745iyKRGFZVVil0aNwDBl5K71Szmy+ZyIyIz5ONng2wn98FrknZBLJfgl+TIeWrYPB//JE7s0+hcMU0aO/VJERJZDJpVg8r2d8cPL/eHvaguNtgKjP4/Dop0nUVmjE7s8ugmGKSN3/GqYYr8UEZHlCPJxxC/TBmBUX18IArBm7zk89skBJGZcEbs0agDDlBGrqNbhdPbVlc+5LAIRkUWxUcixaGgQ1jwbAmdbBU5lF2Popwex4OdUlFVx5XRjwjBlxNI0xajRC3C1U8BTrRK7HCIiEkFkDw/8ET0IT/TyhiAAXxxIx4Mf7cXe07lil0ZXMUwZseRLhQC48jkRkaVztlXgoxHBWD+uD7wdrXHpSjnGfHEY0d8dw5VS3vEnNoYpI1bXLxXIfikiIgJwz51u+P2Vu/Fcfz9IJMAPCZl44KO/8V38Rej1XJdKLAxTRuz6ZRGIiIgAwFYpx9uP9cD3L/VHFzc75JVUYdb3xxG16gCOXigQuzyLxDBlpMqrdDiTUwKAzedERHSjkA5O+GXaQMx5uCvslHIcv1SEYZ/GYtq3icgqLBe7PIvCMGWkUi9rodMLaGevhLsDm8+JiOhGCrkUE+/uhD9fvQcj+/hCIgG2J2XhvqV/4eM/zqC8imtTtQWGKSNV13zOxTqJiOjftLNX4r1hQfh5ygD08XNCRbUeH/1xGvct/Qub4jJQrdOLXaJZY5gyUsmZtZsbB/ISHxERNVKAtxrfvRiOT0b1gpdahctFFZjzYzLuW/oXvou/iBqGqlbBMGWkkjMLAbBfioiImkYikeDRnl7Y8+o9mPdId7jaKXGxoByzvj+OiA//xo+Jl6DjnX8timHKCJVW1uDs1eZzbiNDRES3Q2Ulw/MD/LFv1r2Y83BXONsqcD6/DK9sScKDH9WGqqoazlS1BIYpI5R6WQu9AHg4qOBmz+ZzIiK6fdYKGSbe3Qn7Zt2LWQ/dCUcbK/yTW4pXtiRh4JI9WPnnWS782UwMU0YouW6xTl7iIyKiFmKrlOPlezpj36x78VrknXCzVyJbW4n3fzuFfotiMPuHZJy5uh8sNY1c7ALoRnWLdfJOPiIiamn2KitMvrczJgzsiF+Ss7BufzpSMrX49nAGvj2cgbvvaIfRYe1xX1c3WMk459IYDFNG6PjVZRECODNFREStRCGX4olePogK9saR81ewbv85/J6ajb2nc7H3dC5c7ZQY1tsbw0N90NnNXuxyjRrDlJEpqazBubxSANyTj4iIWp9EIkFff2f09XdGRn4ZNsZdwH8TLiGvpBJr9p7Dmr3n0N3TAY8Fe+HRnl7wdrQWu2SjIxEEgfdH3gatVgu1Wo2ioiI4ODi02PvGncvHiLWH4O1ojQNv3Ndi70tERNRY1To99qTlYGv8Rfx1Khc11y2l0NNHjQd7eODB7u7o7GYHiUQiYqVN1xq/vzkzZWQMmxtzVoqIiERiJZMisocHInt44EppFXamXMZPx7Jw5HwBki4VIelSEd7/7RR8na1xd5d2uPuOdujfyQX2KiuxSxcFw5SROc47+YiIyIg42SowOqwDRod1QE5xBf5IzcHvqRocPJuPiwXl2BiXgY1xGZBKgO5eDujr54K+/k7o6esIDweVyc1c3Q6jaNNfuXIl/Pz8oFKpEBYWhsOHD99y/NatW9G1a1eoVCoEBgZi586d9Z4XBAHz5s2Dp6cnrK2tERERgTNnztQbU1BQgNGjR8PBwQGOjo4YP348SkpKWvyzNVVheTUAzkwREZHxcbNX4emw9lg/ri8S5z2AdWNDMTa8A/xdbaEXgJRMLb44kI6XvklA+KI96PNuDJ5ffwTv/5aGn45lIjVLi4pq89t8WfSeqS1btmDMmDFYvXo1wsLCsGzZMmzduhWnTp2Cm5vbDeMPHjyIu+++G4sWLcIjjzyCTZs2YfHixUhISEBAQAAAYPHixVi0aBE2bNgAf39/vPXWW0hOTkZqaipUqtpFMAcPHozLly9jzZo1qK6uxrhx49CnTx9s2rSpUXW3Vs8UABSVV0NlJYVSLmvR9yUiImotmqIKHD5fgMPp+Yg/fwVnckpuum2Nh4MKvs7W8HWyQTsHJdrZKdHOXglHGwXsVXLYK+WwU8nhYquEQt6y8z6t8ftb9DAVFhaGPn36YMWKFQAAvV4PX19fTJ06FW+88cYN40eMGIHS0lLs2LHDcKxfv34IDg7G6tWrIQgCvLy8MHPmTLz66qsAgKKiIri7u2P9+vUYOXIkTp48ie7du+PIkSMIDQ0FAOzatQsPP/wwLl26BC8vrxu+b2VlJSorKw1fa7Va+Pr6tkqYIiIiMnXlVTqkXtYi+VIhTmWX4Ex2MU5nF0NbUdPo9/h8TCgiuru3aF1m14BeVVWFo0ePYvbs2YZjUqkUERERiI2NbfA1sbGxiI6OrncsMjIS27ZtAwCkp6dDo9EgIiLC8LxarUZYWBhiY2MxcuRIxMbGwtHR0RCkACAiIgJSqRRxcXF44oknbvi+ixYtwjvvvNOcj0tERGQxrBUyhHRwQkgHJ8MxQRBwpawaGQVluFhQhktXypFbXInckkrkFlegqLwGJZXVKKmoQXFFDexVptHaLWqVeXl50Ol0cHevnzrd3d2RlpbW4Gs0Gk2D4zUajeH5umO3GvO/lxDlcjmcnZ0NY/7X7Nmz64W4upkpIiIiahyJRAJnWwWcbRUI9nW85VhTWrnJNCKfEVAqlVAqlWKXQUREZBFM6S5AUe/mc3V1hUwmQ3Z2dr3j2dnZ8PDwaPA1Hh4etxxf9+e/jcnJyan3fE1NDQoKCm76fYmIiIgaImqYUigUCAkJQUxMjOGYXq9HTEwMwsPDG3xNeHh4vfEAsHv3bsN4f39/eHh41Buj1WoRFxdnGBMeHo7CwkIcPXrUMGbPnj3Q6/UICwtrsc9HRERE5k/0y3zR0dEYO3YsQkND0bdvXyxbtgylpaUYN24cAGDMmDHw9vbGokWLAADTp0/HoEGDsHTpUgwZMgSbN29GfHw81q5dC6B2WnDGjBlYuHAhunTpYlgawcvLC1FRUQCAbt264aGHHsKECROwevVqVFdXY8qUKRg5cmSDd/IRERER3YzoYWrEiBHIzc3FvHnzoNFoEBwcjF27dhkayDMyMiCVXptA69+/PzZt2oS5c+dizpw56NKlC7Zt22ZYYwoAZs2ahdLSUkycOBGFhYUYMGAAdu3aZVhjCgA2btyIKVOm4P7774dUKsWwYcOwfPnytvvgREREZBZEX2fKVLXmop1ERETUOlrj97dRbCdDREREZKoYpoiIiIiagWGKiIiIqBkYpoiIiIiagWGKiIiIqBkYpoiIiIiagWGKiIiIqBkYpoiIiIiaQfQV0E1V3VqnWq1W5EqIiIiosep+b7fkmuUMU7epuLgYAODr6ytyJURERNRUxcXFUKvVLfJe3E7mNun1emRlZcHe3h4SiaTF3ler1cLX1xcXL17kNjXg+bgez8U1PBf18Xxcw3NxDc/FNdefC3t7exQXF8PLy6ve3r/NwZmp2ySVSuHj49Nq7+/g4GDxf/mvx/NxDc/FNTwX9fF8XMNzcQ3PxTV156KlZqTqsAGdiIiIqBkYpoiIiIiagWHKyCiVSsyfPx9KpVLsUowCz8c1PBfX8FzUx/NxDc/FNTwX17T2uWADOhEREVEzcGaKiIiIqBkYpoiIiIiagWGKiIiIqBkYpoiIiIiagWHKyKxcuRJ+fn5QqVQICwvD4cOHxS6p1S1atAh9+vSBvb093NzcEBUVhVOnTtUbU1FRgcmTJ8PFxQV2dnYYNmwYsrOzRaq47bz33nuQSCSYMWOG4ZglnYvMzEw888wzcHFxgbW1NQIDAxEfH294XhAEzJs3D56enrC2tkZERATOnDkjYsWtR6fT4a233oK/vz+sra3RqVMn/Oc//6m3v5i5no+9e/fi0UcfhZeXFyQSCbZt21bv+cZ87oKCAowePRoODg5wdHTE+PHjUVJS0oafomXc6lxUV1fj9ddfR2BgIGxtbeHl5YUxY8YgKyur3nuYy7kA/v3vxvVeeuklSCQSLFu2rN7xljgfDFNGZMuWLYiOjsb8+fORkJCAnj17IjIyEjk5OWKX1qr+/vtvTJ48GYcOHcLu3btRXV2NBx98EKWlpYYxr7zyCn7++Wds3boVf//9N7KysjB06FARq259R44cwZo1axAUFFTvuKWciytXruCuu+6ClZUVfv31V6SmpmLp0qVwcnIyjFmyZAmWL1+O1atXIy4uDra2toiMjERFRYWIlbeOxYsX49NPP8WKFStw8uRJLF68GEuWLMEnn3xiGGOu56O0tBQ9e/bEypUrG3y+MZ979OjROHHiBHbv3o0dO3Zg7969mDhxYlt9hBZzq3NRVlaGhIQEvPXWW0hISMAPP/yAU6dO4bHHHqs3zlzOBfDvfzfq/Pjjjzh06BC8vLxueK5FzodARqNv377C5MmTDV/rdDrBy8tLWLRokYhVtb2cnBwBgPD3338LgiAIhYWFgpWVlbB161bDmJMnTwoAhNjYWLHKbFXFxcVCly5dhN27dwuDBg0Spk+fLgiCZZ2L119/XRgwYMBNn9fr9YKHh4fw/vvvG44VFhYKSqVS+Pbbb9uixDY1ZMgQ4fnnn693bOjQocLo0aMFQbCc8wFA+PHHHw1fN+Zzp6amCgCEI0eOGMb8+uuvgkQiETIzM9us9pb2v+eiIYcPHxYACBcuXBAEwXzPhSDc/HxcunRJ8Pb2FlJSUoQOHToIH330keG5ljofnJkyElVVVTh69CgiIiIMx6RSKSIiIhAbGytiZW2vqKgIAODs7AwAOHr0KKqrq+udm65du6J9+/Zme24mT56MIUOG1PvMgGWdi+3btyM0NBTDhw+Hm5sbevXqhc8++8zwfHp6OjQaTb1zoVarERYWZnbnAgD69++PmJgYnD59GgCQlJSE/fv3Y/DgwQAs73zUacznjo2NhaOjI0JDQw1jIiIiIJVKERcX1+Y1t6WioiJIJBI4OjoCsLxzodfr8eyzz+K1115Djx49bni+pc4HNzo2Enl5edDpdHB3d6933N3dHWlpaSJV1fb0ej1mzJiBu+66CwEBAQAAjUYDhUJh+GFQx93dHRqNRoQqW9fmzZuRkJCAI0eO3PCcJZ2Lc+fO4dNPP0V0dDTmzJmDI0eOYNq0aVAoFBg7dqzh8zb034y5nQsAeOONN6DVatG1a1fIZDLodDq8++67GD16NABY3Pmo05jPrdFo4ObmVu95uVwOZ2dnsz43FRUVeP311zFq1CjDRseWdi4WL14MuVyOadOmNfh8S50PhikyKpMnT0ZKSgr2798vdimiuHjxIqZPn47du3dDpVKJXY6o9Ho9QkND8X//938AgF69eiElJQWrV6/G2LFjRa6u7X333XfYuHEjNm3ahB49euDYsWOYMWMGvLy8LPJ80K1VV1fjqaeegiAI+PTTT8UuRxRHjx7Fxx9/jISEBEgkklb9XrzMZyRcXV0hk8luuCsrOzsbHh4eIlXVtqZMmYIdO3bgzz//hI+Pj+G4h4cHqqqqUFhYWG+8OZ6bo0ePIicnB71794ZcLodcLsfff/+N5cuXQy6Xw93d3WLOhaenJ7p3717vWLdu3ZCRkQEAhs9rKf/NvPbaa3jjjTcwcuRIBAYG4tlnn8Urr7yCRYsWAbC881GnMZ/bw8Pjhht5ampqUFBQYJbnpi5IXbhwAbt37zbMSgGWdS727duHnJwctG/f3vDz9MKFC5g5cyb8/PwAtNz5YJgyEgqFAiEhIYiJiTEc0+v1iImJQXh4uIiVtT5BEDBlyhT8+OOP2LNnD/z9/es9HxISAisrq3rn5tSpU8jIyDC7c3P//fcjOTkZx44dMzxCQ0MxevRow/+2lHNx11133bBExunTp9GhQwcAgL+/Pzw8POqdC61Wi7i4OLM7F0DtnVpSaf0f2TKZDHq9HoDlnY86jfnc4eHhKCwsxNGjRw1j9uzZA71ej7CwsDavuTXVBakzZ87gjz/+gIuLS73nLelcPPvsszh+/Hi9n6deXl547bXX8NtvvwFowfNx+33z1NI2b94sKJVKYf369UJqaqowceJEwdHRUdBoNGKX1qomTZokqNVq4a+//hIuX75seJSVlRnGvPTSS0L79u2FPXv2CPHx8UJ4eLgQHh4uYtVt5/q7+QTBcs7F4cOHBblcLrz77rvCmTNnhI0bNwo2NjbCN998Yxjz3nvvCY6OjsJPP/0kHD9+XHj88ccFf39/oby8XMTKW8fYsWMFb29vYceOHUJ6errwww8/CK6ursKsWbMMY8z1fBQXFwuJiYlCYmKiAED48MMPhcTERMMdao353A899JDQq1cvIS4uTti/f7/QpUsXYdSoUWJ9pNt2q3NRVVUlPPbYY4KPj49w7Nixej9PKysrDe9hLudCEP7978b/+t+7+QShZc4Hw5SR+eSTT4T27dsLCoVC6Nu3r3Do0CGxS2p1ABp8fPnll4Yx5eXlwssvvyw4OTkJNjY2whNPPCFcvnxZvKLb0P+GKUs6Fz///LMQEBAgKJVKoWvXrsLatWvrPa/X64W33npLcHd3F5RKpXD//fcLp06dEqna1qXVaoXp06cL7du3F1QqldCxY0fhzTffrPdL0lzPx59//tngz4ixY8cKgtC4z52fny+MGjVKsLOzExwcHIRx48YJxcXFInya5rnVuUhPT7/pz9M///zT8B7mci4E4d//bvyvhsJUS5wPiSBct3wuERERETUJe6aIiIiImoFhioiIiKgZGKaIiIiImoFhioiIiKgZGKaIiIiImoFhioiIiKgZGKaIiIiImoFhioiIiKgZGKaIiIiImoFhiojMWm5uLiZNmoT27dtDqVTCw8MDkZGROHDgAABAIpFg27Zt4hZJRCZNLnYBREStadiwYaiqqsKGDRvQsWNHZGdnIyYmBvn5+WKXRkRmgnvzEZHZKiwshJOTE/766y8MGjTohuf9/Pxw4cIFw9cdOnTA+fPnAQA//fQT3nnnHaSmpsLLywtjx47Fm2++Cbm89t+gEokEq1atwvbt2/HXX3/B09MTS5YswZNPPtkmn42IjAcv8xGR2bKzs4OdnR22bduGysrKG54/cuQIAODLL7/E5cuXDV/v27cPY8aMwfTp05Gamoo1a9Zg/fr1ePfdd+u9/q233sKwYcOQlJSE0aNHY+TIkTh58mTrfzAiMiqcmSIis/bf//4XEyZMQHl5OXr37o1BgwZh5MiRCAoKAlA7w/Tjjz8iKirK8JqIiAjcf//9mD17tuHYN998g1mzZiErK8vwupdeegmffvqpYUy/fv3Qu3dvrFq1qm0+HBEZBc5MEZFZGzZsGLKysrB9+3Y89NBD+Ouvv9C7d2+sX7/+pq9JSkrCggULDDNbdnZ2mDBhAi5fvoyysjLDuPDw8HqvCw8P58wUkQViAzoRmT2VSoUHHngADzzwAN566y288MILmD9/Pp577rkGx5eUlOCdd97B0KFDG3wvIqLrcWaKiCxO9+7dUVpaCgCwsrKCTqer93zv3r1x6tQpdO7c+YaHVHrtx+ahQ4fqve7QoUPo1q1b638AIjIqnJkiIrOVn5+P4cOH4/nnn0dQUBDs7e0RHx+PJUuW4PHHHwdQe0dfTEwM7rrrLiiVSjg5OWHevHl45JFH0L59ezz55JOQSqVISkpCSkoKFi5caHj/rVu3IjQ0FAMGDMDGjRtx+PBhrFu3TqyPS0QiYQM6EZmtyspKvP322/j999/xzz//oLq6Gr6+vhg+fDjmzJkDa2tr/Pzzz4iOjsb58+fh7e1tWBrht99+w4IFC5CYmAgrKyt07doVL7zwAiZMmACgtgF95cqV2LZtG/bu3QtPT08sXrwYTz31lIifmIjEwDBFRHQbGroLkIgsE3umiIiIiJqBYYqIiIioGdiATkR0G9ghQUR1ODNFRERE1AwMU0RERETNwDBFRERE1AwMU0RERETNwDBFRERE1AwMU0RERETNwDBFRERE1AwMU0RERETN8P9tmurV9DpCmQAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.ylabel(\"Learning rate\")\n",
    "plt.xlabel(\"Step\")\n",
    "plt.plot(range(total_training_steps), track_lrs);"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e7512808-b48d-4146-86a1-5931b1e3aec1",
   "metadata": {},
   "source": [
    "## D.3 Gradient clipping"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c0a74f76-8d2b-4974-a03c-d645445cdc21",
   "metadata": {},
   "source": [
    "- Gradient clipping is yet another technique used to stabilize the training when training LLMs\n",
    "- By setting a threshold, gradients exceeding this limit are scaled down to a maximum magnitude to ensure that the updates to the model's parameters during backpropagation remain within a manageable range\n",
    "- For instance, using the `max_norm=1.0` setting in PyTorch's `clip_grad_norm_` method means that the norm of the gradients is clipped such that their maximum norm does not exceed 1.0\n",
    "- the \"norm\" refers to a measure of the gradient vector's length (or magnitude) in the parameter space of the model\n",
    "- Specifically, it's the L2 norm, also known as the Euclidean norm\n",
    "- Mathematically, for a vector $\\mathbf{v}$ with components $\\mathbf{v} = [v_1, v_2, \\ldots, v_n]$, the L2 norm is defined as:\n",
    "$$\n",
    "\\| \\mathbf{v} \\|_2 = \\sqrt{v_1^2 + v_2^2 + \\ldots + v_n^2}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d44838a6-4322-47b2-a935-c00d3a88355f",
   "metadata": {},
   "source": [
    "- The L2 norm is calculated similarly for matrices.\n",
    "- Let's assume our gradient matrix is:\n",
    "$$\n",
    "G = \\begin{bmatrix}\n",
    "1 & 2 \\\\\n",
    "2 & 4\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "- And we want to clip these gradients with a `max_norm` of 1.\n",
    "\n",
    "- First, we calculate the L2 norm of these gradients:\n",
    "$$\n",
    "\\|G\\|_2 = \\sqrt{1^2 + 2^2 + 2^2 + 4^2} = \\sqrt{25} = 5\n",
    "$$\n",
    "\n",
    "- Since $\\|G\\|_2 = 5$ is greater than our `max_norm` of 1, we need to scale down the gradients so that their norm is exactly 1. The scaling factor is calculated as $\\frac{max\\_norm}{\\|G\\|_2} = \\frac{1}{5}$.\n",
    "\n",
    "- Therefore, the scaled gradient matrix $G'$ will be as follows:\n",
    "$$\n",
    "G' = \\frac{1}{5} \\times G = \\begin{bmatrix}\n",
    "\\frac{1}{5} & \\frac{2}{5} \\\\\n",
    "\\frac{2}{5} & \\frac{4}{5}\n",
    "\\end{bmatrix}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eeb0c3c1-2cff-46f5-8127-24412184428c",
   "metadata": {},
   "source": [
    "- Let's see this in action\n",
    "- First, we initialize a new model and calculate the loss for a training batch like we would do in the regular training loop"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "e199e1ff-58c4-413a-855e-5edbe9292649",
   "metadata": {},
   "outputs": [],
   "source": [
    "from previous_chapters import calc_loss_batch\n",
    "\n",
    "torch.manual_seed(123)\n",
    "model = GPTModel(GPT_CONFIG_124M)\n",
    "\n",
    "loss = calc_loss_batch(input_batch, target_batch, model, device)\n",
    "loss.backward()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "76b60f3a-15ec-4846-838d-fdef3df99899",
   "metadata": {},
   "source": [
    "- If we call `.backward()`, PyTorch will calculate the gradients and store them in a `.grad` attribute for each weight (parameter) matrix\n",
    "- Let's define a utility function to calculate the highest gradient based on all model weights"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "e70729a3-24d1-411d-a002-2529cd3a8a9e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(0.0373)\n"
     ]
    }
   ],
   "source": [
    "def find_highest_gradient(model):\n",
    "    max_grad = None\n",
    "    for param in model.parameters():\n",
    "        if param.grad is not None:\n",
    "            grad_values = param.grad.data.flatten()\n",
    "            max_grad_param = grad_values.max()\n",
    "            if max_grad is None or max_grad_param > max_grad:\n",
    "                max_grad = max_grad_param\n",
    "    return max_grad\n",
    "\n",
    "print(find_highest_gradient(model))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "734f30e6-6b24-4d4b-ae91-e9a4b871113f",
   "metadata": {},
   "source": [
    "- Applying gradient clipping, we can see that the largest gradient is now substantially smaller:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "fa81ef8b-4280-400f-a93e-5210f3e62ff0",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(0.0166)\n"
     ]
    }
   ],
   "source": [
    "torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)\n",
    "print(find_highest_gradient(model))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b62c2af0-dac3-4742-be4b-4292c6753099",
   "metadata": {},
   "source": [
    "## D.4 The modified training function"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "76715332-94ec-4185-922a-75cb420819d5",
   "metadata": {},
   "source": [
    "- Now let's add the three concepts covered above (learning rate warmup, cosine decay, and gradient clipping) to the `train_model_simple` function covered in chapter 5 to create the more sophisticated `train_model` function below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "46eb9c84-a293-4016-a523-7ad726e171e9",
   "metadata": {},
   "outputs": [],
   "source": [
    "from previous_chapters import evaluate_model, generate_and_print_sample\n",
    "\n",
    "\n",
    "def train_model(model, train_loader, val_loader, optimizer, device, n_epochs,\n",
    "                eval_freq, eval_iter, start_context, warmup_steps=10,\n",
    "                initial_lr=3e-05, min_lr=1e-6):\n",
    "\n",
    "    train_losses, val_losses, track_tokens_seen, track_lrs = [], [], [], []\n",
    "    tokens_seen, global_step = 0, -1\n",
    "\n",
    "    # Retrieve the maximum learning rate from the optimizer\n",
    "    peak_lr = optimizer.param_groups[0][\"lr\"]\n",
    "\n",
    "    # Calculate the total number of iterations in the training process\n",
    "    total_training_steps = len(train_loader) * n_epochs\n",
    "\n",
    "    # Calculate the learning rate increment during the warmup phase\n",
    "    lr_increment = (peak_lr - initial_lr) / warmup_steps\n",
    "\n",
    "    for epoch in range(n_epochs):\n",
    "        model.train()\n",
    "        for input_batch, target_batch in train_loader:\n",
    "            optimizer.zero_grad()\n",
    "            global_step += 1\n",
    "\n",
    "            # Adjust the learning rate based on the current phase (warmup or cosine annealing)\n",
    "            if global_step < warmup_steps:\n",
    "                # Linear warmup\n",
    "                lr = initial_lr + global_step * lr_increment  \n",
    "            else:\n",
    "                # Cosine annealing after warmup\n",
    "                progress = ((global_step - warmup_steps) / \n",
    "                            (total_training_steps - warmup_steps))\n",
    "                lr = min_lr + (peak_lr - min_lr) * 0.5 * (1 + math.cos(math.pi * progress))\n",
    "\n",
    "            # Apply the calculated learning rate to the optimizer\n",
    "            for param_group in optimizer.param_groups:\n",
    "                param_group[\"lr\"] = lr\n",
    "            track_lrs.append(lr)  # Store the current learning rate\n",
    "\n",
    "            # Calculate and backpropagate the loss\n",
    "            loss = calc_loss_batch(input_batch, target_batch, model, device)\n",
    "            loss.backward()\n",
    "\n",
    "            # Apply gradient clipping after the warmup phase to avoid exploding gradients\n",
    "            if global_step > warmup_steps:\n",
    "                torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)\n",
    "            \n",
    "            optimizer.step()\n",
    "            tokens_seen += input_batch.numel()\n",
    "\n",
    "            # Periodically evaluate the model on the training and validation sets\n",
    "            if global_step % eval_freq == 0:\n",
    "                train_loss, val_loss = evaluate_model(\n",
    "                    model, train_loader, val_loader,\n",
    "                    device, eval_iter\n",
    "                )\n",
    "                train_losses.append(train_loss)\n",
    "                val_losses.append(val_loss)\n",
    "                track_tokens_seen.append(tokens_seen)\n",
    "                # Print the current losses\n",
    "                print(f\"Ep {epoch+1} (Iter {global_step:06d}): \"\n",
    "                      f\"Train loss {train_loss:.3f}, Val loss {val_loss:.3f}\")\n",
    "\n",
    "        # Generate and print a sample from the model to monitor progress\n",
    "        generate_and_print_sample(\n",
    "            model, train_loader.dataset.tokenizer,\n",
    "            device, start_context\n",
    "        )\n",
    "\n",
    "    return train_losses, val_losses, track_tokens_seen, track_lrs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "55fcd247-ba9d-4b93-a757-0f7ce04fee41",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Ep 1 (Iter 000000): Train loss 10.914, Val loss 10.940\n",
      "Ep 1 (Iter 000005): Train loss 8.903, Val loss 9.313\n",
      "Every effort moves you,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,\n",
      "Ep 2 (Iter 000010): Train loss 7.362, Val loss 7.789\n",
      "Ep 2 (Iter 000015): Train loss 6.273, Val loss 6.814\n",
      "Every effort moves you,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,\n",
      "Ep 3 (Iter 000020): Train loss 5.958, Val loss 6.609\n",
      "Ep 3 (Iter 000025): Train loss 5.675, Val loss 6.592\n",
      "Every effort moves you.                                                 \n",
      "Ep 4 (Iter 000030): Train loss 5.607, Val loss 6.565\n",
      "Ep 4 (Iter 000035): Train loss 5.063, Val loss 6.483\n",
      "Every effort moves you, and, and the to the to the to the to the to the, and, and the, and the, and, and, and the, and the, and, and the, and, and, and the, and, and the\n",
      "Ep 5 (Iter 000040): Train loss 4.384, Val loss 6.379\n",
      "Every effort moves you, I was, and I had been.                   \"I, I had the picture, as a little's his pictures, I had been, I was his\n",
      "Ep 6 (Iter 000045): Train loss 4.638, Val loss 6.306\n",
      "Ep 6 (Iter 000050): Train loss 3.690, Val loss 6.196\n",
      "Every effort moves you know the to me a little of his pictures--I had been.  \"I was the's--and, I felt to see a little of his pictures--I had been. \"I of Jack's \"strong. \"I\n",
      "Ep 7 (Iter 000055): Train loss 3.157, Val loss 6.148\n",
      "Ep 7 (Iter 000060): Train loss 2.498, Val loss 6.157\n",
      "Every effort moves you know it was not that, and he was to the fact of the of a and he was--his, the fact of the donkey, in the of the his head to have.   \"I had been his pictures--and by his\n",
      "Ep 8 (Iter 000065): Train loss 2.182, Val loss 6.178\n",
      "Ep 8 (Iter 000070): Train loss 1.998, Val loss 6.193\n",
      "Every effort moves you know,\" was not that my dear, his pictures--so handsome, in a so that he was a year after Jack's resolve had been his painting.     \"Oh, I had the donkey. \"There were, with his\n",
      "Ep 9 (Iter 000075): Train loss 1.824, Val loss 6.211\n",
      "Ep 9 (Iter 000080): Train loss 1.742, Val loss 6.201\n",
      "Every effort moves you know,\" was not that my hostess was \"interesting\": on that point I could have given Miss Croft the fact, and.         \"Oh, as I turned, and down the room, in his\n",
      "Ep 10 (Iter 000085): Train loss 1.285, Val loss 6.234\n",
      "Every effort moves you?\"  \"Yes--quite insensible to the fact with a little: \"Yes--and by me to me to have to see a smile behind his close grayish beard--as if he had the donkey. \"There were days when I\n",
      "Ep 11 (Iter 000090): Train loss 1.256, Val loss 6.236\n",
      "Ep 11 (Iter 000095): Train loss 0.803, Val loss 6.255\n",
      "Every effort moves you?\"  \"Yes--quite insensible to the irony. She wanted him vindicated--and by me!\"  He laughed again, and threw back his head to look up at the sketch of the donkey. \"There were days when I\n",
      "Ep 12 (Iter 000100): Train loss 0.731, Val loss 6.284\n",
      "Ep 12 (Iter 000105): Train loss 0.889, Val loss 6.299\n",
      "Every effort moves you?\"  \"Yes--quite insensible to the irony. She wanted him vindicated--and by me!\"  He laughed again, and threw back his head to look up at the sketch of the donkey. \"There were days when I\n",
      "Ep 13 (Iter 000110): Train loss 0.703, Val loss 6.316\n",
      "Ep 13 (Iter 000115): Train loss 0.517, Val loss 6.315\n",
      "Every effort moves you?\"  \"Yes--quite insensible to the irony. She wanted him vindicated--and by me!\"  He laughed again, and threw back his head to look up at the sketch of the donkey. \"There were days when I\n",
      "Ep 14 (Iter 000120): Train loss 0.594, Val loss 6.324\n",
      "Ep 14 (Iter 000125): Train loss 0.481, Val loss 6.325\n",
      "Every effort moves you?\"  \"Yes--quite insensible to the irony. She wanted him vindicated--and by me!\"  He laughed again, and threw back his head to look up at the sketch of the donkey. \"There were days when I\n",
      "Ep 15 (Iter 000130): Train loss 0.529, Val loss 6.324\n",
      "Every effort moves you?\"  \"Yes--quite insensible to the irony. She wanted him vindicated--and by me!\"  He laughed again, and threw back his head to look up at the sketch of the donkey. \"There were days when I\n"
     ]
    }
   ],
   "source": [
    "torch.manual_seed(123)\n",
    "model = GPTModel(GPT_CONFIG_124M)\n",
    "model.to(device)\n",
    "\n",
    "peak_lr = 5e-4\n",
    "optimizer = torch.optim.AdamW(model.parameters(), lr=peak_lr, weight_decay=0.1)\n",
    "\n",
    "n_epochs = 15\n",
    "train_losses, val_losses, tokens_seen, lrs = train_model(\n",
    "    model, train_loader, val_loader, optimizer, device, n_epochs=n_epochs,\n",
    "    eval_freq=5, eval_iter=1, start_context=\"Every effort moves you\",\n",
    "    warmup_steps=10, initial_lr=1e-5, min_lr=1e-5\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "827e8d5e-0872-4b90-98ac-200c80ee2d53",
   "metadata": {},
   "source": [
    "- Looking at the results above, we can see that the model starts out generating incomprehensible strings of words, whereas, towards the end, it's able to produce grammatically more or less correct sentences\n",
    "- If we were to check a few passages it writes towards the end, we would find that they are contained in the training set verbatim -- it simply memorizes the training data\n",
    "- Note that the overfitting here occurs because we have a very, very small training set, and we iterate over it so many times\n",
    "  - The LLM training here primarily serves educational purposes; we mainly want to see that the model can learn to produce coherent text\n",
    "  - Instead of spending weeks or months on training this model on vast amounts of expensive hardware, we load the pretrained weights"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9decec45-4fdf-4ff6-85a7-1806613f8af7",
   "metadata": {},
   "source": [
    "- A quick check that the learning rate behaves as intended"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "d8ebb8d2-8308-4a83-a2a6-730c3bf84452",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkgAAAGdCAYAAADpBYyuAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAABWSElEQVR4nO3deVxU5f4H8M+ZGZhBVhEZQEHRcCc1EMIlK7lheUvKUonUzCvWldKsNCu1W90ozepa/iLrltV1yzIrXIrQNBVRwX3XTNxgUGQGULaZ5/eHzsgQLhjMmeXzfr3m5W3OM/A951XO5z7neb5HEkIIEBEREZGFQu4CiIiIiOwNAxIRERFRHQxIRERERHUwIBERERHVwYBEREREVAcDEhEREVEdDEhEREREdTAgEREREdWhkrsAe2IymXD69Gl4e3tDkiS5yyEiIqIbIIRAaWkpQkJCoFA0ztwPA1Itp0+fRmhoqNxlEBER0U04ceIEWrdu3Sg/iwGpFm9vbwCXLrCPj4/M1RAREdGNMBgMCA0NtXyPNwYGpFrMt9V8fHwYkIiIiBxMYy6P4SJtIiIiojoYkIiIiIjqYEAiIiIiqoMBiYiIiKgOBiQiIiKiOhiQiIiIiOpgQCIiIiKqgwGJiIiIqA4GJCIiIqI6biogzZ07F23btoVGo0FsbCy2bNlyzfFLly5Fp06doNFoEBkZiZUrV1odF0Jg+vTpCA4OhoeHB+Lj43H48GGrMcXFxUhOToaPjw/8/PwwZswYlJWVWY7/8ccfkCTpT6/NmzffzCkSERGRC2twQFqyZAkmTZqEGTNmIC8vD927d0dCQgJ0Ol294zdt2oSkpCSMGTMG27dvR2JiIhITE7Fnzx7LmJkzZ2LOnDlIT09HTk4OPD09kZCQgIqKCsuY5ORk7N27F5mZmcjIyMD69euRkpLyp9/3yy+/4MyZM5ZXVFRUQ0+RiIiIXJ1ooJiYGDF+/HjLPxuNRhESEiLS0tLqHT906FAxaNAgq/diY2PFuHHjhBBCmEwmERQUJGbNmmU5XlJSItRqtVi0aJEQQoh9+/YJAGLr1q2WMatWrRKSJIlTp04JIYQ4duyYACC2b9/e0FOy0Ov1AoDQ6/U3/TOIiIjItpri+7tBD6utqqpCbm4upk6danlPoVAgPj4e2dnZ9X4mOzsbkyZNsnovISEBy5cvBwAcO3YMBQUFiI+Ptxz39fVFbGwssrOzMXz4cGRnZ8PPzw/R0dGWMfHx8VAoFMjJycGDDz5oef+BBx5ARUUFOnTogMmTJ+OBBx646vlUVlaisrLS8s8Gg+HGLoSD2vpHMX7eWwA3pQJqlRJqNwXUqsv/W6WAj4cbAr3VaOmtRoCXGu4qLlEjIiLX1KCAdPbsWRiNRmi1Wqv3tVotDhw4UO9nCgoK6h1fUFBgOW5+71pjAgMDrQtXqeDv728Z4+XlhdmzZ6NPnz5QKBT49ttvkZiYiOXLl181JKWlpeFf//rXjZy6U3hh6U78ce7CDY9v3swNLS8HpjYtPNG+pRduCfRC+5aeCPH1gELReE9NJiIisicNCkj2LCAgwGqmqlevXjh9+jRmzZp11YA0depUq88YDAaEhoY2ea1yMJkETp6/CABIigmFJEmorDahssaIyhoTKqqN0F+sRlFpJYpKK1FjEjh/oRrnL1TjUGEZNh45Z/XzNG4KtAvwQoTWC91b+6FnmB+6hPhArVLKcXpERESNqkEBKSAgAEqlEoWFhVbvFxYWIigoqN7PBAUFXXO8+c/CwkIEBwdbjenRo4dlTN1F4DU1NSguLr7q7wWA2NhYZGZmXvW4Wq2GWq2+6nFncv5CFWpMApIEvDa4G9yUV799ZjKJS2GprBI6QyUKDBX442w5jujKcLSoDH+cK0dFtQn7zhiw74wB3+84DQBwVynQLcQHPcOao2eYH6Lb+CPIV2OrUyQiImo0DQpI7u7uiIqKQlZWFhITEwEAJpMJWVlZSE1NrfczcXFxyMrKwsSJEy3vZWZmIi4uDgAQHh6OoKAgZGVlWQKRwWBATk4OnnrqKcvPKCkpQW5urmVX2po1a2AymRAbG3vVenfs2GEVulxZoeHSWqsWnu7XDEcAoFBIaO7pjuae7uig9f7T8RqjCfnFF3C0qBz7zxiw40QJtuefx/kL1cjLL0FefollbEetN/p3bIn+HVoium1zzjAREZFDaPAttkmTJmHUqFGIjo5GTEwM3n//fZSXl2P06NEAgJEjR6JVq1ZIS0sDAEyYMAH9+/fH7NmzMWjQICxevBjbtm3DvHnzAACSJGHixIl44403EBERgfDwcEybNg0hISGWENa5c2cMHDgQY8eORXp6Oqqrq5Gamorhw4cjJCQEAPDFF1/A3d0dPXv2BAAsW7YMn332GT799NO/fJGcga70UsuEQO+/PqOjUirQrqUX2rX0wt+6XFo7JoTA8XMXsP3EeeQdL0Fe/nnsO2PAwcJSHCwsxbz1v8PDTYne7Vugf8eWGNBZi1Z+Hn+5FiIioqbQ4IA0bNgwFBUVYfr06SgoKECPHj2wevVqyyLr/Px8KBRXZih69+6NhQsX4pVXXsFLL72EiIgILF++HN26dbOMmTx5MsrLy5GSkoKSkhL07dsXq1evhkZz5ct8wYIFSE1NxYABA6BQKDBkyBDMmTPHqrbXX38dx48fh0qlQqdOnbBkyRI8/PDDDb4ozkh3eQZJ69M0txQlSULbAE+0DfDEgz1bAwDOl1dhw5GzWHeoCOsOFaGotBJZB3TIOqDD9O/3IqpNc/z91mAMigxGoA9vxRERkf2QhBBC7iLshcFggK+vL/R6PXx8fOQup1F9kHUYszMPYVh0KN5++Fab/34hBPafKcW6Q0VYe0CHrceLYf43T5KA2HB/3N89BPd2C4a/p7vN6yMiIsfVFN/fTrOLja5NV9q0M0jXI0kSuoT4oEuID566sz0KDRVYsesMMnadRl5+CTb/XozNvxdj+vd7MaBTIB6NDcMdES3ZSoCIiGTBgOQiCg2X1iC1tJNbWVofDZ7oG44n+obj5PkLWLHrDH7cdRp7Thnw875C/LyvEK2beyApJgyPRLdulLVTREREN4oByUUUmmeQvO2vrUHr5s0wrn97jOvfHocKS7EwJx/L8k7i5PmLmPXTQbyXeQh/66JFcmwb9LmlBSSJs0pERNS0GJBcRNHlGSStncwgXU0HrTdefaArpgzshBW7z2BhznHk5Zdg1Z4CrNpTgM7BPniyfzsMigyG6jrtCoiIiG4Wv2FcgMkkLGuQAmVag9RQHu5KPBzVGsv+2QerJvTDiNvboJm7EvvPGDBh8Q70n/UrPt94DBeqauQulYiInBADkgsortVFO8DLMQJSbZ2DffB6YjdsevFuPH9PB7TwdMepkov414/70PutNXg38xDOlVVe/wcRERHdIAYkF6CzdNFWX7eLtj3za+aO1LsjsPHFu/FGYje0adEMJReqMSfrMPrNXItZPx2A/kK13GUSEZETcNxvS7phhZYu2o43e1QfjZsSj93eBmueuxP/l3wbIlv54kKVEXPXHkW/mWvw4ZrDKK/krTciIrp5DEguoKiJu2jLRamQcF9kMH5I7YN5I6LQUesNQ0UN3vn5EO6YuRaf/vY7KqqNcpdJREQOiAHJBRQ6yA62myVJEu7pGoRVE/rhP8N7oG2LZjhXXoU3VuzHnbN+xZKt+TCa2DCeiIhuHAOSC3C2W2xXo1BIGNyjFTIn9cdbD0UixFeDAkMFpny7G4PnbsDWP4rlLpGIiBwEA5ILMC/SdpUHwropFRgeE4Y1z9+JVwZ1hrdGhT2nDHgkPRupC/NwquSi3CUSEZGdY0ByAeYu2s4+g1SXxk2Jf/Rrh1+fvxOPxoZBkoCMXWdw9zu/4t3MQ+yhREREV8WA5AJ0Tr4G6XpaeKnx5oORyHi6L2LD/VFZY8KcrMMYMHsdVu0+AyG4PomIiKwxIDk5k0mgyPwcNhcNSGZdQ3yxOOV2fJR8G1o398AZfQWeWpCHsV/m4jRvuxERUS0MSE7Ouou2u9zlyE6SJNwbGYxfJvXHM3ffAjelhF/2F+Jv767D/I3HuNuNiIgAMCA5PfMW/xaeaj7ctRaNmxKT7umIFc/0Q1Sb5iivMuLVH/fhoY82Yf8Zg9zlERGRzPiN6eR0pc7ZJLKxdNB6Y+m4OLyR2A3eahV2nijB3z/YgLdWHWCTSSIiF8aA5OTMC7RdbQdbQygUEh67vQ1+ea4/7u0WBKNJIH3dUdz/wQbsOaWXuzwiIpIBA5KT0xm4QPtGaX00+OixKMwbEYUALzUO68qQOHcj5mQdRo3RJHd5RERkQwxITs7SRZsB6Ybd0zUIPz97B+6LDEKNSeDdzEMY8tEmHNGVyV0aERHZCAOSkys0uGaTyL/K39Mdcx+9Df8Z3gM+GhV2ntRj0Jzf8NmGYzBxpxsRkdNjQHJyOvZAummSdOnZbj89ewf6RQSgssaE1zL24bH/5lh2BxIRkXNiQHJyV7pocwbpZgX7euDLJ2LwemI3eLgpsenoOdz3n9/w60Gd3KUREVETYUByYrW7aAd6cwbpr5AkCSNub4OMZ/qic7APzpVX4fHPtyJt5X5UcwE3EZHTYUByYuyi3fjat/TCd//sjZFxbQAAH6//HY+kZ+NE8QWZKyMiosbEgOTE2EW7aWjclHhtcDekPxYFH40KO06U4L45v2Hl7jNyl0ZERI2E35pO7EoPJK4/agoDuwVh5YR+uC3MD6UVNfjngjxM/34Pqmp4y42IyNExIDkxXal5gTbXHzWV1s2bYcm4OPzzzvYAgC+zjyPpk83c5UZE5OAYkJwYeyDZhptSgckDO+G/o6LhrVEh9/h5/P2DDdhyrFju0oiI6CYxIDkx8ywGu2jbxoDOWvyY2hcdtd4oKq3Eo59sxvyNxyAEG0sSETkaBiQndqVJJGeQbKVtgCe+G98b93cPQY1J4NUf9+HZJTtwscood2lERNQADEhOzNwkkj2QbKuZuwpzhvfAtL93gVIhYfmO03joo01sBUBE5EAYkJwYZ5DkI0kSxvQNx4J/xCLAyx37zxgweO5GbP2D65KIiBwBA5KTMpkEn8NmB25v1wI/Pt0X3Vr5oLi8Csmf5OCb3JNyl0VERNfBgOSkzpVXwXi5i3YLT3bRllOwrwe+HheHe7sFocpowvNLdyJt1X4YTVy8TURkrxiQnJS5B1KAF7to24Nm7irMffQ2PH33LQCAj9f9jnFf5aKsskbmyoiIqD785nRSOvZAsjsKhYTn7umI/wzvAXeVAr/sL8TDH23CyfNcvE1EZG8YkJyUuQcS1x/Zn8E9WmFJyu0I8FLjQEEpEuduxM4TJXKXRUREtTAgOSnuYLNvPcOa4/vUPugc7IOzZVUYPm8zsvYXyl0WERFdxoDkpMwzSC3ZA8lutfLzwNIn49AvIgAXq40Y++U2LMg5LndZREQEBiSnZX4OG2eQ7JuXWoXPHu+FR6JawySAl7/bg3d+OsjHkxARyYwByUkVXd7FpuUMkt1zUyow8+FbMWFABADgw7VH8NzSnaiqMclcGRGR62JAclLmGaRAziA5BEmS8OzfOuDtIZFQKiQsyzuFJ+ZvhaGiWu7SiIhcEgOSEzKZBIrK2EXbEQ3rFYZPR0WjmbsSG46cxbCPN1t6WhERke0wIDkhcxdtBbtoO6S7OgZiSUocArzU2H/GgEfSs/mgWyIiG2NAckLmHWwt2EXbYUW29sU3T8ahdXMPHD93AQ+nb8LBglK5yyIichn89nRCReyB5BTaBnji26d6o4PWC4WGSgz9OBt5+eflLouIyCUwIDkh8wxSIHewOTytjwZfj4tDzzA/6C9WI/mTHPx2uEjusoiInB4DkhNiDyTn4tfMHQv+EWtpKPnE/K1YseuM3GURETk1BiQnZN71xBkk59HMXYVPR0VjUGQwqo0CqYvysHhLvtxlERE5LQYkJ8QeSM5JrVJiTlJPPBobBiGAF5ftxheb/pC7LCIip8SA5IR07KLttJQKCf9O7Iax/cIBADN+2It564/KXBURkfNhQHJCOgObRDozSZLw0n2d8fTdtwAA3lx5AB9kHZa5KiIi58KA5GSMtbpo8xab85IkCc/d0xHP/a0DAGB25iHM+ukAH3JLRNRIGJCcTDG7aLuUpwdE4OX7OgMA5q49in+v2M+QRETUCBiQnIy5B1IAu2i7jLF3tMNrg7sCAD7dcAzTv98Lk4khiYjor+A3qJOxbPHn7TWXMjKuLd4eEglJAr7afByvfL+HIYmI6C+4qYA0d+5ctG3bFhqNBrGxsdiyZcs1xy9duhSdOnWCRqNBZGQkVq5caXVcCIHp06cjODgYHh4eiI+Px+HD1otOi4uLkZycDB8fH/j5+WHMmDEoKyur9/cdOXIE3t7e8PPzu5nTc2iWBdrcweZyhvUKw7tDu0MhAQtz8jHt+z283UZEdJMaHJCWLFmCSZMmYcaMGcjLy0P37t2RkJAAnU5X7/hNmzYhKSkJY8aMwfbt25GYmIjExETs2bPHMmbmzJmYM2cO0tPTkZOTA09PTyQkJKCiosIyJjk5GXv37kVmZiYyMjKwfv16pKSk/On3VVdXIykpCf369WvoqTkF9kBybQ/2bI13HukOSQIWMCQREd080UAxMTFi/Pjxln82Go0iJCREpKWl1Tt+6NChYtCgQVbvxcbGinHjxgkhhDCZTCIoKEjMmjXLcrykpESo1WqxaNEiIYQQ+/btEwDE1q1bLWNWrVolJEkSp06dsvrZkydPFo899pj4/PPPha+vb4POTa/XCwBCr9c36HP2ZOqyXaLNlAzx7s8H5S6FZPTNthOi7YsZos2UDPHKd7uFyWSSuyQioibTFN/fDZpBqqqqQm5uLuLj4y3vKRQKxMfHIzs7u97PZGdnW40HgISEBMv4Y8eOoaCgwGqMr68vYmNjLWOys7Ph5+eH6Ohoy5j4+HgoFArk5ORY3luzZg2WLl2KuXPn3tD5VFZWwmAwWL0cHXsgEQAMiWqNWQ93t6xJevWHvZxJIiJqgAYFpLNnz8JoNEKr1Vq9r9VqUVBQUO9nCgoKrjne/Of1xgQGBlodV6lU8Pf3t4w5d+4cHn/8ccyfPx8+Pj43dD5paWnw9fW1vEJDQ2/oc/bsynPYeIvN1T0c1Rozh9wKSQK+yD6Of/24jyGJiOgGOc0utrFjx+LRRx/FHXfcccOfmTp1KvR6veV14sSJJqzQNszb/DmDRADwSHQo3n7oUkiav+kPvJbBkEREdCMaFJACAgKgVCpRWFho9X5hYSGCgoLq/UxQUNA1x5v/vN6YuovAa2pqUFxcbBmzZs0avPPOO1CpVFCpVBgzZgz0ej1UKhU+++yzemtTq9Xw8fGxejkyo0ngbFkVAEDLRdp02dBel0ISAHy+8Q+8vfogQxIR0XU0KCC5u7sjKioKWVlZlvdMJhOysrIQFxdX72fi4uKsxgNAZmamZXx4eDiCgoKsxhgMBuTk5FjGxMXFoaSkBLm5uZYxa9asgclkQmxsLIBL65R27Nhheb322mvw9vbGjh078OCDDzbkNB3WufLKK120vRiQ6IqhvULx5oORAID0dUfxwZojMldERGTfVA39wKRJkzBq1ChER0cjJiYG77//PsrLyzF69GgAwMiRI9GqVSukpaUBACZMmID+/ftj9uzZGDRoEBYvXoxt27Zh3rx5AC49U2rixIl44403EBERgfDwcEybNg0hISFITEwEAHTu3BkDBw7E2LFjkZ6ejurqaqSmpmL48OEICQmxjKlt27ZtUCgU6Nat201fHEdjXqAd4KWGUiHJXA3Zm0djw3Cx2ojXM/bh3cxD0LgpkHJHe7nLIiKySw0OSMOGDUNRURGmT5+OgoIC9OjRA6tXr7Ysss7Pz4dCcWViqnfv3li4cCFeeeUVvPTSS4iIiMDy5cutgsvkyZNRXl6OlJQUlJSUoG/fvli9ejU0mivraBYsWIDU1FQMGDAACoUCQ4YMwZw5c/7KuTsd8wJtrj+iqxnTNxwV1UbM+ukg3lx5AB5uSoyIayt3WUREdkcSXIxgYTAY4OvrC71e75DrkRZtycfUZbsxoFMg/vt4L7nLITv2zk8H8eHaS7fZZj58K4ZGO/4OTiJyXU3x/e00u9joyi22QM4g0XU8d08HPNEnHADw4re78MPO0zJXRERkXxiQnEgheyDRDZIkCdP+3hmPxobBJIBnl+zAz3vr72VGROSKGJCciI49kKgBJEnCG4O74aGerWA0CaQu3I5NR8/KXRYRkV1gQHIiulLzY0Y4g0Q3RqGQMPPhW5HQVYsqowljv9iGXSdL5C6LiEh2DEhOxNxFO9CbM0h041RKBf4zvCfi2rVAeZURj3++FUeLyuQui4hIVgxITsJoEijiDBLdJI2bEvNGRiGylS+Ky6sw4tMcnC65KHdZRESyYUByEufKK2ESYBdtumneGjfMH90L7Vp64rS+AiP+m4Pi8iq5yyIikgUDkpNgF21qDC281PhqTCyCfTU4WlSO0Z9vQVlljdxlERHZHAOSkyjkDjZqJK38PPDVmBg0b+aGnSf1GPfVNlTWGOUui4jIphiQnAR3sFFjuiXQG/NHx8DTXYmNR85h4uIdMJrYdJ+IXAcDkpMwzyC15A42aiTdQ/0wb2Q03JUKrNpTgJe/2w0+mYiIXAUDkpPgDBI1hT63BGBOUg8oJGDx1hOY+dNBuUsiIrIJBiQnwS7a1FQGdgvGmw9GAgA++vUo5q0/KnNFRERNjwHJSRSaH1TL57BRExgeE4YpAzsBAN5ceQDf7zglc0VERE2LAclJ6Eo5g0RN68n+7fBEn3AAwPNLd2LTET63jYicFwOSE6jdRZszSNRUJEnCK4M6Y1BkMKqNAuO+ysX+Mwa5yyIiahIMSE7gXBm7aJNtKBQSZg/tjphwf5RW1mD051v5SBIickoMSE7AvIOtpTe7aFPT07gp8cmIaEQEeqHAUIHHP98C/cVqucsiImpUDEhOwNwDKZA9kMhGfJu5Yf4TMdD6qHGosAwpX7LbNhE5FwYkJ2DewcYeSGRLrfw88PnjMfBSq5BzrBjPfb0TJnbbJiInwYDkBMw72AK5g41srEuIDz4eEQU3pYSMXWeQtmq/3CURETUKBiQnwB5IJKc+twRg1sPdAQCf/HYMn204JnNFRER/HQOSEyhiDySSWWLPVpZGkq+v2IeVu8/IXBER0V/DgOQEuAaJ7MGT/dthxO1tIAQwcckObDlWLHdJREQ3jQHJCXAXG9kDSZLw6gNdcU8XLapqTBj75TYcLSqTuywiopvCgOTgjCaBs2WX1yBxBolkplRImJPUEz3D/KC/WI0n5m9FcXmV3GURETUYA5KDs+qi7cmARPLTuCnxychohPp74Pi5C0j5chsqqtkjiYgcCwOSgzOvP2IXbbInAV5qfP54L3hrVNh2/Dwmf7MLQrBHEhE5DgYkB6fjDjayU7cEeuPjx6KgUkj4YedpvJd5SO6SiIhuGAOSg2MPJLJnvW8JwJsPRQIA5qw5gm9yT8pcERHRjWFAcnCWHWycQSI7NTQ6FP+8sz0AYOqyXcg+ek7mioiIro8BycHpSi/3QOIWf7Jjz9/TEYNuDUa1UeDJ/+Vy+z8R2T0GJAens8wg8RYb2S+FQsLsR7pbtv+P/nwrzl1uT0FEZI8YkBycZQaJAYnsXO3t//nFF5DyVS63/xOR3WJAcnDsok2OpPb2/9zj5/HCN7tgMnH7PxHZHwYkB8Yu2uSIam///3Hnabz3C7f/E5H9YUByYOYu2kqFxC7a5FBqb///YM0RfLed2/+JyL4wIDkwcw+kAC93dtEmhzM0OhRPXd7+P+Wb3cg9fl7mioiIrmBAcmDm9Ufsok2O6oV7OuKeLlpUGU0Y99U2nDx/Qe6SiIgAMCA5NPMONi7QJkelUEh4b1gPdA72wdmyKvzji20or6yRuywiIgYkR1bIHkjkBDzVKnw6KhoBXmocKCjFxCU7uLONiGTHgOTALA+q5QwSObhWfh74eEQU3FUKZO4rxKyfD8pdEhG5OAYkB6YzsEkkOY+oNs0xc8itAICPfj2Kb/lgWyKSEQOSAyss5S02ci6JPVsh9a5bAABTl+1G7vFimSsiIlfFgOTAzDNIXKRNzmTS3zogoeulnW0pX+ZyZxsRyYIByUHVGE2WLtrc5k/OxLyzrUuwD86VX9rZVsadbURkYwxIDupceVWtLtrucpdD1KiauV/a2dbS+/LOtsXbYeTONiKyIQYkB2W+vdbSSw0Fu2iTEwrx88C8yzvbftmvw8zVB+QuiYhcCAOSg2IPJHIFPcOaY9bDl3a2fbz+d+5sIyKbYUByUJYdbFygTU5ucI9aO9u+240dJ0rkLYiIXAIDkoNiDyRyJZP+1gHxnbWoqjEh5cttlhlUIqKmwoDkoHScQSIXcmlnW3dEBHpBV1qJlK9yUVFtlLssInJiDEgOqpAzSORivDVu+HRUNHw93LDzRAle/m4PhODONiJqGgxIDsryHDb2QCIX0qaFJ+Y+ehuUCgnf5p3Efzcck7skInJSDEgOyjyD1NKbM0jkWvpGBODl+zoDAN5cuR/rDxXJXBEROSMGJAdUYzThHLtokwsb3actHolqDZMAUhfm4djZcrlLIiInw4DkgNhFm1ydJEl448FuuC3MD4aKGoz9chtKK6rlLouInMhNBaS5c+eibdu20Gg0iI2NxZYtW645funSpejUqRM0Gg0iIyOxcuVKq+NCCEyfPh3BwcHw8PBAfHw8Dh8+bDWmuLgYycnJ8PHxgZ+fH8aMGYOysjLL8YMHD+Kuu+6CVquFRqNBu3bt8Morr6C62vn+0jRvcWYXbXJlapUS6Y9FIchHgyO6MkxcvIOPIyGiRtPggLRkyRJMmjQJM2bMQF5eHrp3746EhATodLp6x2/atAlJSUkYM2YMtm/fjsTERCQmJmLPnj2WMTNnzsScOXOQnp6OnJwceHp6IiEhARUVV3qdJCcnY+/evcjMzERGRgbWr1+PlJQUy3E3NzeMHDkSP//8Mw4ePIj3338fn3zyCWbMmNHQU7R77IFEdEmgjwYfX34cSdYBHWb/fFDukojIWYgGiomJEePHj7f8s9FoFCEhISItLa3e8UOHDhWDBg2yei82NlaMGzdOCCGEyWQSQUFBYtasWZbjJSUlQq1Wi0WLFgkhhNi3b58AILZu3WoZs2rVKiFJkjh16tRVa3322WdF3759b/jc9Hq9ACD0ev0Nf0YO/9v8h2gzJUOMmb/1+oOJXMB3eSdFmykZos2UDPH9jqv/nUBEzqkpvr8bNINUVVWF3NxcxMfHW95TKBSIj49HdnZ2vZ/Jzs62Gg8ACQkJlvHHjh1DQUGB1RhfX1/ExsZaxmRnZ8PPzw/R0dGWMfHx8VAoFMjJyan39x45cgSrV69G//79G3KKDoE9kIisJfZshXF3tAMATP5mJ/ac0stcERE5ugYFpLNnz8JoNEKr1Vq9r9VqUVBQUO9nCgoKrjne/Of1xgQGBlodV6lU8Pf3/9Pv7d27NzQaDSIiItCvXz+89tprVz2fyspKGAwGq5cjKGIPJKI/mTywE+7s2BIV1ZceR1JUWil3SUTkwJxuF9uSJUuQl5eHhQsXYsWKFXjnnXeuOjYtLQ2+vr6WV2hoqA0rvXnmGaRA9kAislAqJPxneE+0a+mJ0/oKPPW/XFTVmOQui4gcVIMCUkBAAJRKJQoLC63eLywsRFBQUL2fCQoKuuZ485/XG1N3EXhNTQ2Ki4v/9HtDQ0PRpUsXJCUl4a233sKrr74Ko7H+ZzZNnToVer3e8jpx4sS1Tt9umHexcQaJyJqvhxs+GRkNb40K246fx4wf+DgSIro5DQpI7u7uiIqKQlZWluU9k8mErKwsxMXF1fuZuLg4q/EAkJmZaRkfHh6OoKAgqzEGgwE5OTmWMXFxcSgpKUFubq5lzJo1a2AymRAbG3vVek0mE6qrq2Ey1f//ItVqNXx8fKxejkB3+dZBINcgEf1J+5ZemJPUE5IELNpyAl9tPi53SUTkgFQN/cCkSZMwatQoREdHIyYmBu+//z7Ky8sxevRoAMDIkSPRqlUrpKWlAQAmTJiA/v37Y/bs2Rg0aBAWL16Mbdu2Yd68eQAuNXybOHEi3njjDURERCA8PBzTpk1DSEgIEhMTAQCdO3fGwIEDMXbsWKSnp6O6uhqpqakYPnw4QkJCAAALFiyAm5sbIiMjoVarsW3bNkydOhXDhg2Dm5tbY1wru1BjNOFsmfkWG2eQiOpzV8dAvDiwE9JWHcBrP+5DR603Ytu1kLssInIgDQ5Iw4YNQ1FREaZPn46CggL06NEDq1evtiyyzs/Ph0JxZWKqd+/eWLhwIV555RW89NJLiIiIwPLly9GtWzfLmMmTJ6O8vBwpKSkoKSlB3759sXr1amg0VwLAggULkJqaigEDBkChUGDIkCGYM2fOlRNRqfD222/j0KFDEEKgTZs2SE1NxbPPPntTF8ZenS2rgmAXbaLrSrmjHfaeNuCHnacxfmEefkjtixA/D7nLIiIHIQneoLcwGAzw9fWFXq+329ttu06W4IEPNyLYV4PsqQPkLofIrl2sMuKhjzZh/xkDurf2xZJxcdC4KeUui4gaWVN8fzvdLjZnxx1sRDfOw12JeSOi4NfMDTtP6vHKci7aJqIbw4DkYHSXeyAFcgcb0Q0J9W+GD5Nug0ICvsk9iS+zuWibiK6PAcnBcAaJqOH6RgRg6r2dAQCvZ+xDzu/nZK6IiOwdA5KD0bEHEtFN+Ue/cAzuEYIak8A/F+ThdMlFuUsiIjvGgORgzD2Q+Bw2ooaRJAlvPXQrugT74Fx5FZ78Xy4qqutvIktExIDkYMxdtNkDiajhPNyV+HhEFJo3c8Ouk3q8/B0XbRNR/RiQHIxlDRJnkIhuSqh/M3z46KVF29/mncQXm/6QuyQiskMMSA6kxmjCuXLzLTbOIBHdrD63BOCl+y4v2l6xH5u5aJuI6mBAciDmLtoqhQT/ZuyiTfRXjOkbjsQeITCaBMYvyMMpLtomoloYkByIef1RS281FApJ5mqIHJskSUirvWj7Ky7aJqIrGJAciHkHG5tEEjWO2ou2d5/S46XvdnPRNhEBYEByKFd2sHGBNlFjCfVvhrmP3galQsKyvFOYz0XbRAQGJIfCHkhETaP3LQGYem8nAMAbK/Yj+ygXbRO5OgYkB2Lpos0eSESNzmrR9kIu2iZydQxIDsRyi40zSESNzrxou2uID4rLqzDuq21ctE3kwhiQHAgXaRM1LfOibX9Pd+w5ZcBLy7hom8hVMSA5EEsXbS7SJmoyrZs3w4eP9ry0aHv7KXy+8Q+5SyIiGTAgOQh20Saynd7tr3Ta/vfK/dh09KzMFRGRrTEgOQh20SayrSf6tMWDPVvBaBJIXbgdJ89fkLskIrIhBiQHwS7aRLZ1adF2JLq1urRo+8n/sdM2kSthQHIQV3aw8fYaka1o3JT4eES0ZdH2VC7aJnIZDEgOwtIkkgu0iWyqlZ+HpdP2d9tP4TMu2iZyCQxIDkLHHkhEsolr3wIvX160/SYXbRO5BAYkB3FlBom32IjkMLpPWzzERdtELoMByUGY1yBxiz+RPCRJwpu1Fm2P+yoXF6u4aJvIWTEgOQhzk8iWvMVGJJvai7b3njZg6rJdXLRN5KQYkBwEb7ER2Yfai7aX7zjNRdtETooByQFU1+qizUXaRPLjom0i58eA5ADOllWyizaRnRndpy0euo2LtomcFQOSA9DVekgtu2gT2QdJkvDmg5GIbOXLRdtETogByQFYHjPCHWxEdkXjpkT6iCgu2iZyQgxIDqCQXbSJ7FbdRdv/3XBM7pKIqBEwIDmAIvZAIrJrtRdtp606gE1HuGibyNExIDmAwlprkIjIPtVetD1+YR5OFHPRNpEjY0ByALpSziAR2bvai7bPX6jmom0iB8eA5AAsM0jsgURk18yLtlt4umPfGS7aJnJkDEgOwDyDFMgu2kR2r5WfBz7kom0ih8eAZOcuddGuAgBoOYNE5BDi2rfAK4O4aJvIkTEg2bnaXbSbs4s2kcN4vDcXbRM5MgYkO1fILtpEDomLtokcGwOSndNd7oEUyB1sRA6n7qLtF7lom8hhMCDZOXMXbfZAInJMtRdtf89F20QOgwHJzunYRZvI4dVetP3myv3YyEXbRHaPAcnO6S6vQeIONiLHZl60bRJAKhdtE9k9BiQ7V8geSEROgYu2iRwLA5KdYxdtIuehcVPiYy7aJnIIDEh2rojPYSNyKiF+HpibzEXbRPaOAcmOVRtNOFt2qYs2d7EROY/b23HRNpG9Y0CyY2fLLt1ec1OyizaRs+GibSL7xoBkx8zrj1p6sYs2kbPhom0i+8aAZMcK2UWbyKnVXbQ95Vsu2iayFwxIdkxXyh5IRM7OvGhbpZDww87T+OS33+UuiYjAgGTXLM9hYw8kIqd2e7sWmPb3LgCAt1YdwK8HdTJXREQMSHas0PKYEc4gETm7kXFtMCw6FCYBPL1oO44WlcldEpFLY0CyY+ZbbFyDROT8JEnCa4ldEdWmOUorajD2y20wVFTLXRaRy2JAsmOWLtrsgUTkEtQqJdIfi0Kwrwa/F5VjwqLtMJq4aJtIDgxIdkxnYBdtIlfT0luNeSOioVYpsPZgEWb9dFDukohcEgOSnao2mnCu/FIXbQYkItcS2doXMx++FQCQvu4ovt9xSuaKiFzPTQWkuXPnom3bttBoNIiNjcWWLVuuOX7p0qXo1KkTNBoNIiMjsXLlSqvjQghMnz4dwcHB8PDwQHx8PA4fPmw1pri4GMnJyfDx8YGfnx/GjBmDsrIrixh//fVXDB48GMHBwfD09ESPHj2wYMGCmzk9u1BUWruLtpvM1RCRrQ3u0QpP9m8PAJj8zS7sOlkib0FELqbBAWnJkiWYNGkSZsyYgby8PHTv3h0JCQnQ6erflrpp0yYkJSVhzJgx2L59OxITE5GYmIg9e/ZYxsycORNz5sxBeno6cnJy4OnpiYSEBFRUVFjGJCcnY+/evcjMzERGRgbWr1+PlJQUq99z66234ttvv8WuXbswevRojBw5EhkZGQ09RbtgWaDtrYEksYs2kSt6IaEj7u4UiMoaE1K+zIWutOL6HyKiRiGJBrZtjY2NRa9evfDhhx8CAEwmE0JDQ/H000/jxRdf/NP4YcOGoby83Cqo3H777ejRowfS09MhhEBISAiee+45PP/88wAAvV4PrVaL+fPnY/jw4di/fz+6dOmCrVu3Ijo6GgCwevVq3HfffTh58iRCQkLqrXXQoEHQarX47LPPbujcDAYDfH19odfr4ePj05DL0uh+2luAcV/lokeoH5aP7yNrLUQkH0NFNR6cuxFHi8pxW5gfFqXcDrVKKXdZRHalKb6/GzSDVFVVhdzcXMTHx1/5AQoF4uPjkZ2dXe9nsrOzrcYDQEJCgmX8sWPHUFBQYDXG19cXsbGxljHZ2dnw8/OzhCMAiI+Ph0KhQE5OzlXr1ev18Pf3v+rxyspKGAwGq5e90LEHEhEB8NG44ZOR0fDWqJCXX4Lpy/fycSRENtCggHT27FkYjUZotVqr97VaLQoKCur9TEFBwTXHm/+83pjAwECr4yqVCv7+/lf9vV9//TW2bt2K0aNHX/V80tLS4Ovra3mFhoZedaytXXnMCBdoE7m6di298OGjt0EhAUu2ncAXm/6QuyQip+eUu9jWrl2L0aNH45NPPkHXrl2vOm7q1KnQ6/WW14kTJ2xY5bVZHlTLHkhEBKB/h5aYem9nAMDrK/Zj45GzMldE5NwaFJACAgKgVCpRWFho9X5hYSGCgoLq/UxQUNA1x5v/vN6YuovAa2pqUFxc/Kffu27dOtx///147733MHLkyGuej1qtho+Pj9XLXliaRHIGiYgu+0e/cDzYsxWMJoHxC/OQf+6C3CUROa0GBSR3d3dERUUhKyvL8p7JZEJWVhbi4uLq/UxcXJzVeADIzMy0jA8PD0dQUJDVGIPBgJycHMuYuLg4lJSUIDc31zJmzZo1MJlMiI2Ntbz366+/YtCgQXj77betdrg5It5iI6K6JElC2kOR6N7aFyUXqvGPL7eirLJG7rKInFKDb7FNmjQJn3zyCb744gvs378fTz31FMrLyy1rfUaOHImpU6daxk+YMAGrV6/G7NmzceDAAbz66qvYtm0bUlNTAVz6D37ixIl444038MMPP2D37t0YOXIkQkJCkJiYCADo3LkzBg4ciLFjx2LLli3YuHEjUlNTMXz4cMsOtrVr12LQoEF45plnMGTIEBQUFKCgoADFxcV/9RrJQsdbbERUD42bEh+PiEagtxqHCsvw7JIdMPFxJESNT9yEDz74QISFhQl3d3cRExMjNm/ebDnWv39/MWrUKKvxX3/9tejQoYNwd3cXXbt2FStWrLA6bjKZxLRp04RWqxVqtVoMGDBAHDx40GrMuXPnRFJSkvDy8hI+Pj5i9OjRorS01HJ81KhRAsCfXv3797/h89Lr9QKA0Ov1N34xmkBltVG0mZIh2kzJEOfKKmWthYjsU+7xYhHx0krRZkqGmP3TAbnLIZJVU3x/N7gPkjOzlz5Ip0suovdba+CmlHDojXvZKJKI6vVN7kk8v3QnAGBOUk880L3+nnBEzk72PkhkG1d2sLGLNhFd3cNRrZFyRzsAwAtLd2LHiRJ5CyJyIgxIdsjymBE2iSSi65gysBMGXH4cydgvt+GM/qLcJRE5BQYkO2Tpou3NHWxEdG1KhYT3h/dAR603ikorMfbLbbhQxZ1tRH8VA5IdutIDiTNIRHR93ho3fDoqGi083bHnlAHPfb2TO9uI/iIGJDtkfmI3eyAR0Y0K9W+G9BFRcFcqsGpPAd775ZDcJRE5NAYkO2SeQWrJHkhE1AC92vrjzYciAQAfrDmC73eckrkiIsfFgGSHzLvYOINERA31cFRrjOt/eWfbN7uwPf+8zBUROSYGJDtUZHnMCGeQiKjhJid0QnxnLapqTBj7ZS5Ol3BnG1FDMSDZmaoaE86VVwG41AeJiKihzDvbOgV542xZJf7xBXe2ETUUA5KdKSq7NHvkppTQvJmbzNUQkaPyUqvw6ahoBHi5Y98ZA5/ZRtRADEh2Rscu2kTUSFo3b4aPR0TDXanAT3sL8c7PB+UuichhMCDZGfZAIqLGFNWmOd4acmln2//9ehRLt52QuSIix8CAZGeKStlFm4ga10O3tUbqXbcAAKYu241NR8/KXBGR/WNAsjPmGSTuYCOixjTpbx1wf/cQ1JgEnvwqF0d0ZXKXRGTXGJDsjLkHUiB7IBFRI1IoJMx6+FZEtWkOQ0UNRs/fgnOXN4UQ0Z8xINkZ3eUeSIHsok1EjUzjpsS8EVEI82+GE8UXMfbLbaioNspdFpFdYkCyM5xBIqKm1MJLjc8e7wUfjQp5+SV4fikfbEtUHwYkO6NjF20iamK3BHohfUQUVAoJGbvO4N1MPtiWqC4GJDtSVWNC8eUu2tzFRkRNqXf7AKRdfrDth2uP4Gtu/yeywoBkR8xdtN2VCvixizYRNbFHokMt2/9fWrYbm45w+z+RGQOSHTGvP2rprWYXbSKyCavt///j9n8iMwYkO6JjDyQisjFu/yeqHwOSHdGVXnkOGxGRrdTd/v8Pbv8nYkCyJ+ZbbJxBIiJba+GlxuejL23/355fgmcWbYeR2//JhTEg2RGd5UG1nEEiIttr39ILn4yMhrtSgZ/3FeJfP+6FEAxJ5JoYkOxIIbtoE5HMYtu1wHvDekCSgC+zjyN93e9yl0QkCwYkO6Kz3GLjDBIRyWfQrcF4ZVAXAMDbqw9g+fZTMldEZHsMSHbE8hw2rkEiIpmN6RuOf/QNBwC88M1ObGSPJHIxDEh2gl20icjevHRfZ/z91mBUGwXGfZWLfacNcpdEZDMMSHaCXbSJyN4oFBJmD+2O2HB/lFVe6pF0quSi3GUR2QQDkp1gF20iskdqlRLzRkSjg9YLhYZKjPpsC/QXquUui6jJMSDZCR17IBGRnfJt5ob5o2Og9VHjiK4MY79iI0lyfgxIdsK8QJs72IjIHoX4eWD+6Bh4q1XYcqwYzy3dCRMbSZITY0CyE+ZbbOyBRET2qnOwDz4eEQU3pYQVu87gjRX72UiSnBYDkp0oZBdtInIAvW8JwDuPdAcAfLbxGBtJktNiQLITvMVGRI5icI9WePm+zgAuNZJctCVf5oqIGh8Dkp3Q8RYbETmQsXe0w1N3tgcAvPzdbqzafUbmiogaFwOSneAMEhE5mskJHTG8VyhMApiweAe7bZNTYUCyA5U1RksXbc4gEZGjkCQJ/34wEgO7BqHKaELKl9uw80SJ3GURNQoGJDtQVMou2kTkmJQKCf9J6oE+t7RAeZURj3++BUd0ZXKXRfSXMSDZgdoPqWUXbSJyNGqVEh+PiEb31r44f6EaI/6bw0eSkMNjQLIDXKBNRI7OS63C56Nj0L6lJ87oKzDivzk4d/kZk0SOiAHJDph7IHGBNhE5Mn9Pd3w1JhYhvhr8XlSOxz/firLKGrnLIropDEh2QFdqfg4bAxIRObYQPw98OSYW/p7u2H1Kj5Qv+dw2ckwMSHbAPIPUkrfYiMgJ3BLohfmje8HTXYlNR89hwuLtqDGa5C6LqEEYkOyA+TlsnEEiImdxa2s/fDIyGu5KBX7aW4jJ3+ziw23JoTAg2YEiS5NIziARkfPofUsAPny0J5QKCcu2n8LLy/fw4bbkMBiQ7EChZRcbZ5CIyLnc0zUI7w3rAUkCFm3Jx+sZ+xmSyCEwIMmsssaI8xeqAXAGiYic0wPdQ/D2kFsBAJ9tPIbZPx+SuSKi62NAkpmli7ZKAV8PdtEmIuc0NDoUrw/uCgD4cO0RfLjmsMwVEV0bA5LMzDvYAr3ZRZuInNuIuLZ46b5OAIB3fj6ET3/7XeaKiK6OAUlmReyBREQuJOWO9pgYHwEAeGPFfvxv83GZKyKqHwOSzGrPIBERuYIJAyIwrn87AMAry/dg8ZZ8mSsi+jMGJJmxBxIRuRpJkvDiwE54vHdbAMCLy3bj660n5C2KqA4GJJnpLi/SDuQONiJyIZIkYcb9XSwhacqyXfh6G0MS2Q8GJJmxBxIRuSpzSBoV1wZCAFO+3YWlDElkJxiQZKYzsIs2EbkuSZLw6gNdMfJySJr87S58k3tS7rKIGJDkpuMuNiJycZIk4V8PdMVjt4dBCOCFb3ZiWR5DEsnrpgLS3Llz0bZtW2g0GsTGxmLLli3XHL906VJ06tQJGo0GkZGRWLlypdVxIQSmT5+O4OBgeHh4ID4+HocPWzcRKy4uRnJyMnx8fODn54cxY8agrKzMcryiogKPP/44IiMjoVKpkJiYeDOnZlO1u2hzFxsRuTJJkvDaA92QHHspJD23lCGJ5NXggLRkyRJMmjQJM2bMQF5eHrp3746EhATodLp6x2/atAlJSUkYM2YMtm/fjsTERCQmJmLPnj2WMTNnzsScOXOQnp6OnJwceHp6IiEhARUVFZYxycnJ2Lt3LzIzM5GRkYH169cjJSXFctxoNMLDwwPPPPMM4uPjG3pasmAXbSKiKxQKCa8P7oZHa4UktgAguUiigU8NjI2NRa9evfDhhx8CAEwmE0JDQ/H000/jxRdf/NP4YcOGoby8HBkZGZb3br/9dvTo0QPp6ekQQiAkJATPPfccnn/+eQCAXq+HVqvF/PnzMXz4cOzfvx9dunTB1q1bER0dDQBYvXo17rvvPpw8eRIhISFWv/Pxxx9HSUkJli9f3qCLYTAY4OvrC71eDx8fnwZ99mbkHj+PIR9tQuvmHtgw5e4m/31ERI7AZBJ49ce9+DL7UhPJ1wZ3xci4tvIWRXatKb6/GzSDVFVVhdzcXKsZGoVCgfj4eGRnZ9f7mezs7D/N6CQkJFjGHzt2DAUFBVZjfH19ERsbaxmTnZ0NPz8/SzgCgPj4eCgUCuTk5DTkFKxUVlbCYDBYvWxJxx5IRER/olBcWpM0tl84AGD693sxb/1RmasiV9OggHT27FkYjUZotVqr97VaLQoKCur9TEFBwTXHm/+83pjAwECr4yqVCv7+/lf9vTciLS0Nvr6+lldoaOhN/6ybYe6BxB1sRETWJEnCS/d1xtN33wIAeHPlAXyQxQfcku249C62qVOnQq/XW14nTti2/wZ7IBERXZ0kSXjuno547m8dAACzMw/hnZ8OooErQ4huSoMCUkBAAJRKJQoLC63eLywsRFBQUL2fCQoKuuZ485/XG1N3EXhNTQ2Ki4uv+ntvhFqtho+Pj9XLlizPYeMMEhHRVT09IAIv39cZAPDh2iP494r9DEnU5BoUkNzd3REVFYWsrCzLeyaTCVlZWYiLi6v3M3FxcVbjASAzM9MyPjw8HEFBQVZjDAYDcnJyLGPi4uJQUlKC3Nxcy5g1a9bAZDIhNja2IadgVyw9kDiDRER0TWPvaIfXBncFAHy64Rhe+m43jCaGJGo6qoZ+YNKkSRg1ahSio6MRExOD999/H+Xl5Rg9ejQAYOTIkWjVqhXS0tIAABMmTED//v0xe/ZsDBo0CIsXL8a2bdswb948AJemUCdOnIg33ngDERERCA8Px7Rp0xASEmLpZdS5c2cMHDgQY8eORXp6Oqqrq5Gamorhw4db7WDbt28fqqqqUFxcjNLSUuzYsQMA0KNHj79wiZqOjjNIREQ3bGRcW6hVCkxdthuLtpyA4WIN3h3WHWqVUu7SyAk1OCANGzYMRUVFmD59OgoKCtCjRw+sXr3assg6Pz8fCsWVianevXtj4cKFeOWVV/DSSy8hIiICy5cvR7du3SxjJk+ejPLycqSkpKCkpAR9+/bF6tWrodFcmVlZsGABUlNTMWDAACgUCgwZMgRz5syxqu2+++7D8ePHLf/cs2dPALDbqdhCdtEmImqQYb3C4K1xw8TFO7Bi9xkYKqqR/lgUPNUN/jojuqYG90FyZrbsg1RZY0THV1YDAHZOvwe+zdgokojoRm04fBYpX23DhSojeoT6Yf7oXvBr5i53WSQT2fsgUeMx315zVyng48H/50NE1BB9IwKw4B+x8Gvmhh0nSjD042wU6Cuu/0GiG8SAJJMrD6lVQ5IkmashInI8PcOa4+txcdD6qHGosAwPp2/CsbPlcpdFToIBSSbmGSTuYCMiunkdtN745sneaNuiGU6ev4hH0jdh18kSucsiJ8CAJBNLk0juYCMi+ktC/Zth6ZO90SXYB2fLqjB83masPVD/A9SJbhQDkkzMjxlhF20ior+upbcaS8bdjn4RAbhQZcQ/vtyGRVvy5S6LHBgDkkzYRZuIqHF5a9zw2eO9MOS21jCaBKYu2413f+ajSejmMCDJhF20iYgan5tSgXceuRXPDIgAAMxZcwTPL92FqhqTzJWRo2FAkollkTabRBIRNSpJkjDpbx3w1kORUCokfJt3EmO+2IrSimq5SyMHwoAkE3MXbd5iIyJqGsNjwvDpqGg0c1fit8Nn8Uh6Nk6evyB3WeQgGJBkUFFtRMmFS/9PhrfYiIiazl0dA7EkJQ4BXmocKChF4tyNyD1+Xu6yyAEwIMmg6PIONjW7aBMRNbnI1r74IbUPOl9uA5A0bzO+235S7rLIzjEgyUBX6/Yau2gTETW9ED8PfPNkHO7pokWV0YRnl+zEzNUHYDJxhxvVjwFJBoXsok1EZHOeahXSH4vCP+9sDwD4v1+P4qkFubhQVSNzZWSPGJBkoDOYn8PGgEREZEsKhYTJAzvh3aHd4a5U4Ke9hXj4o2ycKrkod2lkZxiQZFB4eQ1SS2/uYCMiksNDt7XGopRYtPB0x74zBjzwwQZsOnpW7rLIjjAgyYA9kIiI5BfVxh/Lx/dBl2AfnCuvwmOf5mDe+qPsvE0AGJBkYVmkzRkkIiJZhfo3w7J/9sZDt7WCSQBvrjyA1IXbUVbJdUmujgFJBoVcg0REZDc0bkrMfqQ7Xh/cFSqFhBW7z+DBuRtxtKhM7tJIRgxIMtCVmm+xcQaJiMgeSJKEEXFtsWTc7Qj0VuOwrgyDP9yIn/YWyF0ayYQBycZqd9EO5DZ/IiK7EtXGHxnP9EVMW3+UVdZg3Fe5+PeKfXzYrQtiQLIxdtEmIrJvgd4aLBgbiyf6hAMAPvntGB75OBv55/gcN1fCgGRj5gXaWh8Nu2gTEdkpN6UC0+/vgo9HRMHXww07T5Rg0JzfsGLXGblLIxthQLIxcxdt7mAjIrJ/CV2DsHJCP0S1aY7SyhqMX5iHqct2o6LaKHdp1MQYkGyMO9iIiBxLKz8PLEm5HePvag9JAhZtycfgDzficGGp3KVRE2JAsjHzDrZA7mAjInIYKqUCLyR0wldPxCLAS42DhaX4+wcb8PnGY3zgrZNiQLIx8wwSd7ARETmevhEBWDWhH/pFBKCyxoR//bgPj/03h89yc0IMSDZWxB5IREQOraW3Gl8+EYPXB3eFxk2BTUfPYeB76/FN7kk+psSJMCDZGGeQiIgcn7mx5KoJd6BnmB9KK2vw/NKdSPkqF2fLKuUujxoBA5KNFRo4g0RE5CzCAzyxdFwcXkjoCDelhMx9hUh4bz1W7j7D2SQHx4BkQxXVRugvXu6izV1sREROQaVUYPxdt2D5+D7oqPXGufIq/HNBHsZ+mYvTXJvksBiQbMiqi7aGXbSJiJxJ1xBf/PB0H6TedQtUCgm/7C/E395dh/kbj8HInW4OhwHJhmr3QGIXbSIi56NWKfF8QkeseKYfbgvzQ3mVEa/+uA8PfbQJ+88Y5C6PGoAByYZ03MFGROQSOgZ545sne+P1xG7wVquw80QJ/v7BBqSt2o8LVTVyl0c3gAHJhriDjYjIdSgUEkbc3ga/PNcf93YLgtEk8PG63zFg9jp8v+MUF3HbOQYkG7I8h40zSERELkPro8FHj0Xhk5HRaOXngTP6CkxYvAMPp2dj18kSucujq2BAsiFdKZ/DRkTkqv7WRYus5/rj+Xs6wMNNidzj5/HAhxvx/NKd0F2+w0D2gwHJhnTmGSRvziAREbkijZsSqXdHYO3zd+Khnq0AAN/knsRd7/yK//v1CCqqjTJXSGYMSDbEGSQiIgKAIF8N3h3WA8v+2RvdQy/tdpu5+iDumLkWX2X/gaoak9wlujwGJBsq5AwSERHVcltYc3z3VG+8O7Q7Wvl5QFdaiWnf78Xds3/F0m0nUGNkUJILA5KNsIs2ERHVR6GQ8NBtrbHm+f54bXBXtPRW4+T5i3jhm1245/31yNh1GiY2mrQ5BiQbMXfR1rixizYREf2ZWqXEyLi2WP/CXZh6byf4NXPD70XlSF24HffN+Q3fbT+Jas4o2QwDko3U7oHELtpERHQ1Hu5KjOvfHr9NvgsT4yPgpVbhQEEpnl2yE3fO+hWfbTiG8ko2m2xqDEg2Yl5/xC7aRER0I7w1bpgY3wEbp9yNFxI6IsDLHadKLuK1jH3o/dYazP75IM6WVcpdptNiQLIR8w42rj8iIqKG8G3mhvF33YINU+7Gmw9Gom2LZtBfrMYHa46gz1trMPmbndhxooSduRsZF8PYCHewERHRX6FxU+LR2DAM6xWKn/cWIH3dUew8qcfX207i620n0SXYB8m3h2Fwj1bwUvPr/a/iFbQRc5dU9kAiIqK/QqmQcG9kMAZ2C8K24+exMCcfK3afwb4zBrz83R68uWI/HujRCsmxYega4sN1rzeJAclGdKVcg0RERI1HkiT0auuPXm39Mf3vXfBt3kks3JKP34vKsWhLPhZtycctgV64/9YQ/L17MNq39JK7ZIfCgGQjtXexERERNabmnu74R792GNM3HJt/L8bCLfn4aU8BjujK8N4vh/DeL4fQJdgHf+8ejPtvDUGofzO5S7Z7DEg2Umi5xcYZJCIiahqSJCGufQvEtW8BQ0U1MvcW4sddp7Hh8FnsO2PAvjMGzFx9ELe29sWdHQPRv0NL9Aj1g1LB23B1MSDZQEW1EYaKSz0rWnIGiYiIbMBH44YhUa0xJKo1zpdXYfXeAvy48zQ2/34Ou07qseukHnOyDsPXww19IwLQv0NL3NmhJXdbX8aAZAM6A7toExGRfJp7uiMpJgxJMWHQlVbg1wNFWHeoCL8dLoL+YjVW7DqDFbvOAADat/TEbWHN0TOsOXqG+aGD1tslZ5j4bW0D5h5IWh920SYiInkFemswtFcohvYKRY3RhJ0nS7Du4KXAtOuUHkeLynG0qBxLc08CADzdlege6oceoX7oFOyD9i090S7ACx7uSpnPpGkxINkAeyAREZE9UikViGrjj6g2/ph0T0cUl1dhe/55bM8vwfYT57EjvwTlVUZsOnoOm46es3xOkoBWfh5o39IL7Vt6IbylJ7TearS8/ArwUkPj5tgBigHJBiw72Hhfl4iI7Ji/pzsGdNZiQGctAMBoEjiiK0Ne/nnsPFGCI7oyHCkqQ8mFapw8fxEnz1/EukNF9f4sXw83tPRWw7+ZO5qplfB0V6GZuxL3Rgbh7k5aW57WTWFAsgFLDyQu0CYiIgeiVEjoGOSNjkHeSIoJs7xfXF6FI7oyHC0qw1FdGf44dwFFZZU4W1qJotJKVBlN0F+shv5i9Z9+ZruWXgxIdEnfWwKgVADRbf3lLoWIiOgv8/d0R0y4P2LC//y9JoSA4WINdKUV0JVWQn+xGuWVNbhQZUR5VQ1i2znGd6Ek+HQ7C4PBAF9fX+j1evj4+MhdDhEREd2Apvj+VtzMh+bOnYu2bdtCo9EgNjYWW7Zsueb4pUuXolOnTtBoNIiMjMTKlSutjgshMH36dAQHB8PDwwPx8fE4fPiw1Zji4mIkJyfDx8cHfn5+GDNmDMrKyqzG7Nq1C/369YNGo0FoaChmzpx5M6dHRERELq7BAWnJkiWYNGkSZsyYgby8PHTv3h0JCQnQ6XT1jt+0aROSkpIwZswYbN++HYmJiUhMTMSePXssY2bOnIk5c+YgPT0dOTk58PT0REJCAioqKixjkpOTsXfvXmRmZiIjIwPr169HSkqK5bjBYMA999yDNm3aIDc3F7NmzcKrr76KefPmNfQUiYiIyNWJBoqJiRHjx4+3/LPRaBQhISEiLS2t3vFDhw4VgwYNsnovNjZWjBs3TgghhMlkEkFBQWLWrFmW4yUlJUKtVotFixYJIYTYt2+fACC2bt1qGbNq1SohSZI4deqUEEKI//u//xPNmzcXlZWVljFTpkwRHTt2vOFz0+v1AoDQ6/U3/BkiIiKSV1N8fzdoBqmqqgq5ubmIj4+3vKdQKBAfH4/s7Ox6P5OdnW01HgASEhIs448dO4aCggKrMb6+voiNjbWMyc7Ohp+fH6Kjoy1j4uPjoVAokJOTYxlzxx13wN3d3er3HDx4EOfPn6+3tsrKShgMBqsXERERUYMC0tmzZ2E0GqHVWm/P02q1KCgoqPczBQUF1xxv/vN6YwIDA62Oq1Qq+Pv7W42p72fU/h11paWlwdfX1/IKDQ2t/8SJiIjIpdzUIm1nMXXqVOj1esvrxIkTcpdEREREdqBBASkgIABKpRKFhYVW7xcWFiIoKKjezwQFBV1zvPnP642puwi8pqYGxcXFVmPq+xm1f0ddarUaPj4+Vi8iIiKiBgUkd3d3REVFISsry/KeyWRCVlYW4uLi6v1MXFyc1XgAyMzMtIwPDw9HUFCQ1RiDwYCcnBzLmLi4OJSUlCA3N9cyZs2aNTCZTIiNjbWMWb9+Paqrq61+T8eOHdG8efOGnCYRERG5uoau6l68eLFQq9Vi/vz5Yt++fSIlJUX4+fmJgoICIYQQI0aMEC+++KJl/MaNG4VKpRLvvPOO2L9/v5gxY4Zwc3MTu3fvtox56623hJ+fn/j+++/Frl27xODBg0V4eLi4ePGiZczAgQNFz549RU5OjtiwYYOIiIgQSUlJluMlJSVCq9WKESNGiD179ojFixeLZs2aiY8//viGz4272IiIiBxPU3x/NzggCSHEBx98IMLCwoS7u7uIiYkRmzdvthzr37+/GDVqlNX4r7/+WnTo0EG4u7uLrl27ihUrVlgdN5lMYtq0aUKr1Qq1Wi0GDBggDh48aDXm3LlzIikpSXh5eQkfHx8xevRoUVpaajVm586dom/fvkKtVotWrVqJt956q0HnxYBERETkeJri+5uPGqmFjxohIiJyPHbzqBEiIiIiZ8aARERERFSHSu4C7In5biM7ahMRETkO8/d2Y64aYkCqpbS0FADYUZuIiMgBlZaWwtfXt1F+Fhdp12IymXD69Gl4e3tDkqRG/dkGgwGhoaE4ceKEyy8A57W4gtfiCl4La7weV/BaXMFrYc18PfLz8yFJEkJCQqBQNM7qIc4g1aJQKNC6desm/R3s2H0Fr8UVvBZX8FpY4/W4gtfiCl4La76+vo1+PbhIm4iIiKgOBiQiIiKiOhiQbEStVmPGjBlQq9VylyI7XosreC2u4LWwxutxBa/FFbwW1pryenCRNhEREVEdnEEiIiIiqoMBiYiIiKgOBiQiIiKiOhiQiIiIiOpgQLKBuXPnom3bttBoNIiNjcWWLVvkLqnJpaWloVevXvD29kZgYCASExNx8OBBqzEVFRUYP348WrRoAS8vLwwZMgSFhYUyVWw7b731FiRJwsSJEy3vudq1OHXqFB577DG0aNECHh4eiIyMxLZt2yzHhRCYPn06goOD4eHhgfj4eBw+fFjGipuG0WjEtGnTEB4eDg8PD7Rv3x6vv/661fOknPVarF+/Hvfffz9CQkIgSRKWL19udfxGzru4uBjJycnw8fGBn58fxowZg7KyMhueReO51vWorq7GlClTEBkZCU9PT4SEhGDkyJE4ffq01c9wlutxvX83anvyySchSRLef/99q/cb41owIDWxJUuWYNKkSZgxYwby8vLQvXt3JCQkQKfTyV1ak1q3bh3Gjx+PzZs3IzMzE9XV1bjnnntQXl5uGfPss8/ixx9/xNKlS7Fu3TqcPn0aDz30kIxVN72tW7fi448/xq233mr1vitdi/Pnz6NPnz5wc3PDqlWrsG/fPsyePRvNmze3jJk5cybmzJmD9PR05OTkwNPTEwkJCaioqJCx8sb39ttv46OPPsKHH36I/fv34+2338bMmTPxwQcfWMY467UoLy9H9+7dMXfu3HqP38h5JycnY+/evcjMzERGRgbWr1+PlJQUW51Co7rW9bhw4QLy8vIwbdo05OXlYdmyZTh48CAeeOABq3HOcj2u9++G2XfffYfNmzcjJCTkT8ca5VoIalIxMTFi/Pjxln82Go0iJCREpKWlyViV7el0OgFArFu3TgghRElJiXBzcxNLly61jNm/f78AILKzs+Uqs0mVlpaKiIgIkZmZKfr37y8mTJgghHC9azFlyhTRt2/fqx43mUwiKChIzJo1y/JeSUmJUKvVYtGiRbYo0WYGDRoknnjiCav3HnroIZGcnCyEcJ1rAUB89913ln++kfPet2+fACC2bt1qGbNq1SohSZI4deqUzWpvCnWvR322bNkiAIjjx48LIZz3elztWpw8eVK0atVK7NmzR7Rp00a89957lmONdS04g9SEqqqqkJubi/j4eMt7CoUC8fHxyM7OlrEy29Pr9QAAf39/AEBubi6qq6utrk2nTp0QFhbmtNdm/PjxGDRokNU5A653LX744QdER0fjkUceQWBgIHr27IlPPvnEcvzYsWMoKCiwuh6+vr6IjY11uuvRu3dvZGVl4dChQwCAnTt3YsOGDbj33nsBuNa1qO1Gzjs7Oxt+fn6Ijo62jImPj4dCoUBOTo7Na7Y1vV4PSZLg5+cHwLWuh8lkwogRI/DCCy+ga9eufzreWNeCD6ttQmfPnoXRaIRWq7V6X6vV4sCBAzJVZXsmkwkTJ05Enz590K1bNwBAQUEB3N3dLf9xm2m1WhQUFMhQZdNavHgx8vLysHXr1j8dc7Vr8fvvv+Ojjz7CpEmT8NJLL2Hr1q145pln4O7ujlGjRlnOub7/bpzterz44oswGAzo1KkTlEoljEYj/v3vfyM5ORkAXOpa1HYj511QUIDAwECr4yqVCv7+/k59bYBLaxanTJmCpKQkywNaXel6vP3221CpVHjmmWfqPd5Y14IBiZrc+PHjsWfPHmzYsEHuUmRx4sQJTJgwAZmZmdBoNHKXIzuTyYTo6Gi8+eabAICePXtiz549SE9Px6hRo2Suzra+/vprLFiwAAsXLkTXrl2xY8cOTJw4ESEhIS53LejGVFdXY+jQoRBC4KOPPpK7HJvLzc3Ff/7zH+Tl5UGSpCb9XbzF1oQCAgKgVCr/tBupsLAQQUFBMlVlW6mpqcjIyMDatWvRunVry/tBQUGoqqpCSUmJ1XhnvDa5ubnQ6XS47bbboFKpoFKpsG7dOsyZMwcqlQpardZlrgUABAcHo0uXLlbvde7cGfn5+QBgOWdX+O/mhRdewIsvvojhw4cjMjISI0aMwLPPPou0tDQArnUtaruR8w4KCvrTZpeamhoUFxc77bUxh6Pjx48jMzPTMnsEuM71+O2336DT6RAWFmb5+/T48eN47rnn0LZtWwCNdy0YkJqQu7s7oqKikJWVZXnPZDIhKysLcXFxMlbW9IQQSE1NxXfffYc1a9YgPDzc6nhUVBTc3Nysrs3BgweRn5/vdNdmwIAB2L17N3bs2GF5RUdHIzk52fK/XeVaAECfPn3+1PLh0KFDaNOmDQAgPDwcQUFBVtfDYDAgJyfH6a7HhQsXoFBY/zWsVCphMpkAuNa1qO1GzjsuLg4lJSXIzc21jFmzZg1MJhNiY2NtXnNTM4ejw4cP45dffkGLFi2sjrvK9RgxYgR27dpl9fdpSEgIXnjhBfz0008AGvFa3PzacroRixcvFmq1WsyfP1/s27dPpKSkCD8/P1FQUCB3aU3qqaeeEr6+vuLXX38VZ86csbwuXLhgGfPkk0+KsLAwsWbNGrFt2zYRFxcn4uLiZKzadmrvYhPCta7Fli1bhEqlEv/+97/F4cOHxYIFC0SzZs3E//73P8uYt956S/j5+Ynvv/9e7Nq1SwwePFiEh4eLixcvylh54xs1apRo1aqVyMjIEMeOHRPLli0TAQEBYvLkyZYxznotSktLxfbt28X27dsFAPHuu++K7du3W3Zl3ch5Dxw4UPTs2VPk5OSIDRs2iIiICJGUlCTXKf0l17oeVVVV4oEHHhCtW7cWO3bssPo7tbKy0vIznOV6XO/fjbrq7mITonGuBQOSDXzwwQciLCxMuLu7i5iYGLF582a5S2pyAOp9ff7555YxFy9eFP/85z9F8+bNRbNmzcSDDz4ozpw5I1/RNlQ3ILnatfjxxx9Ft27dhFqtFp06dRLz5s2zOm4ymcS0adOEVqsVarVaDBgwQBw8eFCmapuOwWAQEyZMEGFhYUKj0Yh27dqJl19+2epLz1mvxdq1a+v9O2LUqFFCiBs773PnzomkpCTh5eUlfHx8xOjRo0VpaakMZ/PXXet6HDt27Kp/p65du9byM5zlelzv34266gtIjXEtJCFqtWwlIiIiIq5BIiIiIqqLAYmIiIioDgYkIiIiojoYkIiIiIjqYEAiIiIiqoMBiYiIiKgOBiQiIiKiOhiQiIiIiOpgQCIiIiKqgwGJiIiIqA4GJCIiIqI6GJCIiIiI6vh/02lODEDK4jUAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(range(len(lrs)), lrs)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e80ac790-f9c3-45b8-9ea4-d2e5bf8fbf28",
   "metadata": {},
   "outputs": [],
   "source": [
    "- And a quick look at the loss curves"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "445d8155-6eae-4b50-a381-d0820ebc27cc",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAnUAAAHWCAYAAAARl3+JAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAAB0m0lEQVR4nO3dd3wT9f8H8NcladKku6WTTqDQwZ5CkS1DRAERRH4MURQEAQfiAsGFICriAOWr4EBRlKWyyoayV6G0lN0W6KCU7p3c74+jaUMLFGh7afp6Ph73aPK59c4Z2xd39/mcIIqiCCIiIiKq1RRyF0BERERED46hjoiIiMgCMNQRERERWQCGOiIiIiILwFBHREREZAEY6oiIiIgsAEMdERERkQVgqCMiIiKyAAx1RERERBaAoY6Iap1Lly5BEAQcP35c7lKIiMwGQx0RyUIQhDtOs2bNkrtEIqJaRSV3AURUNyUmJhpf//HHH5g5cyZiY2ONbba2tnKURURUa/FMHRHJwsPDwzg5ODhAEATjezc3N3z++efw9vaGRqNBy5YtsXHjxttuS6/XY+zYsQgKCkJ8fDwAYO3atWjdujWsra3RoEEDzJ49G8XFxcZ1BEHA//73PwwaNAg6nQ6BgYFYt26dcf6NGzcwYsQIuLq6QqvVIjAwEEuXLr1tDX/99ReaNWsGrVYLFxcX9OrVCzk5Ocb5//vf/xAcHAxra2sEBQXh22+/NVk/ISEBQ4cOhaOjI5ydnfHEE0/g0qVLxvljxozBwIEDMX/+fHh6esLFxQUTJ05EUVFRpY85EVk2hjoiMjtffvklPvvsM8yfPx8nTpxAnz598Pjjj+Ps2bPlli0oKMBTTz2F48ePY/fu3fD19cXu3bsxatQoTJkyBdHR0fjuu++wbNkyfPTRRybrzp49G0OHDsWJEyfw6KOPYsSIEUhLSwMAzJgxA9HR0diwYQNiYmKwaNEi1KtXr8J6ExMTMXz4cIwdOxYxMTHYsWMHBg8eDFEUAQDLly/HzJkz8dFHHyEmJgYff/wxZsyYgZ9++gkAUFRUhD59+sDOzg67d+9GREQEbG1t0bdvXxQWFhr3s337dpw/fx7bt2/HTz/9hGXLlmHZsmVVcciJyBKIREQyW7p0qejg4GB87+XlJX700Ucmy7Rr10586aWXRFEUxYsXL4oAxN27d4s9e/YUO3fuLKanpxuX7dmzp/jxxx+brP/LL7+Inp6exvcAxHfffdf4Pjs7WwQgbtiwQRRFURwwYID47LPPVqr+I0eOiADES5cuVTi/YcOG4m+//WbS9sEHH4gdO3Y01takSRPRYDAY5xcUFIharVbctGmTKIqiOHr0aNHPz08sLi42LvPUU0+Jw4YNq1SNRGT5eE8dEZmVzMxMXL16FWFhYSbtYWFhiIyMNGkbPnw4vL29sW3bNmi1WmN7ZGQkIiIiTM7M6fV65OfnIzc3FzqdDgDQvHlz43wbGxvY29sjJSUFADBhwgQ8+eSTOHr0KHr37o2BAweiU6dOFdbcokUL9OzZE82aNUOfPn3Qu3dvDBkyBE5OTsjJycH58+fx3HPPYdy4ccZ1iouL4eDgYKz33LlzsLOzM9lufn4+zp8/b3wfGhoKpVJpfO/p6YmTJ0/e4WgSUV3CUEdEtdajjz6KX3/9Ffv27UOPHj2M7dnZ2Zg9ezYGDx5cbh1ra2vjaysrK5N5giDAYDAAAPr164e4uDisX78e4eHh6NmzJyZOnIj58+eX26ZSqUR4eDj27t2LzZs346uvvsI777yDAwcOGAPkkiVL0KFDh3LrldTbpk0bLF++vNy2XV1dK1UvERFDHRGZFXt7e3h5eSEiIgJdu3Y1tkdERKB9+/Ymy06YMAFNmzbF448/jv/++8+4fOvWrREbG4tGjRo9UC2urq4YPXo0Ro8ejYcffhjTpk2rMNQBUsAKCwtDWFgYZs6cCT8/P6xevRqvvvoqvLy8cOHCBYwYMaLCdVu3bo0//vgDbm5usLe3f6CaiajuYqgjIrMzbdo0vPfee2jYsCFatmyJpUuX4vjx4xWeyXr55Zeh1+vx2GOPYcOGDejcuTNmzpyJxx57DL6+vhgyZAgUCgUiIyMRFRWFDz/8sFI1zJw5E23atEFoaCgKCgrw77//Ijg4uMJlDxw4gK1bt6J3795wc3PDgQMHcO3aNePys2fPxuTJk+Hg4IC+ffuioKAAhw8fxo0bN/Dqq69ixIgR+PTTT/HEE0/g/fffh7e3N+Li4rBq1Sq88cYb8Pb2vv+DSUR1BkMdEZmdyZMnIyMjA6+99hpSUlIQEhKCdevWITAwsMLlp06dCoPBgEcffRQbN25Enz598O+//+L999/H3LlzYWVlhaCgIDz//POVrkGtVuOtt97CpUuXoNVq8fDDD2PFihUVLmtvb49du3ZhwYIFyMzMhJ+fHz777DP069cPAPD8889Dp9Ph008/xbRp02BjY4NmzZph6tSpAACdToddu3Zh+vTpGDx4MLKyslC/fn307NmTZ+6IqNIEUbzZ556IiIiIai2OU0dERERkARjqiIiIiCwAQx0RERGRBWCoIyIiIrIADHVEREREFoChjoiIiMgCMNTdo2+++Qb+/v6wtrZGhw4dcPDgQblLqlK7du3CgAED4OXlBUEQsGbNGpP5oihi5syZ8PT0hFarRa9evXD27FmTZdLS0jBixAjY29vD0dERzz33HLKzs02WOXHiBB5++GFYW1vDx8cH8+bNK1fLypUrERQUBGtrazRr1gzr16+v8s97v+bMmYN27drBzs4Obm5uGDhwIGJjY02Wyc/Px8SJE+Hi4gJbW1s8+eSTSE5ONlkmPj4e/fv3h06ng5ubG6ZNm4bi4mKTZXbs2IHWrVtDo9GgUaNGWLZsWbl6zPV7uWjRIjRv3hz29vawt7dHx44dsWHDBuN8HqOKffLJJxAEwTiOHcBjBQCzZs2CIAgmU1BQkHE+j5GpK1eu4P/+7//g4uICrVaLZs2a4fDhw8b5/H0O+Pv7l/tOCYKAiRMnAqiF3ymRKm3FihWiWq0Wf/zxR/HUqVPiuHHjREdHRzE5OVnu0qrM+vXrxXfeeUdctWqVCEBcvXq1yfxPPvlEdHBwENesWSNGRkaKjz/+uBgQECDm5eUZl+nbt6/YokULcf/+/eLu3bvFRo0aicOHDzfOz8jIEN3d3cURI0aIUVFR4u+//y5qtVrxu+++My4TEREhKpVKcd68eWJ0dLT47rvvilZWVuLJkyer/RhURp8+fcSlS5eKUVFR4vHjx8VHH31U9PX1FbOzs43LjB8/XvTx8RG3bt0qHj58WHzooYfETp06GecXFxeLTZs2FXv16iUeO3ZMXL9+vVivXj3xrbfeMi5z4cIFUafTia+++qoYHR0tfvXVV6JSqRQ3btxoXMacv5fr1q0T//vvP/HMmTNibGys+Pbbb4tWVlZiVFSUKIo8RhU5ePCg6O/vLzZv3lycMmWKsZ3HShTfe+89MTQ0VExMTDRO165dM87nMSqVlpYm+vn5iWPGjBEPHDggXrhwQdy0aZN47tw54zL8fS6KKSkpJt+n8PBwEYC4fft2URRr33eKoe4etG/fXpw4caLxvV6vF728vMQ5c+bIWFX1uTXUGQwG0cPDQ/z000+Nbenp6aJGoxF///13URRFMTo6WgQgHjp0yLjMhg0bREEQxCtXroiiKIrffvut6OTkJBYUFBiXmT59utikSRPj+6FDh4r9+/c3qadDhw7iiy++WKWfsaqkpKSIAMSdO3eKoigdFysrK3HlypXGZWJiYkQA4r59+0RRlAK0QqEQk5KSjMssWrRItLe3Nx6bN954QwwNDTXZ17Bhw8Q+ffoY39e276WTk5P4v//9j8eoAllZWWJgYKAYHh4udu3a1RjqeKwk7733ntiiRYsK5/EYmZo+fbrYuXPn287n7/OKTZkyRWzYsKFoMBhq5XeKl18rqbCwEEeOHEGvXr2MbQqFAr169cK+fftkrKzmXLx4EUlJSSbHwMHBAR06dDAeg3379sHR0RFt27Y1LtOrVy8oFAocOHDAuEyXLl2gVquNy/Tp0wexsbG4ceOGcZmy+ylZxlyPdUZGBgDA2dkZAHDkyBEUFRWZfIagoCD4+vqaHKtmzZrB3d3duEyfPn2QmZmJU6dOGZe503GoTd9LvV6PFStWICcnBx07duQxqsDEiRPRv3//cp+Hx6rU2bNn4eXlhQYNGmDEiBGIj48HwGN0q3Xr1qFt27Z46qmn4ObmhlatWmHJkiXG+fx9Xl5hYSF+/fVXjB07FoIg1MrvFENdJaWmpkKv15v8hwMAd3d3JCUlyVRVzSr5nHc6BklJSXBzczOZr1Kp4OzsbLJMRdsou4/bLWOOx9pgMGDq1KkICwtD06ZNAUj1q9VqODo6mix767G63+OQmZmJvLy8WvG9PHnyJGxtbaHRaDB+/HisXr0aISEhPEa3WLFiBY4ePYo5c+aUm8djJenQoQOWLVuGjRs3YtGiRbh48SIefvhhZGVl8Rjd4sKFC1i0aBECAwOxadMmTJgwAZMnT8ZPP/0EgL/PK7JmzRqkp6djzJgxAGrn/3eqe1qaiMqZOHEioqKisGfPHrlLMUtNmjTB8ePHkZGRgb/++gujR4/Gzp075S7LrCQkJGDKlCkIDw+HtbW13OWYrX79+hlfN2/eHB06dICfnx/+/PNPaLVaGSszPwaDAW3btsXHH38MAGjVqhWioqKwePFijB49WubqzNMPP/yAfv36wcvLS+5S7hvP1FVSvXr1oFQqy/V6SU5OhoeHh0xV1aySz3mnY+Dh4YGUlBST+cXFxUhLSzNZpqJtlN3H7ZYxt2M9adIk/Pvvv9i+fTu8vb2N7R4eHigsLER6errJ8rceq/s9Dvb29tBqtbXie6lWq9GoUSO0adMGc+bMQYsWLfDll1/yGJVx5MgRpKSkoHXr1lCpVFCpVNi5cycWLlwIlUoFd3d3HqsKODo6onHjxjh37hy/T7fw9PRESEiISVtwcLDxcjV/n5uKi4vDli1b8PzzzxvbauN3iqGuktRqNdq0aYOtW7ca2wwGA7Zu3YqOHTvKWFnNCQgIgIeHh8kxyMzMxIEDB4zHoGPHjkhPT8eRI0eMy2zbtg0GgwEdOnQwLrNr1y4UFRUZlwkPD0eTJk3g5ORkXKbsfkqWMZdjLYoiJk2ahNWrV2Pbtm0ICAgwmd+mTRtYWVmZfIbY2FjEx8ebHKuTJ0+a/NIMDw+Hvb298Zfx3Y5DbfxeGgwGFBQU8BiV0bNnT5w8eRLHjx83Tm3btsWIESOMr3msysvOzsb58+fh6enJ79MtwsLCyg2zdObMGfj5+QHg7/NbLV26FG5ubujfv7+xrVZ+p+6pW0Udt2LFClGj0YjLli0To6OjxRdeeEF0dHQ06fVS22VlZYnHjh0Tjx07JgIQP//8c/HYsWNiXFycKIpSF3hHR0dx7dq14okTJ8Qnnniiwi7wrVq1Eg8cOCDu2bNHDAwMNOkCn56eLrq7u4sjR44Uo6KixBUrVog6na5cF3iVSiXOnz9fjImJEd977z2z6QIviqI4YcIE0cHBQdyxY4dJd/jc3FzjMuPHjxd9fX3Fbdu2iYcPHxY7duwoduzY0Ti/pCt87969xePHj4sbN24UXV1dK+wKP23aNDEmJkb85ptvKuwKb67fyzfffFPcuXOnePHiRfHEiRPim2++KQqCIG7evFkURR6jOynb+1UUeaxEURRfe+01cceOHeLFixfFiIgIsVevXmK9evXElJQUURR5jMo6ePCgqFKpxI8++kg8e/asuHz5clGn04m//vqrcRn+Ppfo9XrR19dXnD59erl5te07xVB3j7766ivR19dXVKvVYvv27cX9+/fLXVKV2r59uwig3DR69GhRFKVu8DNmzBDd3d1FjUYj9uzZU4yNjTXZxvXr18Xhw4eLtra2or29vfjss8+KWVlZJstERkaKnTt3FjUajVi/fn3xk08+KVfLn3/+KTZu3FhUq9ViaGio+N9//1Xb575XFR0jAOLSpUuNy+Tl5YkvvfSS6OTkJOp0OnHQoEFiYmKiyXYuXbok9uvXT9RqtWK9evXE1157TSwqKjJZZvv27WLLli1FtVotNmjQwGQfJcz1ezl27FjRz89PVKvVoqurq9izZ09joBNFHqM7uTXU8VhJw0B4enqKarVarF+/vjhs2DCTcdd4jEz9888/YtOmTUWNRiMGBQWJ33//vcl8/j6XbNq0SQRQ7rOLYu37TgmiKIr3dm6PiIiIiMwN76kjIiIisgAMdUREREQWgKGOiIiIyAIw1BERERFZAIY6IiIiIgvAUEdERERkARjq7kNBQQFmzZqFgoICuUsxazxOlcPjVHk8VpXD41Q5PE6Vx2NVOXIfJ45Tdx8yMzPh4OCAjIwM2Nvby12O2eJxqhwep8rjsaocHqfK4XGqPB6rypH7OPFMHREREZEFYKgjIiIisgAquQuobsXFxTh27Bjc3d2hUFRNhs3KygIAXLlyBZmZmVWyTUvE41Q5PE6Vx2NVOTxOlcPjVHk8VpVT9jilp6cjOTkZrVq1gkpVM3HL4u+pO3ToENq3by93GURERFQHHTx4EO3atauRfVn8mTp3d3cA0kH19PSUuRoiIiKqCxITE9G+fXtjDqkJFh/qSi65enp6wtvbW+ZqiIiIqC6pqlu/KrWvGtsTEREREVUbhjoiIiIiC8BQR0RERGQBLP6eOiIiorL0ej2KiorkLoNqOSsrKyiVSrnLMMFQR0REdYIoikhKSkJ6errcpZCFcHR0hIeHBwRBkLsUAAx1RERUR5QEOjc3N+h0OrP5Q0y1jyiKyM3NRUpKCgCYzZBpDHVERGTx9Hq9MdC5uLjIXQ5ZAK1WCwBISUmBm5ubWVyKZUcJIiKyeCX30Ol0OpkrIUtS8n0yl3s0GeqIiKjO4CVXqkrm9n1iqCMiIiKyAAx1REREdYi/vz8WLFhQ6eV37NgBQRCqvdfwsmXL4OjoWK37sHQMdURERGZIEIQ7TrNmzbqv7R46dAgvvPBCpZfv1KkTEhMT4eDgcF/7o5rD3q9ERERmKDEx0fj6jz/+wMyZMxEbG2tss7W1Nb4WRRF6vR4q1d3/rLu6ut5THWq1Gh4eHve0DsmDZ+oeUH6RHjsiz+Hs4S1yl0JERBbEw8PDODk4OEAQBOP706dPw87ODhs2bECbNm2g0WiwZ88enD9/Hk888QTc3d1ha2uLdu3aYcsW079Pt15+FQQB//vf/zBo0CDodDoEBgZi3bp1xvm3Xn4tuUy6adMmBAcHw9bWFn379jUJocXFxZg8eTIcHR3h4uKC6dOnY/To0Rg4cOA9HYNFixahYcOGUKvVaNKkCX755RfjPFEUMWvWLPj6+kKj0cDLywuTJ082zv/2228RGBgIa2truLu7Y8iQIfe079qIoe4BLV/7Hx5a9RA8N4wFigvlLoeIiCpJFEXkFhbX+CSKYpV9hjfffBOffPIJYmJi0Lx5c2RnZ+PRRx/F1q1bcezYMfTt2xcDBgxAfHz8Hbcze/ZsDB06FCdOnMCjjz6KESNGIC0t7bbL5+bmYv78+fjll1+wa9cuxMfH4/XXXzfOnzt3LpYvX46lS5ciIiICmZmZWLNmzT19ttWrV2PKlCl47bXXEBUVhRdffBHPPvsstm/fDgD4+++/8cUXX+C7777D2bNnsWbNGjRr1gwAcPjwYUyePBnvv/8+YmNjsXHjRnTp0uWe9l8b8fLrA2rasgMyTtrAXZ8O/dnNUAY/JndJRERUCXlFeoTM3FTj+41+vw906qr58/v+++/jkUceMb53dnZGixYtjO8/+OADrF69GuvWrcOkSZNuu50xY8Zg+PDhAICPP/4YCxcuxMGDB9G3b98Kly8qKsLixYvRsGFDAMCkSZPw/vvvG+d/9dVXeOuttzBo0CAAwNdff43169ff02ebP38+xowZg5deegkA8Oqrr2L//v2YP38+unfvjvj4eHh4eKBXr16wsrKCr68v2rdvDwCIj4+HjY0NHnvsMdjZ2cHPzw+tWrW6p/3XRjxT94DaBLhig/AwACBz/y93WZqIiKjqtG3b1uR9dnY2Xn/9dQQHB8PR0RG2traIiYm565m65s2bG1/b2NjA3t7e+Aisiuh0OmOgA6THZJUsn5GRgeTkZGPAAgClUok2bdrc02eLiYlBWFiYSVtYWBhiYmIAAE899RTy8vLQoEEDjBs3DqtXr0ZxcTEA4JFHHoGfnx8aNGiAkSNHYvny5cjNzb2n/ddGPFP3gFRKBRL9BwGX/oF9/BYgNw3QOctdFhER3YXWSono9/vIst+qYmNjY/L+9ddfR3h4OObPn49GjRpBq9ViyJAhKCy88+1BVlZWJu8FQYDBYLin5avysnJl+Pj4IDY2Flu2bEF4eDheeuklfPrpp9i5cyfs7Oxw9OhR7NixA5s3b8bMmTMxa9YsHDp0yKKHTeGZuioQ0vIhnDL4QSkWA6dWyV0OERFVgiAI0KlVNT5V51MIIiIiMGbMGAwaNAjNmjWDh4cHLl26VG37q4iDgwPc3d1x6NAhY5ter8fRo0fvaTvBwcGIiIgwaYuIiEBISIjxvVarxYABA7Bw4ULs2LED+/btw8mTJwEAKpUKvXr1wrx583DixAlcunQJ27Zte4BPZv54pq4KdG3siq8NXRCq+AUFR36Dpt3zcpdERER1UGBgIFatWoUBAwZAEATMmDHjjmfcqsvLL7+MOXPmoFGjRggKCsJXX32FGzdu3FOgnTZtGoYOHYpWrVqhV69e+Oeff7Bq1Spjb95ly5ZBr9ejQ4cO0Ol0+PXXX6HVauHn54d///0XFy5cQJcuXeDk5IT169fDYDCgSZMm1fWRzQLP1FUBR50acV79UCwqoEk6AqSek7skIiKqgz7//HM4OTmhU6dOGDBgAPr06YPWrVvXeB3Tp0/H8OHDMWrUKHTs2BG2trbo06cPrK2tK72NgQMH4ssvv8T8+fMRGhqK7777DkuXLkW3bt0AAI6OjliyZAnCwsLQvHlzbNmyBf/88w9cXFzg6OiIVatWoUePHggODsbixYvx+++/IzQ0tJo+sXkQxJq+CF7DLl++DB8fHyQkJMDb27va9vPdzvMI3DIWPZTHgS7TgB7vVtu+iIjo3uTn5+PixYsICAi4p2BBVcNgMCA4OBhDhw7FBx98IHc5VeZO36uayh9l8UxdFekZ7IZVeqkXrCFyBSDD6W4iIiJzEBcXhyVLluDMmTM4efIkJkyYgIsXL+KZZ56RuzSLxlBXRRq62uK0Q2dkilooMhKA+L1yl0RERCQLhUKBZcuWoV27dggLC8PJkyexZcsWBAcHy12aRWNHiSoiCAI6B/vgv4MPYbhqOxD5O+DfWe6yiIiIapyPj0+5nqtU/XimrgqVvQQrnloLFFr+QIdERERkHhjqqlD7AGfEWAXjL30XJIR9DCiqboBJIiIiojthqKtCGpUSnQPd8XrReKwqeghQaeQuiYiIiOoIhroq1iPYDQCw7fTtn5lHREREVNUY6qpY9yZSqEu+fBHZW+YBkX/IXBERERHVBQx1VczVToMWPo7oqTwG2z0fAXsXyl0SERER1QEMddWgZ5Ab/tV3QIy2NdBhPAciJiIi2XTr1g1Tp041vvf398eCBQvuuI4gCFizZs0D77uqtnMns2bNQsuWLat1H7UFQ1016BHkhkzY4smc6chv9gyg4GEmIqJ7M2DAAPTt27fCebt374YgCDhx4sQ9b/fQoUN44YUXHrQ8E7cLVomJiejXr1+V7otuj2mjGoR62cPdXoPcQj0OXEyTuxwiIqqFnnvuOYSHh+Py5cvl5i1duhRt27ZF8+bN73m7rq6u0Ol0VVHiXXl4eECj4UgQNYWhrhoIgoAeQe4AgAMnTgP7FwHx+2WuioiIapPHHnsMrq6uWLZsmUl7dnY2Vq5cieeeew7Xr1/H8OHDUb9+feh0OjRr1gy///77Hbd76+XXs2fPokuXLrC2tkZISAjCw8PLrTN9+nQ0btwYOp0ODRo0wIwZM1BUVAQAWLZsGWbPno3IyEgIggBBEIw133r59eTJk+jRowe0Wi1cXFzwwgsvIDs72zh/zJgxGDhwIObPnw9PT0+4uLhg4sSJxn1VhsFgwPvvvw9vb29oNBq0bNkSGzduNM4vLCzEpEmT4OnpCWtra/j5+WHOnDkAAFEUMWvWLPj6+kKj0cDLywuTJ0+u9L7lxseEVZOeQW74/WA8AmIWAVH/AU2fBHwfkrssIiK6VWHOva+j1ADKm39C9cWAvgAQFICV9s7bVdtUehcqlQqjRo3CsmXL8M4770AQBADAypUrodfrMXz4cGRnZ6NNmzaYPn067O3t8d9//2HkyJFo2LAh2rdvf9d9GAwGDB48GO7u7jhw4AAyMjJM7r8rYWdnh2XLlsHLywsnT57EuHHjYGdnhzfeeAPDhg1DVFQUNm7ciC1btgAAHBwcym0jJycHffr0QceOHXHo0CGkpKTg+eefx6RJk0yC6/bt2+Hp6Ynt27fj3LlzGDZsGFq2bIlx48ZV6rh9+eWX+Oyzz/Ddd9+hVatW+PHHH/H444/j1KlTCAwMxMKFC7Fu3Tr8+eef8PX1RUJCAhISEgAAf//9N7744gusWLECoaGhSEpKQmRkZKX2aw4Y6qpJWKN60KgU+Cm3I57S/Aec/g/IzwCsy3/RiYhIRh973fs6Ty0DQgdJr0//A6wcA/h1Bp79r3SZBc2A3Oum683KuKfdjB07Fp9++il27tyJbt26AZAuvT755JNwcHCAg4MDXn/9dePyL7/8MjZt2oQ///yzUqFuy5YtOH36NDZt2gQvL+k4fPzxx+Xug3v33XeNr/39/fH6669jxYoVeOONN6DVamFrawuVSgUPD4/b7uu3335Dfn4+fv75Z9jYSOH266+/xoABAzB37ly4u0tXuJycnPD1119DqVQiKCgI/fv3x9atWysd6ubPn4/p06fj6aefBgDMnTsX27dvx4IFC/DNN98gPj4egYGB6Ny5MwRBgJ+fn3Hd+Ph4eHh4oFevXrCysoKvr2+ljqO54OXXaqJVK9GpoQuixACk6RoAxflA9Fq5yyIiolokKCgInTp1wo8//ggAOHfuHHbv3o3nnnsOAKDX6/HBBx+gWbNmcHZ2hq2tLTZt2oT4+PhKbT8mJgY+Pj7GQAcAHTt2LLfcH3/8gbCwMHh4eMDW1hbvvvtupfdRdl8tWrQwBjoACAsLg8FgQGxsrLEtNDQUSmXpYzY9PT2RklK5Af0zMzNx9epVhIWFmbSHhYUhJiYGgHSJ9/jx42jSpAkmT56MzZs3G5d76qmnkJeXhwYNGmDcuHFYvXo1iouL7+lzyoln6qpRj2B3bI+9hvWKrvg/XAAiVwCtR8ldFhERlfX21XtfR1nm5v+gAdI2hFvOk0w9+WB13fTcc8/h5ZdfxjfffIOlS5eiYcOG6Nq1KwDg008/xZdffokFCxagWbNmsLGxwdSpU1FYWFgl+waAffv2YcSIEZg9ezb69OkDBwcHrFixAp999lmV7aMsKysrk/eCIMBQhUODtW7dGhcvXsSGDRuwZcsWDB06FL169cJff/0FHx8fxMbGYsuWLQgPD8dLL71kPFN6a13mSNYzdbt27cKAAQPg5eVV4Vg2oihi5syZ8PT0hFarRa9evXD27Fl5ir0PPYKkp0t8e701RAhAXARw45K8RRERkSm1zb1PyjLnRJQqqa3s/XS32+59GDp0KBQKBX777Tf8/PPPGDt2rPH+uoiICDzxxBP4v//7P7Ro0QINGjTAmTNnKr3t4OBgJCQkIDEx0di2f79px769e/fCz88P77zzDtq2bYvAwEDExcWZflS1Gnq9/q77ioyMRE5O6b2GERERUCgUaNKkSaVrvhN7e3t4eXkhIiLCpD0iIgIhISEmyw0bNgxLlizBH3/8gb///htpadJoFVqtFgMGDMDChQuxY8cO7Nu3DydPVk1Ar26yhrqcnBy0aNEC33zzTYXz582bh4ULF2Lx4sU4cOAAbGxs0KdPH+Tn59dwpfenvqMWQR52uCq64JrrzU4SJ/6UtygiIqpVbG1tMWzYMLz11ltITEzEmDFjjPMCAwMRHh6OvXv3IiYmBi+++CKSk5Mrve1evXqhcePGGD16NCIjI7F792688847JssEBgYiPj4eK1aswPnz57Fw4UKsXr3aZBl/f39cvHgRx48fR2pqKgoKCsrta8SIEbC2tsbo0aMRFRWF7du34+WXX8bIkSON99NVhWnTpmHu3Ln4448/EBsbizfffBPHjx/HlClTAACff/45fv/9d5w+fRpnzpzBypUr4eHhAUdHRyxbtgw//PADoqKicOHCBfz666/QarUm992ZM1lDXb9+/fDhhx9i0KBB5eaJoogFCxbg3XffxRNPPIHmzZvj559/xtWrV6t9dOqq1DNYOlu3QdlNaoj8HRBF+QoiIqJa57nnnsONGzfQp08fk/vf3n33XbRu3Rp9+vRBt27d4OHhgYEDB1Z6uwqFAqtXr0ZeXh7at2+P559/Hh999JHJMo8//jheeeUVTJo0CS1btsTevXsxY8YMk2WefPJJ9O3bF927d4erq2uFw6rodDps2rQJaWlpaNeuHYYMGYKePXvi66+/vreDcReTJ0/Gq6++itdeew3NmjXDxo0bsW7dOgQGBgKQevLOmzcPbdu2Rbt27XDp0iWsX78eCoUCjo6OWLJkCcLCwtC8eXNs2bIF//zzD1xcXKq0xuoiiKJ5JAxBELB69Wrjl/HChQto2LAhjh07ZjJKddeuXdGyZUt8+eWXldru5cuX4ePjg4SEBHh7e1dD5Xd2JO4Gnly0F27WxThgNQFCUQ7wXDjgU3t60xAR1Xb5+fm4ePEiAgICYG1tLXc5ZCHu9L2SI3+Ybe/XpKQkACh3Stbd3d04ryIFBQXIzMw0TllZWdVa59209HGEs40aKfkqXPPpIzVG3nlgSCIiIqJ7Zbah7n7NmTPHOHaPg4ODyY2RclAqBHRr4goACLfqJjVG/Q0Ul7/fgIiIiOh+mW2oKxnA8NYbPpOTk+84uOFbb72FjIwM4xQdHV2tdVZGz5uPDFt6xRuwry8NQnxm413WIiIiIqo8sw11AQEB8PDwwNatW41tmZmZOHDgQIUDI5bQaDSwt7c3TnZ2djVR7h11aVwPKoWAc6n5SG90s1NIzD/yFkVEREQWRdbBh7Ozs3Hu3Dnj+5Lu0M7OzvD19cXUqVPx4YcfIjAwEAEBAZgxYwa8vLzuqWePObCztkKHBs6IOHcdm7T9MGx4Z6BRL7nLIiIiIgsia6g7fPgwunfvbnz/6quvAgBGjx6NZcuW4Y033kBOTg5eeOEFpKeno3Pnzti4cWOt7LnUI8gdEeeuY12cCsMe6Xf3FYiIqMpV5ZMJiMzt+2Q2Q5pUF7mHNClxKTUH3ebvgEoh4NjMR2BnbSWNV3dzVHAiIqo+BoMBZ8+ehVKphKurK9RqtfGpDET3ShRFFBYW4tq1a9Dr9QgMDIRCYXpHmxz5g89+rSH+9WzQwNUGF67lYPfZVDx6/Sfg2HJgxErALUju8oiILJpCoUBAQAASExNx9ep9POuVqAI6nQ6+vr7lAp1cGOpqUM8gN1y4dhFbY1LwqD4SyIgHTq4Ees64+8pERPRA1Go1fH19UVxcfNfnlBLdjVKphEqlMqszvgx1NahHkDuW7L6IHbEp0I98GcpmTwFNeH8dEVFNEQQBVlZWsLKykrsUoipnHucL64i2/k6ws1bhek4hIhXBQNPBgJVW7rKIiIjIAjDU1SArpQJdG0tPl9gWkyJzNURERGRJGOpqWM9gNwDA1tMp0qPCdn0KfNcFKMiWuTIiIiKqzRjqaljXxm5QCEBMYiauZumB478BiZHA6X/lLo2IiIhqMYa6GuZso0ZrXycAwLbYa0CL4dKMyN9lrIqIiIhqO4Y6GfS4eQl22+kUoPlQqfHCTiDjioxVERERUW3GUCeDnkHuAICIc6nIs/EB/MIAiMDJP+UtjIiIiGothjoZNHa3RX1HLQqKDdh7PhVo8bQ0I3KF9OgwIiIionvEUCcDQRBMe8GGPAGorIFrp4HE4/IWR0RERLUSQ51MegTdvK8uJgWixh4I6i/NiFwhY1VERERUWzHUyeShBi7QWimRlJmP6MTM0l6wJ1cC+iJ5iyMiIqJah6FOJtZWSnQOrAcA2BqTAjToDti4AbnXgXNbZK6OiIiIahuGOhn1DCpzX51SVTq8yfHlMlZFREREtRFDnYy63wx1kQnpuJZVALR8RpoRuwHIviZjZURERFTbMNTJyN3eGs3qOwAAtsemAO6hQP02gK0HkHZe5uqIiIioNmGok1nZXrAAgGHLgaknAN+HZKyKiIiIahuGOpmVjFe3++w1FBTrAXtPQKGUuSoiIiKqbRjqZNbUywGudhrkFOpx8GJa6Qx9EXD1mHyFERERUa3CUCczhUJAjyY3e8GWXILNvAp8HgL82A/IS5evOCIiIqo1GOrMQA/jI8OSIYoiYOcJ6FwAjR2Qelbm6oiIiKg2UMldAAGdG9WDWqlAQloezl/LRiM3O2D474CDN6C0krs8IiIiqgV4ps4M2GhU6NjQBUCZS7DOAQx0REREVGkMdWaiZ3CZp0uUZdADKadlqIiIiIhqE4Y6M9H9ZmeJI3E3kJ5bKDWmxwMLmgM/PAIU5shYHREREZk7hjoz4eOsQxN3O+gNInaeufmIMHtvacy6gkwgeq28BRIREZFZY6gzIyW9YDdHJ0sNCgXQepT0+ujPMlVFREREtQFDnRl5tKknACA8Orn0EmzLEYCgAOL3AdfOyFgdERERmTOGOjPStL49gj3tUVhswJpjV6RGe08gsI/0+hjP1hEREVHFGOrMiCAIGNbWGwDwx+HL0kDEANBmtPTz+O9AcaFM1REREZE5Y6gzMwNb1YdapUBMYiairmRKjY0eAWw9gNxUIHa9vAUSERGRWWKoMzOOOjX6hnoAAP44HC81KlVAqxHSa3aYICIiogow1JmhYe18AABrj11FXqFeamw1Uvp5fps0fh0RERFRGQx1ZqhjAxf4OGuRVVCMDVGJUqNzABDQFYAIHFsua31ERERkfhjqzJBCIeCpNtLZuj8OJZTOKBmz7tiv0uPDiIiIiG5iqDNTQ9p4QxCAAxfTcCn15iPCgh4DtE5A5mXpMiwRERHRTQx1ZsrLUYsuga4AgD8P3zxbZ2UNhE0Fes4EPFvIVxwRERGZHYY6M/b0zQ4Tfx25jGK9QWrsPBV4+DXA1k2+woiIiMjsMNSZsZ7B7nCxUSMlqwA7Yq/JXQ4RERGZMYY6M6ZWKTCoVX0AwB+Hy3SYKC4Eov4GVk8ASp46QURERHUaQ52ZKxmzbtvpFKRk5UuN+kJg3WQg8jcgbq+M1REREZG5YKgzc4Hudmjl6wi9QcSqo1ekRo0t8NAEoOt0afw6IiIiqvMY6mqBYW2ls3V/HkqAWHK5tce7QPe3AXsvGSsjIiIic8FQVws81sILOrUSF1JzcOjSDbnLISIiIjPEUFcL2GpUeKy5J4BbnjChLwZOrwc2TGeHCSIiojqOoa6WKOkwsf5kIrLyi6TGgkxg5RjgwGIg8bhstREREZH8GOpqida+TmjoaoO8Ij3+iUyUGnXOQPAA6fWRn+QrjoiIiGTHUFdLCIJgPFtnMmZdm9HSz5N/AYU5MlRGRERE5oChrhYZ3NobKoWAyIR0nE7KlBr9OgNOAUBhFnBqjaz1ERERkXwY6mqRerYa9Ap2B1Cmw4RCAbQeKb0+ykuwREREdRVDXS1Tcgl29bErKCjWS40tRwCCEkg4AKSclrE6IiIikotZhzq9Xo8ZM2YgICAAWq0WDRs2xAcffFA6AG8d1KWxKzzsrZGeW4Tw6GSp0c4DaNxXen3sF/mKIyIiItmYdaibO3cuFi1ahK+//hoxMTGYO3cu5s2bh6+++kru0mSjVAgY0sYbwC1j1rUeJf2M/B0oLpChMiIiIpKTWYe6vXv34oknnkD//v3h7++PIUOGoHfv3jh48KDcpclq6M3Hhu05l4rLN3Klxka9ADtPIPc6cPo/GasjIiIiOZh1qOvUqRO2bt2KM2fOAAAiIyOxZ88e9OvXT+bK5OXrokPHBi4QReCvI5elRqUKaPV/0uujP8tXHBEREcnCrEPdm2++iaeffhpBQUGwsrJCq1atMHXqVIwYMeK26xQUFCAzM9M4ZWVl1WDFNefp9tLZupWHL0NvuHmPYUmou7AduHFJnsKIiIhIFmYd6v78808sX74cv/32G44ePYqffvoJ8+fPx08/3X7ojjlz5sDBwcE4hYSE1GDFNadPqAfsrVW4kp6HiHOpUqOTP9CgG6CwAi4flrM8IiIiqmGCaMZdSX18fPDmm29i4sSJxrYPP/wQv/76K06frnjojoKCAhQUlHYUuHLlCkJCQpCQkABvb+9qr7kmzVwbhZ/3xaF/c09880xrqfHaGenxYTb15C2OiIioDrt8+TJ8fHxqNH+Y9Zm63NxcKBSmJSqVShgMhtuuo9FoYG9vb5zs7Oyqu0zZlHSYCD+VjBs5hVKja2MGOiIiojrIrEPdgAED8NFHH+G///7DpUuXsHr1anz++ecYNGiQ3KWZhab1HRDqZY9CvQGrj10pv0BmYs0XRURERLIw61D31VdfYciQIXjppZcQHByM119/HS+++CI++OADuUszGyVPmPjjUELpoMyFucCP/YAFTYGsZBmrIyIioppi1qHOzs4OCxYsQFxcHPLy8nD+/Hl8+OGHUKvVcpdmNp5oUR8alQKxyVmIvJwhNap1gKEYEA1AXIS8BRIREVGNMOtQR3fnoLNCv6YeAG55wkT/z4BXTgFNB8tUGREREdUkhjoLMPTmJdh/Iq8it7BYavRsDth7yVgVERER1SSGOgvwUIALfJ11yC4oxvqTSeUXSDgIXDlS84URERFRjWGoswAKhYChbaUxcP4sewkWACL/AH54BFg7CdAXyVAdERER1QSGOgsxpI0PFAJw8FIazl/LLp0R+AigcwFSooH938pXIBEREVUrhjoL4eFgjW5N3AAAfx4uc7ZO5ww8cnMImB2fAOkJFaxNREREtR1DnQUpecLE30euoEhf5qkbLYYDvh2Bolxg45syVUdERETViaHOgvQMdkM9WzVSswuw/XRK6QyFAuj/OaBQAaf/BWI3ylckERERVQuGOgtipVRgcOubHSYO33KZ1T0EeOgl6fWGadJTJ4iIiMhiMNRZmJJLsNtjryE5M990ZtfpgL03kB4P7J4vQ3VERERUXRjqLEwjN1u08XOC3iDiryOXTWdqbIF+c6XXEQuBa7E1XyARERFVC4Y6CzTs5hMmVh5OgCiKpjOD+gOBfQBDEfDfa8Ct84mIiKhWYqizQP2becJGrcSl67k4cDHNdKYgAI/OA1Ra4NJu4MSf8hRJREREVYqhzgLZaFQY0EJ67usv++PKL+DkD3R5XXq9dTafNEFERGQBGOos1KiO/gCA/04k4uTljPILdJoMtPo/YORqQGlVs8URERFRlWOos1AhXvYY1Ko+AODj9THl761TqYEnvgFcm8hQHREREVU1hjoL9lrvxlArFdh34Tp2xF6788LJpwCDvmYKIyIioirHUGfBvJ10GBPmDwCYsyEGesNterpu/QBY3Bk4/GPNFUdERERViqHOwk3s1ggOWiucSc7G37eOW1fCzgMQDUBKdM0WR0RERFWGoc7COeis8HKPRgCAz8JjkVtYXH6htmOBsZuBx76o4eqIiIioqjDU1QEjO/rB20mL5MwC/LjnYvkFFErAt0PNF0ZERERVhqGuDtColJjWR+rlunjnBaRmF9x+4YwrwIbpQPEdliEiIiKzw1BXRwxo7oVm9R2QXVCMhVvPVryQwQD89BhwYLH0bFgiIiKqNRjq6giFQsBbjwYBAH47EI8L17IrWgjo9rb0evd8IK2CS7VERERklhjq6pBODeuhR5Abig0i5m2MrXihZkOAgK5AcT6wfhpw66DFREREZJYY6uqYN/sFQSEAG08l4UhcWvkFBAHo/xmgVAPnwoGYf2q+SCIiIrpnDHV1TGN3Owxt6wMA+Hj96fKPDwOAeoFA2BTp9cY3gYIKLtUSERGRWWGoq4NeeaQxtFZKHIm7gU2nkipe6OHXAEc/IPMKsGNOzRZIRERE94yhrg5yt7fGuIcDAABzN8aiSG8ov5CVFnh0vvR6/yIgKaoGKyQiIqJ7xVBXR73QtSFcbNS4mJqDFQfjK16ocW8geAAg6oH/XpWGPCEiIiKzxFBXR9lqVJjaKxAAsGDLWWTlF1W8YN9PACsbIOEAsOltIO1CDVZJRERElcVQV4c93d4XDerZ4HpOIb7fdZuw5uANdL85dt2BRcBvT9dcgURERFRpDHV1mJVSgTf6SgMSL9l9AUkZ+RUv2HEiMOh7oEF3oPnQ0vbCXODngcDBJYD+Nmf6iIiIqEYw1NVxfULd0dbPCflFBnwRfqbihQQBaDEMGLVG6hVb4uxm4MJ26ZFiClVpe3FhtdZMRERE5THU1XGCIOCtR4MBACuPJCA2KetuK5S+9n0I6P0R0HlKaXtxIbCgqXSZ9sRKoOAu2yMiIqIqwVBHaOPnhH5NPWAQgU82xFR+RTsPoNMkoN3zpW0J+4HsZODMBmDV88CnjYA/RwHRa4GivKovnoiIiAAw1NFNb/QNgkohYHvsNew9l3r/GwroArx0AOg6HXBuKD1DNnqtFOw+bQT8PQ6I3QBcPw+kJwDZ1/jECiIioiogiBU+J8pyXL58GT4+PkhISIC3t7fc5Zi199ZG4ad9cWhW3wFrJ4ZBoRDuvtKdiCKQdAKI+huIWgVkJFS8nHd74Pnw0vdftwPybgCj1gLuoVLbseXA0Z8BlUaalGpAZS29ttICahtp6BW1DrDSAWpbwMYFaNSrdLsZVwCFEtC5AEqrB/tsREREdyBH/lDdfRGqKyb3DMTfR6/g5JUM/HPiKp5oWf/BNigIgGcLaeo1G7h8SAp4seuBvHTpLJ6+UApmZeWkAnlpgKAsbUuPky7t3ot6jYFJZULd8iFASrQUFht0k9qO/w5s/1gKg2obwM4TcPIHnAMApwDAuQHg4AMo+b8KERGZN/6lIiMXWw0mdGuITzfFYt7GWPRt6gGNSnn3FStDEACf9tLUb25pu8EAGIpNl31+ixT4nPxL25oOAdybAsUFgL5Aml9cKP0sygOKcoHCbGmYlaJcoDAHsK8olArSGb0SualAxm2eqFFCoZKCXUnQcwsG2o+71yNARERUrRjqyMTYsAD8si8OV9Lz8PPeOIzr0qB6d6hQAAq1aZtLw/LLuTaWpgfx0j7pknBZzZ8GfDsBRTnSvX0Zl4EbF4G0i6U/9QXS6xsXb9ZyS6j7ZbAUJB/9FPBoJrXlpkmXeq0dHqxmIiKiSmKoIxNatRKv9m6MN/46ga+2ncVTbb3hqFPffcXaQrjlPkFbV2m6HYMByEo0DXplg5ooAgkHgcIs6T6/Eod/ALZ9KJ3hcwuRzu65h0qv6zUGVBZ0TImIyCww1FE5T7b2xg+7LyI2OQvf7jiPt2+OY1cnKRSAQ31p8u9c8TKj10qBr+zl4pzr0s+MBGk6u6nMNlWASyMp4LmHAG6hUuhz9JP2R0REdB/Y+5UqtD02Bc8uPQS1UoGtr3WFj7NO7pJqn7wbQEoMkHxK6qCRHC29L8ioePnG/YBnVpS+j9sL1Gsi9eIlIqJahb1fyWx0a+yKsEYuiDh3HZ9tjsWCp1vJXVLto3UC/DpJUwlRBDKv3Ax4p0qDXmqs6b2EuWnA0n7S67cuAxo76XX0WiksOvoCDr6AgzdgZV1zn4mIiMwWQx1VSBAEvNUvGI99tQdrjl/F8w83QNP6vOn/gQmCFMQcvIHGvUvb9UWmT9zISiq9nFsS6ADgwPdA3B7Tbdq6S/fuOfrcDHs+0qVcRx/ptca22j4OEZGRKALizRENDMXSmKEl9zEXZEmjF1jppCGkAOn3Xm4aALF0fenFLe/LtAHS77ySsUbzM6TRDtQ2pfc7G/TS0FhAmfuohfLvtY5ShzYLwlBHt9W0vgMGtaqP1ceu4OP1MVj+fAcIt3Y0oKqhtDIdENk9BJgSKQ3bUpZ/mDTYckYCkB4v9brNTpamK4cr3nbYVOCR2dLr/ExprECPZkD9NuU7jhBR1cq+Jv1/auteelY94wqQdl76/1tfcHOopkLTnyXjeJa0qW2BHu+UbnfjW9JZ/h7vAt5tpbbYjcDWm/+vo8z/2+WCDUrfK62AcdtKl930DhAXAXR5Awh6VGq7tAdYO7F0CCrjpJd+ivrStrLeulL6j8qNbwLHfgV6vgc8/KrUlnwK+L7rvR/TycekMUQBYPfnQMQC4KGJQN+PpbasROCL0Ltv55Vo6X5pC8JQR3f0Wu/G+O9EIvaev45Np5LRt6mH3CXVLbf2ku3+dulrUZT+lZsRLwW89ITSsJd+82dBBmDrVrrO9XPAv1Olp2pMO1/aHrVK+peue1PA3othjyyXvlgawsigB3TOpe1nNgMFmUDjvqVBJHYDcDb85hiYOaU/i/LLhLGim68LpfDlFgS8uKt0uz/0Am5cAp4Ll8bpBIBTq4HNZQJaZdh5moa6y4eBywdNn72dnyHdv3svlLf8jrl+Hrh6DMhJKW0rLpA+w726NeQBMDnjBgBCSeewCs6kmbwv02YSWBVS57Nbf2cJiptn+iy620A5DHV0R95OOowJ88f3uy5g/K9H8FhzT7zWuwkC6tncfWWqXoIgdaKwcQG8bnPPY34GTH4BKq2Ahj2k+/3K/hLcPAPIvCy91joDHk0B92Y3fzYFXIM4DAuVJ4rSGaWCLOkPa0lI0hdJoQMAfB8q/a6lxADZKQDE0j+4xp+45f3Ns0I2boBvB2l9gwGI+EIKUGFTSi/jHVsOnNtSeoarop8lA5QX50vrNOgmPV2mxKpxQH46MPFQ6ZiYV45KwxPdi1vPrqu00lQ24Ni4Sp2gVOrSRx4q1bc8AlENKDWlr60dTbfbZZp0f61Xy9K2Bt2AUetw98uZYumPW//91vUNoO1YqUd+ifptgLGbpf/GCuXNn6o7vL/ZZlWmg93jXwNPfGO6L6+WwHs3KjyMldbrPWkqy8H79tsVxdLjYIH/eGWoo7ua2isQ17IKsPrYFfx7IhEbopIwtK0PpvQMhIcDb9I3a7cOfuzRDBi52rRNXyx15kg6CaSekR7RdnGXNJVQqKQ/Qh5NpW24NwXqt+bgyvfLoL/lEtutQaQAcPKT7pEEpEt4p/+V/sC3HF66ncgV0qV3kz+qZSZl2fdW0h9bRz+gXiNp/eJC4NppKUCVDQdRq6SzugWZUmArN5VpLwkr7Z4H+n8mvS7IApb2lV7PSC29tWDXp9Ll/3sR9Bjgu1x6LQjS+I+iAWj7bGmoSzoBnFp1b9stew8rIIXPwhzTe6wCHpbO+KhtpEljJ/0see60UiN9tpIwplSbBhkAmFjB4w1bDJOmB1H2ntwSdu7S9CDqty7fpnUsDdb3y1wClCCYTy3VgKGO7kqnVuGLYS0x7uEGmL85FttOp+D3g/FYdfQyxnTyx/iuDeFkw7M4tZZSBTy5RHpdlA9ciwGSooDkqJs/T968rHNKmk78IS07bDkQ/Jj0OuZf4MhS6Sxgx4ml2z79n3Tmz6aedMnX2tE8xuIrygNyr0vPF7b3LG0/t+Xm5bU8aSoJWSWX2PSFpZfZ9DfbW42U/vgDQMIhYPO70v0+gxaVbndJT2ng6pLAZii6e429ZgOdp0qvM+Kly+YOPqah7uD3wJUj9/bZO04C+nwkvc5OBr57GFDbAW9fLl3m2K/A+a33tl19mTNUChXg3LD8H097L+msL0r+sJb9ifLtSjVQL7B0fUEA2oyR5pe9bBg8QDrmKo1p4DK+t5buRdXYSvemqW3KP3P6mT/Kf6aALtJEVEsw1FGlhXjZ48cx7XDoUhrmbTyNQ5du4LtdF/DbgXi80KUBxnYOgI2GX6lazcpaupRb9nKuKEr36hmD3knpZ8kj0QBpSJZzWwDbMvdcFuYCK54x3b6gkEKezuVm0Lv5WldPOuNhKAZajyo9Q3Vxt3T/kWcLoM1oqc2gB/4ae/PmbH0FN27rb4ax/NJgVpQPPPm/0rMbp9YAa8ZLIbTsmcs/x0hPB7kXvh1LQ11hFpCwX7rMV1beDSlEVkRQVBxEyp4F1bkATfqXf/pJ477SGVRDUekx0BffckxK2oukY2PnaboNWw/p8p4oloawwEek3tMaO0Bjf/MMle3N92XaSia1rWlYt7YHJh8t/1l7fyhND+KxL8q3+Xe+/eDgRHUI/wLTPWvn74w/X+yIHbHXMHfjaZxOysJn4Wfw075LmNS9EYZ38IVGZVndxOs0QZBClqNvaW+4WzV5VOrd5+hX2laUC3i3k8JMznWp04ZoAHJTpSk1tuJtNeheGupSYqR7mkIHlYY6CED0mnv/HGWDlpW1dDlSuOWsoXdbKQSWnNkpOctTcu9T2amkzbtd6fruzYChv5jegA8Aw3+XPntJYCsb3pSV+DXs5A8M/618e9c3Kv3xK+ToA7xewX+HhyY82HaJSBZm/0SJK1euYPr06diwYQNyc3PRqFEjLF26FG3btq3U+nyiRPUyGET8c+IqPtt8BvFpuQAAbyctXunVGANb1YdSYbn3LtA9Ki6U7tfLvS6NIZV7vXTKSZXOvClUQPsXS+/5unwEOLtZumk7dGDptg58V8EN2jdv0haUN0OZtemlNzvP0l6NZc9KERFVAznyh1mHuhs3bqBVq1bo3r07JkyYAFdXV5w9exYNGzZEw4YN774BMNTVlMJiA/44nICFW8/iWlYBAKCxuy1e790Ej4S4c3w7IiKqU2pNqEtISIAgCMYiDx48iN9++w0hISF44YUXqqy4N998ExEREdi9e/d9b4OhrmblFeqxbO8lLNpxDpn5Uq+4Vr6OeKNPEDo25DNMiYiobpAjf9xXN7RnnnkG27dvBwAkJSXhkUcewcGDB/HOO+/g/fffr7Li1q1bh7Zt2+Kpp56Cm5sbWrVqhSVLltxxnYKCAmRmZhqnrKx7vOmZHohWrcSEbg2x+40eeKlbQ1hbKXAsPh3Dl+zHyB8O4OTl2zzMnoiIiB7IfYW6qKgotG8vjYz9559/omnTpti7dy+WL1+OZcuWVVlxFy5cwKJFixAYGIhNmzZhwoQJmDx5Mn766afbrjNnzhw4ODgYp5CQkCqrhyrPQWeFN/oGYde07hj5kB9UCgG7z6ZiwNd7MHH5UZxLYdgmIiKqSvd1+dXW1hZRUVHw9/fH448/jrCwMEyfPh3x8fFo0qQJ8vLy7r6RSlCr1Wjbti327t1rbJs8eTIOHTqEffv2VbhOQUEBCgoKjO+vXLmCkJAQXn6VWdz1HHwRfgZrI68aB/PuHeKOl7o3QksfR1lrIyIiqmq15vJraGgoFi9ejN27dyM8PBx9+0ojh1+9ehUuLlV335Snp2e5M23BwcGIj4+/7ToajQb29vbGyc7Orsrqofvn52KDBU+3wvrJD6N3iDTi+eboZAz8JgIj/rcfe86mwoz77BAREZm9+wp1c+fOxXfffYdu3bph+PDhaNGiBQDpHriSy7JVISwsDLGxpmMonTlzBn5+frdZg8xdsKc9vh/VFuGvdMHg1tKQJxHnruP/fjiAgd9EYGNUIgwGhjsiIqJ7dd9Dmuj1emRmZsLJycnYdunSJeh0Ori5uVVJcYcOHUKnTp0we/ZsDB06FAcPHsS4cePw/fffY8SIEZXaBnu/mrfLN3KxZNcFrDiUgIJiAwCgoasNxndtiIGt6sNKaQaPlCIiIrpHtWZIk7y8PIiiCJ1OenBxXFwcVq9ejeDgYPTp06dKC/z333/x1ltv4ezZswgICMCrr76KcePGVXp9hrraITW7AEsjLuLnfXHIujkUipeDNcZ1aYCn2/lCq+YTKoiIqPaoNaGud+/eGDx4MMaPH4/09HQEBQXBysoKqamp+PzzzzFhgvk8YoahrnbJyi/Cr/vj8cOei0jNljq8ONuo8Wwnf4zq6A8HnZXMFRIREd1drekocfToUTz8sPQA67/++gvu7u6Ii4vDzz//jIULF1ZpgVS32FlbYUK3htgzvTs+HNgUPs5apOUU4rPwMwibuw1z1scgJTNf7jKJiIjMzn2FutzcXGOv0s2bN2Pw4MFQKBR46KGHEBcXV6UFUt1kbaXE/z3kh+2vdcOXT7dEkIcdsguK8d2uC+g8bzveXn0Scddz5C6TiIjIbNxXqGvUqBHWrFmDhIQEbNq0Cb179wYApKSkwN7evkoLpLpNpVTgiZb1sWHKw/hhdFu08XNCYbEBvx2IR/f5OzD592NYe/wKDl5MQ0JaLor0BrlLJiIikoXqflaaOXMmnnnmGbzyyivo0aMHOnbsCEA6a9eqVasqLZAIAARBQM9gd/QIcsPBi2n4dsd57DxzDesir2Jd5NUyywFudhp4Omjh5WgNLwctPB218HKwhpejFp6O1qhno4FCIcj4aYiIiKrefQ9pkpSUhMTERLRo0QIKhXTC7+DBg7C3t0dQUFCVFvkg2FHCcp26moFf98fjYmo2EjPykZiej8JKnKlTKxVwd9DAy0ErBT0Ha3g6auHtqMVDDVzY05aIiB5Yren9Wtbly5cBwGwDE0Nd3WEwiLieU4jEjDxcTc/D1fR86XVGPq6m5yExPR8pWfm409jGAfVs8PPY9vBx1tVc4UREZHHkyB/3dfnVYDDgww8/xGeffYbs7GwAgJ2dHV577TW88847xjN3RDVJoRDgaqeBq50Gzb0dK1ymSG9AcmY+EkuC3s2fV9PzcTzhBi6m5uDJRXvx83PtEeTB+0OJiKj2uK9Q98477+CHH37AJ598grCwMADAnj17MGvWLOTn5+Ojjz6q0iKJqoqVUgFvJx28ncqfiUvKyMfoHw8iNjkLQxfvww9j2qGdv7MMVRIREd27+7r86uXlhcWLF+Pxxx83aV+7di1eeuklXLlypcoKfFC8/Er3IiO3CM/9dAiH425Ao1Lgm2dao1eIu9xlERFRLVNrBh9OS0ursDNEUFAQ0tLSHrgoIrk46Kzwy3Md0DPIDQXFBrz46xH8eThB7rKIiIju6r5CXYsWLfD111+Xa//666/RvHnzBy6KSE5atRKLR7bBk629oTeIeOOvE1i88zwesE8RERFRtbqve+rmzZuH/v37Y8uWLcYx6vbt24eEhASsX7++SgskkoOVUoH5TzVHPVs1vtt1AZ9sOI3r2QV4q18wx7gjIiKzdF9n6rp27YozZ85g0KBBSE9PR3p6OgYPHoxTp07hl19+qeoaiWQhCALeejQYbz8q3WqwZPdFvL4ykk+tICIis/TA49SVFRkZidatW0Ov11fVJh8YO0pQVfj7yGW88fcJ6A0iujdxxTcjWkOnvq8T3UREVAfUmo4SRHXNk228sWRUG1hbKbA99hr+738HkJ5bKHdZRERERgx1RJXUI8gdy5/vAHtrFY7Gp+OpxfuQmJEnd1lEREQAGOqI7kkbP2esHN8J7vYanE3JxpPf7sW5lGy5yyIiIrq33q+DBw++4/z09PQHqYWoVmjiYYe/J3TCqB8P4sK1HDy1eC9+HNMOrXyd5C6NiIjqsHs6U+fg4HDHyc/PD6NGjaquWonMhreTDn+N74QW3g64kVuEZ5YcwM4z1+Qui4iI6rAq7f1qjtj7lapTTkExxv96BLvPpkKlEPDZ0BZ4omV9ucsiIiKZsfcrUS1jo1Hhh9HtMKCFF4oNIqasOI6lERflLouIiOoghjqiB6RWKfDlsJYY08kfADD7n2h8uuk0HytGREQ1iqGOqAooFALeGxCC13s3BgB8s/083lt3isGOiIhqDEMdURURBAGTegTi40HNIAjAz/vi8O2O83KXRUREdQRDHVEVe6aDL957LAQA8OmmWKw+dlnmioiIqC5gqCOqBmPCAjDu4QAAwBt/ncDec6kyV0RERJaOoY6omrzVLxj9m3uiSC/ixV+O4HRSptwlERGRBWOoI6omCoWAz55qgfb+zsgqKMaYHw/xWbFERFRtGOqIqpG1lRLfj2qDhq42SMrMx7NLDyEzv0jusoiIyAIx1BFVM0edGsuebQ9XOw1OJ2Vhwq9HUFhskLssIiKyMAx1RDXAx1mHpWPaQadWIuLcdbz59wmOYUdERFWKoY6ohjSt74BvR7SGUiFg1bEr+GzzGblLIiIiC8JQR1SDujVxw8eDmgIAvt5+Dr8diJe5IiIishQMdUQ1bFg7X0zuGQgAeHfNSWw7nSxzRUREZAkY6ohk8EqvQAxp4w2DCExcfgwnLqfLXRIREdVyDHVEMhAEAXMGN8PDgfWQV6TH2GWHEH89t1r3WVCsx/bTKbiazrHyiIgsEUMdkUyslAp8O6I1gj3tkZpdiDFLD+JGTmGV7ycpIx+fbY5F2Cfb8OyyQxiyaC+yC4qrfD9ERCQvhjoiGdlZW2HZs+3g5WCNC6k5GPfzYeQX6R94u6Io4tClNEz87Sg6z92Gr7adQ2q2FBiv3gx5RERkWRjqiGTmbm+NZWPbw85ahcNxN/DKH8dhMNzfGHb5RXr8cSge/RfuwVOL9+G/E4koNohoH+CMb0e0xtIx7QAAP+29xPv4iIgsjEruAogIaOxuh+9HtsXoHw9iQ1QSPvwvBjMHhFR6/cs3cvHL/jj8cSgB6bnSY8isrRQY2LI+RnX0R4iXvXHZJ1p6Ye3xq3jz75NYNykMKiX/bUdEZAkY6ojMRMeGLvj0qeaYsuI4foy4CC9Hazz/cIPbLi+KIvaev46f9l7ClphklJzc83bSYuRDfhjWzgeOOnW59WY8FoIdsdcQnZiJpRGXMK7L7fdBRES1B0MdkRl5omV9XE3Px9yNp/HR+hh4OWrxaDNPk2VyCoqx6tgV/Lz3Es6mZBvbOzeqh1Ed/dAz2B1KhXDbfdSz1eCdR4Pxxt8n8Hn4GfRt6gEfZ121fSYiIqoZDHVEZmZ81wa4mp6HX/bHYeofx+Fqp0E7f2dcSs3Bz/visPJIArLypd6rOrUST7b2xuhOfmjkZlfpfTzV1ht/H72MAxfTMGNtFJaOaQdBuH0QJCIi88dQR2RmBEHArMdDkZiRjy0xyXj+p8No5euIHbHXjMv4u+gwqqM/hrT1hr211X3t4+PBzdBvwW7siL2Gf08kYkALr6r8GEREVMN4hzSRGVIqBHw1vBVa+DgiI6/IGOi6NXHF0mfbYdtr3TC2c8B9BboSDV1tMbF7IwDA7H+ikXGzgwUREdVOPFNHZKa0aiV+GN0Ws9adgpudNUZ29ENAPZsq3cf4bg2wLvIKzl/LwScbYzBncPMq3T4REdUcnqkjMmP1bDX4+pnWmDkgpMoDHQBoVEpjkPv9YAIOXkyr8n0QEVHNYKgjquPaBzjj6XY+AIC3V59EQfGDP9GCiIhqHkMdEeGtfsGoZ6vGuZRsfLfzgtzlEBHRfWCoIyI46Kwwc0AoAODrbedw/lr2XdYgIiJzw1BHRACAAc090bWxKwr1Bryz+iRE8f6eP0tERPJgqCMiANLYdR8ObAprKwX2X0jDyiOX5S6JiIjuQa0KdZ988gkEQcDUqVPlLoXIIvk46/BKr8YAgI/XxyA1u0DmioiIqLJqTag7dOgQvvvuOzRvznG0iKrT2M4BCPa0R3puET78N1rucoiIqJJqRajLzs7GiBEjsGTJEjg5OcldDpFFs1Iq8MngZhAEYM3xq9h99trdVyIiItnVilA3ceJE9O/fH7169ZK7FKI6oYWPI0Z39AcAvLM6CnmFHLuOiMjcmX2oW7FiBY4ePYo5c+ZUavmCggJkZmYap6ysrGqukMgyvd6nCTwdrBGflouF287KXQ4REd2FWYe6hIQETJkyBcuXL4e1tXWl1pkzZw4cHByMU0hISDVXSWSZbDUqzH5cGrvu+10XEJOYKXNFRER0J4JoxoNRrVmzBoMGDYJSqTS26fV6CIIAhUKBgoICk3mAdKauoKC0x96VK1cQEhKChIQEeHt711jtRJZi/C9HsPFUElr6OOLvCZ2gVAhyl0REZPYuX74MHx+fGs0fqhrZy33q2bMnTp48adL27LPPIigoCNOnTy8X6ABAo9FAo9EY32dm8uwC0YOY9Xgo9pxLxfGEdCw/EIdRN++1IyIi82LWoc7Ozg5NmzY1abOxsYGLi0u5diKqHh4O1nijbxPMXHsK8zbGoneIBzwcKnc7BBER1RyzvqeOiMzDiA5+aOnjiOyCYry3LkrucoiIqAK1LtTt2LEDCxYskLsMojpFqRAwZ3AzqBQCNp1KxuZTSXKXREREt6h1oY6I5BHsaY9xXRoAAGauPYWs/CKZKyIiorIY6oio0qb0DISvsw5Jmfn4bPMZucshIqIyGOqIqNKsrZT4aJDUSemnfZdwPCFd3oKIiMiIoY6I7snDga4Y2NILogi8/PtRxF3PkbskIiICQx0R3YcZj4XAx1mLhLQ8DP52L05cTpe7JCKiOo+hjojumYutBn9P6IRQL3tczynE09/vx47YFLnLIiKq0xjqiOi+uNlZY8ULD6Fzo3rILdTj+Z8O468jl+Uui4iozmKoI6L7ZmdthR/HtMPAll4oNoh4fWUkvtl+Dmb8SGkiIovFUEdED0StUuDzoS3xYldpDLtPN8XivXWnoDcw2BER1SSGOiJ6YAqFgLf6BeO9ASEQBODnfXGYuPwo8ov0cpdGRFRnMNQRUZV5NiwAXw1vBbVSgY2nkjDqh4PIyOWTJ4iIagJDHRFVqceae+Gnse1hZ63CwUtpGLJ4L66m58ldFhGRxWOoI6Iq17GhC1aO7wh3ew3OpmRj8Ld7EZuUJXdZREQWjaGOiKpFkIc9Vr0UhkZutkjKzMeQxXux/8J1ucsiIrJYDHVEVG3qO2rx1/iOaOvnhKz8Yoz64SDWn0yUuywiIovEUEdE1cpRp8avz3dAn1B3FOoNmPjbUSyLuCh3WUREFoehjoiqnbWVEt+OaIORD/lBFIFZ/0Tjkw2nOUgxEVEVYqgjohqhVAh4/4lQTOvTBACweOd5vPZnJAqLDTJXRkRkGRjqiKjGCIKAid0b4dMhzaFUCFh17Aqe++kQsguK5S6NiKjWY6gjohr3VFsf/G90W2itlNh9NhVPf78P17IK5C6LiKhWY6gjIll0b+KGFS88BBcbNaKuZGLYd/twPZvBjojofjHUEZFsWvg44u8JnVDfUYsLqTkY+9Nh5PBSLBHRfWGoIyJZ+dezwU9j28NJZ4XIhHRMWH6UnSeIiO4DQx0Rya6Rmy1+HNMOWisldp25hjf+ioTBwOFOiIjuBUMdEZmFVr5OWPR/raFSCFhz/Co+Wh/DceyIiO4BQx0RmY1uTdwwb0hzAMAPey7i+10XZK6IiKj2YKgjIrMyuLU33nk0GAAwZ8Np/HXksswVERHVDgx1RGR2xnVpgBe7NAAATP/7BLadTpa5IiIi88dQR0RmaXrfIAxuXR96g4iXlh/FkbgbcpdERGTWGOqIyCwpFALmPtkc3Zq4Ir/IgLHLDuFscpbcZRERmS2GOiIyW1ZKBb4d0RotfRyRkVeEUT8exNX0PLnLIiIySwx1RGTWdGoVlo5ph4auNkjMyMeoHw8iPbdQ7rKIiMwOQx0RmT0nGzV+fq4DPOytcS4lG2OXHUJeoV7usoiIzApDHRHVCvUdtfj5ufawt1bhaHw6Jv52FEV6Pk6MiKgEQx0R1RqN3e3w45h20KgU2HY6BW+tOsmnThAR3cRQR0S1Slt/Z3zzTGsoFQL+OnIZczfGyl0SEZFZYKgjolqnV4g75gxqBgBYvPM8/rebjxMjImKoI6JaaWg7H7zRtwkA4MP/YrDm2BWZKyIikhdDHRHVWhO6NsSzYf4AgNdXRmLXmWvyFkREJCOGOiKqtQRBwIz+IXi8hReKDSLG/3oExxPS5S6LiEgWDHVEVKspFALmP9UCnRvVQ26hHmOXHcL5a9lyl0VEVOMY6oio1lOrFFg8sg2aezsgLacQT3wdgQVbziC7oFju0oiIagxDHRFZBFuN9DixFj6OyC4oxoItZ9Fl3nb8sOci8ov49AkisnwMdURkMVxsNVg9oRO+eaY1GtSzQVpOIT74Nxo9P9uJPw8noJhPoCAiC8ZQR0QWRaEQ0L+5Jza/0gWfDG4GD3trXEnPwxt/nUDfL3djY1Qin0JBRBaJoY6ILJJKqcDT7X2xY1o3vPNoMBx1VjiXko3xvx7FwG/3Yu+5VLlLJCKqUgx1RGTRrK2UGNelAXa90R2TezSCTq1EZEI6nvnfAfzf/w4gkkOgEJGFYKgjojrB3toKr/Zugp3TumNMJ39YKQXsOZeKJ76JwIRfj+BcCodBIaLajaGOiOoUVzsNZj0eim2vdcPg1vUhCMCGqCT0/mIn3vgrElfS8+QukYjovjDUEVGd5OOsw+dDW2LjlC54JMQdBhH48/BldJ+/Ax/8G43r2QVyl0hEdE8Y6oioTmviYYclo9pi1Uud8FADZxQWG/DDnovoMm87Fmw5g6z8IrlLJCKqFEG08L79ly9fho+PDxISEuDt7S13OURkxkRRxO6zqZi36TSirmQCANRKBTo1csEjIe7oFewOd3trmaskotpAjvzBUEdEdAuDQcSGqCQs2HIGZ2/pQNHCxxGPBLvhkRAPNHa3hSAIMlVJROaMoe4Wc+bMwapVq3D69GlotVp06tQJc+fORZMmTSq9DYY6IrpfoijiXEo2NkcnIzw6GcdvGf7E11lnPIPXzt8JKiXvaCEiCUPdLfr27Yunn34a7dq1Q3FxMd5++21ERUUhOjoaNjY2ldoGQx0RVZWUrHxsjUlBeHQy9pxLRWFx6WPHHHVW6NHEDY+EuOPhxq6w1ahkrJSI5MZQdxfXrl2Dm5sbdu7ciS5dulRqHYY6IqoOOQXF2H02FeHRydh2Ohk3cks7VPA+PCKSI3/Uqn9KZmRkAACcnZ1lroSI6jobjQp9m3qgb1MPFOsNOBJ3A+HRyQiPSUbc9VzsiL2GHbHX8M7qKLTwdsAjIe7oGeyOIA873odHRNWi1pypMxgMePzxx5Geno49e/bcdrmCggIUFJSOL3XlyhWEhITwTB0R1Yi73Yfn5WCNbkFu6NHEDZ0auUCnrlX/tiaiSuLl1zuYMGECNmzYgD179tzx4MyaNQuzZ88u185QR0RyKHsfXsS5VBSUuQ9PrVKgYwMX9AhyQ/cmbvB10clYKRFVJYa625g0aRLWrl2LXbt2ISAg4I7L8kwdEZmr/CI99p2/jm2nU7DtdEq5R5I1dLWRAl6QG9r5O8OKvWmJai2GuluIooiXX34Zq1evxo4dOxAYGHjP22BHCSIyRyWXaUsC3uG4G9AbSn8d22lU6BxYD92D3NCtiSvc7NjZgqg2YUeJW0ycOBG//fYb1q5dCzs7OyQlJQEAHBwcoNVqZa6OiOj+CYKAQHc7BLrb4cWuDZGRV4TdZ69h++lr2BGbgus5hdgQlYQNUdLvvebeDujexA09gtzQrL4DFAp2tiAiU2Z9pu52PcSWLl2KMWPGVGobPFNHRLWNwSDixJUMbDudgu2nU3DySobJ/Hq2arT2dYKfiw5+LjbST2cbeDlacwBkIjPBM3W3MOO8SURUbRQKAS19HNHSxxGvPtIYKVn52BF7DdtPp2D32VSkZhdic3RyufVUCgH1nbTwddbB/2bY83WWgp+vsw5atVKGT0NENcWsQx0REQFudtYY2tYHQ9v6oLBYGhPvTHIW4q7nIj4tB3HXcxGXlovCYoP0+noudp9NrWA7Gvi72MDXRQc/Z53008UGQR52sLZi4COq7RjqiIhqEbVKgY4NXdCxoYtJu8EgIjkrXwp613MRl5aDSyWvr+cgM78YKVkFSMkqwMFLaSbrejpY4+tnWqGNHwd2J6rNGOqIiCyAQiHA00ELTwctHmrgUm5+em4hLt0MePE3z+zFX8/FmZQsJGbkY9h3+/FmvyA81zmAT7wgqqUY6oiI6gBHnRotdWq09HE0ac8uKMabf5/AvycS8eF/MTh86QbmPdUc9tZW8hRKRPeN3aSIiOowW40KXw1vhfefCIWVUsDGU0l4/Ks9OHU14+4rE5FZYagjIqrjBEHAqI7+WDm+E+o7anHpei4Gf7sXfx5KkLs0IroHDHVERAQAaOnjiH9f7ozuTVxRUGzAG3+fwLSVkcgr1MtdGhFVAkMdEREZOdmo8cPodpjWpwkUArDyyGUM+jYCF1Nz5C6NiO6CoY6IiEwoFAImdm+EX5/vgHq2apxOysKAr/Zg/clEuUsjojtgqCMiogp1algP/01+GO39nZFdUIyXlh/F7H9OobDYIHdpRFQBhjoiIrotd3tr/DauA17s2gAAsDTiEoZ9vw9X0/NkroyIbsVQR0REd6RSKvBWv2AsGdUWdtYqHItPR/+Fu7HzzDW5SyOiMhjqiIioUh4Jccd/Lz+MpvXtcSO3CGOWHsQX4WegN4hyl0ZEYKgjIqJ74Ouiw1/jO+GZDr4QReDLrWcx+seDuJ5dIHdpRHUeQx0REd0TayslPh7UDF8MawGtlRJ7zqWi/8I9OHwpTe7SiOo0PvuViIjuy6BW3gj1csD4X4/gwrUcPP39fgxuXR8uthrYalTQqZWw0ahgo1bBRqO82aaSft58r1EpIAiC3B+FyCIw1BER0X1r7G6HdZM6461VJ/FP5FX8efjyPa2vVAjQqZXGEFgS/Gw0KrTzd8JznQOgUvKiElFlMNQREdEDsdWosPDplujfzAOnrmYiu6AYuQV6ZBcWI7egGDkFeuQUFiOnoBjZBXrkFhYj9+ajx/QGEVn5xcjKLy633S0xydh2OgVfDW8FN3vrmv5YRLUOQx0RET0wQRDQt6kn+jb1rNTyBoOI3CI9cgqKb06lwS+nUI/E9Dx8te0cDlxMw6MLd2Ph063QqVG9av4URLUbQx0REdU4hUKArUa6v+52Hglxx0vLj+J0Uhb+74cDmNqrMSZ1bwSFgvfgEVWENyoQEZFZauBqizUTwzCsrQ8MIvB5+BmMWXaIw6cQ3QZDHRERmS1rKyXmDmmO+U+1gLWVArvOXEP/hXtwJI7DpxDdiqGOiIjM3pA23lg7sTMauNogKTMfw77bjyW7LkAU+TQLohIMdUREVCs08ZCGTxnQwgvFBhEfrY/BC78cQUZekdylEZkFhjoiIqo1SoZP+WBgU6iVCoRHJ+Oxr3bj5OUMuUsjkh1DHRER1SqCIGDkQ374e0In+DhrkZCWhycX7cUv++N4OZbqNIY6IiKqlZp5O+DfSQ/jkRB3FOoNmLEmClNWHEd2QfmBjInqAoY6IiKqtRx0Vvh+ZBu82z8YKoWAdZFX8fjXexCblCV3aUQ1joMPExFRrSYIAp5/uAFa+jhi0m/HcOFaDp74Zg8+HNgMQ9p4y12eibScQuw7fx17zqVi3/lUKBQCega54ZEQD7Txc4KSAyvTAxBEC78B4fLly/Dx8UFCQgK8vc3rf24iIqpa17ML8Mqfkdh15hoAYFhbH8x+IhTWVkpZ6skr1OPQpTREnEvFnnOpiE7MxO3+6rrYqNEjyA29Qz3wcGA92WqmqiFH/mCoIyIii2IwiPhm+zl8seUMDCIQ5GGHOYObobG7HWzu8FiyqlCsN+DklQxEnEtFxLnrOBJ3A4V6g8kyjd1tEdaoHsIa1kOR3oDw6GRsPZ1iMjSLtZUCXQJd0TvUAz2C3OBso67WuqnqMdRVA4Y6IqK6ae+5VExecQyp2YXGNkedFeo7aqXJSfrp7aSF1802Zxs1BKHyl0BFUcT5azk3Q1wq9l24jqx8044ang7WCGtUD50b1UOnhi5ws7cut50ivQGHLqZhc3QywqOTcSU9zzhPIQDt/J3xSIg7eod4wNdFdx9Hg2oaQ101YKgjIqq7UjLz8fbqKBy4WD5sVURrpYSXozXqO+nKBD5r1HfUob6TFu52GqTlFCLifCr2nL2OiHOpSMrMN9mGvbUKnRrWQ1gjF4Q1qoeAejb3HBSjEzOx+ZQU8KITM03mB3nYoXeIOx4J8UDT+vb3tG2qOQx11YChjoiIACAzvwhX0/Nw5UYertz8eTk9z9iWklVw120oFQL0BtM/m2qVAu38ndCpoXQ2rml9hyrt8JCQlostMcnYfCoZBy+lmezfy8EavW6ewevQwBlWSg5qYS4Y6qoBQx0REVVGQbEeien5xsB3JT3P5HViRh6K9CIEAWhW38F4X1xbf6ca69SQnluIbadTsPlUMnaeuYa8Ir1xnp21ytiTtmsTV9hW8/2DdGcMddWAoY6IiKqCwSDiWnYBrK2UcNBayV0O8ov0iDiXivDoZGyJSTa5d1CtVCCskQt6h3qgZ7Ab3OzK38dH1UuO/MEYT0REVAkKhQD3Cjo5yMXaSomewe7oGewOvUHEsfgbCI9OxqZTSbh0PRfbY69he+w1CALQ2tfpZkcLdzRwtZW7dKomPFNHRERkQURRxLmUbGyOTsbmU0mIvJxhMr+Rmy16h7ijd6gHmtd3gKIK7/8r1huQmJGPhLRcJNzIRUKa1Is32NMeoV728HXWVen+zBkvv1YDhjoiIqrLEjPysCU6GZujk7Hv/HUUl+lo4W6vQa9gKeB1bOACterOHS1EUboEnZCWh8s3cqXwlpaHhBu5iE/LRWJGfrmOJGXZalQI9rRDiKc9Qr0cEOJlj0B3W2hUljfQMkNdNWCoIyIikmTkFWFHbAo2Rydjx+kU5BSW6WihUaFbkBt6h7gjoJ4NLt8oE9xu5BnPvuUXGe6wB6k3sLeTFj5OOvg4a1GsFxGTmImYpCwUFpdfV6UQ0MjN1hjyQr3sEexpXyX3LYqiiOyCYtzIKUJabiHScgqQllNk/DmpR6Nq61DCUFcNGOqIiIjKKyjWY+/568bx8FKz7z6kCwAIAuBpbw1vZx18nXXG8OZz87WbnabCS6zFegPOX8tBdGIGTl3JRHRiJk5dzTR5kkZZPs7a0jN6nvYIrW+PerYa3MgtRFpO6XQjpxDXb/5Myy0yCW43corKPdGjrG2vda22ewwZ6qoBQx0REdGdGQwijl9OvxnwknAjtwg+Tlp4lwltJQHOy1F718u0lSWKIq5m5OPUlQxjyIu+mmnyRI2qoLVSwtlGDWcbNZxs1HCxUcNJp8a4LgHwdNBW6b5KsPcrERER1TiFQkBrXye09nXCm/2Camy/giAYH9vWO9TD2J6eW4joRCngRV+Vwt65a9nQG6RxAp10ajjprOBio4GTjRWcbTRwvvWnTg1nWzWcdWpo1ZZ3z15FGOqIiIjIrDjq1OjUsB46NaxnbMsv0iO3UA8HrVWVPrHDkjDUERERkdmztlLW2JM7ais+JI6IiIjIAjDUEREREVkAhjoiIiIiC8BQR0RERGQBGOqIiIiILABDHREREZEFYKgjIiIisgAMdUREREQWoFaEum+++Qb+/v6wtrZGhw4dcPDgQblLIiIiIjIrZh/q/vjjD7z66qt47733cPToUbRo0QJ9+vRBSkqK3KURERERmQ2zD3Wff/45xo0bh2effRYhISFYvHgxdDodfvzxR7lLIyIiIjIbZh3qCgsLceTIEfTq1cvYplAo0KtXL+zbt6/CdQoKCpCZmWmcsrKyaqpcIiIiItmYdahLTU2FXq+Hu7u7Sbu7uzuSkpIqXGfOnDlwcHAwTiEhITVRKhEREZGsVHIXUNXeeustvPrqq8b3CQkJaNq0KRITE2WsioiIiOqSktxhMBhqbJ9mHerq1asHpVKJ5ORkk/bk5GR4eHhUuI5Go4FGozG+z83NBQC0b9+++golIiIiqkBycjJ8fX1rZF9mHerUajXatGmDrVu3YuDAgQCkxLt161ZMmjSpUtto1aoVDh48CHd3dygUZn21uUpkZWUhJCQE0dHRsLOzk7scs8JjUzEel9vjsbk9HpuK8bjcXl07NgaDAcnJyWjVqlWN7dOsQx0AvPrqqxg9ejTatm2L9u3bY8GCBcjJycGzzz5bqfVVKhXatWtXzVWaj8zMTABA/fr1YW9vL3M15oXHpmI8LrfHY3N7PDYV43G5vbp4bGrqDF0Jsw91w4YNw7Vr1zBz5kwkJSWhZcuW2LhxY7nOE0RERER1mdmHOgCYNGlSpS+3EhEREdVFln+TWR2j0Wjw3nvvmXQWIQmPTcV4XG6Px+b2eGwqxuNyezw21U8QRVGUuwgiIiIiejA8U0dERERkARjqiIiIiCwAQx0RERGRBWCosxBz5sxBu3btYGdnBzc3NwwcOBCxsbFyl2V2PvnkEwiCgKlTp8pdilm4cuUK/u///g8uLi7QarVo1qwZDh8+LHdZstLr9ZgxYwYCAgKg1WrRsGFDfPDBB6iLtx/v2rULAwYMgJeXFwRBwJo1a0zmi6KImTNnwtPTE1qtFr169cLZs2flKbaG3enYFBUVYfr06WjWrBlsbGzg5eWFUaNG4erVq/IVXIPu9r0pa/z48RAEAQsWLKix+iwZQ52F2LlzJyZOnIj9+/cjPDwcRUVF6N27N3JycuQuzWwcOnQI3333HZo3by53KWbhxo0bCAsLg5WVFTZs2IDo6Gh89tlncHJykrs0Wc2dOxeLFi3C119/jZiYGMydOxfz5s3DV199JXdpNS4nJwctWrTAN998U+H8efPmYeHChVi8eDEOHDgAGxsb9OnTB/n5+TVcac2707HJzc3F0aNHMWPGDBw9ehSrVq1CbGwsHn/8cRkqrXl3+96UWL16Nfbv3w8vL68aqqwOEMkipaSkiADEnTt3yl2KWcjKyhIDAwPF8PBwsWvXruKUKVPkLkl206dPFzt37ix3GWanf//+4tixY03aBg8eLI4YMUKmiswDAHH16tXG9waDQfTw8BA//fRTY1t6erqo0WjE33//XYYK5XPrsanIwYMHRQBiXFxczRRlJm53bC5fvizWr19fjIqKEv38/MQvvviixmuzRDxTZ6EyMjIAAM7OzjJXYh4mTpyI/v37o1evXnKXYjbWrVuHtm3b4qmnnoKbmxtatWqFJUuWyF2W7Dp16oStW7fizJkzAIDIyEjs2bMH/fr1k7ky83Lx4kUkJSWZ/D/l4OCADh06YN++fTJWZp4yMjIgCAIcHR3lLkV2BoMBI0eOxLRp0xAaGip3ORalVjxRgu6NwWDA1KlTERYWhqZNm8pdjuxWrFiBo0eP4tChQ3KXYlYuXLiARYsW4dVXX8Xbb7+NQ4cOYfLkyVCr1Rg9erTc5cnmzTffRGZmJoKCgqBUKqHX6/HRRx9hxIgRcpdmVpKSkgCg3CMb3d3djfNIkp+fj+nTp2P48OF15pmndzJ37lyoVCpMnjxZ7lIsDkOdBZo4cSKioqKwZ88euUuRXUJCAqZMmYLw8HBYW1vLXY5ZMRgMaNu2LT7++GMAQKtWrRAVFYXFixfX6VD3559/Yvny5fjtt98QGhqK48ePY+rUqfDy8qrTx4XuT1FREYYOHQpRFLFo0SK5y5HdkSNH8OWXX+Lo0aMQBEHuciwOL79amEmTJuHff//F9u3b4e3tLXc5sjty5AhSUlLQunVrqFQqqFQq7Ny5EwsXLoRKpYJer5e7RNl4enoiJCTEpC04OBjx8fEyVWQepk2bhjfffBNPP/00mjVrhpEjR+KVV17BnDlz5C7NrHh4eAAAkpOTTdqTk5ON8+q6kkAXFxeH8PBwnqUDsHv3bqSkpMDX19f4OzkuLg6vvfYa/P395S6v1uOZOgshiiJefvllrF69Gjt27EBAQIDcJZmFnj174uTJkyZtzz77LIKCgjB9+nQolUqZKpNfWFhYuWFvzpw5Az8/P5kqMg+5ublQKEz/vatUKmEwGGSqyDwFBATAw8MDW7duRcuWLQEAmZmZOHDgACZMmCBvcWagJNCdPXsW27dvh4uLi9wlmYWRI0eWu7e5T58+GDlyJJ599lmZqrIcDHUWYuLEifjtt9+wdu1a2NnZGe9pcXBwgFarlbk6+djZ2ZW7r9DGxgYuLi51/n7DV155BZ06dcLHH3+MoUOH4uDBg/j+++/x/fffy12arAYMGICPPvoIvr6+CA0NxbFjx/D5559j7NixcpdW47Kzs3Hu3Dnj+4sXL+L48eNwdnaGr68vpk6dig8//BCBgYEICAjAjBkz4OXlhYEDB8pXdA2507Hx9PTEkCFDcPToUfz777/Q6/XG38nOzs5Qq9VylV0j7va9uTXgWllZwcPDA02aNKnpUi2P3N1vqWoAqHBaunSp3KWZHQ5pUuqff/4RmzZtKmo0GjEoKEj8/vvv5S5JdpmZmeKUKVNEX19f0draWmzQoIH4zjvviAUFBXKXVuO2b99e4e+V0aNHi6IoDWsyY8YM0d3dXdRoNGLPnj3F2NhYeYuuIXc6NhcvXrzt7+Tt27fLXXq1u9v35lYc0qTqCKJYB4dJJyIiIrIw7ChBREREZAEY6oiIiIgsAEMdERERkQVgqCMiIiKyAAx1RERERBaAoY6IiIjIAjDUEREREVkAhjoiIiIiC8BQR0T0gARBwJo1a+Qug4jqOIY6IqrVxowZA0EQyk19+/aVuzQiohqlkrsAIqIH1bdvXyxdutSkTaPRyFQNEZE8eKaOiGo9jUYDDw8Pk8nJyQmAdGl00aJF6NevH7RaLRo0aIC//vrLZP2TJ0+iR48e0Gq1cHFxwQsvvIDs7GyTZX788UeEhoZCo9HA09MTkyZNMpmfmpqKQYMGQafTITAwEOvWrTPOu3HjBkaMGAFXV1dotVoEBgaWC6FERA+KoY6ILN6MGTPw5JNPIjIyEiNGjMDTTz+NmJgYAEBOTg769OkDJycnHDp0CCtXrsSWLVtMQtuiRYswceJEvPDCCzh58iTWrVuHRo0amexj9uzZGDp0KE6cOIFHH30UI0aMQFpamnH/0dHR2LBhA2JiYrBo0SLUq1ev5g4AEdUNIhFRLTZ69GhRqVSKNjY2JtNHH30kiqIoAhDHjx9vsk6HDh3ECRMmiKIoit9//73o5OQkZmdnG+f/999/okKhEJOSkkRRFEUvLy/xnXfeuW0NAMR3333X+D47O1sEIG7YsEEURVEcMGCA+Oyzz1bNByYiug3eU0dEtV737t2xaNEikzZnZ2fj644dO5rM69ixI44fPw4AiImJQYsWLWBjY2OcHxYWBoPBgNjYWAiCgKtXr6Jnz553rKF58+bG1zY2NrC3t0dKSgoAYMKECXjyySdx9OhR9O7dGwMHDkSnTp3u67MSEd0OQx0R1Xo2NjblLodWFa1WW6nlrKysTN4LggCDwQAA6NevH+Li4rB+/XqEh4ejZ8+emDhxIubPn1/l9RJR3cV76ojI4u3fv7/c++DgYABAcHAwIiMjkZOTY5wfEREBhUKBJk2awM7ODv7+/ti6desD1eDq6orRo0fj119/xYIFC/D9998/0PaIiG7FM3VEVOsVFBQgKSnJpE2lUhk7I6xcuRJt27ZF586dsXz5chw8eBA//PADAGDEiBF47733MHr0aMyaNQvXrl3Dyy+/jJEjR8Ld3R0AMGvWLIwfPx5ubm7o168fsrKyEBERgZdffrlS9c2cORNt2rRBaGgoCgoK8O+//xpDJRFRVWGoI6Jab+PGjfD09DRpa9KkCU6fPg1A6pm6YsUKvPTSS/D09MTvv/+OkJAQAIBOp8OmTZswZcoUtGvXDjqdDk8++SQ+//xz47ZGjx6N/Px8fPHFF3j99ddRr149DBkypNL1qdVqvPXWW7h06RK0Wi0efvhhrFixogo+ORFRKUEURVHuIoiIqosgCFi9ejUGDhwodylERNWK99QRERERWQCGOiIiIiILwHvqiMii8Q4TIqoreKaOiIiIyAIw1BERERFZAIY6IiIiIgvAUEdERERkARjqiIiIiCwAQx0RERGRBWCoIyIiIrIADHVEREREFoChjoiIiMgC/D8YPsYDaCYvWQAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from previous_chapters import plot_losses\n",
    "\n",
    "epochs_tensor = torch.linspace(1, n_epochs, len(train_losses))\n",
    "plot_losses(epochs_tensor, tokens_seen, train_losses, val_losses)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}