LLMs-from-scratch/appendix-D/01_main-chapter-code/appendix-D.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "af53bcb1-ff9d-49c7-a0bc-5b8d32ff975b",
   "metadata": {},
   "source": [
    "## Appendix D: Adding Bells and Whistles to the Training Loop"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4f58c142-9434-49af-b33a-356b80a45b86",
   "metadata": {},
   "source": [
    "- In this appendix, we add a few more advanced features to the training function, which are used in typical pretraining and finetuning; finetuning is covered in chapters 6 and 7\n",
    "- The next three sections below discuss learning rate warmup, cosine decay, and gradient clipping\n",
    "- The final section adds these techniques to the training function"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "744def4f-c03f-42ee-97bb-5d7d5b89b723",
   "metadata": {},
   "source": [
    "- We start by initializing a model reusing the code from chapter 5:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "8755bd5e-bc06-4e6e-9e63-c7c82b816cbe",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch version: 2.2.1\n",
      "tiktoken version: 0.5.1\n"
     ]
    }
   ],
   "source": [
    "from importlib.metadata import version\n",
    "import torch\n",
    "import tiktoken\n",
    "\n",
    "print(\"torch version:\", version(\"torch\"))\n",
    "print(\"tiktoken version:\", version(\"tiktoken\"))\n",
    "\n",
    "\n",
    "from previous_chapters import GPTModel\n",
    "\n",
    "GPT_CONFIG_124M = {\n",
    "    \"vocab_size\": 50257,  # Vocabulary size\n",
    "    \"ctx_len\": 256,       # Shortened context length (orig: 1024)\n",
    "    \"emb_dim\": 768,       # Embedding dimension\n",
    "    \"n_heads\": 12,        # Number of attention heads\n",
    "    \"n_layers\": 12,       # Number of layers\n",
    "    \"drop_rate\": 0.1,     # Dropout rate\n",
    "    \"qkv_bias\": False     # Query-key-value bias\n",
    "}\n",
    "\n",
    "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
    "\n",
    "torch.manual_seed(123)\n",
    "model = GPTModel(GPT_CONFIG_124M)\n",
    "model.eval();  # Disable dropout during inference"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "51574e57-a098-412c-83e8-66dafa5a0b99",
   "metadata": {},
   "source": [
    "- Next, using the same code we used in chapter 5, we initialize the data loaders:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "386ca110-2bb4-42f1-bd54-8836df80acaa",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import urllib.request\n",
    "\n",
    "file_path = \"the-verdict.txt\"\n",
    "url = \"https://raw.githubusercontent.com/rasbt/LLMs-from-scratch/main/ch02/01_main-chapter-code/the-verdict.txt\"\n",
    "\n",
    "if not os.path.exists(file_path):\n",
    "    with urllib.request.urlopen(url) as response:\n",
    "        text_data = response.read().decode('utf-8')\n",
    "    with open(file_path, \"w\", encoding=\"utf-8\") as file:\n",
    "        file.write(text_data)\n",
    "else:\n",
    "    with open(file_path, \"r\", encoding=\"utf-8\") as file:\n",
    "        text_data = file.read()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "ae96992b-536a-4684-a924-658b9ffb7e9c",
   "metadata": {},
   "outputs": [],
   "source": [
    "from previous_chapters import create_dataloader_v1\n",
    "\n",
    "# Train/validation ratio\n",
    "train_ratio = 0.90\n",
    "split_idx = int(train_ratio * len(text_data))\n",
    "\n",
    "\n",
    "torch.manual_seed(123)\n",
    "\n",
    "train_loader = create_dataloader_v1(\n",
    "    text_data[:split_idx],\n",
    "    batch_size=2,\n",
    "    max_length=GPT_CONFIG_124M[\"ctx_len\"],\n",
    "    stride=GPT_CONFIG_124M[\"ctx_len\"],\n",
    "    drop_last=True,\n",
    "    shuffle=True\n",
    ")\n",
    "\n",
    "val_loader = create_dataloader_v1(\n",
    "    text_data[split_idx:],\n",
    "    batch_size=2,\n",
    "    max_length=GPT_CONFIG_124M[\"ctx_len\"],\n",
    "    stride=GPT_CONFIG_124M[\"ctx_len\"],\n",
    "    drop_last=False,\n",
    "    shuffle=False\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "939c08d8-257a-41c6-b842-019f7897ac74",
   "metadata": {},
   "source": [
    "## D.1 Learning rate warmup"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7fafcd30-ddf7-4a9f-bcf4-b13c052b3133",
   "metadata": {},
   "source": [
    "- When training complex models like LLMs, implementing learning rate warmup can help stabilize the training\n",
    "- In learning rate warmup, we gradually increase the learning rate from a very low value (`initial_lr`) to a user-specified maximum (`peak_lr`)\n",
    "- This way, the model will start the training with small weight updates, which helps decrease the risk of large destabilizing updates during the training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "2bb4790b-b8b6-4e9e-adf4-704a04b31ddf",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "135\n"
     ]
    }
   ],
   "source": [
    "n_epochs = 15\n",
    "peak_lr = 0.01\n",
    "initial_lr = 0.0001\n",
    "\n",
    "optimizer = torch.optim.AdamW(model.parameters(), lr=peak_lr, weight_decay=0.1)\n",
    "total_training_steps = len(train_loader) * n_epochs\n",
    "\n",
    "print(total_training_steps)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5bf3a8da-abc4-4b80-a5d8-f1cc1c7cc5f3",
   "metadata": {},
   "source": [
    "- Typically, the number of warmup steps is between 10% and 20% of the total number of steps\n",
    "- We can compute the increment as the difference between the `peak_lr` and `initial_lr` divided by the number of warmup steps"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "e075f80e-a398-4809-be1d-8019e1d31c90",
   "metadata": {},
   "outputs": [],
   "source": [
    "warmup_steps = 20\n",
    "lr_increment = (peak_lr - initial_lr) / warmup_steps\n",
    "\n",
    "global_step = -1\n",
    "track_lrs = []\n",
    "\n",
    "for epoch in range(n_epochs):\n",
    "    for input_batch, target_batch in train_loader:\n",
    "        optimizer.zero_grad()\n",
    "        global_step += 1\n",
    "    \n",
    "        if global_step < warmup_steps:\n",
    "            lr = initial_lr + global_step * lr_increment\n",
    "        else:\n",
    "            lr = peak_lr\n",
    "        \n",
    "        # Apply the calculated learning rate to the optimizer\n",
    "        for param_group in optimizer.param_groups:\n",
    "            param_group[\"lr\"] = lr\n",
    "        track_lrs.append(optimizer.param_groups[0][\"lr\"])\n",
    "    \n",
    "        # Calculate loss and update weights"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "cb6da121-eeed-4023-bdd8-3666c594b4ed",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlMAAAGwCAYAAACNeeBZAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAA9hAAAPYQGoP6dpAAA9s0lEQVR4nO3dfVyV9eH/8fdB5cYb8G6BCCoVm5UoCYqY+7omRWUryjV0ftPM6dqsNCynplCtxrIsZ1nkfn2zfZfTuZWV32I5rKxJqIgKmTctCxUP3g1QSu7O9fvDnSNnonHkHK5z83o+HjyM63zO4XN9VvLedb3P51gMwzAEAACAixJk9gQAAAB8GWEKAACgDQhTAAAAbUCYAgAAaAPCFAAAQBsQpgAAANqAMAUAANAGHc2egK+y2WyqqKhQt27dZLFYzJ4OAABoBcMwdPLkSUVHRysoyD3XlAhTF6miokKxsbFmTwMAAFyEAwcOKCYmxi2vRZi6SN26dZN05n+M8PBwk2cDAABao6amRrGxsY7f4+5AmLpI9lt74eHhhCkAAHyMOys6FNABAADagDAFAADQBoQpAACANiBMAQAAtAFhCgAAoA0IUwAAAG1AmAIAAGgDwhQAAEAbEKYAAADagDAFAADQBqaHqWXLlmnAgAEKDQ1VSkqKNm/efMHxa9as0cCBAxUaGqqEhAS98847To+//vrruv7669WrVy9ZLBZt3779nNc4ffq0ZsyYoV69eqlr164aN26cKisr3XlaAAAgQJgaplavXq2srCzl5ORo27ZtGjJkiNLT03XkyJEWx2/atEkTJkzQ1KlTVVJSooyMDGVkZKisrMwxpra2VqNGjdKTTz553p/7wAMP6O2339aaNWv04YcfqqKiQrfffrvbzw8AAPg/i2EYhlk/PCUlRcOGDdPzzz8vSbLZbIqNjdV9992nuXPnnjM+MzNTtbW1WrdunePYiBEjlJiYqLy8PKexX375peLi4lRSUqLExETH8erqan3nO9/RypUr9eMf/1iStHv3bl1xxRUqLCzUiBEjWjX3mpoaRUREqLq6mg869qDqrxt0sq7B7GkAAEzQu2uIQjt1cOtreuL3d0e3vMpFqK+vV3FxsebNm+c4FhQUpLS0NBUWFrb4nMLCQmVlZTkdS09P19q1a1v9c4uLi9XQ0KC0tDTHsYEDB6pfv34XDFN1dXWqq6tzfF9TU9Pqn4mLs+mfx/Tf/69INtPiPgDATH+4e7j+67vfMXsa38q0MHXs2DE1NTUpMjLS6XhkZKR2797d4nOsVmuL461Wa6t/rtVqVXBwsLp37+7S6+Tm5urRRx9t9c9B2/191xHZDKlDkEUdgyxmTwcA0M6CLL7xd79pYcrXzJs3z+mqWE1NjWJjY02ckf8rO1QtSVo0brDGJcWYPBsAAFpmWpjq3bu3OnTocM676CorKxUVFdXic6Kiolwaf77XqK+vV1VVldPVqW97nZCQEIWEhLT656BtmmyGPq04E6YSYiJMng0AAOdn2rv5goODlZSUpIKCAscxm82mgoICpaamtvic1NRUp/GStH79+vOOb0lSUpI6derk9Dp79uxReXm5S68Dz9p/7JRq65sU1qmDLvtOV7OnAwDAeZl6my8rK0uTJ09WcnKyhg8friVLlqi2tlZTpkyRJE2aNEl9+/ZVbm6uJGnmzJkaPXq0Fi9erLFjx2rVqlXaunWrli9f7njNEydOqLy8XBUVFZLOBCXpzBWpqKgoRUREaOrUqcrKylLPnj0VHh6u++67T6mpqa1+Jx88r/Tft/iujA5XB/pSAAAvZmqYyszM1NGjR5WdnS2r1arExETl5+c7Subl5eUKCjp78WzkyJFauXKlFixYoPnz5ys+Pl5r167VoEGDHGPeeustRxiTpPHjx0uScnJy9Mgjj0iSnn32WQUFBWncuHGqq6tTenq6XnjhhXY4Y7RW6cEz75ZM6MstPgCAdzN1nylfxj5TnnVH3iZt+fJfWnzHEMrnAAC38cTvb9M/Tgb4T2fK5/++MkX5HADg5QhT8Dr7j53S15TPAQA+gjAFr2Mvn19F+RwA4AMIU/A6Ow+eCVODKJ8DAHwAYQpex77zOe/kAwD4AsIUvErz8vlgyucAAB9AmIJX+eLomfJ55+AOupTyOQDABxCm4FUcO5/3oXwOAPANhCl4FXuYonwOAPAVhCl4FXv5nL4UAMBXEKbgNZpshsoO8Zl8AADfQpiC1/ji6Cl900D5HADgWwhT8BrsfA4A8EWEKXgNyucAAF9EmILXKD3IzucAAN9DmIJXaL7zOWEKAOBLCFPwCpTPAQC+ijAFr7DzIOVzAIBvIkzBK1A+BwD4KsIUvIJ953P6UgAAX0OYgumal8/5GBkAgK8hTMF0/2xWPo/rTfkcAOBbCFMwXSnlcwCADyNMwXSljr5Ud3MnAgDARSBMwXSO8nlMuMkzAQDAdYQpmIqdzwEAvo4wBVNRPgcA+DrCFExlL58Pio6gfA4A8EmEKZiKnc8BAL6OMAVTlVI+BwD4OMIUTNNkM7SL8jkAwMcRpmAae/m8C+VzAIAPI0zBNDsdO59TPgcA+C7CFExTRvkcAOAHCFMwjb18PjiGMAUA8F2EKZiiscmmTyu4MgUA8H2EKZjin0drdbrBpi7BHXRp7y5mTwcAgItGmIIp7Lf4roqOUBDlcwCADyNMwRRljs06ucUHAPBthCmYYufBKkls1gkA8H2EKbS7xiabdh0+s/M55XMAgK8jTKHdUT4HAPgTwhTanaN83pfyOQDA9xGm0O5K6UsBAPwIYQrtzn5lijAFAPAHhCm0q+blc7ZFAAD4A8IU2tXnR0/pdINNXUM6Kq4X5XMAgO8jTKFdlR48c4vvyuhwyucAAL9AmEK7KqMvBQDwM4QptCt7+XwwfSkAgJ8gTKHdsPM5AMAfEabQbiifAwD8EWEK7cZePr+K8jkAwI8QptBuKJ8DAPwRYQrtZqc9TFE+BwD4EcIU2kVjk02fUT4HAPghwhTaBeVzAIC/Mj1MLVu2TAMGDFBoaKhSUlK0efPmC45fs2aNBg4cqNDQUCUkJOidd95xetwwDGVnZ6tPnz4KCwtTWlqa9u3b5zRm7969uvXWW9W7d2+Fh4dr1KhRev/9991+bjhrJ+VzAICfMjVMrV69WllZWcrJydG2bds0ZMgQpaen68iRIy2O37RpkyZMmKCpU6eqpKREGRkZysjIUFlZmWPMokWLtHTpUuXl5amoqEhdunRRenq6Tp8+7Rhz8803q7GxURs2bFBxcbGGDBmim2++WVar1ePnHKgonwMA/JXFMAzDrB+ekpKiYcOG6fnnn5ck2Ww2xcbG6r777tPcuXPPGZ+Zmana2lqtW7fOcWzEiBFKTExUXl6eDMNQdHS0Zs+erQcffFCSVF1drcjISK1YsULjx4/XsWPH9J3vfEcbN27U97//fUnSyZMnFR4ervXr1ystLa1Vc6+pqVFERISqq6sVHh7e1qXwe7e98A+VlFfpd+MTdWtiX7OnAwAIUJ74/W3alan6+noVFxc7hZegoCClpaWpsLCwxecUFhaeE3bS09Md4/fv3y+r1eo0JiIiQikpKY4xvXr10ve+9z394Q9/UG1trRobG/XSSy/pkksuUVJS0nnnW1dXp5qaGqcvtE5jk027Ks6sF1emAAD+xrQwdezYMTU1NSkyMtLpeGRk5Hlvt1mt1guOt/95oTEWi0V///vfVVJSom7duik0NFTPPPOM8vPz1aNHj/PONzc3VxEREY6v2NhY1044gO07ckp1jWfK5wMonwMA/IzpBfT2ZhiGZsyYoUsuuUQfffSRNm/erIyMDP3oRz/S4cOHz/u8efPmqbq62vF14MCBdpy1b7N/uDHlcwCAPzItTPXu3VsdOnRQZWWl0/HKykpFRUW1+JyoqKgLjrf/eaExGzZs0Lp167Rq1Spdc801Gjp0qF544QWFhYXp1VdfPe98Q0JCFB4e7vSF1rGXzwezWScAwA+ZFqaCg4OVlJSkgoICxzGbzaaCggKlpqa2+JzU1FSn8ZK0fv16x/i4uDhFRUU5jampqVFRUZFjzNdffy3pTD+ruaCgINlstrafGM5h3xaBzToBAP6oo5k/PCsrS5MnT1ZycrKGDx+uJUuWqLa2VlOmTJEkTZo0SX379lVubq4kaebMmRo9erQWL16ssWPHatWqVdq6dauWL18u6UwfatasWXr88ccVHx+vuLg4LVy4UNHR0crIyJB0JpD16NFDkydPVnZ2tsLCwvT73/9e+/fv19ixY01ZB3/WfOdzyucAAH9kapjKzMzU0aNHlZ2dLavVqsTEROXn5zsK5OXl5U5XkEaOHKmVK1dqwYIFmj9/vuLj47V27VoNGjTIMWbOnDmqra3V9OnTVVVVpVGjRik/P1+hoaGSztxezM/P18MPP6wf/vCHamho0FVXXaU333xTQ4YMad8FCACUzwEA/s7UfaZ8GftMtc6ftx7QnL/s1IhLe2rV9JZv3wIA0F78ap8pBIbSg+x8DgDwb4QpeJR9WwTK5wAAf0WYgsc0L58Pjulu7mQAAPAQwhQ8xl4+7xbSUf17djZ7OgAAeARhCh5j70td1ZedzwEA/oswBY+x96UonwMA/BlhCh7jCFP0pQAAfowwBY9oaLJpFzufAwACAGEKHrGv8pTqKZ8DAAIAYQoeUXaI8jkAIDAQpuAR9r4U+0sBAPwdYQoesZOdzwEAAYIwBbdraLbzOeVzAIC/I0zB7Rzl81DK5wAA/0eYgtvZy+eDoiMonwMA/B5hCm6381CVJCkhhlt8AAD/R5iC25UeOtOXonwOAAgEhCm4VfPy+WDCFAAgABCm4FZ7K0+eLZ/3onwOAPB/hCm4VfPyucVC+RwA4P8IU3Ar+87nlM8BAIGCMAW3spfP2awTABAoCFNwG3Y+BwAEIsIU3IbyOQAgEBGm4Db28nlCX8rnAIDAQZiC2+w8eDZMAQAQKAhTcBvHtgiEKQBAACFMwS0ammz6zHpSElemAACBhTAFt6B8DgAIVIQpuEXpQcrnAIDARJiCW5QeonwOAAhMhCm4RRkfIwMACFCEKbRZfSPlcwBA4CJMoc3s5fPw0I7q15PyOQAgsBCm0GbN95eifA4ACDSEKbRZKX0pAEAAI0yhzXgnHwAgkBGm0Cb1jTbtPkz5HAAQuAhTaJO9lSdV30T5HAAQuAhTaJPm+0tRPgcABCLCFNpkZ7N38gEAEIgIU2iTMsrnAIAAR5jCRWtePh/ct7u5kwEAwCSEKVw0e/k8IqyTYnuGmT0dAABMQZjCRSt19KXCKZ8DAAIWYQoXrZTyOQAAhClcPHv5nL4UACCQEaZwUdj5HACAMy4qTDU2Nurvf/+7XnrpJZ08eeYXakVFhU6dOuXWycF7UT4HAOCMjq4+4auvvtINN9yg8vJy1dXV6brrrlO3bt305JNPqq6uTnl5eZ6YJ7wM5XMAAM5w+crUzJkzlZycrH/9618KCzt7ReK2225TQUGBWycH71Xq2Kyzu7kTAQDAZC5fmfroo4+0adMmBQcHOx0fMGCADh065LaJwbuVHmTncwAApIu4MmWz2dTU1HTO8YMHD6pbt25umRS8W32jTXuslM8BAJAuIkxdf/31WrJkieN7i8WiU6dOKScnRzfddJM75wYvRfkcAICzXL7Nt3jxYqWnp+vKK6/U6dOn9dOf/lT79u1T79699ac//ckTc4SX2dnsFh/lcwBAoHM5TMXExGjHjh1avXq1duzYoVOnTmnq1KmaOHGiUyEd/oudzwEAOMvlMLVx40aNHDlSEydO1MSJEx3HGxsbtXHjRv3Xf/2XWycI71N2iPI5AAB2Lnemrr32Wp04ceKc49XV1br22mtdnsCyZcs0YMAAhYaGKiUlRZs3b77g+DVr1mjgwIEKDQ1VQkKC3nnnHafHDcNQdna2+vTpo7CwMKWlpWnfvn3nvM7//d//KSUlRWFhYerRo4cyMjJcnnsgqmts0m5rjSRpcAxhCgAAl8OUYRgt9mSOHz+uLl26uPRaq1evVlZWlnJycrRt2zYNGTJE6enpOnLkSIvjN23apAkTJmjq1KkqKSlRRkaGMjIyVFZW5hizaNEiLV26VHl5eSoqKlKXLl2Unp6u06dPO8b89a9/1Z133qkpU6Zox44d+sc//qGf/vSnLs09UO21nlJDk6GIsE6K6cFtXQAALIZhGK0ZePvtt0uS3nzzTd1www0KCQlxPNbU1KSdO3fqe9/7nvLz81v9w1NSUjRs2DA9//zzks5suxAbG6v77rtPc+fOPWd8ZmamamtrtW7dOsexESNGKDExUXl5eTIMQ9HR0Zo9e7YefPBBSWeumEVGRmrFihUaP368GhsbNWDAAD366KOaOnVqq+f6n2pqahQREaHq6mqFh4df9Ov4mpVF5Zr/RqlGXd5bf/xZitnTAQDAJZ74/d3qK1MRERGKiIiQYRjq1q2b4/uIiAhFRUVp+vTp+uMf/9jqH1xfX6/i4mKlpaWdnUxQkNLS0lRYWNjicwoLC53GS1J6erpj/P79+2W1Wp3GREREKCUlxTFm27ZtOnTokIKCgnT11VerT58+uvHGG52ubrWkrq5ONTU1Tl+ByLHzObf4AACQ5EIB/ZVXXpF0ZqfzBx980OVbev/p2LFjampqUmRkpNPxyMhI7d69u8XnWK3WFsdbrVbH4/Zj5xvzxRdfSJIeeeQRPfPMMxowYIAWL16sH/zgB9q7d6969uzZ4s/Ozc3Vo48+6uJZ+h/K5wAAOHO5M5WTk9PmIGUmm80mSXr44Yc1btw4JSUl6ZVXXpHFYtGaNWvO+7x58+apurra8XXgwIH2mrLXaF4+J0wBAHCGy1sjSNJf/vIX/fnPf1Z5ebnq6+udHtu2bVurXqN3797q0KGDKisrnY5XVlYqKiqqxedERUVdcLz9z8rKSvXp08dpTGJioiQ5jl955ZWOx0NCQnTppZeqvLz8vPMNCQlx6okFIsrnAACcy+UrU0uXLtWUKVMUGRmpkpISDR8+XL169dIXX3yhG2+8sdWvExwcrKSkJBUUFDiO2Ww2FRQUKDU1tcXnpKamOo2XpPXr1zvGx8XFKSoqymlMTU2NioqKHGOSkpIUEhKiPXv2OMY0NDToyy+/VP/+/Vs9/0Bk70sNjmHncwAA7Fy+MvXCCy9o+fLlmjBhglasWKE5c+bo0ksvVXZ2dov7T11IVlaWJk+erOTkZA0fPlxLlixRbW2tpkyZIkmaNGmS+vbtq9zcXEnSzJkzNXr0aC1evFhjx47VqlWrtHXrVi1fvlzSmc8JnDVrlh5//HHFx8crLi5OCxcuVHR0tGMfqfDwcN1zzz3KyclRbGys+vfvr6eeekqSdMcdd7i6HAGl9FCVJHY+BwCgOZfDVHl5uUaOHClJCgsL08mTJyVJd955p0aMGOHY5qA1MjMzdfToUWVnZ8tqtSoxMVH5+fmOAnl5ebmCgs5ePBs5cqRWrlypBQsWaP78+YqPj9fatWs1aNAgx5g5c+aotrZW06dPV1VVlUaNGqX8/HyFhoY6xjz11FPq2LGj7rzzTn3zzTdKSUnRhg0b1KNHD1eXI6CUUj4HAOAcrd5nyu7SSy/VX//6V1199dVKTk7WtGnT9POf/1zvvfeexo8f7/LVKV8VaPtM1TU2aVDO39TQZOijOdcqtmdns6cEAIDLTN1nyu6HP/yh3nrrLUnSlClT9MADD+i6665TZmambrvtNrdMCt7HXj7v3pnyOQAAzbl8m2/58uWO7QVmzJihXr16adOmTbrlllv085//3O0ThHfY+e++VEJfyucAADTnUphqbGzUb37zG919992KiYmRJI0fP17jx4/3yOTgPeybdVI+BwDAmUu3+Tp27KhFixapsbHRU/OBl3Jsi0CYAgDAicudqTFjxujDDz/0xFzgpeoam7THeuZdm1yZAgDAmcudqRtvvFFz585VaWmpkpKSzvlomVtuucVtk4N32GM9SfkcAIDzcDlM/fKXv5QkPfPMM+c8ZrFY1NTU1PZZwas031+K8jkAAM5cDlP2d/IhcJSxWScAAOflcmcKgWfnQcIUAADnQ5jCBdU1NmlvJeVzAADOhzCFC6J8DgDAhRGmcEGUzwEAuDDCFC6olL4UAAAX5PK7+Wpqalo8brFYFBISouDg4DZPCt6jlHfyAQBwQS6Hqe7du1/wdk9MTIzuuusu5eTkKCiIC1++7HTD2fJ5QgxhCgCAlrgcplasWKGHH35Yd911l4YPHy5J2rx5s1599VUtWLBAR48e1dNPP62QkBDNnz/f7RNG+7GXz3t07qS+3SmfAwDQEpfD1KuvvqrFixfrJz/5iePYj370IyUkJOill15SQUGB+vXrpyeeeIIw5ePst/gGUT4HAOC8XL4Pt2nTJl199dXnHL/66qtVWFgoSRo1apTKy8vbPjuYip3PAQD4di6HqdjYWL388svnHH/55ZcVGxsrSTp+/Lh69OjR9tnBVPYrU4PpSwEAcF4u3+Z7+umndccdd+jdd9/VsGHDJElbt27V7t279Ze//EWStGXLFmVmZrp3pmhXpxuatMfKzucAAHwbl8PULbfcot27d+ull17S3r17JUk33nij1q5dqwEDBkiSfvGLX7h1kmh/e6wn1WijfA4AwLdxOUxJUlxcnH7729+6ey7wIpTPAQBonYsKU1VVVdq8ebOOHDkim83m9NikSZPcMjGYq4y+FAAAreJymHr77bc1ceJEnTp1SuHh4U5XLSwWC2HKT+zkY2QAAGgVl9/NN3v2bN199906deqUqqqq9K9//cvxdeLECU/MEe2s+c7nlM8BALgwl8PUoUOHdP/996tz586emA+8gL183rNLMOVzAAC+hcthKj09XVu3bvXEXOAldlI+BwCg1VzuTI0dO1YPPfSQdu3apYSEBHXq1Mnp8VtuucVtk4M5yhx9qXCTZwIAgPdzOUxNmzZNkvTYY4+d85jFYlFTU1PbZwVTlfIxMgAAtJrLYeo/t0KAf2lePk+I6W7uZAAA8AEud6bg33Y3K59HR4SaPR0AALxeq65MLV26VNOnT1doaKiWLl16wbH333+/WyYGc7DzOQAArmlVmHr22Wc1ceJEhYaG6tlnnz3vOIvFQpjycfby+WD6UgAAtEqrwtT+/ftb/Gf4n+ZXpgAAwLejMwUH5/I5YQoAgNZw+d18TU1NWrFihQoKClr8oOMNGza4bXJoX5TPAQBwncthaubMmVqxYoXGjh2rQYMGUVL2I833l+J/VwAAWsflMLVq1Sr9+c9/1k033eSJ+cBEpQerJLFZJwAArnC5MxUcHKzLL7/cE3OByUoP1UiifA4AgCtcDlOzZ8/W7373OxmG4Yn5wCSnG5q0j/I5AAAuc/k238cff6z3339f7777rq666qpzPuj49ddfd9vk0H7s5fNelM8BAHCJy2Gqe/fuuu222zwxF5jI3pdi53MAAFzjUphqbGzUtddeq+uvv15RUVGemhNM0PydfAAAoPVc6kx17NhR99xzj+rq6jw1H5jEXj6nLwUAgGtcLqAPHz5cJSUlnpgLTOK08zlXpgAAcInLnalf/vKXmj17tg4ePKikpCR16dLF6fHBgwe7bXJoH58drlHTv8vnfSifAwDgEpfD1Pjx4yVJ999/v+OYxWKRYRiyWCxqampy3+zQLsqafbgx5XMAAFzjcpjav3+/J+YBE9nL54PpSwEA4DKXw1T//v09MQ+YaOfBs1emAACAa1wOU3a7du1SeXm56uvrnY7fcsstbZ4U2s/phibtO3JKEuVzAAAuhsth6osvvtBtt92m0tJSR1dKkqNrQ2fKt1A+BwCgbVzeGmHmzJmKi4vTkSNH1LlzZ3366afauHGjkpOT9cEHH3hgivAke/k8IYbyOQAAF8PlK1OFhYXasGGDevfuraCgIAUFBWnUqFHKzc3V/fffzx5UPsbel+IWHwAAF8flK1NNTU3q1q2bJKl3796qqKiQdKaYvmfPHvfODh5XeojyOQAAbeHylalBgwZpx44diouLU0pKihYtWqTg4GAtX75cl156qSfmCA9pXj5nWwQAAC6Oy2FqwYIFqq2tlSQ99thjuvnmm/X9739fvXr10urVq90+QXjOrn+Xz3t3DVZUOOVzAAAuhsthKj093fHPl19+uXbv3q0TJ06oR48eFJh9DDufAwDQdi53puw+//xz/e1vf9M333yjnj17tmkSy5Yt04ABAxQaGqqUlBRt3rz5guPXrFmjgQMHKjQ0VAkJCXrnnXecHjcMQ9nZ2erTp4/CwsKUlpamffv2tfhadXV1SkxMlMVi0fbt29t0Hr6mlPI5AABt5nKYOn78uMaMGaPvfve7uummm3T48GFJ0tSpUzV79myXJ7B69WplZWUpJydH27Zt05AhQ5Senq4jR460OH7Tpk2aMGGCpk6dqpKSEmVkZCgjI0NlZWWOMYsWLdLSpUuVl5enoqIidenSRenp6Tp9+vQ5rzdnzhxFR0e7PG9/YC+fE6YAALh4LoepBx54QJ06dVJ5ebk6d+7sOJ6Zman8/HyXJ/DMM89o2rRpmjJliq688krl5eWpc+fO+p//+Z8Wx//ud7/TDTfcoIceekhXXHGFfv3rX2vo0KF6/vnnJZ25KrVkyRItWLBAt956qwYPHqw//OEPqqio0Nq1a51e691339V7772np59+2uV5+zqnnc8pnwMAcNFcDlPvvfeennzyScXExDgdj4+P11dffeXSa9XX16u4uFhpaWlnJxQUpLS0NBUWFrb4nMLCQqfx0pkel338/v37ZbVancZEREQoJSXF6TUrKys1bdo0/e///q9TKDyfuro61dTUOH35MsrnAAC4h8thqra2tsXwceLECYWEhLj0WseOHVNTU5MiIyOdjkdGRspqtbb4HKvVesHx9j8vNMYwDN1111265557lJyc3Kq55ubmKiIiwvEVGxvbqud5q7Jmt/gonwMAcPFcDlPf//739Yc//MHxvcVikc1m06JFi3Tttde6dXKe8txzz+nkyZOaN29eq58zb948VVdXO74OHDjgwRl6HuVzAADcw+WtERYtWqQxY8Zo69atqq+v15w5c/Tpp5/qxIkT+sc//uHSa/Xu3VsdOnRQZWWl0/HKykpFRUW1+JyoqKgLjrf/WVlZqT59+jiNSUxMlCRt2LBBhYWF51xJS05O1sSJE/Xqq6+e83NDQkJcvvLmzdj5HAAA93D5ytSgQYO0d+9ejRo1Srfeeqtqa2t1++23q6SkRJdddplLrxUcHKykpCQVFBQ4jtlsNhUUFCg1NbXF56SmpjqNl6T169c7xsfFxSkqKsppTE1NjYqKihxjli5dqh07dmj79u3avn27Y2uF1atX64knnnDpHHwR5XMAANzH5StT0plC98MPP+x07ODBg5o+fbqWL1/u0mtlZWVp8uTJSk5O1vDhw7VkyRLV1tZqypQpkqRJkyapb9++ys3NlSTNnDlTo0eP1uLFizV27FitWrVKW7dudfxci8WiWbNm6fHHH1d8fLzi4uK0cOFCRUdHKyMjQ5LUr18/pzl07dpVknTZZZedU6z3R2fL5yGUzwEAaKOLClMtOX78uF5++WWXw1RmZqaOHj2q7OxsWa1WJSYmKj8/31EgLy8vV1DQ2QtoI0eO1MqVK7VgwQLNnz9f8fHxWrt2rQYNGuQYM2fOHNXW1mr69OmqqqrSqFGjlJ+fr9BQgoPUvC8VTvkcAIA2shiGYbjjhXbs2KGhQ4eqqanJHS/n9WpqahQREaHq6mqFh4ebPR2XPLhmh/5SfFD3//ByZV3/PbOnAwBAu/HE7++L/jgZ+K4yyucAALgNYSrAfFN/tnw+OKa7uZMBAMAPtLozdfvtt1/w8aqqqrbOBe2gefk8Mtx/tnoAAMAsrQ5TEREXviUUERGhSZMmtXlC8KyzO59TPgcAwB1aHaZeeeUVT84D7cS+WWcCt/gAAHALOlMBho+RAQDAvQhTAeRM+fykJMIUAADuQpgKILsO18hmiPI5AABuRJgKIPby+eCYCMrnAAC4CWEqgOw8yGadAAC4G2EqgJzdFoEwBQCAuxCmAgTlcwAAPIMwFSDs5fPvdKN8DgCAOxGmAkTpwSpJZ65KUT4HAMB9CFMBovRQjSTK5wAAuBthKkA4tkUgTAEA4FaEqQDwdX3j2fJ5DGEKAAB3IkwFgM+cyuehZk8HAAC/QpgKAHy4MQAAnkOYCgD28jlhCgAA9yNMBYDSQ1WSCFMAAHgCYcrPfV3fqM+PnJJE+RwAAE8gTPk5e/n8EsrnAAB4BGHKz1E+BwDAswhTfm7nvzfrZOdzAAA8gzDl5+w7n3NlCgAAzyBM+THK5wAAeB5hyo/tqqB8DgCApxGm/Fgpt/gAAPA4wpQfK6V8DgCAxxGm/Ji9fD6YvhQAAB5DmPJTTuVzrkwBAOAxhCk/1bx8fgnlcwAAPIYw5adKucUHAEC7IEz5KfvHyFA+BwDAswhTfoptEQAAaB+EKT/0dX2j/nmU8jkAAO2BMOWH7OXzyHDK5wAAeBphyg/tPMgtPgAA2gthyg+VsfM5AADthjDlhyifAwDQfghTfqa2jvI5AADtiTDlZ3YdpnwOAEB7Ikz5mVLK5wAAtCvClJ8pc/Slups7EQAAAgRhys/stIepmHCTZwIAQGAgTPmR5uVztkUAAKB9EKb8yK7DNTLs5fNulM8BAGgPhCk/crZ83t3ciQAAEEAIU36EzToBAGh/hCk/Ukr5HACAdkeY8hOUzwEAMAdhyk/Yy+dR4aGUzwEAaEeEKT+x89/lc65KAQDQvghTfqKM8jkAAKYgTPkJe/l8cAxhCgCA9kSY8gOnKJ8DAGAarwhTy5Yt04ABAxQaGqqUlBRt3rz5guPXrFmjgQMHKjQ0VAkJCXrnnXecHjcMQ9nZ2erTp4/CwsKUlpamffv2OR7/8ssvNXXqVMXFxSksLEyXXXaZcnJyVF9f75Hz87RdFWfL59/pFmL2dAAACCimh6nVq1crKytLOTk52rZtm4YMGaL09HQdOXKkxfGbNm3ShAkTNHXqVJWUlCgjI0MZGRkqKytzjFm0aJGWLl2qvLw8FRUVqUuXLkpPT9fp06clSbt375bNZtNLL72kTz/9VM8++6zy8vI0f/78djlnd7Pf4uOqFAAA7c9iGIZh5gRSUlI0bNgwPf/885Ikm82m2NhY3XfffZo7d+454zMzM1VbW6t169Y5jo0YMUKJiYnKy8uTYRiKjo7W7Nmz9eCDD0qSqqurFRkZqRUrVmj8+PEtzuOpp57Siy++qC+++KLFx+vq6lRXV+f4vqamRrGxsaqurlZ4uLmbZD6werveKDmkrOu+q/vHxJs6FwAAvFlNTY0iIiLc+vvb1CtT9fX1Ki4uVlpamuNYUFCQ0tLSVFhY2OJzCgsLncZLUnp6umP8/v37ZbVancZEREQoJSXlvK8pnQlcPXv2PO/jubm5ioiIcHzFxsa26hzbw86DVZJ4Jx8AAGYwNUwdO3ZMTU1NioyMdDoeGRkpq9Xa4nOsVusFx9v/dOU1P//8cz333HP6+c9/ft65zps3T9XV1Y6vAwcOXPjk2smpukZ9caxWErf5AAAwQ0ezJ2C2Q4cO6YYbbtAdd9yhadOmnXdcSEiIQkK8r9xtL5/3iaB8DgCAGUy9MtW7d2916NBBlZWVTscrKysVFRXV4nOioqIuON7+Z2tes6KiQtdee61Gjhyp5cuXt+lczGK/xcdVKQAAzGFqmAoODlZSUpIKCgocx2w2mwoKCpSamtric1JTU53GS9L69esd4+Pi4hQVFeU0pqamRkVFRU6veejQIf3gBz9QUlKSXnnlFQUFmf7GxovCzucAAJjL9Nt8WVlZmjx5spKTkzV8+HAtWbJEtbW1mjJliiRp0qRJ6tu3r3JzcyVJM2fO1OjRo7V48WKNHTtWq1at0tatWx1XliwWi2bNmqXHH39c8fHxiouL08KFCxUdHa2MjAxJZ4NU//799fTTT+vo0aOO+Zzvipi3KiVMAQBgKtPDVGZmpo4ePars7GxZrVYlJiYqPz/fUSAvLy93umo0cuRIrVy5UgsWLND8+fMVHx+vtWvXatCgQY4xc+bMUW1traZPn66qqiqNGjVK+fn5Cg0NlXTmStbnn3+uzz//XDExMU7zMXmnCJdQPgcAwHym7zPlqzyxT4Wrir44rszln6hPRKgK540xZQ4AAPgSv9tnCm3DzucAAJiPMOXDKJ8DAGA+wpQPc5TPYwhTAACYhTDlo5qXz7kyBQCAeQhTPurTQ9WOnc97d2XncwAAzEKY8lHsLwUAgHcgTPkowhQAAN6BMOWjHNsiUD4HAMBUhCkfdKquUfspnwMA4BUIUz7IXj6PpnwOAIDpCFM+iJ3PAQDwHoQpH0T5HAAA70GY8kGUzwEA8B6EKR9z8nSDvjhK+RwAAG9BmPIxn1bUSKJ8DgCAtyBM+ZgyyucAAHgVwpSPsfelBtOXAgDAKxCmfEzpQa5MAQDgTQhTPuTk6QZ9wc7nAAB4FcKUD2lePu9F+RwAAK9AmPIh9vJ5An0pAAC8BmHKh+w8yM7nAAB4G8KUD2FbBAAAvA9hykdQPgcAwDsRpnxE2aEz5fO+3cMonwMA4EUIUz7i7C2+cJNnAgAAmiNM+Qj7zufc4gMAwLsQpnzE2W0Rups7EQAA4IQw5QNqKJ8DAOC1CFM+4NNm5fOeXYJNng0AAGiOMOUDKJ8DAOC9CFM+wF4+H0xfCgAAr0OY8gGl7HwOAIDXIkx5uZrTDdpP+RwAAK9FmPJylM8BAPBuhCkvV3qoShJXpQAA8FaEKS9X+u8rUwkxhCkAALwRYcrLlVE+BwDAqxGmvBjlcwAAvB9hyovZr0pRPgcAwHsRpryY48ONuSoFAIDXIkx5McrnAAB4P8KUFys9WCWJK1MAAHgzwpSXqjndoC+Pfy2JMAUAgDcjTHmp5uXzHpTPAQDwWoQpL2UPU4PpSwEA4NUIU15q50E26wQAwBcQprwU2yIAAOAbCFNeiPI5AAC+gzDlhexXpWJ6UD4HAMDbEaa8UOlBbvEBAOArCFNeqPQQ5XMAAHwFYcoLsS0CAAC+gzDlZaq/OVs+HxRNmAIAwNsRprzMp5TPAQDwKYQpL1PK/lIAAPgUwpSXcYQp+lIAAPgErwhTy5Yt04ABAxQaGqqUlBRt3rz5guPXrFmjgQMHKjQ0VAkJCXrnnXecHjcMQ9nZ2erTp4/CwsKUlpamffv2OY05ceKEJk6cqPDwcHXv3l1Tp07VqVOn3H5urjrd0KQOQRauTAEA4CNMD1OrV69WVlaWcnJytG3bNg0ZMkTp6ek6cuRIi+M3bdqkCRMmaOrUqSopKVFGRoYyMjJUVlbmGLNo0SItXbpUeXl5KioqUpcuXZSenq7Tp087xkycOFGffvqp1q9fr3Xr1mnjxo2aPn26x8/32/y/ycNU9ki6UuJ6mT0VAADQChbDMAwzJ5CSkqJhw4bp+eeflyTZbDbFxsbqvvvu09y5c88Zn5mZqdraWq1bt85xbMSIEUpMTFReXp4Mw1B0dLRmz56tBx98UJJUXV2tyMhIrVixQuPHj9dnn32mK6+8Ulu2bFFycrIkKT8/XzfddJMOHjyo6Ojoc35uXV2d6urqHN/X1NQoNjZW1dXVCg8Pd+uaAAAAz6ipqVFERIRbf3+bemWqvr5excXFSktLcxwLCgpSWlqaCgsLW3xOYWGh03hJSk9Pd4zfv3+/rFar05iIiAilpKQ4xhQWFqp79+6OICVJaWlpCgoKUlFRUYs/Nzc3VxEREY6v2NjYiztpAADgV0wNU8eOHVNTU5MiIyOdjkdGRspqtbb4HKvVesHx9j+/bcwll1zi9HjHjh3Vs2fP8/7cefPmqbq62vF14MCBVp4lAADwZx3NnoCvCAkJUUhIiNnTAAAAXsbUK1O9e/dWhw4dVFlZ6XS8srJSUVFRLT4nKirqguPtf37bmP8suDc2NurEiRPn/bkAAAAtMTVMBQcHKykpSQUFBY5jNptNBQUFSk1NbfE5qampTuMlaf369Y7xcXFxioqKchpTU1OjoqIix5jU1FRVVVWpuLjYMWbDhg2y2WxKSUlx2/kBAAD/Z/ptvqysLE2ePFnJyckaPny4lixZotraWk2ZMkWSNGnSJPXt21e5ubmSpJkzZ2r06NFavHixxo4dq1WrVmnr1q1avny5JMlisWjWrFl6/PHHFR8fr7i4OC1cuFDR0dHKyMiQJF1xxRW64YYbNG3aNOXl5amhoUH33nuvxo8f3+I7+QAAAM7H9DCVmZmpo0ePKjs7W1arVYmJicrPz3cUyMvLyxUUdPYC2siRI7Vy5UotWLBA8+fPV3x8vNauXatBgwY5xsyZM0e1tbWaPn26qqqqNGrUKOXn5ys0NNQx5rXXXtO9996rMWPGKCgoSOPGjdPSpUvb78QBAIBfMH2fKV/liX0qAACAZ/ndPlMAAAC+jjAFAADQBoQpAACANiBMAQAAtAFhCgAAoA1M3xrBV9nfBFlTU2PyTAAAQGvZf2+7czMDwtRFOnnypCQpNjbW5JkAAABXnTx5UhEREW55LfaZukg2m00VFRXq1q2bLBaL2163pqZGsbGxOnDgAPtXifVojrU4i7VwxnqcxVqcxVqc1XwtunXrppMnTyo6OtppU/C24MrURQoKClJMTIzHXj88PDzg/+VvjvU4i7U4i7VwxnqcxVqcxVqcZV8Ld12RsqOADgAA0AaEKQAAgDYgTHmZkJAQ5eTkKCQkxOypeAXW4yzW4izWwhnrcRZrcRZrcZan14ICOgAAQBtwZQoAAKANCFMAAABtQJgCAABoA8IUAABAGxCmvMyyZcs0YMAAhYaGKiUlRZs3bzZ7Sh6Xm5urYcOGqVu3brrkkkuUkZGhPXv2OI05ffq0ZsyYoV69eqlr164aN26cKisrTZpx+/ntb38ri8WiWbNmOY4F0locOnRI//3f/61evXopLCxMCQkJ2rp1q+NxwzCUnZ2tPn36KCwsTGlpadq3b5+JM/acpqYmLVy4UHFxcQoLC9Nll12mX//6106fL+av67Fx40b96Ec/UnR0tCwWi9auXev0eGvO+8SJE5o4caLCw8PVvXt3TZ06VadOnWrHs3CPC61FQ0ODfvWrXykhIUFdunRRdHS0Jk2apIqKCqfX8Je1kL79343m7rnnHlksFi1ZssTpuDvWgzDlRVavXq2srCzl5ORo27ZtGjJkiNLT03XkyBGzp+ZRH374oWbMmKFPPvlE69evV0NDg66//nrV1tY6xjzwwAN6++23tWbNGn344YeqqKjQ7bffbuKsPW/Lli166aWXNHjwYKfjgbIW//rXv3TNNdeoU6dOevfdd7Vr1y4tXrxYPXr0cIxZtGiRli5dqry8PBUVFalLly5KT0/X6dOnTZy5Zzz55JN68cUX9fzzz+uzzz7Tk08+qUWLFum5555zjPHX9aitrdWQIUO0bNmyFh9vzXlPnDhRn376qdavX69169Zp48aNmj59enudgttcaC2+/vprbdu2TQsXLtS2bdv0+uuva8+ePbrlllucxvnLWkjf/u+G3RtvvKFPPvlE0dHR5zzmlvUw4DWGDx9uzJgxw/F9U1OTER0dbeTm5po4q/Z35MgRQ5Lx4YcfGoZhGFVVVUanTp2MNWvWOMZ89tlnhiSjsLDQrGl61MmTJ434+Hhj/fr1xujRo42ZM2cahhFYa/GrX/3KGDVq1Hkft9lsRlRUlPHUU085jlVVVRkhISHGn/70p/aYYrsaO3ascffddzsdu/32242JEycahhE46yHJeOONNxzft+a8d+3aZUgytmzZ4hjz7rvvGhaLxTh06FC7zd3d/nMtWrJ582ZDkvHVV18ZhuG/a2EY51+PgwcPGn379jXKysqM/v37G88++6zjMXetB1emvER9fb2Ki4uVlpbmOBYUFKS0tDQVFhaaOLP2V11dLUnq2bOnJKm4uFgNDQ1OazNw4ED169fPb9dmxowZGjt2rNM5S4G1Fm+99ZaSk5N1xx136JJLLtHVV1+t3//+947H9+/fL6vV6rQWERERSklJ8bu1kKSRI0eqoKBAe/fulSTt2LFDH3/8sW688UZJgbcedq0578LCQnXv3l3JycmOMWlpaQoKClJRUVG7z7k9VVdXy2KxqHv37pICby1sNpvuvPNOPfTQQ7rqqqvOedxd68EHHXuJY8eOqampSZGRkU7HIyMjtXv3bpNm1f5sNptmzZqla665RoMGDZIkWa1WBQcHO/4ysIuMjJTVajVhlp61atUqbdu2TVu2bDnnsUBaiy+++EIvvviisrKyNH/+fG3ZskX333+/goODNXnyZMf5tvTfjL+thSTNnTtXNTU1GjhwoDp06KCmpiY98cQTmjhxoiQF3HrYtea8rVarLrnkEqfHO3bsqJ49e/r12pw+fVq/+tWvNGHCBMcHHQfaWjz55JPq2LGj7r///hYfd9d6EKbgVWbMmKGysjJ9/PHHZk/FFAcOHNDMmTO1fv16hYaGmj0dU9lsNiUnJ+s3v/mNJOnqq69WWVmZ8vLyNHnyZJNn1/7+/Oc/67XXXtPKlSt11VVXafv27Zo1a5aio6MDcj1wYQ0NDfrJT34iwzD04osvmj0dUxQXF+t3v/udtm3bJovF4tGfxW0+L9G7d2916NDhnHdlVVZWKioqyqRZta97771X69at0/vvv6+YmBjH8aioKNXX16uqqsppvD+uTXFxsY4cOaKhQ4eqY8eO6tixoz788EMtXbpUHTt2VGRkZMCsRZ8+fXTllVc6HbviiitUXl4uSY7zDZT/Zh566CHNnTtX48ePV0JCgu6880498MADys3NlRR462HXmvOOioo65408jY2NOnHihF+ujT1IffXVV1q/fr3jqpQUWGvx0Ucf6ciRI+rXr5/j79OvvvpKs2fP1oABAyS5bz0IU14iODhYSUlJKigocByz2WwqKChQamqqiTPzPMMwdO+99+qNN97Qhg0bFBcX5/R4UlKSOnXq5LQ2e/bsUXl5ud+tzZgxY1RaWqrt27c7vpKTkzVx4kTHPwfKWlxzzTXnbJGxd+9e9e/fX5IUFxenqKgop7WoqalRUVGR362FdOadWkFBzn9ld+jQQTabTVLgrYdda847NTVVVVVVKi4udozZsGGDbDabUlJS2n3OnmQPUvv27dPf//539erVy+nxQFqLO++8Uzt37nT6+zQ6OloPPfSQ/va3v0ly43pcfG8e7rZq1SojJCTEWLFihbFr1y5j+vTpRvfu3Q2r1Wr21DzqF7/4hREREWF88MEHxuHDhx1fX3/9tWPMPffcY/Tr18/YsGGDsXXrViM1NdVITU01cdbtp/m7+QwjcNZi8+bNRseOHY0nnnjC2Ldvn/Haa68ZnTt3Nv74xz86xvz2t781unfvbrz55pvGzp07jVtvvdWIi4szvvnmGxNn7hmTJ082+vbta6xbt87Yv3+/8frrrxu9e/c25syZ4xjjr+tx8uRJo6SkxCgpKTEkGc8884xRUlLieIdaa877hhtuMK6++mqjqKjI+Pjjj434+HhjwoQJZp3SRbvQWtTX1xu33HKLERMTY2zfvt3p79O6ujrHa/jLWhjGt/+78Z/+8918huGe9SBMeZnnnnvO6NevnxEcHGwMHz7c+OSTT8yeksdJavHrlVdecYz55ptvjF/+8pdGjx49jM6dOxu33XabcfjwYfMm3Y7+M0wF0lq8/fbbxqBBg4yQkBBj4MCBxvLly50et9lsxsKFC43IyEgjJCTEGDNmjLFnzx6TZutZNTU1xsyZM41+/foZoaGhxqWXXmo8/PDDTr8k/XU93n///Rb/jpg8ebJhGK077+PHjxsTJkwwunbtaoSHhxtTpkwxTp48acLZtM2F1mL//v3n/fv0/fffd7yGv6yFYXz7vxv/qaUw5Y71sBhGs+1zAQAA4BI6UwAAAG1AmAIAAGgDwhQAAEAbEKYAAADagDAFAADQBoQpAACANiBMAQAAtAFhCgAAoA0IUwAAAG1AmALg144ePapf/OIX6tevn0JCQhQVFaX09HT94x//kCRZLBatXbvW3EkC8GkdzZ4AAHjSuHHjVF9fr1dffVWXXnqpKisrVVBQoOPHj5s9NQB+gs/mA+C3qqqq1KNHD33wwQcaPXr0OY8PGDBAX331leP7/v3768svv5Qkvfnmm3r00Ue1a9cuRUdHa/LkyXr44YfVseOZ/w9qsVj0wgsv6K233tIHH3ygPn36aNGiRfrxj3/cLucGwHtwmw+A3+ratau6du2qtWvXqq6u7pzHt2zZIkl65ZVXdPjwYcf3H330kSZNmqSZM2dq165deumll7RixQo98cQTTs9fuHChxo0bpx07dmjixIkaP368PvvsM8+fGACvwpUpAH7tr3/9q6ZNm6ZvvvlGQ4cO1ejRozV+/HgNHjxY0pkrTG+88YYyMjIcz0lLS9OYMWM0b948x7E//vGPmjNnjioqKhzPu+eee/Tiiy86xowYMUJDhw7VCy+80D4nB8ArcGUKgF8bN26cKioq9NZbb+mGG27QBx98oKFDh2rFihXnfc6OHTv02GOPOa5sde3aVdOmTdPhw4f19ddfO8alpqY6PS81NZUrU0AAooAOwO+Fhobquuuu03XXXaeFCxfqZz/7mXJycnTXXXe1OP7UqVN69NFHdfvtt7f4WgDQHFemAAScK6+8UrW1tZKkTp06qampyenxoUOHas+ePbr88svP+QoKOvvX5ieffOL0vE8++URXXHGF508AgFfhyhQAv3X8+HHdcccduvvuuzV48GB169ZNW7du1aJFi3TrrbdKOvOOvoKCAl1zzTUKCQlRjx49lJ2drZtvvln9+vXTj3/8YwUFBWnHjh0qKyvT448/7nj9NWvWKDk5WaNGjdJrr72mzZs36+WXXzbrdAGYhAI6AL9VV1enRx55RO+9957++c9/qqGhQbGxsbrjjjs0f/58hYWF6e2331ZWVpa+/PJL9e3b17E1wt/+9jc99thjKikpUadOnTRw4ED97Gc/07Rp0ySdKaAvW7ZMa9eu1caNG9WnTx89+eST+slPfmLiGQMwA2EKAC5CS+8CBBCY6EwBAAC0AWEKAACgDSigA8BFoCEBwI4rUwAAAG1AmAIAAGgDwhQAAEAbEKYAAADagDAFAADQBoQpAACANiBMAQAAtAFhCgAAoA3+P8MbL070/c55AAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "plt.ylabel(\"Learning rate\")\n",
    "plt.xlabel(\"Step\")\n",
    "plt.plot(range(total_training_steps), track_lrs);"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7b3996b6-3f7a-420a-8584-c5760249f3d8",
   "metadata": {},
   "source": [
    "## D.2 Cosine decay"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c5216214-de79-40cf-a733-b1049a73023c",
   "metadata": {},
   "source": [
    "- Another popular technique for training complex deep neural networks is cosine decay, which also adjusts the learning rate across training epochs\n",
    "- In cosine decay, the learning rate follows a cosine curve, decreasing from its initial value to near zero following a half-cosine cycle\n",
    "- This gradual reduction is designed to slow the pace of learning as the model begins to improve its weights; it reduces the risk of overshooting minima as the training progresses,  which is crucial for stabilizing the training in its later stages\n",
    "- Cosine decay is often preferred over linear decay for its smoother transition in learning rate adjustments, but linear decay is also used in practice (for example, [OLMo: Accelerating the Science of Language Models](https://arxiv.org/abs/2402.00838))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "4e8d2068-a057-4abf-b478-f02cc37191f6",
   "metadata": {},
   "outputs": [],
   "source": [
    "import math\n",
    "\n",
    "min_lr = 0.1 * initial_lr\n",
    "track_lrs = []\n",
    "\n",
    "lr_increment = (peak_lr - initial_lr) / warmup_steps\n",
    "global_step = -1\n",
    "\n",
    "for epoch in range(n_epochs):\n",
    "    for input_batch, target_batch in train_loader:\n",
    "        optimizer.zero_grad()\n",
    "        global_step += 1\n",
    "    \n",
    "        # Adjust the learning rate based on the current phase (warmup or cosine annealing)\n",
    "        if global_step < warmup_steps:\n",
    "            # Linear warmup\n",
    "            lr = initial_lr + global_step * lr_increment  \n",
    "        else:\n",
    "            # Cosine annealing after warmup\n",
    "            progress = ((global_step - warmup_steps) / \n",
    "                        (total_training_steps - warmup_steps))\n",
    "            lr = min_lr + (peak_lr - min_lr) * 0.5 * (1 + math.cos(math.pi * progress))\n",
    "        \n",
    "        # Apply the calculated learning rate to the optimizer\n",
    "        for param_group in optimizer.param_groups:\n",
    "            param_group[\"lr\"] = lr\n",
    "        track_lrs.append(optimizer.param_groups[0][\"lr\"])\n",
    "    \n",
    "        # Calculate loss and update weights"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "0e779e33-8a44-4984-bb23-be0603dc4158",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlMAAAGwCAYAAACNeeBZAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAA9hAAAPYQGoP6dpAABiTUlEQVR4nO3deVhUZf8G8HsWZoZ12GRHwaVcAFFQxDRbKDJbSDM1SzPTMtewLM20fO1nWpaZmlqWVppmb5mZWYaVG6IIIoi4JIqCwyoM+zJzfn8go7yigSxnlvtzXXP5cuaZ4TvnNbh9zvc8j0QQBAFEREREdFukYhdAREREZMoYpoiIiIiagWGKiIiIqBkYpoiIiIiagWGKiIiIqBkYpoiIiIiagWGKiIiIqBnkYhdgqvR6PbKysmBvbw+JRCJ2OURERNQIgiCguLgYXl5ekEpbZk6JYeo2ZWVlwdfXV+wyiIiI6DZcvHgRPj4+LfJeDFO3yd7eHkDt/xkODg4iV0NERESNodVq4evra/g93hIYpm5T3aU9BwcHhikiIiIT05ItOmxAJyIiImoGhikiIiKiZmCYIiIiImoGhikiIiKiZmCYIiIiImoGhikiIiKiZmCYIiIiImoGhikiIiKiZmCYIiIiImoGhikiIiKiZhA9TK1cuRJ+fn5QqVQICwvD4cOHbzl+69at6Nq1K1QqFQIDA7Fz5856z//www948MEH4eLiAolEgmPHjt3wHhUVFZg8eTJcXFxgZ2eHYcOGITs7uyU/FhEREVkIUcPUli1bEB0djfnz5yMhIQE9e/ZEZGQkcnJyGhx/8OBBjBo1CuPHj0diYiKioqIQFRWFlJQUw5jS0lIMGDAAixcvvun3feWVV/Dzzz9j69at+Pvvv5GVlYWhQ4e2+OcjIiIi8ycRBEEQ65uHhYWhT58+WLFiBQBAr9fD19cXU6dOxRtvvHHD+BEjRqC0tBQ7duwwHOvXrx+Cg4OxevXqemPPnz8Pf39/JCYmIjg42HC8qKgI7dq1w6ZNm/Dkk08CANLS0tCtWzfExsaiX79+japdq9VCrVajqKiIGx23oqKyapRV10AulcJKJoHKSgalXNqiG1QSEZHlaI3f3/IWeZfbUFVVhaNHj2L27NmGY1KpFBEREYiNjW3wNbGxsYiOjq53LDIyEtu2bWv09z169Ciqq6sRERFhONa1a1e0b9/+lmGqsrISlZWVhq+1Wm2jvyfdnoP/5OGZz+Og/5+4byWTwE4ph4O1FVztlGhnp0Q7eyW8HK3h62yN9s426OBiC7W1lTiFExGRRREtTOXl5UGn08Hd3b3ecXd3d6SlpTX4Go1G0+B4jUbT6O+r0WigUCjg6OjYpPdZtGgR3nnnnUZ/H2q+P1JzoBcAiQS4fv60WifgSlk1rpRV40J+2U1f72avxB3u9rjD3R4B3g4I8lHD39UOMilntYiIqOWIFqZMzezZs+vNimm1Wvj6+opYkflLziwEAHzwZE8M7e2NGr2AimodSiprUFxRg6LyauQVVyK3pBLZ2gpkXinHxSvluFhQhpziSsNj/9k8w3vaKmQIbu+Ivn4u6OPvhN7tnaCykon0CYmIyByIFqZcXV0hk8luuIsuOzsbHh4eDb7Gw8OjSeNv9h5VVVUoLCysNzv1b++jVCqhVCob/X2oeXR6ASmZtZdSg3zUkEgksJJJYCWTwl5lBU/1rV9fXFGNszklOJNdgpMaLZIvFeFElhalVTocOJuPA2fzAQAKuRR9/Zxx9x2uuPuOdrjT3Z79WERE1CSihSmFQoGQkBDExMQgKioKQG0DekxMDKZMmdLga8LDwxETE4MZM2YYju3evRvh4eGN/r4hISGwsrJCTEwMhg0bBgA4deoUMjIymvQ+1LrO5ZagvFoHG4UMHdvZNfn19ior9GrvhF7tnQzHanR6nMkpQfyFKziSXoDD6QXQaCuw/2we9p/Nw//tTIOPkzUe7O6BB3u4o4+fMy8JEhHRvxL1Ml90dDTGjh2L0NBQ9O3bF8uWLUNpaSnGjRsHABgzZgy8vb2xaNEiAMD06dMxaNAgLF26FEOGDMHmzZsRHx+PtWvXGt6zoKAAGRkZyMrKAlAblIDaGSkPDw+o1WqMHz8e0dHRcHZ2hoODA6ZOnYrw8PBG38lHre/4pSIAQA8vhxYLNHKZFN08HdDN0wHP9usAQRBwLq8Uf5/Kxd4zuTh0Lh+XrpTjiwPp+OJAOlztFHgkyAuP9vRC7/aOnLEiIqIGiRqmRowYgdzcXMybNw8ajQbBwcHYtWuXock8IyMDUum1pbD69++PTZs2Ye7cuZgzZw66dOmCbdu2ISAgwDBm+/bthjAGACNHjgQAzJ8/H2+//TYA4KOPPoJUKsWwYcNQWVmJyMhIrFq1qg0+MTVWcmZtmAr0dmy17yGRSNCpnR06tbPD8wP8UV6lw94zufj9RDZi0rKRV1KF9QfPY/3B8/B1tsaTvX3xZKgPvB2tW60mIiIyPaKuM2XKuM5U6xq66gASMgqxbEQwonp5t/n3r9bpsf9MHrYnZeG3ExqUVekA1N5ZOLBLOzwT1h73d3PnZUAiIhNjVutMEd1MjU6P1Mu1zeeBPv/Sad5KrGRS3NvVDfd2dUN5lQ67TlzGd0cuIfZcPvaezsXe07no4GKD5/r7YXioL+yU/E+JiMhScWbqNnFmqvWkabR4aNk+2CnlOD7/QUiNaPbnQn4pvj18Ed8ezkBReTUAwF4px1N9fPFcfz/4OtuIXCEREd1Ka/z+Fn2jY6L/dX3zuTEFKQDo4GKLNwZ3Rezs+7AwKgAd29miuLIG6/anY9D7f+Klr48i5Wq/FxERWQZemyCjk3w1TAWJdImvMWwUcjzTrwOe7tsef5/JxRf707HvTB52ndBg1wkNInu445UH7kBXD85aEhGZO4YpMjqGO/l8HMUtpBGkUgnuvdMN997phtPZxVj151n8lJSF305k47cT2RgS5IlXIrqgs5u92KUSEVEr4WU+MirV1zefexvvzFRD7nC3x7KRvfD7jLsxJMgTAPDL8ct44KO9mLE5EedyS0SukIiIWgPDFBmV09nFqKrRw14lRwcTbebu4m6PlU/3xq/TByKyhzsEAdh2LAsPfrQX7/x8AkVl1WKXSERELYhhioxKimGxTrXRNZ83VTdPB6x5NhQ7pg7AfV3dUKMX8OWB87jngz/xzaEL0Ol5Iy0RkTlgmCKjUncnn1jrS7WGAG81vniuD74e3xdd3Oxwpawac7elYMjyfTj4T57Y5RERUTMxTJFRSb5uZsrcDOzSDr9OH4h3HusBtbUV0jTFePqzOLz09VFcLCgTuzwiIrpNDFNkNKpq9Ei7XAwACGrFPfnEJJdJMba/H/569R6MCe8AqQTYdUKDBz76G5/vO8dLf0REJohhiozG6exiVOn0UFtbwdfZvDcTdrJVYMHjAfh1+t0I83dGRbUeC385iaGrDuDk1bsZiYjINDBMkdEw9Et5qyGRmHbzeWPd6WGPzRP7YdHQQNir5Ei6VIRHP9mPpb+fQmWNTuzyiIioERimyGgkZxYCMK/m88aQSCQY1bc9/ogehAe7u6NGL+CTPWfx8Mf7EH++QOzyiIjoXzBMkdGom5kKMsPm88Zwd1BhzbMh+HR0b7jaKfFPbimGr4nFgp9TUVHNWSoiImPFMEVGoaJah9PZtc3nljYzdT2JRILBgZ6IiR6Ep0J9IAjAFwfSEbXyAE5pisUuj4iIGsAwRUbhlKYY1ToBTjZW8HY07+bzxlDbWGHJkz2xbmwoXGwVSNMU49EV+/HlgXQIAu/4IyIyJgxTZBSOX7e5saU0nzfG/d3csWvG3bj3znaoqtHjnZ9TMfbLI8jRVohdGhERXcUwRUYh+VIhAMvtl7qVdvZKfPFcHyx4vAeUcin2ns7FQx/vw+8nNGKXRkREYJgiI5GcWbu2kiX3S92KRCLBmHA/7Jg6AN08HVBQWoWJXx/Fgp9TUa3Ti10eEZFFY5gi0dVrPufM1C11cbfHtsn9MWGgP4Da5vSRaw9BU8TLfkREYmGYItGlXtZCpxfgaqeAp1oldjlGTymX4c0h3bH22RDYq+Q4euFK7abJZ7lpMhGRGBimSHQpmZa38nlLeLCHh+GyX35pFZ5ZF4eVf56Fnvv7ERG1KYYpEp1hGxkfR3ELMUEdXGzx48v9MTzEB3oBeP+3U5jwVTyKyqrFLo2IyGIwTJHokq/bk4+aTmUlw/vDe2LxsEAo5FLEpOXg0RX7DX1oRETUuhimSFRlVTU4k1P7Sz+Id/I1y4g+7fHDpP7wdbZGRkEZhq46iD1p2WKXRURk9himSFQnL2uhFwA3eyXcHdh83lwB3mr8NHkAwvydUVJZg/Eb4vH5vnNcNZ2IqBUxTJGojvMSX4tztlXg6/FhGNXXF4IALPzlJF7/73FU1XA9KiKi1sAwRaIy9EvxEl+LUsil+L8nAjH/0e6QSoDv4i/hmc/jkF9SKXZpRERmh2GKRFW3Jx/7pVqeRCLBuLv88eW4vrBXyXH4fAEeX3kApzRsTCciakkMUySa0soa/JNbAqC214dax6A72uHHl++Cn4sNLl0px5OfHsTBf7jAJxFRS2GYItGcyNJCEAAPBxXc7Nl83po6u9lh2+S70NffGcWVNXjuiyPYnpQldllERGaBYYpEc/xSIQD2S7UVRxsFvnq+L4YEeqJKp8e0bxPx+b5zYpdFRGTyGKZINHXbyATxEl+bUVnJ8MmoXhh3lx+A2jv9Fvycyi1oiIiagWGKRFPXfM6ZqbYllUow75HuePPhbgCALw6kY+rmRFRU60SujIjINDFMkSiKK6pxLrcUANeYEoNEIsGEuzvi45HBsJJJ8MvxyxjzxWEUlXNPPyKipmKYIlGkZGoBAN6O1nCxU4pcjeV6PNgbG8b1hb1SjsPpBRi59hDyuBYVEVGTMEyRKOr6pTgrJb7+nV2x5cVwuNopcfKyFk+tiUVWYbnYZRERmQyGKRIF+6WMS3cvB3z3Yj94qVU4l1uK4atjcT6vVOyyiIhMAsMUiSK5blkEzkwZjY7t7LB1Un/4u9ois7Acw9fEcrV0IqJGYJiiNldUXo3z+WUAGKaMjbejNb57MRxdPeyRW1yJEWtjkXSxUOyyiIiMGsMUtbkTVy/x+Tpbw8lWIXI19L/a2SuxZWI4gn0dUVhWjdGfxyHuXL7YZRERGS2GKWpzx9l8bvTUNlb45oUwhHd0QUllDcZ8cRj7zuSKXRYRkVFimKI2l3ypLkw5ilsI3ZKdUo4vx/XB/V3dUFmjxwsb4rH3NAMVEdH/YpiiNnc8sxAAEMQ7+YyeykqGT58JQUS3q4HqKwYqIqL/xTBFbaqwrAoXC2rXMArwYpgyBQq5FKtGhyCimzuqrgaqvxmoiIgMGKaoTSVf7Zfq4GIDtY2VyNVQY9UGqt54oHttoJrwVTz+OpUjdllEREaBYYra1PFLbD43VQq5FCuf7o0HrwaqiV8fZaAiIgLDFLWxum1k2C9lmhRyKVZcH6i+Ooo/GaiIyMIxTFGbOs47+UxeXaCK7OGOKp0eL359FAfO5oldFhGRaBimqM3kl1Qi8+oGuj28HUSuhpqjLlDV9VC9sCEe8ecLxC6LiEgUDFPUZuqazzu62sJBxeZzU2clk2LF070wsIsryqt1GPflERy/uuciEZElYZiiNlPXLxXIfimzoZTLsPbZUPT1d0bx1ZXSuTkyEVkahilqM7yTzzxZK2T44rk+6HndXn7nckvELouIqM0wTFGbSeaefGbLTinHhnF90NXDHnkllRj9eRwuFpSJXRYRUZtgmKI2kVtcictFFZBIgB4MU2bJ0UaBb14IQ6d2trhcVIHRn8dBU1QhdllERK2OYYraRF2/VKd2drBTykWuhlqLq50SG1/oB19na2QUlOHZdXEoLKsSuywiolYlephauXIl/Pz8oFKpEBYWhsOHD99y/NatW9G1a1eoVCoEBgZi586d9Z4XBAHz5s2Dp6cnrK2tERERgTNnztQbc/r0aTz++ONwdXWFg4MDBgwYgD///LPFPxtdw34py+GhVmHTC/3g7qDEmZwSPL/+CMqqasQui4io1YgaprZs2YLo6GjMnz8fCQkJ6NmzJyIjI5GT0/CKygcPHsSoUaMwfvx4JCYmIioqClFRUUhJSTGMWbJkCZYvX47Vq1cjLi4Otra2iIyMREXFtcsNjzzyCGpqarBnzx4cPXoUPXv2xCOPPAKNRtPqn9lSJWcWAmCYshS+zjb46vkwOKjkSMgoxMsbE1Ct04tdFhFRq5AIgiCI9c3DwsLQp08frFixAgCg1+vh6+uLqVOn4o033rhh/IgRI1BaWoodO3YYjvXr1w/BwcFYvXo1BEGAl5cXZs6ciVdffRUAUFRUBHd3d6xfvx4jR45EXl4e2rVrh71792LgwIEAgOLiYjg4OGD37t2IiIhosNbKykpUVlYavtZqtfD19UVRUREcHLgA5b8J+78/kK2txPcvhSPUz1nscqiNxJ8vwDPr4lBRrccTvbyxdHhPSKUSscsiIgum1WqhVqtb9Pe3aDNTVVVVOHr0aL3wIpVKERERgdjY2AZfExsbe0PYiYyMNIxPT0+HRqOpN0atViMsLMwwxsXFBXfeeSe++uorlJaWoqamBmvWrIGbmxtCQkJuWu+iRYugVqsND19f39v+7JYmW1uBbG0lpBKguxeDpyUJ9XPGqtG9IZNK8GNiJhb+chIi/vuNiKhViBam8vLyoNPp4O7uXu+4u7v7TS+3aTSaW46v+/NWYyQSCf744w8kJibC3t4eKpUKH374IXbt2gUnJ6eb1jt79mwUFRUZHhcvXmzaB7ZgyVf7pTq72cFGweZzS3NfV3e8/2QQAOCLA+lY9dc/IldERNSyRG9Ab2uCIGDy5Mlwc3PDvn37cPjwYURFReHRRx/F5cuXb/o6pVIJBweHeg9qnOOZ3NzY0g3t7YO5Q7oBAN7/7RQ2H84QuSIiopYjWphydXWFTCZDdnZ2vePZ2dnw8PBo8DUeHh63HF/3563G7NmzBzt27MDmzZtx1113oXfv3li1ahWsra2xYcOGFvlsVF/dsghB3EbGor0wsCMm3dMJADDnx2T8doI3fBCReRAtTCkUCoSEhCAmJsZwTK/XIyYmBuHh4Q2+Jjw8vN54ANi9e7dhvL+/Pzw8POqN0Wq1iIuLM4wpK6tdlVkqrf/RpVIp9HrebdTSBEG4tiwCw5TFmxV5J0aE+kIvANO+TURCxhWxSyIiajZRL/NFR0fjs88+w4YNG3Dy5ElMmjQJpaWlGDduHABgzJgxmD17tmH89OnTsWvXLixduhRpaWl4++23ER8fjylTpgCo7YeaMWMGFi5ciO3btyM5ORljxoyBl5cXoqKiANQGMicnJ4wdOxZJSUk4ffo0XnvtNaSnp2PIkCFtfg7MnUZbgbySSsikEnT35KVRSyeRSPDuEwG49852qKzR44UN8UjPKxW7LCKiZhE1TI0YMQIffPAB5s2bh+DgYBw7dgy7du0yNJBnZGTU62Pq378/Nm3ahLVr16Jnz574/vvvsW3bNgQEBBjGzJo1C1OnTsXEiRPRp08flJSUYNeuXVCpVABqLy/u2rULJSUluO+++xAaGor9+/fjp59+Qs+ePdv2BFiAulmpLm52UFnJRK6GjIFcJsWKp3sj0FuNgtIqPPflYeSVVP77C4mIjJSo60yZstZYp8IcLf39FD7ZcxZPhfpgyZMMq3RNbnElhn56ABcLytHT1xGbJ/SDtYKBm4hal1mtM0WW4Vq/lKO4hZDRaWevxPpxfeFoY4Wki4WY+m0idHr+246ITA/DFLUaQRCQnMk9+ejmOrWzw+djQqGQS/HHyWy8vf0EF/UkIpPDMEWtJquoAgWlVZBLJejqYS92OWSkQv2c8fGIYEgkwNeHLmDN3nNil0RE1CQMU9Rqki8VAgDu9LBn8znd0uBAT7w1pDsA4L1f07DjeJbIFRERNR7DFLWaun4pLtZJjfH8AH88f5c/ACD6uySuQUVEJoNhilpNXb9UAPulqJHeHNINEd3cUFWjx8Sv4nGxoEzskoiI/hXDFLWK65vPg7gnHzWSTCrBxyN7obunA/JKqjB+wxFoK6rFLouI6JYYpqhVXLpSjsKyaihkUtzhYSd2OWRCbJVyrHsuFG72SpzOLsGUTYmo0XGrJyIyXgxT1Crq+qXu9LCHUs7mc2oaT7U11o3tA2srGfaezsXbP3PJBCIyXgxT1CqOZxYC4ObGdPsCfdRYNrJ2yYRvDmXgywPnxS6JiKhBDFPUKlIM/VIMU3T7Int4YM7gbgCA//ySij9Ss0WuiIjoRgxT1OIEQbhuGxmGKWqeFwb6Y1Tf9hAEYPrmRJzSFItdEhFRPQxT1OIu5JehuKIGCrkUd7hz5XNqHolEggWP90D/Ti4ordLhha+O4EppldhlEREZMExRizt+9RJfN08HWMn4V4yaz0omxcqne6O9sw0uFpTj5Y0JqOYdfkRkJPibjloc+6WoNTjZKvD52FDYKmSIPZeP/+xIFbskIiIADFPUCo5f3ZOP/VLU0u5wt8fHI3tBIgG+ir2AjXEXxC6JiIhhilqWXi8gJVMLAAjkzBS1goju7nj1wTsBAPN/OoG4c/kiV0RElo5hilrU+fxSlFTWQCmXoosbVz6n1vHyPZ3waE8v1OgFTNqYwD38iEhUDFPUour24+vh5QA5m8+plUgkEiwZFoRAbzUKSqsw4at4lFbWiF0WEVko/rajFlW3vlSQj6O4hZDZs1bIsHZMCFztlEjTFCP6u2PQ67nlDBG1PYYpalHJV8NUAPulqA14qq2x5tkQKGRS/HYiGx/HnBG7JCKyQAxT1GJ0egEnsupmphimqG2EdHDCu08EAAA+jjmDncmXRa6IiCwNwxS1mPS8EpRW6WBtJUOndmw+p7YzPNQX4wf4AwBmfpdkCPVERG2BYYpaTF2/VA8vB8ikEpGrIUsze3BXDOziivJqHSZ+dRT5JZVil0REFoJhiloMNzcmMcllUqwY1Rv+rrbILCzHtM2JqOGWM0TUBhimqMUYtpFhmCKRqG2ssPqZENgoZDhwNh/v/35K7JKIyAIwTFGLqNHpcSKrbuVzR3GLIYt2p4c9ljwZBABY8/c5/HKcDelE1LoYpqhF/JNbivJqHWwVMnR0tRW7HLJwjwR54cW7OwIAXvs+Caezi0WuiIjMGcMUtQjDyufeakjZfE5G4LXIO9G/kwvKqnR48euj0FZUi10SEZkphilqEcmXCgEAQVysk4yEXCbFJ6N6wdvRGul5pYjewhXSiah1MExRizieyTv5yPi42Cnx6TO9oZBL8cfJHKz486zYJRGRGWKYomar0emRamg+Z5gi4xLk44iFUbUrpH/0x2n8mZYjckVEZG4YpqjZzuSUoLJGD3ulHH4ubD4n4/NUqC9Gh7WHIADTNyfifF6p2CURkRlhmKJmu35zYzafk7Ga/2gP9GrvCG1FDV765ijKqmrELomIzATDFDXb8cxCAFysk4ybQi7Fp6ND4GqnRJqmGG/8NxmCwIZ0Imo+hilqtutnpoiMmYdahVWje0MulWB7UhbW7U8XuyQiMgMMU9QsVTV6nNTULojImSkyBX39nTF3SDcAwKJf0xB3Ll/kiojI1DFMUbOczi5GVY0eDio52jvbiF0OUaOM7e+HqGAv6PQCpnybiJziCrFLIiITxjBFzZJ83fpSEgmbz8k0SCQS/N/QQHRxs0NucSWmfZuIGp1e7LKIyEQxTFGzHL/aL8XNjcnU2Cjk+PSZENgoZDh0rgAf7j4tdklEZKIYpqhZUq7OTLFfikxRZzc7LB4WBABY9dc/+CM1W+SKiMgUMUzRbaus0SFNw5XPybQ92tMLz/X3AwBEf3cMFwvKxC2IiEwOwxTdtlOaYlTrBDjaWMHHyVrscohu25yHuyHYt3ZBz0kbj6KiWid2SURkQhim6LYZms+92XxOpk0hl2Ll6N5wsrFCSqYWC3akil0SEZkQhim6bXWLdbJfisyBt6M1lo3sBYkE2BSXgR8SLoldEhGZCIYpum28k4/MzaA72mHafV0AAHN+TDb0BBIR3QrDFN2WimodTmfXrnweyJkpMiPT7u+CgV1cUVGtx8vfJKC4olrskojIyDFM0W1J0xSjRi/AxVYBL7VK7HKIWoxMKsGyEcHwVKtwLq+UGyIT0b+6rTBVU1ODP/74A2vWrEFxce3sRFZWFkpKSlq0ODJeyZcKAXDlczJPLnZKrHi6dkPkX5Iv48sD58UuiYiMWJPD1IULFxAYGIjHH38ckydPRm5uLgBg8eLFePXVV1u8QDJOdf1SQVxfisxUSAcnvHl1Q+T/23kSCRlXRK6IiIxVk8PU9OnTERoaiitXrsDa+traQk888QRiYmJatDgyXnXLIgQwTJEZe66/H4YEeqJGL2DqpkQUllWJXRIRGaEmh6l9+/Zh7ty5UCgU9Y77+fkhMzOzxQoj41VepcOZnNpLukE+juIWQ9SKJBIJFg0LRAcXG2QWluPVrcfZP0VEN2hymNLr9dDpblwd+NKlS7C3t2+Rosi4pV7WQqcX0M5eCXcHpdjlELUqB5UVVj7dGwqZFH+czMa6/elil0RERqbJYerBBx/EsmXLDF9LJBKUlJRg/vz5ePjhh1uyNjJShuZzrnxOFiLAW423Hqntn3rv1zQksn+KiK7T5DC1dOlSHDhwAN27d0dFRQWefvppwyW+xYsXt0aNZGSSM7m5MVmeZ/p1wMOBHqjRC5iyKRFFZVx/iohqyZv6Ah8fHyQlJWHLli1ISkpCSUkJxo8fj9GjR9drSCfzlZxZCIDbyJBlkUgkeG9YEFIytcgoKMOr3ydh7bMhnJ0loqbPTO3duxcAMHr0aCxZsgSrVq3CCy+8ACsrK8NzTbFy5Ur4+flBpVIhLCwMhw8fvuX4rVu3omvXrlCpVAgMDMTOnTvrPS8IAubNmwdPT09YW1sjIiICZ86cueF9fvnlF4SFhcHa2hpOTk6Iiopqcu2WqLSyBmevNp9zZooszfX9U7tTs/EF158iItxGmLr33ntRUFBww/GioiLce++9TXqvLVu2IDo6GvPnz0dCQgJ69uyJyMhI5OTkNDj+4MGDGDVqFMaPH4/ExERERUUhKioKKSkphjFLlizB8uXLsXr1asTFxcHW1haRkZGoqKgwjPnvf/+LZ599FuPGjUNSUhIOHDiAp59+ukm1W6rUy1roBcDdQQk3B658TpYn0EeNuYb+qZM4drFQ3IKISHQSoYn3+UqlUmRnZ6Ndu3b1jp8+fRqhoaHQahu/MWhYWBj69OmDFStWAKi9U9DX1xdTp07FG2+8ccP4ESNGoLS0FDt27DAc69evH4KDg7F69WoIggAvLy/MnDnTsIBoUVER3N3dsX79eowcORI1NTXw8/PDO++8g/Hjxze61srKSlRWVhq+1mq18PX1RVFRERwcHBr9Pqbui/3pWLAjFRHd3PH52FCxyyEShSAImLwpATuTNfBxssYvUwdCbWMldllE1AharRZqtbpFf383emZq6NChGDp0KCQSCZ577jnD10OHDsXjjz+OyMhI9O/fv9HfuKqqCkePHkVERMS1YqRSREREIDY2tsHXxMbG1hsPAJGRkYbx6enp0Gg09cao1WqEhYUZxiQkJCAzMxNSqRS9evWCp6cnBg8eXG92qyGLFi2CWq02PHx9fRv9Wc1J3WKd7JciS1bXP9Xe2QaXrpTjte+TuP4UkQVrdJiqCxGCIMDe3r5esPDw8MDEiRPxzTffNPob5+XlQafTwd3dvd5xd3d3aDSaBl+j0WhuOb7uz1uNOXfuHADg7bffxty5c7Fjxw44OTnhnnvuafDyZZ3Zs2ejqKjI8Lh48WKjP6s5OX7dnnxEluz6/qnfU7O5fx+RBWv03XxffvklgNqVzl999VXY2tq2WlGtSa/XAwDefPNNDBs2DEDtZ/Px8cHWrVvx4osvNvg6pVIJpdKyF6gsqazBubxSAGw+JwJq/1Hx5pBumL/9BBb9ehIhHZzQ09dR7LKIqI01uQF9/vz5LRKkXF1dIZPJkJ2dXe94dnY2PDw8GnyNh4fHLcfX/XmrMZ6engCA7t27G55XKpXo2LEjMjIymvGJzN+JzCIIAuClVsHVzrKDJVGdMeEdMDjAA9W62j6qonKuP0VkaZocpgDg+++/x1NPPYV+/fqhd+/e9R6NpVAoEBISUm9zZL1ej5iYGISHhzf4mvDw8Bs2U969e7dhvL+/Pzw8POqN0Wq1iIuLM4wJCQmBUqnEqVOnDGOqq6tx/vx5dOjQodH1W6K6file4iO6RiKRYPGTQfB1tsalK+WYxf4pIovT5DC1fPlyjBs3Du7u7khMTETfvn3h4uKCc+fOYfDgwU16r+joaHz22WfYsGEDTp48iUmTJqG0tBTjxo0DAIwZMwazZ882jJ8+fTp27dqFpUuXIi0tDW+//Tbi4+MxZcoUALU/1GbMmIGFCxdi+/btSE5OxpgxY+Dl5WVYR8rBwQEvvfQS5s+fj99//x2nTp3CpEmTAADDhw9v6umwKMcv1TWfO4pbCJGRqeufspJJ8NuJbKw/eF7skoioDTV5BfRVq1Zh7dq1GDVqFNavX49Zs2ahY8eOmDdv3i0buBsyYsQI5ObmYt68edBoNAgODsauXbsMDeQZGRmQSq/lvf79+2PTpk2YO3cu5syZgy5dumDbtm0ICAgwjJk1axZKS0sxceJEFBYWYsCAAdi1axdUqmtrIr3//vuQy+V49tlnUV5ejrCwMOzZswdOTk5NPR0WpW5mKoD9UkQ3CPJxxJsPd8PbP6fi/3aeRO/27J8ishRNXmfKxsYGJ0+eRIcOHeDm5obdu3ejZ8+eOHPmDPr164f8/PzWqtWotMY6FcZMW1GNoLd/BwAkvPUAnG0VIldEZHwEQcCkbxKw64QGvs7W2DF1INTWXH+KyJiIus5UHQ8PD8MMVPv27XHo0CEAtWs8sU/AfKVcnZXycbJmkCK6iev7py4WlOON/x7nz0UiC9DkMHXfffdh+/btAIBx48bhlVdewQMPPIARI0bgiSeeaPECyTgkX+JinUSNoba2wopRtf1Tv6ZosDGOdwkTmbsm90ytXbvWsFbT5MmT4eLigoMHD+Kxxx676RpNZPrYL0XUeD19HfH6Q12x8JeTWLAjFSEdnNDN0/zbAYgsVZNmpmpqarBw4cJ6K5SPHDkSy5cvx9SpU6FQ8PKPuTJsI+PtKG4hRCZi/AB/3NfVDVU1ekzZlICyqhqxSyKiVtKkMCWXy7FkyRLU1PCHgiUpKqvGhfwyAFz5nKixJBIJ3n8yCO4OSvyTW4q3t58QuyQiaiVN7pm6//778ffff7dGLWSk6mal2jvbQG3DO5OIGsvFTollI3pBKgG+i7+En45lil0SEbWCJvdMDR48GG+88QaSk5MREhJyw9Yyjz32WIsVR8aBK58T3b7wTi6Yel8XfBxzBnN+SEZPH0f4uZrm3qZE1LAmh6mXX34ZAPDhhx/e8JxEIoFOp2t+VWRUkjMLAQBBvMRHdFum3tcZsefycTi9AFO+TcB/J/WHUi4TuywiaiFNvsyn1+tv+mCQMk9128hwZoro9shlUnw8MhiONlZIydRi8a+n/v1FRGQybmujY7IcV0qrcOlKOQAui0DUHJ5qa3zwZE8AwBcH0vFHarbIFRFRS2GYoluq65fyd7WFg4rN50TNEdHdHc/f5Q8AeO37JFwuKhe5IiJqCQxTdEuG5nPOShG1iNcH34kAbwdcKavG9M3HUKPTi10SETUTwxTd0vFLhQC4jQxRS1HKZfhkVG/YKmQ4nF6AT/acFbskImomhim6pZRMLQD2SxG1JH9XW/zf0EAAwCd7ziD2n3yRKyKi5mhymNJqtQ0+iouLUVVV1Ro1kkjySiqRWVgOiQTo4cV9xYha0uPB3hge4gO9AMzYkoiCUv78JDJVTQ5Tjo6OcHJyuuHh6OgIa2trdOjQAfPnzzdshkymq65fqqOrLezZfE7U4t55vAc6tbNFtrYSr25NgiAIYpdERLehyWFq/fr18PLywpw5c7Bt2zZs27YNc+bMgbe3Nz799FNMnDgRy5cvx3vvvdca9VIbSr66vlSQj6O4hRCZKRuFHCue7g2FXIo9aTlYtz9d7JKI6DY0eQX0DRs2YOnSpXjqqacMxx599FEEBgZizZo1iImJQfv27fHuu+9izpw5LVosta26mSn2SxG1nm6eDnjrke54a1sKFu9KQ19/Z/4DhsjENHlm6uDBg+jVq9cNx3v16oXY2FgAwIABA5CRkdH86khU12amGKaIWtMzYe3xUA8PVOsETP02EcUV1WKXRERN0OQw5evri3Xr1t1wfN26dfD19QUA5Ofnw8nJqfnVkWhytBXQaCsglQDdPdl8TtSaJBIJFg8LgrejNS7kl2HOjynsnyIyIU2+zPfBBx9g+PDh+PXXX9GnTx8AQHx8PNLS0vD9998DAI4cOYIRI0a0bKXUpuou8XVqZwdbZZP/mhBRE6ltrLB8VC88tSYWPydlYWBnVzzVx1fssoioEZo8M/XYY48hLS0NgwcPRkFBAQoKCjB48GCkpaXhkUceAQBMmjQJH374YYsXS23HsPI5L/ERtZmQDk6Y+eAdAIB521NwJrtY5IqIqDFua8rB39+fd+uZOUO/FJvPidrUS3d3Quw/+dh3Jg9TNiXipyl3QWUlE7ssIrqF2wpThYWFOHz4MHJycm5YT2rMmDEtUhiJRxAEHDfMTDmKWwyRhZFKJfjwqWAM/ngfTmUX4z87UvHuE4Fil0VEt9DkMPXzzz9j9OjRKCkpgYODAyQSieE5iUTCMGUGsrWVyC2uZPM5kUja2Svx0YieeHbdYWyMy0D/Tq4YEuQpdllEdBNN7pmaOXMmnn/+eZSUlKCwsBBXrlwxPAoKClqjRmpjdf1Sd7jbw1rBywtEYhjYpR0m3dMJAPDGD8dxsaBM5IqI6GaaHKYyMzMxbdo02NjYtEY9ZASSLxUCAALZL0UkqugH7kDv9o4orqjB1G8TUa3jNl1ExqjJYSoyMhLx8fGtUQsZibp+KS7WSSQuK5kUH4/sBQeVHMcuFuKD30+JXRIRNaDJPVNDhgzBa6+9htTUVAQGBsLKqv4GuI899liLFUdtTxAEpHAbGSKj4etsgyVPBuGlbxKw5u9zCO/ognvudBO7LCK6jkRo4jK7UunNJ7MkEgl0Ol2zizIFWq0WarUaRUVFcHAwnybtrMJy9H9vD+RSCVLeieQt2URG4q1tKfj60AW42Crw6/SBcHNQiV0SkUlqjd/fTb7Mp9frb/qwlCBlzo5futZ8ziBFZDzeHNINXT3skV9ahRlbjkGn53YzRMaiyWGKzFtyZiEA9ksRGRuVlQwrnu4NaysZDv6Tj1V/nhW7JCK6qlE9U8uXL8fEiROhUqmwfPnyW46dNm1aixRG4kjO1AJgvxSRMersZof/RAXg1a1J+OiP0wjr6IK+/s5il0Vk8RrVM+Xv74/4+Hi4uLjA39//5m8mkeDcuXMtWqCxMseeKUEQ0Ps/u3GlrBrbp9yFIK5+TmSUorccww+JmfBUq7Bz2kA42SrELonIZLTG7+9GzUylp6c3+L/JvFy6Uo4rZdWwkklwp4e92OUQ0U0siApA4sVCpOeV4rXvj+OzMSH1dqMgorbFnikyqFv5/E4PeyjlbD4nMlZ2SjlWPN0LCpkUf5zMxpcHzotdEpFFa/I6UzqdDuvXr0dMTEyDGx3v2bOnxYqjtlUXpgK9HcUthIj+VQ8vNd4c0g3zt5/Aol9Poo+fMwJ54wiRKJocpqZPn47169djyJAhCAgI4NSyGUm+xJXPiUzJmPAOOHA2D7+nZmPqtwnYMW0g7JRN/rFORM3U5P/qNm/ejO+++w4PP/xwa9RDIhEE4bqZKYYpIlMgkUiw5MkgnFi+H+fzy/Dmj8lYNiKY/8glamNN7plSKBTo3Llza9RCIrpYUI6i8mooZFLc4c7mcyJT4WijwPJRwZBJJfjpWBa2Hr0kdklEFqfJYWrmzJn4+OOP0cRdaMjIHb+6WGc3T3so5LwvgciUhHRwRvQDdwAA5v90AmdzikWuiMiyNPky3/79+/Hnn3/i119/RY8ePW7Y6PiHH35oseKo7dT1S7GBlcg0TRrUCbH/5GP/2TxM3piIn6bcxS2hiNpIk6cgHB0d8cQTT2DQoEFwdXWFWq2u9yDTVLcnXxDv5CMySVKpBB+O6AlXOwVOZRdjwY5UsUsishhNmpmqqanBvffeiwcffBAeHh6tVRO1Mb1eQEpWbZjiNjJEpsvNXoWPRgRjzBeHsSkuA3d1csWQIE+xyyIye02amZLL5XjppZdQWVnZWvWQCC4UlKG4ogZKuRRd3O3ELoeImmFgl3aYNKgTAOCN/x7HxYIykSsiMn9NvszXt29fJCYmtkYtJJLjlwoBAN29HGAlY/M5kal75YE7ENLBCcWVNZjybSKqavT//iIium1NbkB/+eWXMXPmTFy6dAkhISGwtbWt93xQUFCLFUdtw7BYJy/xEZkFK5kUH48MxsMf70PSxUJ88PspzHm4m9hlEZmtJoepkSNHAgCmTZtmOCaRSCAIAiQSCXQ6XctVR22ibrFO9ksRmQ8fJxu8P7wnXvz6KNbuPYfwTi649043scsiMktNDlPp6emtUQeJRK8XkJJZt42Mo7jFEFGLiuzhgbHhHbAh9gJmfpeEX6cPhLuDSuyyiMxOk8NUhw4dWqMOEsm5vFKUVulgbSVDp3a2//4CIjIpsx/uhiPnryD1shYzNh/DNy+EQSbldjNELem2d8RMTU1FRkYGqqqq6h1/7LHHml0UtZ26WanuXg6Qs/mcyOyorGRY8XQvPPLJfsSey8fKP89i2v1dxC6LyKw0OUydO3cOTzzxBJKTkw29UgAMG2uyZ8q01C3Wyc2NicxXx3Z2WBgVgOjvkrDsj9MI83dGWEcXscsiMhtNnoqYPn06/P39kZOTAxsbG5w4cQJ79+5FaGgo/vrrr1YokVpT8tU9+YK4jQyRWRva2wfDevtALwDTNx9DQWnVv7+IiBqlyWEqNjYWCxYsgKurK6RSKaRSKQYMGIBFixbVu8OPjJ9OLyAlUwuAYYrIEix4vAc6trOFRluB17YmccN6ohbS5DCl0+lgb28PAHB1dUVWVhaA2sb0U6dOtWx11KrO5ZagvFoHG4UM/q5c+ZzI3Nkq5VgxqjcUcili0nLwxYHzYpdEZBaaHKYCAgKQlJQEAAgLC8OSJUtw4MABLFiwAB07dmzxAqn11PVLBXipeXcPkYXo7uWAt4bULuD53q8ncexiobgFEZmBJoepuXPnQq+v3ZpgwYIFSE9Px8CBA7Fz504sX778topYuXIl/Pz8oFKpEBYWhsOHD99y/NatW9G1a1eoVCoEBgZi586d9Z4XBAHz5s2Dp6cnrK2tERERgTNnzjT4XpWVlQgODoZEIsGxY8duq35TVbdYZyAv8RFZlGf6dcDDgR6o1gmYvDEBRWXVYpdEZNKaHKYiIyMxdOhQAEDnzp2RlpaGvLw85OTk4L777mtyAVu2bEF0dDTmz5+PhIQE9OzZE5GRkcjJyWlw/MGDBzFq1CiMHz8eiYmJiIqKQlRUFFJSUgxjlixZguXLl2P16tWIi4uDra0tIiMjUVFRccP7zZo1C15eXk2u2xzU7cnHfikiyyKRSPDesCB0cLFBZmE5Xv2e/VNEzXHbCwudPXsWv/32G8rLy+Hs7HzbBXz44YeYMGECxo0bh+7du2P16tWwsbHBF1980eD4jz/+GA899BBee+01dOvWDf/5z3/Qu3dvrFixAkDtrNSyZcswd+5cPP744wgKCsJXX32FrKwsbNu2rd57/frrr/j999/xwQcf3Hb9pqpGp0fq5drmc24jQ2R5HFRWWPl0byhkUuxOzca6/dzdguh2NTlM5efn4/7778cdd9yBhx9+GJcvXwYAjB8/HjNnzmzSe1VVVeHo0aOIiIi4VpBUioiICMTGxjb4mtjY2HrjgdrZsrrx6enp0Gg09cao1WqEhYXVe8/s7GxMmDABX3/9NWxsbP611srKSmi12noPU3Y2twQV1XrYKeXwd+HK50SWKMBbjbce7Q4AeO/XNCRkXBG5IiLT1OQw9corr8DKygoZGRn1QsiIESOwa9euJr1XXl4edDod3N3d6x13d3eHRqNp8DUajeaW4+v+vNUYQRDw3HPP4aWXXkJoaGijal20aBHUarXh4evr26jXGStD87m3A6RsPieyWM+EtccjQZ6o0QuYsjEBV7j+FFGTNTlM/f7771i8eDF8fHzqHe/SpQsuXLjQYoW1pk8++QTFxcWYPXt2o18ze/ZsFBUVGR4XL15sxQpbX/Ilbm5MRLX9U4uGBsLf1RZZRRWYuTUJej37p4iaoslhqrS0tMHLYgUFBVAqlU16L1dXV8hkMmRnZ9c7np2dDQ8PjwZf4+HhccvxdX/easyePXsQGxsLpVIJuVyOzp07AwBCQ0MxduzYBr+vUqmEg4NDvYcpq7uTj/1SRGSvssKKp3tBIZdiT1oO1u47J3ZJRCalyWFq4MCB+OqrrwxfSyQS6PV6LFmyBPfee2+T3kuhUCAkJAQxMTGGY3q9HjExMQgPD2/wNeHh4fXGA8Du3bsN4/39/eHh4VFvjFarRVxcnGHM8uXLkZSUhGPHjuHYsWOGpRW2bNmCd999t0mfwRRVX9d8HsQwRUQAenip8fajPQAA7/92CkfOF4hcEZHpaPJGx0uWLMH999+P+Ph4VFVVYdasWThx4gQKCgpw4MCBJhcQHR2NsWPHIjQ0FH379sWyZctQWlqKcePGAQDGjBkDb29vLFq0CEDt3oCDBg3C0qVLMWTIEGzevBnx8fFYu3YtgNpwN2PGDCxcuBBdunSBv78/3nrrLXh5eSEqKgoA0L59+3o12NnVrv7dqVOnGy5fmqPT2cWoqtHDXiVHB5d/b74nIsswqq8v4tLz8dOxLEzdlIhfpg2Ai13TrjgQWaImh6mAgACcPn0aK1asgL29PUpKSjB06FBMnjwZnp6eTS5gxIgRyM3Nxbx586DRaBAcHIxdu3YZGsgzMjIglV6bQOvfvz82bdqEuXPnYs6cOejSpQu2bduGgIAAw5hZs2ahtLQUEydORGFhIQYMGIBdu3ZBpVI1uT5zlFK3WKe3GhIJm8+JqJZEIsH/PRGI5MwinMstxSvfJWH9c314kwrRv5AILbRS26VLl7BgwQLDDJG502q1UKvVKCoqMrn+qTd/TMbGuAy8OKgjZg/uJnY5RGRk0jRaRK08gIpqPV6LvBOT7+0sdklELaY1fn/f9qKd/ys/Px/r1q1rqbejVlTXfB7k7ShuIURklLp6OGDBY7Wz/Ut/P4VD5/JFrojIuLVYmCLTUFWjR9rlYgDcRoaIbm54qA+G9vaGXgCmfZuI3OJKsUsiMloMUxbmdHYxqnR6qK2t4ONkLXY5RGSkJBIJFkYFoIubHXKKK/HKlmPQcf0pogYxTFmY44bFOtl8TkS3ZqOQY9Xo3rC2kmH/2Tys2HNW7JKIjFKj7+YbOnToLZ8vLCxsbi3UBpIzCwHU3slHRPRvurjbY2FUAGZuTcKymNPo4+eE/p1dxS6LyKg0Okyp1bf+5atWqzFmzJhmF0St6/qZKSKixhgW4oO49Hx8F38J0zYfw87pA+Bmz6VmiOo0Okx9+eWXrVkHtYGKah1OZ9c2n3MbGSJqinceC0DSxSKcyi7G1E2J2PhCGOQydooQAeyZsiinNMWo1glwtlXA25HN50TUeNYKGVY90xu2Chni0gvw/m+nxC6JyGgwTFmQ41z5nIiaoVM7O7w/vCcAYM3ec9iVclnkioiMA8OUBUlhvxQRNdPDgZ54YYA/AODVrcdxLrdE5IqIxMcwZUHqZqbYL0VEzfH64K7o6+eMksoaTPomAWVVNWKXRCQqhikLcX3zOWemiKg5rGRSrHi6F1ztlDiVXYw3f0xBC23zSmSSGKYsROplLXR6Aa52Sng48JZmImoeNwcVVj7dCzKpBD8mZuKbuAyxSyISDcOUhUjJ5MrnRNSywjq64PWH7gQALPj5BI5dLBS3ICKRMExZiLrFOtkvRUQtacLAjniohweqdQJe/uYoCkqrxC6JqM0xTFmI5Lo7+RimiKgFSSQSvD88CP6utsgqqsD0zYncEJksDsOUBSirqsGZnNrm80A2nxNRC7NXWWH1MyGwtpJh35k8fPzHabFLImpTDFMW4ORlLfQC4GavhDubz4moFdzpYY9FQwMBAMv3nMWetGyRKyJqOwxTFoCbGxNRW4jq5Y0x4R0AAK9sScLFgjKRKyJqGwxTFqCuXyrQ21HcQojI7L05pBuCfR1RVF6NF78+ivIqndglEbU6hikLcDyTM1NE1DaUchlWje4NVzsFUi9r8fp/j3NBTzJ7DFNmrrSyBv9c3TuLyyIQUVvwcrTGyqd7Qy6VYHtSFj7fly52SUStimHKzJ3I0kIQAE+1Cu3slWKXQ0QWIqyjC956pDsAYNGvJ7H/TJ7IFRG1HoYpM3f8UiEAIJCzUkTUxsaEd8Cw3j7QC8DUbxPYkE5mi2HKzKWwX4qIRCKRSPDuEwEI8lHjShkb0sl8MUyZubrmc/ZLEZEYVFYyrH4mBC62tQ3pb/zAhnQyPwxTZqy4ohrncksB8DIfEYnHy9EaK0fXNqT/dCwL6/azIZ3MC8OUGUvJ1AIAvB2t4WLH5nMiEk+/ji6YO6QbAOD/dp7EgbNsSCfzwTBlxtgvRUTGZGx/P0ND+pRNbEgn88EwZcbYL0VExqSuIT3Qmw3pZF4YpsxY8tVlETgzRUTGQmUlw5pn2ZBO5oVhykwVlVfjfH7tFDqbz4nImNQ1pMuuNqRzhXQydQxTZurE1Ut8vs7WcLRRiFwNEVF91zekL/r1JP48lSNyRUS3j2HKTBk2N/Z2FLcQIqKbeK6/H0aE+kIvANM2JeJsTonYJRHdFoYpM5V8qTZMBbJfioiMlEQiwX+iAtDHzwnFlTV4YcMRFJZViV0WUZMxTJmpZMPMFMMUERkvhVyKT58JgbejNc7nl2HKpkTU6PRil0XUJAxTZqiwrAoZV9dv6cEwRURGztVOic/GhMJGIcP+s3lY+MtJsUsiahKGKTNUNyvl52IDtbWVyNUQEf277l4O+PCpYADA+oPnsSkuQ9yCiJqAYcoMHTf0SzmKWwgRURM8FOCBmQ/cAQCY91MKDp3LF7kiosZhmDJDKeyXIiITNeW+zni0pxdq9AImfXMUGfnccoaMH8OUGaqbmeI2MkRkaiQSCd5/MghBPrVbzjy/4QiKyqvFLovolhimzEx+SSUyC8sBAAHeDiJXQ0TUdCorGdY+GwoPBxXO5pRg8sYEVPMOPzJiDFNmpq75vGM7W9ir2HxORKbJQ63C52Ov3eE3f/sJ7uFHRothysywX4qIzEWAtxrLR/aCRAJsisvAuv3cw4+ME8OUmWG/FBGZk4ju7pg7pDsA4N2dJ/H7CY3IFRHdiGHKzBhWPueyCERkJp6/yw/P9GsPQQCmbz5mmIEnMhYMU2Ykt7gSl4sqIJEAPbzYfE5E5kEikeDtR3tgYBdXlFfrMH7DEVwuKhe7LCIDhikzUvevtU7t7GCrlItcDRFRy5HLpFg5ujfucLdDtrYS49fHo6SyRuyyiAAwTJmVun4pNp8TkTlyUFlh3dg+cLVTIPWylksmkNFgmDIjyZmFAIBAH4YpIjJPvs42WDe2D6ytZPj7dC7e/DGZSyaQ6BimzMi15nOGKSIyXz19HbHi6V6QSoDv4i9hecxZsUsiC8cwZSaytRXI1lZCKgG6ezJMEZF5u7+bO/4TFQAA+OiP0/gu/qLIFZElY5gyE8lX+6W6uNnDWiETuRoiotY3OqwDXr6nEwBg9g/J+Pt0rsgVkaVimDITx69e4mO/FBFZktci70RUsBd0egEvf3OUa1CRKBimzEQK+6WIyAJJJBIsebIn+ndyQWmVDuPWH8GlK2Vil0UWhmHKDAiCwG1kiMhiKeRSrH42BHe62yO3uBJjvziMgtIqscsiC2IUYWrlypXw8/ODSqVCWFgYDh8+fMvxW7duRdeuXaFSqRAYGIidO3fWe14QBMybNw+enp6wtrZGREQEzpw5Y3j+/PnzGD9+PPz9/WFtbY1OnTph/vz5qKoyzf/4NNoK5JVUQiaVoLsnVz4nIsvjoLLCl+P6wFOtwj+5pXh+/RGUclFPaiOih6ktW7YgOjoa8+fPR0JCAnr27InIyEjk5OQ0OP7gwYMYNWoUxo8fj8TERERFRSEqKgopKSmGMUuWLMHy5cuxevVqxMXFwdbWFpGRkaioqAAApKWlQa/XY82aNThx4gQ++ugjrF69GnPmzGmTz9zS6mal7nC3h8qKzedEZJm8HK3x1fN94WhjhWMXC/HSN0dRVcNFPan1SQSRVzsLCwtDnz59sGLFCgCAXq+Hr68vpk6dijfeeOOG8SNGjEBpaSl27NhhONavXz8EBwdj9erVEAQBXl5emDlzJl599VUAQFFREdzd3bF+/XqMHDmywTref/99fPrppzh37lyj6tZqtVCr1SgqKoKDg7izQUt/P4VP9pzFiFBfLH4ySNRaiIjElpBxBaM/i0N5tQ6P9vTCxyOCIZVKxC6LjERr/P4WdWaqqqoKR48eRUREhOGYVCpFREQEYmNjG3xNbGxsvfEAEBkZaRifnp4OjUZTb4xarUZYWNhN3xOoDVzOzs43fb6yshJarbbew1gY+qXYfE5EhN7tnbD62RDIpRL8nJSFd34+wVXSqVWJGqby8vKg0+ng7u5e77i7uzs0Gk2Dr9FoNLccX/dnU97z7Nmz+OSTT/Diiy/etNZFixZBrVYbHr6+vrf+cG1EEIRrK5+z+ZyICAAw6I52WPpUTwDAhtgLWLGHq6RT6xG9Z0psmZmZeOihhzB8+HBMmDDhpuNmz56NoqIiw+PiReNYbTerqAIFpVWwkknQ1dNe7HKIiIzG48HeePvR7gCApbtPY2PcBZErInMlaphydXWFTCZDdnZ2vePZ2dnw8PBo8DUeHh63HF/3Z2PeMysrC/feey/69++PtWvX3rJWpVIJBweHeg9jkHypEEBt87lSzuZzIqLrPXeXP6bd1xkAMHdbCn45flnkisgciRqmFAoFQkJCEBMTYzim1+sRExOD8PDwBl8THh5ebzwA7N692zDe398fHh4e9cZotVrExcXVe8/MzEzcc889CAkJwZdffgmp1DQn6er6pbhYJxFRw1554A6MDmsPQQCmb05EzMnsf38RUROIniCio6Px2WefYcOGDTh58iQmTZqE0tJSjBs3DgAwZswYzJ492zB++vTp2LVrF5YuXYq0tDS8/fbbiI+Px5QpUwDUroY7Y8YMLFy4ENu3b0dycjLGjBkDLy8vREVFAbgWpNq3b48PPvgAubm50Gg0N+2pMmZ1/VKB3o7iFkJEZKQkEgkWPB6Ax4O9UKMXMGljAg6czRO7LDIjcrELGDFiBHJzczFv3jxoNBoEBwdj165dhgbyjIyMerNG/fv3x6ZNmzB37lzMmTMHXbp0wbZt2xAQEGAYM2vWLJSWlmLixIkoLCzEgAEDsGvXLqhUKgC1M1lnz57F2bNn4ePjU68eU7rjo17zOWemiIhuSiaV4IPhPVFepcPvqdl4YUM8vh7fF6F+N7+Lm6ixRF9nylQZwzpTFwvKMHDJn1DIpEh5JxIKuegTjURERq2yRoeJXx3F36dzYa+UY+OEMAT5OIpdFrUhs1tnipqnrl+qq6c9gxQRUSMo5TKsfiYEYf7OKK6swZgvDiNNYzzrBpJp4m9gE3Y8sxAAEMj1pYiIGs1aIcO65/og2NcRhWXVeObzwziXWyJ2WWTCGKZMWAr7pYiIboudUo4N4/qiu6cD8koqMfrzOGTkl4ldFpkohikTJQjCtW1kODNFRNRkahsrfD2+Lzq72eFyUQVGro1loKLbwjBloi7kl6G4ogYKuRR3uHPlcyKi2+Fip8SmF8LQsZ0tsq4Gqgv5pWKXRSaGYcpEHb96ia+7pwOsZPy/kYjodrk5qLB5Qj90MgSqQwxU1CT8LWyi2C9FRNRy3BxU+HZibaC6fDVQnc9joKLGYZgyUcev7snHfikiopbhZl8/UI36jIGKGodhygTp9QJSMmvXReHMFBFRy6kLVNea0hmo6N8xTJmg8/mlKKmsgcpKis7t7MQuh4jIrLjZq/DthNpApdFWYMTaWJzNKRa7LDJiDFMmKPm65nM5m8+JiFpcO3slvp3QD13c7JCtrcRTaw4ZelWJ/hd/E5uguvWluJ8UEVHraWevxJYXwxHorUZBaRVGfXYI8ecLxC6LjBDDlAlKvhqmuI0MEVHrcrZVYOOEMPTxc0JxRQ2eXXcY+87kil0WGRmGKROj0ws4kcVlEYiI2oqDygpfPR+GQXe0Q3m1DuPXx+O3ExqxyyIjwjBlYtLzSlBapYONQoaObD4nImoT1goZPhsTisEBHqjS6fHyxgT8mHhJ7LLISDBMmZi6fqkeXg6QSSUiV0NEZDkUcik+GdULT4b4QKcXEP1dEr6KPS92WWQEGKZMzHFDv5SjuIUQEVkguUyKJcOC8Fx/PwgCMO+nE3j/tzQIgiB2aSQihikTw21kiIjEJZVKMP/R7ngl4g4AwMo//8HMrUmo1ulFrozEwjBlQmp0epzIql35nNvIEBGJRyKRYHpEFywZFgSZVIIfEjLx/PojKKmsEbs0EgHDlAn5J7cU5dU62Cpk6OhqK3Y5REQW76k+vvh8bCisrWTYdyYPI9bEIqe4QuyyqI0xTJmQupXPA7zVkLL5nIjIKNx7pxs2T+wHF1sFTmRpMXTVQfyTWyJ2WdSGGKZMSPKlQgDslyIiMjY9fR3xw8v94edig0tXyvHkpwdxhKulWwyGKRNy/LqZKSIiMi4dXGzx30n90dPXEVfKqvH0Z4ewNf6i2GVRG2CYMhE1Oj1Srzafc08+IiLj5GKnxLcTwjA4wAPVOgGvfX8c/7fzJHR6Lp1gzhimTMSZnBJU1uhhr5Kjg7ON2OUQEdFN2CjkWPl0b0y7rzMAYO3ec5j4VTyKK6pFroxaC8OUibh+c2M2nxMRGTepVILoB+/E8lG9oJRLEZOWg2GfHsTFgjKxS6NWwDBlIo5nFgKoDVNERGQaHuvphS0vhsPNXonT2SV4fOUBHE5nY7q5YZgyEYaZKd7JR0RkUoJ9HbF9ygAEeqtRUFqF0Z8fwoaD57kFjRlhmDIBVTV6nNQUAwCCuCcfEZHJ8VCr8N2L4XgkyBPVOgHzt5/AtM3HUMoV080Cw5QJOJ1djKoaPdTWVvB1tha7HCIiug3WChk+GdULbz3SHXKpBD8nZeHxlQdwNqdY7NKomRimTEDdyueB3mpIJGw+JyIyVRKJBOMH+GPzxH5wd1DibE4JHltxAD8nZYldGjUDw5QJMIQp9ksREZmFUD9n7Jg6EOEdXVBWpcPUbxPx9vYTqKrRi10a3QaGKRNQ13wexDv5iIjMRjt7Jb4e3xcv39MJALD+4Hk8tSYWF/JLRa6MmophyshV1uiQpqld+ZzbyBARmRe5TIpZD3XFZ2NC4aCS49jFQjz88T5sjb/Iu/1MCMOUkTulKUa1ToCTjRV8nNh8TkRkjh7o7o6d0weir78zSqt0eO3745iyKRGFZVVil0aNwDBl5K71Szmy+ZyIyIz5ONng2wn98FrknZBLJfgl+TIeWrYPB//JE7s0+hcMU0aO/VJERJZDJpVg8r2d8cPL/eHvaguNtgKjP4/Dop0nUVmjE7s8ugmGKSN3/GqYYr8UEZHlCPJxxC/TBmBUX18IArBm7zk89skBJGZcEbs0agDDlBGrqNbhdPbVlc+5LAIRkUWxUcixaGgQ1jwbAmdbBU5lF2Popwex4OdUlFVx5XRjwjBlxNI0xajRC3C1U8BTrRK7HCIiEkFkDw/8ET0IT/TyhiAAXxxIx4Mf7cXe07lil0ZXMUwZseRLhQC48jkRkaVztlXgoxHBWD+uD7wdrXHpSjnGfHEY0d8dw5VS3vEnNoYpI1bXLxXIfikiIgJwz51u+P2Vu/Fcfz9IJMAPCZl44KO/8V38Rej1XJdKLAxTRuz6ZRGIiIgAwFYpx9uP9cD3L/VHFzc75JVUYdb3xxG16gCOXigQuzyLxDBlpMqrdDiTUwKAzedERHSjkA5O+GXaQMx5uCvslHIcv1SEYZ/GYtq3icgqLBe7PIvCMGWkUi9rodMLaGevhLsDm8+JiOhGCrkUE+/uhD9fvQcj+/hCIgG2J2XhvqV/4eM/zqC8imtTtQWGKSNV13zOxTqJiOjftLNX4r1hQfh5ygD08XNCRbUeH/1xGvct/Qub4jJQrdOLXaJZY5gyUsmZtZsbB/ISHxERNVKAtxrfvRiOT0b1gpdahctFFZjzYzLuW/oXvou/iBqGqlbBMGWkkjMLAbBfioiImkYikeDRnl7Y8+o9mPdId7jaKXGxoByzvj+OiA//xo+Jl6DjnX8timHKCJVW1uDs1eZzbiNDRES3Q2Ulw/MD/LFv1r2Y83BXONsqcD6/DK9sScKDH9WGqqoazlS1BIYpI5R6WQu9AHg4qOBmz+ZzIiK6fdYKGSbe3Qn7Zt2LWQ/dCUcbK/yTW4pXtiRh4JI9WPnnWS782UwMU0YouW6xTl7iIyKiFmKrlOPlezpj36x78VrknXCzVyJbW4n3fzuFfotiMPuHZJy5uh8sNY1c7ALoRnWLdfJOPiIiamn2KitMvrczJgzsiF+Ss7BufzpSMrX49nAGvj2cgbvvaIfRYe1xX1c3WMk459IYDFNG6PjVZRECODNFREStRCGX4olePogK9saR81ewbv85/J6ajb2nc7H3dC5c7ZQY1tsbw0N90NnNXuxyjRrDlJEpqazBubxSANyTj4iIWp9EIkFff2f09XdGRn4ZNsZdwH8TLiGvpBJr9p7Dmr3n0N3TAY8Fe+HRnl7wdrQWu2SjIxEEgfdH3gatVgu1Wo2ioiI4ODi02PvGncvHiLWH4O1ojQNv3Ndi70tERNRY1To99qTlYGv8Rfx1Khc11y2l0NNHjQd7eODB7u7o7GYHiUQiYqVN1xq/vzkzZWQMmxtzVoqIiERiJZMisocHInt44EppFXamXMZPx7Jw5HwBki4VIelSEd7/7RR8na1xd5d2uPuOdujfyQX2KiuxSxcFw5SROc47+YiIyIg42SowOqwDRod1QE5xBf5IzcHvqRocPJuPiwXl2BiXgY1xGZBKgO5eDujr54K+/k7o6esIDweVyc1c3Q6jaNNfuXIl/Pz8oFKpEBYWhsOHD99y/NatW9G1a1eoVCoEBgZi586d9Z4XBAHz5s2Dp6cnrK2tERERgTNnztQbU1BQgNGjR8PBwQGOjo4YP348SkpKWvyzNVVheTUAzkwREZHxcbNX4emw9lg/ri8S5z2AdWNDMTa8A/xdbaEXgJRMLb44kI6XvklA+KI96PNuDJ5ffwTv/5aGn45lIjVLi4pq89t8WfSeqS1btmDMmDFYvXo1wsLCsGzZMmzduhWnTp2Cm5vbDeMPHjyIu+++G4sWLcIjjzyCTZs2YfHixUhISEBAQAAAYPHixVi0aBE2bNgAf39/vPXWW0hOTkZqaipUqtpFMAcPHozLly9jzZo1qK6uxrhx49CnTx9s2rSpUXW3Vs8UABSVV0NlJYVSLmvR9yUiImotmqIKHD5fgMPp+Yg/fwVnckpuum2Nh4MKvs7W8HWyQTsHJdrZKdHOXglHGwXsVXLYK+WwU8nhYquEQt6y8z6t8ftb9DAVFhaGPn36YMWKFQAAvV4PX19fTJ06FW+88cYN40eMGIHS0lLs2LHDcKxfv34IDg7G6tWrIQgCvLy8MHPmTLz66qsAgKKiIri7u2P9+vUYOXIkTp48ie7du+PIkSMIDQ0FAOzatQsPP/wwLl26BC8vrxu+b2VlJSorKw1fa7Va+Pr6tkqYIiIiMnXlVTqkXtYi+VIhTmWX4Ex2MU5nF0NbUdPo9/h8TCgiuru3aF1m14BeVVWFo0ePYvbs2YZjUqkUERERiI2NbfA1sbGxiI6OrncsMjIS27ZtAwCkp6dDo9EgIiLC8LxarUZYWBhiY2MxcuRIxMbGwtHR0RCkACAiIgJSqRRxcXF44oknbvi+ixYtwjvvvNOcj0tERGQxrBUyhHRwQkgHJ8MxQRBwpawaGQVluFhQhktXypFbXInckkrkFlegqLwGJZXVKKmoQXFFDexVptHaLWqVeXl50Ol0cHevnzrd3d2RlpbW4Gs0Gk2D4zUajeH5umO3GvO/lxDlcjmcnZ0NY/7X7Nmz64W4upkpIiIiahyJRAJnWwWcbRUI9nW85VhTWrnJNCKfEVAqlVAqlWKXQUREZBFM6S5AUe/mc3V1hUwmQ3Z2dr3j2dnZ8PDwaPA1Hh4etxxf9+e/jcnJyan3fE1NDQoKCm76fYmIiIgaImqYUigUCAkJQUxMjOGYXq9HTEwMwsPDG3xNeHh4vfEAsHv3bsN4f39/eHh41Buj1WoRFxdnGBMeHo7CwkIcPXrUMGbPnj3Q6/UICwtrsc9HRERE5k/0y3zR0dEYO3YsQkND0bdvXyxbtgylpaUYN24cAGDMmDHw9vbGokWLAADTp0/HoEGDsHTpUgwZMgSbN29GfHw81q5dC6B2WnDGjBlYuHAhunTpYlgawcvLC1FRUQCAbt264aGHHsKECROwevVqVFdXY8qUKRg5cmSDd/IRERER3YzoYWrEiBHIzc3FvHnzoNFoEBwcjF27dhkayDMyMiCVXptA69+/PzZt2oS5c+dizpw56NKlC7Zt22ZYYwoAZs2ahdLSUkycOBGFhYUYMGAAdu3aZVhjCgA2btyIKVOm4P7774dUKsWwYcOwfPnytvvgREREZBZEX2fKVLXmop1ERETUOlrj97dRbCdDREREZKoYpoiIiIiagWGKiIiIqBkYpoiIiIiagWGKiIiIqBkYpoiIiIiagWGKiIiIqBkYpoiIiIiaQfQV0E1V3VqnWq1W5EqIiIiosep+b7fkmuUMU7epuLgYAODr6ytyJURERNRUxcXFUKvVLfJe3E7mNun1emRlZcHe3h4SiaTF3ler1cLX1xcXL17kNjXg+bgez8U1PBf18Xxcw3NxDc/FNdefC3t7exQXF8PLy6ve3r/NwZmp2ySVSuHj49Nq7+/g4GDxf/mvx/NxDc/FNTwX9fF8XMNzcQ3PxTV156KlZqTqsAGdiIiIqBkYpoiIiIiagWHKyCiVSsyfPx9KpVLsUowCz8c1PBfX8FzUx/NxDc/FNTwX17T2uWADOhEREVEzcGaKiIiIqBkYpoiIiIiagWGKiIiIqBkYpoiIiIiagWHKyKxcuRJ+fn5QqVQICwvD4cOHxS6p1S1atAh9+vSBvb093NzcEBUVhVOnTtUbU1FRgcmTJ8PFxQV2dnYYNmwYsrOzRaq47bz33nuQSCSYMWOG4ZglnYvMzEw888wzcHFxgbW1NQIDAxEfH294XhAEzJs3D56enrC2tkZERATOnDkjYsWtR6fT4a233oK/vz+sra3RqVMn/Oc//6m3v5i5no+9e/fi0UcfhZeXFyQSCbZt21bv+cZ87oKCAowePRoODg5wdHTE+PHjUVJS0oafomXc6lxUV1fj9ddfR2BgIGxtbeHl5YUxY8YgKyur3nuYy7kA/v3vxvVeeuklSCQSLFu2rN7xljgfDFNGZMuWLYiOjsb8+fORkJCAnj17IjIyEjk5OWKX1qr+/vtvTJ48GYcOHcLu3btRXV2NBx98EKWlpYYxr7zyCn7++Wds3boVf//9N7KysjB06FARq259R44cwZo1axAUFFTvuKWciytXruCuu+6ClZUVfv31V6SmpmLp0qVwcnIyjFmyZAmWL1+O1atXIy4uDra2toiMjERFRYWIlbeOxYsX49NPP8WKFStw8uRJLF68GEuWLMEnn3xiGGOu56O0tBQ9e/bEypUrG3y+MZ979OjROHHiBHbv3o0dO3Zg7969mDhxYlt9hBZzq3NRVlaGhIQEvPXWW0hISMAPP/yAU6dO4bHHHqs3zlzOBfDvfzfq/Pjjjzh06BC8vLxueK5FzodARqNv377C5MmTDV/rdDrBy8tLWLRokYhVtb2cnBwBgPD3338LgiAIhYWFgpWVlbB161bDmJMnTwoAhNjYWLHKbFXFxcVCly5dhN27dwuDBg0Spk+fLgiCZZ2L119/XRgwYMBNn9fr9YKHh4fw/vvvG44VFhYKSqVS+Pbbb9uixDY1ZMgQ4fnnn693bOjQocLo0aMFQbCc8wFA+PHHHw1fN+Zzp6amCgCEI0eOGMb8+uuvgkQiETIzM9us9pb2v+eiIYcPHxYACBcuXBAEwXzPhSDc/HxcunRJ8Pb2FlJSUoQOHToIH330keG5ljofnJkyElVVVTh69CgiIiIMx6RSKSIiIhAbGytiZW2vqKgIAODs7AwAOHr0KKqrq+udm65du6J9+/Zme24mT56MIUOG1PvMgGWdi+3btyM0NBTDhw+Hm5sbevXqhc8++8zwfHp6OjQaTb1zoVarERYWZnbnAgD69++PmJgYnD59GgCQlJSE/fv3Y/DgwQAs73zUacznjo2NhaOjI0JDQw1jIiIiIJVKERcX1+Y1t6WioiJIJBI4OjoCsLxzodfr8eyzz+K1115Djx49bni+pc4HNzo2Enl5edDpdHB3d6933N3dHWlpaSJV1fb0ej1mzJiBu+66CwEBAQAAjUYDhUJh+GFQx93dHRqNRoQqW9fmzZuRkJCAI0eO3PCcJZ2Lc+fO4dNPP0V0dDTmzJmDI0eOYNq0aVAoFBg7dqzh8zb034y5nQsAeOONN6DVatG1a1fIZDLodDq8++67GD16NABY3Pmo05jPrdFo4ObmVu95uVwOZ2dnsz43FRUVeP311zFq1CjDRseWdi4WL14MuVyOadOmNfh8S50PhikyKpMnT0ZKSgr2798vdimiuHjxIqZPn47du3dDpVKJXY6o9Ho9QkND8X//938AgF69eiElJQWrV6/G2LFjRa6u7X333XfYuHEjNm3ahB49euDYsWOYMWMGvLy8LPJ80K1VV1fjqaeegiAI+PTTT8UuRxRHjx7Fxx9/jISEBEgkklb9XrzMZyRcXV0hk8luuCsrOzsbHh4eIlXVtqZMmYIdO3bgzz//hI+Pj+G4h4cHqqqqUFhYWG+8OZ6bo0ePIicnB71794ZcLodcLsfff/+N5cuXQy6Xw93d3WLOhaenJ7p3717vWLdu3ZCRkQEAhs9rKf/NvPbaa3jjjTcwcuRIBAYG4tlnn8Urr7yCRYsWAbC881GnMZ/bw8Pjhht5ampqUFBQYJbnpi5IXbhwAbt37zbMSgGWdS727duHnJwctG/f3vDz9MKFC5g5cyb8/PwAtNz5YJgyEgqFAiEhIYiJiTEc0+v1iImJQXh4uIiVtT5BEDBlyhT8+OOP2LNnD/z9/es9HxISAisrq3rn5tSpU8jIyDC7c3P//fcjOTkZx44dMzxCQ0MxevRow/+2lHNx11133bBExunTp9GhQwcAgL+/Pzw8POqdC61Wi7i4OLM7F0DtnVpSaf0f2TKZDHq9HoDlnY86jfnc4eHhKCwsxNGjRw1j9uzZA71ej7CwsDavuTXVBakzZ87gjz/+gIuLS73nLelcPPvsszh+/Hi9n6deXl547bXX8NtvvwFowfNx+33z1NI2b94sKJVKYf369UJqaqowceJEwdHRUdBoNGKX1qomTZokqNVq4a+//hIuX75seJSVlRnGvPTSS0L79u2FPXv2CPHx8UJ4eLgQHh4uYtVt5/q7+QTBcs7F4cOHBblcLrz77rvCmTNnhI0bNwo2NjbCN998Yxjz3nvvCY6OjsJPP/0kHD9+XHj88ccFf39/oby8XMTKW8fYsWMFb29vYceOHUJ6errwww8/CK6ursKsWbMMY8z1fBQXFwuJiYlCYmKiAED48MMPhcTERMMdao353A899JDQq1cvIS4uTti/f7/QpUsXYdSoUWJ9pNt2q3NRVVUlPPbYY4KPj49w7Nixej9PKysrDe9hLudCEP7978b/+t+7+QShZc4Hw5SR+eSTT4T27dsLCoVC6Nu3r3Do0CGxS2p1ABp8fPnll4Yx5eXlwssvvyw4OTkJNjY2whNPPCFcvnxZvKLb0P+GKUs6Fz///LMQEBAgKJVKoWvXrsLatWvrPa/X64W33npLcHd3F5RKpXD//fcLp06dEqna1qXVaoXp06cL7du3F1QqldCxY0fhzTffrPdL0lzPx59//tngz4ixY8cKgtC4z52fny+MGjVKsLOzExwcHIRx48YJxcXFInya5rnVuUhPT7/pz9M///zT8B7mci4E4d//bvyvhsJUS5wPiSBct3wuERERETUJe6aIiIiImoFhioiIiKgZGKaIiIiImoFhioiIiKgZGKaIiIiImoFhioiIiKgZGKaIiIiImoFhioiIiKgZGKaIiIiImoFhiojMWm5uLiZNmoT27dtDqVTCw8MDkZGROHDgAABAIpFg27Zt4hZJRCZNLnYBREStadiwYaiqqsKGDRvQsWNHZGdnIyYmBvn5+WKXRkRmgnvzEZHZKiwshJOTE/766y8MGjTohuf9/Pxw4cIFw9cdOnTA+fPnAQA//fQT3nnnHaSmpsLLywtjx47Fm2++Cbm89t+gEokEq1atwvbt2/HXX3/B09MTS5YswZNPPtkmn42IjAcv8xGR2bKzs4OdnR22bduGysrKG54/cuQIAODLL7/E5cuXDV/v27cPY8aMwfTp05Gamoo1a9Zg/fr1ePfdd+u9/q233sKwYcOQlJSE0aNHY+TIkTh58mTrfzAiMiqcmSIis/bf//4XEyZMQHl5OXr37o1BgwZh5MiRCAoKAlA7w/Tjjz8iKirK8JqIiAjcf//9mD17tuHYN998g1mzZiErK8vwupdeegmffvqpYUy/fv3Qu3dvrFq1qm0+HBEZBc5MEZFZGzZsGLKysrB9+3Y89NBD+Ouvv9C7d2+sX7/+pq9JSkrCggULDDNbdnZ2mDBhAi5fvoyysjLDuPDw8HqvCw8P58wUkQViAzoRmT2VSoUHHngADzzwAN566y288MILmD9/Pp577rkGx5eUlOCdd97B0KFDG3wvIqLrcWaKiCxO9+7dUVpaCgCwsrKCTqer93zv3r1x6tQpdO7c+YaHVHrtx+ahQ4fqve7QoUPo1q1b638AIjIqnJkiIrOVn5+P4cOH4/nnn0dQUBDs7e0RHx+PJUuW4PHHHwdQe0dfTEwM7rrrLiiVSjg5OWHevHl45JFH0L59ezz55JOQSqVISkpCSkoKFi5caHj/rVu3IjQ0FAMGDMDGjRtx+PBhrFu3TqyPS0QiYQM6EZmtyspKvP322/j999/xzz//oLq6Gr6+vhg+fDjmzJkDa2tr/Pzzz4iOjsb58+fh7e1tWBrht99+w4IFC5CYmAgrKyt07doVL7zwAiZMmACgtgF95cqV2LZtG/bu3QtPT08sXrwYTz31lIifmIjEwDBFRHQbGroLkIgsE3umiIiIiJqBYYqIiIioGdiATkR0G9ghQUR1ODNFRERE1AwMU0RERETNwDBFRERE1AwMU0RERETNwDBFRERE1AwMU0RERETNwDBFRERE1AwMU0RERETN8P9tmurV9DpCmQAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.ylabel(\"Learning rate\")\n",
    "plt.xlabel(\"Step\")\n",
    "plt.plot(range(total_training_steps), track_lrs);"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e7512808-b48d-4146-86a1-5931b1e3aec1",
   "metadata": {},
   "source": [
    "## D.3 Gradient clipping"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c0a74f76-8d2b-4974-a03c-d645445cdc21",
   "metadata": {},
   "source": [
    "- Gradient clipping is yet another technique used to stabilize the training when training LLMs\n",
    "- By setting a threshold, gradients exceeding this limit are scaled down to a maximum magnitude to ensure that the updates to the model's parameters during backpropagation remain within a manageable range\n",
    "- For instance, using the `max_norm=1.0` setting in PyTorch's `clip_grad_norm_` method means that the norm of the gradients is clipped such that their maximum norm does not exceed 1.0\n",
    "- the \"norm\" refers to a measure of the gradient vector's length (or magnitude) in the parameter space of the model\n",
    "- Specifically, it's the L2 norm, also known as the Euclidean norm\n",
    "- Mathematically, for a vector $\\mathbf{v}$ with components $\\mathbf{v} = [v_1, v_2, \\ldots, v_n]$, the L2 norm is defined as:\n",
    "$$\n",
    "\\| \\mathbf{v} \\|_2 = \\sqrt{v_1^2 + v_2^2 + \\ldots + v_n^2}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d44838a6-4322-47b2-a935-c00d3a88355f",
   "metadata": {},
   "source": [
    "- The L2 norm is calculated similarly for matrices.\n",
    "- Let's assume our gradient matrix is:\n",
    "$$\n",
    "G = \\begin{bmatrix}\n",
    "1 & 2 \\\\\n",
    "2 & 4\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "- And we want to clip these gradients with a `max_norm` of 1.\n",
    "\n",
    "- First, we calculate the L2 norm of these gradients:\n",
    "$$\n",
    "\\|G\\|_2 = \\sqrt{1^2 + 2^2 + 2^2 + 4^2} = \\sqrt{25} = 5\n",
    "$$\n",
    "\n",
    "- Since $\\|G\\|_2 = 5$ is greater than our `max_norm` of 1, we need to scale down the gradients so that their norm is exactly 1. The scaling factor is calculated as $\\frac{max\\_norm}{\\|G\\|_2} = \\frac{1}{5}$.\n",
    "\n",
    "- Therefore, the scaled gradient matrix $G'$ will be as follows:\n",
    "$$\n",
    "G' = \\frac{1}{5} \\times G = \\begin{bmatrix}\n",
    "\\frac{1}{5} & \\frac{2}{5} \\\\\n",
    "\\frac{2}{5} & \\frac{4}{5}\n",
    "\\end{bmatrix}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eeb0c3c1-2cff-46f5-8127-24412184428c",
   "metadata": {},
   "source": [
    "- Let's see this in action\n",
    "- First, we initialize a new model and calculate the loss for a training batch like we would do in the regular training loop"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "e199e1ff-58c4-413a-855e-5edbe9292649",
   "metadata": {},
   "outputs": [],
   "source": [
    "from previous_chapters import calc_loss_batch\n",
    "\n",
    "torch.manual_seed(123)\n",
    "model = GPTModel(GPT_CONFIG_124M)\n",
    "\n",
    "loss = calc_loss_batch(input_batch, target_batch, model, device)\n",
    "loss.backward()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "76b60f3a-15ec-4846-838d-fdef3df99899",
   "metadata": {},
   "source": [
    "- If we call `.backward()`, PyTorch will calculate the gradients and store them in a `.grad` attribute for each weight (parameter) matrix\n",
    "- Let's define a utility function to calculate the highest gradient based on all model weights"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "e70729a3-24d1-411d-a002-2529cd3a8a9e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(0.0373)\n"
     ]
    }
   ],
   "source": [
    "def find_highest_gradient(model):\n",
    "    max_grad = None\n",
    "    for param in model.parameters():\n",
    "        if param.grad is not None:\n",
    "            grad_values = param.grad.data.flatten()\n",
    "            max_grad_param = grad_values.max()\n",
    "            if max_grad is None or max_grad_param > max_grad:\n",
    "                max_grad = max_grad_param\n",
    "    return max_grad\n",
    "\n",
    "print(find_highest_gradient(model))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "734f30e6-6b24-4d4b-ae91-e9a4b871113f",
   "metadata": {},
   "source": [
    "- Applying gradient clipping, we can see that the largest gradient is now substantially smaller:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "fa81ef8b-4280-400f-a93e-5210f3e62ff0",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(0.0166)\n"
     ]
    }
   ],
   "source": [
    "torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)\n",
    "print(find_highest_gradient(model))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b62c2af0-dac3-4742-be4b-4292c6753099",
   "metadata": {},
   "source": [
    "## D.4 The modified training function"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "76715332-94ec-4185-922a-75cb420819d5",
   "metadata": {},
   "source": [
    "- Now let's add the three concepts covered above (learning rate warmup, cosine decay, and gradient clipping) to the `train_model_simple` function covered in chapter 5 to create the more sophisticated `train_model` function below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "46eb9c84-a293-4016-a523-7ad726e171e9",
   "metadata": {},
   "outputs": [],
   "source": [
    "from previous_chapters import evaluate_model, generate_and_print_sample\n",
    "\n",
    "\n",
    "def train_model(model, train_loader, val_loader, optimizer, device, n_epochs,\n",
    "                eval_freq, eval_iter, start_context, warmup_steps=10,\n",
    "                initial_lr=3e-05, min_lr=1e-6):\n",
    "\n",
    "    train_losses, val_losses, track_tokens_seen, track_lrs = [], [], [], []\n",
    "    tokens_seen, global_step = 0, -1\n",
    "\n",
    "    # Retrieve the maximum learning rate from the optimizer\n",
    "    peak_lr = optimizer.param_groups[0][\"lr\"]\n",
    "\n",
    "    # Calculate the total number of iterations in the training process\n",
    "    total_training_steps = len(train_loader) * n_epochs\n",
    "\n",
    "    # Calculate the learning rate increment during the warmup phase\n",
    "    lr_increment = (peak_lr - initial_lr) / warmup_steps\n",
    "\n",
    "    for epoch in range(n_epochs):\n",
    "        model.train()\n",
    "        for input_batch, target_batch in train_loader:\n",
    "            optimizer.zero_grad()\n",
    "            global_step += 1\n",
    "\n",
    "            # Adjust the learning rate based on the current phase (warmup or cosine annealing)\n",
    "            if global_step < warmup_steps:\n",
    "                # Linear warmup\n",
    "                lr = initial_lr + global_step * lr_increment  \n",
    "            else:\n",
    "                # Cosine annealing after warmup\n",
    "                progress = ((global_step - warmup_steps) / \n",
    "                            (total_training_steps - warmup_steps))\n",
    "                lr = min_lr + (peak_lr - min_lr) * 0.5 * (1 + math.cos(math.pi * progress))\n",
    "\n",
    "            # Apply the calculated learning rate to the optimizer\n",
    "            for param_group in optimizer.param_groups:\n",
    "                param_group[\"lr\"] = lr\n",
    "            track_lrs.append(lr)  # Store the current learning rate\n",
    "\n",
    "            # Calculate and backpropagate the loss\n",
    "            loss = calc_loss_batch(input_batch, target_batch, model, device)\n",
    "            loss.backward()\n",
    "\n",
    "            # Apply gradient clipping after the warmup phase to avoid exploding gradients\n",
    "            if global_step > warmup_steps:\n",
    "                torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)\n",
    "            \n",
    "            optimizer.step()\n",
    "            tokens_seen += input_batch.numel()\n",
    "\n",
    "            # Periodically evaluate the model on the training and validation sets\n",
    "            if global_step % eval_freq == 0:\n",
    "                train_loss, val_loss = evaluate_model(\n",
    "                    model, train_loader, val_loader,\n",
    "                    device, eval_iter\n",
    "                )\n",
    "                train_losses.append(train_loss)\n",
    "                val_losses.append(val_loss)\n",
    "                track_tokens_seen.append(tokens_seen)\n",
    "                # Print the current losses\n",
    "                print(f\"Ep {epoch+1} (Iter {global_step:06d}): \"\n",
    "                      f\"Train loss {train_loss:.3f}, Val loss {val_loss:.3f}\")\n",
    "\n",
    "        # Generate and print a sample from the model to monitor progress\n",
    "        generate_and_print_sample(\n",
    "            model, train_loader.dataset.tokenizer,\n",
    "            device, start_context\n",
    "        )\n",
    "\n",
    "    return train_losses, val_losses, track_tokens_seen, track_lrs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "55fcd247-ba9d-4b93-a757-0f7ce04fee41",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Ep 1 (Iter 000000): Train loss 10.914, Val loss 10.940\n",
      "Ep 1 (Iter 000005): Train loss 8.903, Val loss 9.313\n",
      "Every effort moves you,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,\n",
      "Ep 2 (Iter 000010): Train loss 7.362, Val loss 7.789\n",
      "Ep 2 (Iter 000015): Train loss 6.273, Val loss 6.814\n",
      "Every effort moves you,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,\n",
      "Ep 3 (Iter 000020): Train loss 5.958, Val loss 6.609\n",
      "Ep 3 (Iter 000025): Train loss 5.675, Val loss 6.592\n",
      "Every effort moves you.                                                 \n",
      "Ep 4 (Iter 000030): Train loss 5.607, Val loss 6.565\n",
      "Ep 4 (Iter 000035): Train loss 5.063, Val loss 6.483\n",
      "Every effort moves you, and, and the to the to the to the to the to the, and, and the, and the, and, and, and the, and the, and, and the, and, and, and the, and, and the\n",
      "Ep 5 (Iter 000040): Train loss 4.384, Val loss 6.379\n",
      "Every effort moves you, I was, and I had been.                   \"I, I had the picture, as a little's his pictures, I had been, I was his\n",
      "Ep 6 (Iter 000045): Train loss 4.638, Val loss 6.306\n",
      "Ep 6 (Iter 000050): Train loss 3.690, Val loss 6.196\n",
      "Every effort moves you know the to me a little of his pictures--I had been.  \"I was the's--and, I felt to see a little of his pictures--I had been. \"I of Jack's \"strong. \"I\n",
      "Ep 7 (Iter 000055): Train loss 3.157, Val loss 6.148\n",
      "Ep 7 (Iter 000060): Train loss 2.498, Val loss 6.157\n",
      "Every effort moves you know it was not that, and he was to the fact of the of a and he was--his, the fact of the donkey, in the of the his head to have.   \"I had been his pictures--and by his\n",
      "Ep 8 (Iter 000065): Train loss 2.182, Val loss 6.178\n",
      "Ep 8 (Iter 000070): Train loss 1.998, Val loss 6.193\n",
      "Every effort moves you know,\" was not that my dear, his pictures--so handsome, in a so that he was a year after Jack's resolve had been his painting.     \"Oh, I had the donkey. \"There were, with his\n",
      "Ep 9 (Iter 000075): Train loss 1.824, Val loss 6.211\n",
      "Ep 9 (Iter 000080): Train loss 1.742, Val loss 6.201\n",
      "Every effort moves you know,\" was not that my hostess was \"interesting\": on that point I could have given Miss Croft the fact, and.         \"Oh, as I turned, and down the room, in his\n",
      "Ep 10 (Iter 000085): Train loss 1.285, Val loss 6.234\n",
      "Every effort moves you?\"  \"Yes--quite insensible to the fact with a little: \"Yes--and by me to me to have to see a smile behind his close grayish beard--as if he had the donkey. \"There were days when I\n",
      "Ep 11 (Iter 000090): Train loss 1.256, Val loss 6.236\n",
      "Ep 11 (Iter 000095): Train loss 0.803, Val loss 6.255\n",
      "Every effort moves you?\"  \"Yes--quite insensible to the irony. She wanted him vindicated--and by me!\"  He laughed again, and threw back his head to look up at the sketch of the donkey. \"There were days when I\n",
      "Ep 12 (Iter 000100): Train loss 0.731, Val loss 6.284\n",
      "Ep 12 (Iter 000105): Train loss 0.889, Val loss 6.299\n",
      "Every effort moves you?\"  \"Yes--quite insensible to the irony. She wanted him vindicated--and by me!\"  He laughed again, and threw back his head to look up at the sketch of the donkey. \"There were days when I\n",
      "Ep 13 (Iter 000110): Train loss 0.703, Val loss 6.316\n",
      "Ep 13 (Iter 000115): Train loss 0.517, Val loss 6.315\n",
      "Every effort moves you?\"  \"Yes--quite insensible to the irony. She wanted him vindicated--and by me!\"  He laughed again, and threw back his head to look up at the sketch of the donkey. \"There were days when I\n",
      "Ep 14 (Iter 000120): Train loss 0.594, Val loss 6.324\n",
      "Ep 14 (Iter 000125): Train loss 0.481, Val loss 6.325\n",
      "Every effort moves you?\"  \"Yes--quite insensible to the irony. She wanted him vindicated--and by me!\"  He laughed again, and threw back his head to look up at the sketch of the donkey. \"There were days when I\n",
      "Ep 15 (Iter 000130): Train loss 0.529, Val loss 6.324\n",
      "Every effort moves you?\"  \"Yes--quite insensible to the irony. She wanted him vindicated--and by me!\"  He laughed again, and threw back his head to look up at the sketch of the donkey. \"There were days when I\n"
     ]
    }
   ],
   "source": [
    "torch.manual_seed(123)\n",
    "model = GPTModel(GPT_CONFIG_124M)\n",
    "model.to(device)\n",
    "\n",
    "peak_lr = 5e-4\n",
    "optimizer = torch.optim.AdamW(model.parameters(), lr=peak_lr, weight_decay=0.1)\n",
    "\n",
    "n_epochs = 15\n",
    "train_losses, val_losses, tokens_seen, lrs = train_model(\n",
    "    model, train_loader, val_loader, optimizer, device, n_epochs=n_epochs,\n",
    "    eval_freq=5, eval_iter=1, start_context=\"Every effort moves you\",\n",
    "    warmup_steps=10, initial_lr=1e-5, min_lr=1e-5\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "827e8d5e-0872-4b90-98ac-200c80ee2d53",
   "metadata": {},
   "source": [
    "- Looking at the results above, we can see that the model starts out generating incomprehensible strings of words, whereas, towards the end, it's able to produce grammatically more or less correct sentences\n",
    "- If we were to check a few passages it writes towards the end, we would find that they are contained in the training set verbatim -- it simply memorizes the training data\n",
    "- Note that the overfitting here occurs because we have a very, very small training set, and we iterate over it so many times\n",
    "  - The LLM training here primarily serves educational purposes; we mainly want to see that the model can learn to produce coherent text\n",
    "  - Instead of spending weeks or months on training this model on vast amounts of expensive hardware, we load the pretrained weights"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9decec45-4fdf-4ff6-85a7-1806613f8af7",
   "metadata": {},
   "source": [
    "- A quick check that the learning rate behaves as intended"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "d8ebb8d2-8308-4a83-a2a6-730c3bf84452",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlwAAAGwCAYAAAB8crvUAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAA9hAAAPYQGoP6dpAABjaUlEQVR4nO3deVhU5dsH8O8MAzPIqiKrKLjlhqKiiEtWUriUolZq5BZJmbiE5fa6lC2kZfWjzC1LK7dssTKlCM0Vkc0VtxR3WQRhAGWbOe8fNgcnUUFnODPD93NdcyFnnhnu82TM7XPu5z4yQRAEEBEREZHRyKUOgIiIiMjSMeEiIiIiMjImXERERERGxoSLiIiIyMiYcBEREREZGRMuIiIiIiNjwkVERERkZAqpA7BkWq0WV65cgYODA2QymdThEBERUTUIgoDCwkJ4enpCLjfM2hQTLiO6cuUKvL29pQ6DiIiIHsDFixfRuHFjg7wXEy4jcnBwAHDrP5ijo6PE0RAREVF1qNVqeHt7i5/jhsCEy4h0lxEdHR2ZcBEREZkZQ5YDsWieiIiIyMiYcBEREREZGRMuIiIiIiNjwkVERERkZEy4iIiIiIyMCRcRERGRkTHhIiIiIjIyJlxERERERsaEi4iIiMjImHARERERGZlJJFxLliyBj48PVCoVAgMDceDAgXuO37RpE1q3bg2VSgU/Pz9s3bpV73lBEDBv3jx4eHjA1tYWwcHBOH36tN6YvLw8hIWFwdHREc7OzggPD0dRUZH4/Llz5yCTye547N+/33AnTkRERHWC5AnXxo0bERUVhfnz5yM1NRUdO3ZESEgIsrOzqxy/b98+jBw5EuHh4UhLS0NoaChCQ0Nx9OhRccyiRYsQExODZcuWITExEXZ2dggJCUFJSYk4JiwsDMeOHUNcXBy2bNmCXbt2ISIi4o6f99dff+Hq1avio0uXLoafBCIiIrJoMkEQBCkDCAwMRNeuXfH5558DALRaLby9vTFp0iTMnDnzjvHDhw9HcXExtmzZIh7r3r07/P39sWzZMgiCAE9PT0ybNg1vvPEGAKCgoABubm5YvXo1RowYgePHj6Nt27ZISkpCQEAAACA2NhYDBgzApUuX4OnpiXPnzsHX1xdpaWnw9/d/oHNTq9VwcnJCQUGBRd68uqi0AjdKK6BUWEFpLYeNlRxyueFu9ElERCQFY3x+KwzyLg+orKwMKSkpmDVrlnhMLpcjODgYCQkJVb4mISEBUVFResdCQkKwefNmAEBGRgYyMzMRHBwsPu/k5ITAwEAkJCRgxIgRSEhIgLOzs5hsAUBwcDDkcjkSExMxZMgQ8figQYNQUlKCVq1aYfr06Rg0aNBdz6e0tBSlpaXi92q1unoTYYb+yS7CgJjdKKvQ6h23sZJDqZBDZWOFhnY2aOSghKuDCo0clP/+WQkPJxWaNbJHAzsbiaInIiKqXZImXNeuXYNGo4Gbm5vecTc3N5w4caLK12RmZlY5PjMzU3xed+xeY1xdXfWeVygUaNCggTjG3t4eixcvRs+ePSGXy/Hjjz8iNDQUmzdvvmvSFR0djbfffrs6p272Us9fvyPZAoAyjRZlGi0KSyuQU1iKE5mFd32P+vWs0cLVHs0b/ftwtUMbD0d4ONkaM3QiIqJaJ2nCZcpcXFz0VtK6du2KK1eu4MMPP7xrwjVr1iy916jVanh7exs9VilkqW/Vwz0f0BjvD/FDSYUWpeUalFZoUVqhRXFpBXKLy5CtLkFOUSlyCisfl67fxOX8m7h+oxxJ564j6dx1vfd2d1ShUxNndGrijM5N6qO9lxNU1lZSnCYREZFBSJpwubi4wMrKCllZWXrHs7Ky4O7uXuVr3N3d7zle9zUrKwseHh56Y3S1WO7u7ncU5VdUVCAvL++uPxe4VW8WFxd31+eVSiWUSuVdn7ck2YW3Lp26OaqgsJLD3koOe2X1/zrdLNPg7LUinMkpxpnsIvyTU4Qz2UU4nV2ETHUJth3NxLajt1YbFXIZ2no6Iqh5Q/Rp1QgBTRvARiH5fg8iIqJqkzThsrGxQZcuXRAfH4/Q0FAAt4rm4+PjERkZWeVrgoKCEB8fj6lTp4rH4uLiEBQUBADw9fWFu7s74uPjxQRLrVYjMTEREyZMEN8jPz8fKSkp4q7D7du3Q6vVIjAw8K7xHjx4UC+Jq8t0K1yujqoHer2tjRXaeTqhnaeT3vEbZRU4cqkAaRfzkXbhOlIv5COnsBSHLxXg8KUCLN95FnY2Vghq7oI+jzTCY60awbtBvYc+HyIiImOS/JJiVFQUxowZg4CAAHTr1g2ffvopiouLMW7cOADA6NGj4eXlhejoaADAlClT0KdPHyxevBgDBw7Ehg0bkJycjBUrVgAAZDIZpk6dinfffRctW7aEr68v5s6dC09PTzGpa9OmDfr164fx48dj2bJlKC8vR2RkJEaMGAFPT08AwJo1a2BjY4NOnToBAH766Sd89dVX+PLLL2t5hkxT1r8rXK4Ohl3Rq2ejQGCzhghs1hDArZ5ql/NvIuX8dew8lYNdp67hWlEp/jqehb+O31rpbNbIDgPae+CZjp54xN3BoPEQEREZguQJ1/Dhw5GTk4N58+YhMzMT/v7+iI2NFYveL1y4ALm88vJRjx49sG7dOsyZMwezZ89Gy5YtsXnzZrRv314cM336dBQXFyMiIgL5+fno1asXYmNjoVJVrsasXbsWkZGR6Nu3L+RyOYYNG4aYmBi92N555x2cP38eCoUCrVu3xsaNG/Hss88aeUbMQ86/K1xuD7jCVV0ymQyN69dD4/r1MNjfC1qtgPSrauw8lYOdp3KQcv46zuYU4/Md/+DzHf+gpas9nunoiac7eKBZI3ujxkZERFRdkvfhsmSW2odLqxXQas42VGgFJMx6QtJdheqScuw4kY0th69i58kclGkqd06283TEYH9PDO3cGC72daO2joiIHp7F9eEi85R3owwVWgEyGSRPZBxV1hjs74XB/l4ouFmOuPQs/HboCvb8cw3Hrqhx7IoaH/5xEiHt3PFCYBMENWsImYzNWYmIqHYx4aIay1bfqt9qaGcDayvT2S3oZGuNZ7s0xrNdGiOvuAzbjl7F90kXcehSAbYcvooth6+imYsdRnZrgmFdGrPxKhER1RomXFRjWYW36rcaORi3futhNLCzQVhgU4QFNsXRywVYd+ACfkm7jLPXivHe1uP48I+TeKajJyIebcZCeyIiMjrTWZ4gs5Gj1vXgMo+6qPZeTnh/iB8S/y8Y7w/xQztPR5RptPgx9RJCPt2Fl1YnIfFsLljOSERExsIVLqoxXQ8uNxNe4aqKvVKBFwKbYGQ3bxy8mI8Vu84i9lgmtp/IxvYT2fD3dsarfZrhybbusOJNuImIyIC4wkU1pruk6GomK1z/JZPJ0KlJfSx9sQu2T3sMLwQ2gY1CjoMX8/Hqd6l48uOd+DHlEjRarngREZFhMOGiGtMVzT9ol3lT4utih/eH+GHvjCcQ+XgLOKoUOHutGNM2HULIp7uw9chVaJl4ERHRQ2LCRTWm6zLvZuAu81Jq5KDEGyGPYN+svpjRrzWcbK3xT3YRXlubikFL9mDHyWzWeBER0QNjwkU1lv2Q91E0ZfZKBSY81hy7ZzyOyX1bws7GCkcvqzHu6yQ8tywB+8/mSh0iERGZISZcVCNarYCcQvPapfggHFXWiHqyFXbPeAIRjzaDUiFH8vnrGLFiPyK+Scb53GKpQyQiIjPChItqxJS6zNeGBnY2mD2gDXZNfxwvdm8CK7kMf6Zn4cmPd+GDbSdQVFohdYhERGQGmHBRjehaQphal3ljc3NU4d1QP8RO6Y3eLV1QptFi2c4zeOzDv/F98kUW1hMR0T3VnU9MMojsfy8nuppZDy5DaenmgG9e6oZVYwLg62KHa0WlmP7DYQxeshcp5/OkDo+IiEwUEy6qkcqCecu/nHg3MpkMfdu44Y+pj+L/BrSBg1KBI5cLMGxpAmb9dAQFN8ulDpGIiEwMEy6qkSzdbX3q6ArX7WwUcox/tBl2vPkYhgd4AwDWH7iA4I934vfDV9lGgoiIREy4qEay/+0yb8k7FGvKxV6Jhc92wMaI7mjWyA45haWYuC4VL69JxuX8m1KHR0REJoAJF9WIboWrkQX24HpYgc0aYtuU3pjStyWsrWSIP5GNJz/eiVV7MnibICKiOo4JF9VItgV2mTckpcIKrz/ZCtum9EY3nwa4UabBO1vSMWzpPpzJKZI6PCIikggTLqoRXdG8G1e47qmFqwM2RHRH9FA/OKgUOHgxHwNjdmP13gy2kCAiqoOYcFG13d5lvi7vUqwuuVyGkd2a4I+pj6JXCxeUlGvx1m/pGPVVIq6wtouIqE5hwkXVVte6zBuKp7MtvnmpGxYMbgeVtRx7/8lFyCe78EPKJe5kJCKqI5hwUbVVdplX1qku84Ygl8swOsgH26Y8ik5NnFFYWoE3Nh3CK9+mILeoVOrwiIjIyPipSdWWrdZ1mefq1oPydbHDpleC8GbII7C2unVfxgExu5FwJlfq0IiIyIiYcFG1sQeXYSis5Jj4eAtsntgTLVztkaUuxQtf7sfHcafYPoKIyEIx4aJqy1LX7fsoGlo7Tyf8GtkTzwc0hiAAMfGnMXLlfmQWlEgdGhERGRgTLqq2LDVXuAytno0Ci57tiP+N8IedjRUOZOSh//92YfuJLKlDIyIiA2LCRdWWLbaE4AqXoQ3298KWyb3R3ssR12+U46XVyXh3SzrKNVqpQyMiIgNgwkXVpmt6yqJ54/B1scOPE3pgXE8fAMCXezLwwsr94rwTEZH5YsJF1Sbe1ocrXEajVFhh/jPtsGJUFzgoFUg6dx1Pf7YHKefzpA6NiIgeAhMuqhatVmDCVYueaueOXyf1Qis3e2QXlmL48v34JuEcG6USEZkpJlxULbnFZdCIXeZtpA6nTvB1scPPr/XE0x08UKEVMO+XY5i26RBKyjVSh0ZERDXEhIuqRdeDq6GdEgp2ma81dkoFPhvZCXMGtoGVXIafUi9j6Bf7cDHvhtShERFRDfCTk6pF12WeLSFqn0wmw8u9m+Hb8G5oaGeD9KtqPPP5Huw/y+70RETmggkXVUsWdyhKrkdzF/w2qRc6NnZC/o1yvPhlIjYmXZA6LCIiqgYmXFQtLJg3DZ7Ottj4SpBY1zXjxyN4Z0s6bwlERGTimHBRtXCFy3SorK3w2chOeD24FQBg1Z4MvLwmCYUl5RJHRkREd8OEi6pFvI8iV7hMgkwmw5TglljyQmeorOXYcTIHw5buw4VcFtMTEZkiJlxULTmFuvsoMuEyJQM7eOD7V4Lg5qjEqawihH6xF0nn2CSViMjUMOGiahFXuHhJ0eR0aOyMXyb2gp+XE/KKyxD2ZSJ+P3xV6rCIiOg2TLjovrRaATlFLJo3Ze5OKnz/ShCeauuGsgotIten4svdZ6UOi4iI/sWEi+6LXebNg62NFZa+2AVjgppCEIB3fz+Ot387xh2MREQmgAkX3ZduhyK7zJs+K7kMbw1qh9kDWgMAvt57DpHrUnk7ICIiifHTk+4rp5Bd5s2JTCZDxKPNETOyE2ys5Nh2NBMvfpmI68VlUodGRFRnMeGi+9KtcLF+y7wM6uiJb8K7wVGlQPL56xi2jPdgJCKSChMuui/uUDRf3Zs1xA8TesDTSYWzOcV4blkCTmcVSh0WEVGdw4SL7iv73x5cbHpqnlq5OeCn13qilZs9MtUleG55AtIuXJc6LCKiOoUJF90XV7jMn65thL+3M/JvlCPsy0TsOX1N6rCIiOoMJlx0X9nsMm8RnOvZYO3Lgejd0gU3yjR4aXUSth1hg1QiotrAhIvuK1vNXYqWwk6pwJdjAjDQzwNlGi0mrkvF+gMXpA6LiMjiMeGie9Lc1mXe1YErXJZAqbBCzMhOGNmtCbQCMOunI1j69xmpwyIismhMuOie8thl3iJZyWV4f0h7vPZYcwDAwtgTWPznSQgCu9ITERkDEy66J10PLhd7dpm3NDKZDNP7tcas/re60n+2/R+8v/U4ky4iIiPgJyjdk9gSgjsULdYrfZrj7UHtAAArd2dg/q/HoOX9F4mIDIoJF91TZcE867cs2ZgePvhgqB9kMuCbhPOY/fMR3vSaiMiAmHDRPWVxh2KdMaJbEyx+riPkMmBD0kW8sekQKjRaqcMiIrIITLjonrL+vaTYiDsU64ShnRsjZmQnKOQy/Jx2GVM2HEQ5ky4ioofGhIvuiT246p6nO3jii7DOsLGS4/cjVzHhu1SUVTDpIiJ6GEy46J4qi+a5wlWXPNXOHStGd4FSIcdfx7MwcR2TLiKih2ESCdeSJUvg4+MDlUqFwMBAHDhw4J7jN23ahNatW0OlUsHPzw9bt27Ve14QBMybNw8eHh6wtbVFcHAwTp8+rTcmLy8PYWFhcHR0hLOzM8LDw1FUVFTlz/vnn3/g4OAAZ2fnhzpPc6RrC8EVrrrnsUdcsXJ0AGwUcsSlZyGSSRcR0QOTPOHauHEjoqKiMH/+fKSmpqJjx44ICQlBdnZ2leP37duHkSNHIjw8HGlpaQgNDUVoaCiOHj0qjlm0aBFiYmKwbNkyJCYmws7ODiEhISgpKRHHhIWF4dixY4iLi8OWLVuwa9cuRERE3PHzysvLMXLkSPTu3dvwJ2/iNFoB14rKAHCXYl31aKtGYtL1Z3oWJq1PZU0XEdEDkAkSdzkMDAxE165d8fnnnwMAtFotvL29MWnSJMycOfOO8cOHD0dxcTG2bNkiHuvevTv8/f2xbNkyCIIAT09PTJs2DW+88QYAoKCgAG5ubli9ejVGjBiB48ePo23btkhKSkJAQAAAIDY2FgMGDMClS5fg6ekpvveMGTNw5coV9O3bF1OnTkV+fv5dz6W0tBSlpaXi92q1Gt7e3igoKICjo+NDzZMUsgtL0O29eMhkwOl3+7PxaR3298lsRHybgrIKLfq1c8dnL3SCNf8+EJGFUqvVcHJyMujnt6S/McvKypCSkoLg4GDxmFwuR3BwMBISEqp8TUJCgt54AAgJCRHHZ2RkIDMzU2+Mk5MTAgMDxTEJCQlwdnYWky0ACA4OhlwuR2Jionhs+/bt2LRpE5YsWVKt84mOjoaTk5P48Pb2rtbrTJWuYJ5d5umxR1yxfFQX2FjJEXssE5PXp3Gli4ioBiT9FL127Ro0Gg3c3Nz0jru5uSEzM7PK12RmZt5zvO7r/ca4urrqPa9QKNCgQQNxTG5uLsaOHYvVq1dXO7udNWsWCgoKxMfFixer9TpTpSuYZ/0WAcDjtyVd245mYsoGJl1ERNXFZYu7GD9+PF544QU8+uij1X6NUqmEo6Oj3sOc6Zqecoci6Tze2hXLRnWGtZUMW49k4vWNB9mRnoioGiRNuFxcXGBlZYWsrCy941lZWXB3d6/yNe7u7vccr/t6vzH/LcqvqKhAXl6eOGb79u346KOPoFAooFAoEB4ejoKCAigUCnz11VcPeMbmhT24qCpPtHbDshe7wNpKhi2Hr2LGj4d570UiovuQNOGysbFBly5dEB8fLx7TarWIj49HUFBQla8JCgrSGw8AcXFx4nhfX1+4u7vrjVGr1UhMTBTHBAUFIT8/HykpKeKY7du3Q6vVIjAwEMCtOq+DBw+KjwULFsDBwQEHDx7EkCFDDDMBJi6LPbjoLvq2cUPMiE6Qy4AfUi5h/q/HIPH+GyIik6aQOoCoqCiMGTMGAQEB6NatGz799FMUFxdj3LhxAIDRo0fDy8sL0dHRAIApU6agT58+WLx4MQYOHIgNGzYgOTkZK1asAADIZDJMnToV7777Llq2bAlfX1/MnTsXnp6eCA0NBQC0adMG/fr1w/jx47Fs2TKUl5cjMjISI0aMEHcotmnTRi/O5ORkyOVytG/fvpZmRnrZ//bgcuUKF1Whv58HFj/fEVHfH8K3+89DZS3H7AFtIJPJpA6NiMjkSJ5wDR8+HDk5OZg3bx4yMzPh7++P2NhYsej9woULkMsrF+J69OiBdevWYc6cOZg9ezZatmyJzZs36yVC06dPR3FxMSIiIpCfn49evXohNjYWKlXlSs3atWsRGRmJvn37Qi6XY9iwYYiJiam9EzcD2YX/XlLkChfdxZBOjVFSrsWsn45g5e4M2NooEPVkK6nDIiIyOZL34bJkxujjUZsC3/8LWepS/BbZC36NnaQOh0zYV3sysGBLOgBgRr/WmPBYc4kjIiJ6cBbXh4tMl0YrIOffFS5eUqT7eamXL94MeQQAsDD2BFbvzZA4IiIi08KEi6qUW1wKrQDIZUBDOxupwyEzMPHxFpj0RAsAwFu/pWNTsnn3oSMiMiQmXFQlXUuIhuwyTzUQ9WQrhPfyBQDM/OkI/jxWdQNjIqK6hp+kVKUsNbvMU83JZDLMGdgGz3ZpDI1WQOT6NCScyZU6LCIiyTHhoipxhyI9KJlMhg+G+iG4jRvKKrQY/00yjl4ukDosIiJJMeGiKmWxBxc9BIWVHJ+/0AmBvg1QVFqBMV8dwNmcIqnDIiKSDBMuqpJuhYtd5ulBqayt8OWYALT3ckRucRlGrTqAqwU3pQ6LiEgSTLioStliDRcTLnpwDiprrB7XDb4udricfxOjVx3A9eIyqcMiIqp1TLioSllq3QoXLynSw3GxV+Lb8G5wd1ThdHYRxq5OQnFphdRhERHVKiZcVKXsQq5wkeE0rl8P34Z3g3M9axy6mI9Xv0tBaYVG6rCIiGoNEy66w+1d5tkWggylpZsDvh7bFfVsrLD79DVEbTwEjZZ3FiOiuoEJF90ht+i2LvP2TLjIcDo1qY/lo7rA2kqG349cxdxfjoK3cyWiuoAJF91Bt0PRxV4JK7lM4mjI0vRu2QifDu8EmQxYl3gBn8SdkjokIiKjY8JFd2APLjK2gR088G5oewBAzPZ/sP7ABYkjIiIyLiZcdAfdDkV2mSdjCgtsKt7ses7mo9h+IkviiIiIjIcJF91Bt0PRlTsUyciinmwl3ndx4to0HLqYL3VIRERGwYSL7sAeXFRbZDIZoof64dFWjXCzXIOXVifh3LViqcMiIjI4Jlx0hxz24KJaZG0lxxdhncVbAI39+gByi0qlDouIyKCYcNEdxBouFs1TLbFXKvDV2K5oXN8W53Jv4KU1ybhZxsaoRGQ5mHDRHcRdiiyap1rk6qDCmpcqu9FPWp+KCo1W6rCIiAyCCRfp0WgFXCviChdJo3kje6waEwClQo6/jmdj3q/H2BiViCwCEy7Swy7zJLUuTRvgfyMqG6Mu2fGP1CERET00JlykR1e/xS7zJKV+7d3x1jPtAAAf/XkKP6RckjgiIqKHw4SL9GRzhyKZiDE9fPBKn2YAgJk/Hsbu0zkSR0RE9OCYcJEe9uAiUzIjpDVC/T1RoRXw2nepOJlZKHVIREQPhAkX6am8jyJXuEh6crkMC5/tgG6+DVBYWoGXVieJq7BEROaECRfpyS7kDkUyLUqFFVaM6oJmLna4nH8T49mji4jMEBMu0pPNHlxkgpzr2eCrsV1Rv541Dl0qwOsbD0KrZbsIIjIfTLhID1e4yFT5uNhhxegA2FjJEXssEwtjT0gdEhFRtTHhIj26Gi7uUiRT1NWnAT58rgMAYPmus1iXeEHiiIiIqocJF4lu7zLPXYpkqgb7eyHqyVYAgLm/HMWuU2wXQUSmjwkXidhlnszFpCdaYGhnL2i0Al5by3YRRGT6mHCRSNeDq5EDu8yTaZPJZIge6odA3wYoYrsIIjIDTLhIlMUdimRGlAorLGe7CCIyE0y4SMQdimRu/tsuYurGNLaLICKTxISLRLoVrkZc4SIzcnu7iD+OZbFdBBGZJCZcJKq8cTVXuMi8/LddxKbkixJHRESkjwkXibLVukuKXOEi8zPY3wuT+7YEAMz++QiSzuVJHBERUSUmXCTKKtQVzXOFi8zT1L4tMcDPHeUaAa98m4KLeTekDomICAATLrpNFle4yMzJ5TIsfs4ffl5OyCsuQ/iaJBSWlEsdFhEREy66pUKjRa6uyzxruMiM2dpYYeXoALg6KHEqqwiT16dBw52LRCQxJlwEAMgtLqvsMm/HhIvMm7uTCitHB0CpkGPHyRx8sO241CERUR3HhIsAVBbMs8s8WYqO3s5Y/HxHAMDK3RnYmMQbXRORdJhwEYDKHlys3yJL8nQHT0z5d+finM1Hsf9srsQREVFdxYSLAHCHIlmuKX1bYmAHD5RrBEz4LgUXcrlzkYhqHxMuAlB5SdGVK1xkYeRyGT56tiM6NHbC9RvlCF+TBDV3LhJRLWPCRQAqu8xzhYsskW7nopujEqezizBpXRoqNFqpwyKiOuSBEq6Kigr89ddfWL58OQoLCwEAV65cQVFRkUGDo9rDHlxk6dwcb+1cVFnLsfNUDt7fynsuElHtqXHCdf78efj5+WHw4MGYOHEicnJyAAALFy7EG2+8YfAAqXbwPopUF3Ro7IzFz/kDAL7am8F7LhJRralxwjVlyhQEBATg+vXrsLW1FY8PGTIE8fHxBg2Oao9uhcvVgStcZNkGdvDA5CdaAAD+7+ejSDl/XeKIiKguqHHCtXv3bsyZMwc2NjZ6x318fHD58mWDBUa1p0KjxTV2mac6ZGpwKzzV1g1lGi1e/S4FmQUlUodERBauxgmXVquFRqO54/ilS5fg4OBgkKCoduUWl0EQACu5jF3mqU6Qy2X4eLg/WrnZI6ewFK98m4yS8jt/rxERGUqNE66nnnoKn376qfi9TCZDUVER5s+fjwEDBhgyNqoluqanLvY27DJPdYa9UoEvR3eFcz1rHLpUgFk/HYEg8J6LRGQcNU64Fi9ejL1796Jt27YoKSnBCy+8IF5OXLhwoTFiJCPL5g5FqqOaNKyHL17oDCu5DD+nXcbK3WelDomILJSipi9o3LgxDh06hI0bN+LQoUMoKipCeHg4wsLC9IroyXxUdplnwkV1T48WLpg7sA3e+i0dH2w7gVZuDnjsEVepwyIiC1PjhGvXrl3o0aMHwsLCEBYWJh6vqKjArl278Oijjxo0QDI+cYciC+apjhrTwwfHrxZiY/JFTFqfhl8m9kSzRvZSh0VEFqTGlxQff/xx5OXl3XG8oKAAjz/+uEGCotqVo+vBxRUuqqNkMhkWhLZDl6b1UVhSgZe/Sebtf4jIoGqccAmCAJnszsLq3Nxc2NnZPVAQS5YsgY+PD1QqFQIDA3HgwIF7jt+0aRNat24NlUoFPz8/bN269Y4Y582bBw8PD9ja2iI4OBinT5/WG5OXl4ewsDA4OjrC2dkZ4eHhep3yT548iccffxxubm5QqVRo1qwZ5syZg/Jyy/slzBUuIkCpsMLSFzvDw0mFsznFmLI+DRoti+iJyDCqfUlx6NChAG79S3Ds2LFQKis/nDUaDQ4fPowePXrUOICNGzciKioKy5YtQ2BgID799FOEhITg5MmTcHW9s45i3759GDlyJKKjo/H0009j3bp1CA0NRWpqKtq3bw8AWLRoEWJiYrBmzRr4+vpi7ty5CAkJQXp6OlSqW6s4YWFhuHr1KuLi4lBeXo5x48YhIiIC69atAwBYW1tj9OjR6Ny5M5ydnXHo0CGMHz8eWq0W77//fo3P05TpdimyyzzVda4OKqwYFYBnl+3DjpM5+PCPk5jZv7XUYRGRBZAJ1dwHPW7cOADAmjVr8Pzzz+sVyNvY2MDHxwfjx4+Hi4tLjQIIDAxE165d8fnnnwO41efL29sbkyZNwsyZM+8YP3z4cBQXF2PLli3ise7du8Pf3x/Lli2DIAjw9PTEtGnTxFsNFRQUwM3NDatXr8aIESNw/PhxtG3bFklJSQgICAAAxMbGYsCAAbh06RI8PT2rjDUqKgpJSUnYvXt3tc5NrVbDyckJBQUFcHR0rNG81Kau7/2FnMJSbJnUC+29nKQOh0hyvxy8jCkbDgIA/jfCH4P9vaQNiIhqlTE+v6u9wvX1118DuNVR/o033njgy4e3KysrQ0pKCmbNmiUek8vlCA4ORkJCQpWvSUhIQFRUlN6xkJAQbN68GQCQkZGBzMxMBAcHi887OTkhMDAQCQkJGDFiBBISEuDs7CwmWwAQHBwMuVyOxMREDBky5I6f+88//yA2NlZc6atKaWkpSktLxe/VavW9J8AEsMs80Z0G+3vh+NVCLNt5BtN/OIxmLvbwa8x/jBDRg6txDdf8+fMNkmwBwLVr16DRaODm5qZ33M3NDZmZmVW+JjMz857jdV/vN+a/lysVCgUaNGhwx8/t0aMHVCoVWrZsid69e2PBggV3PZ/o6Gg4OTmJD29v77uONRXXithlnqgqb4Y8gidau6K0QouIb5PFG7wTET2IGidcAPDDDz/g+eefR/fu3dG5c2e9h6XZuHEjUlNTsW7dOvz+++/46KOP7jp21qxZKCgoEB8XL16sxUgfjO5DpJG9kl3miW5jJZfh0xH+aN7IDlcLSjDhu1SUVvD2P0T0YGqccMXExGDcuHFwc3NDWloaunXrhoYNG+Ls2bPo379/jd7LxcUFVlZWyMrK0juelZUFd3f3Kl/j7u5+z/G6r/cbk52drfd8RUUF8vLy7vi53t7eaNu2LUaOHIkPPvgAb731VpX3kgQApVIJR0dHvYep4w5FortzVFlj5egAOKgUSDl/HfM2H+Ptf4jogdQ44friiy+wYsUKfPbZZ7CxscH06dMRFxeHyZMno6CgoEbvZWNjgy5duiA+Pl48ptVqER8fj6CgoCpfExQUpDceAOLi4sTxvr6+cHd31xujVquRmJgojgkKCkJ+fj5SUlLEMdu3b4dWq0VgYOBd49VqtSgvL4dWq63ReZqybHaZJ7qnZo3s8fkLnSGXARuTL+Lb/eelDomIzFCNO81fuHBBbP9ga2uLwsJCAMCoUaPQvXt3cbdhdUVFRWHMmDEICAhAt27d8Omnn6K4uFjcFTl69Gh4eXkhOjoaADBlyhT06dMHixcvxsCBA7FhwwYkJydjxYoVAG61rZg6dSreffddtGzZUmwL4enpidDQUABAmzZt0K9fP4wfPx7Lli1DeXk5IiMjMWLECHGH4tq1a2FtbQ0/Pz8olUokJydj1qxZGD58OKytrWs6bSYrS7yPIle4iO6mT6tGmNGvNaK3ncCC39LxiJsDAps1lDosIjIjNU643N3dkZeXh6ZNm6JJkybYv38/OnbsiIyMjAdaah8+fDhycnIwb948ZGZmwt/fH7GxsWLR+4ULFyCXVy7E9ejRA+vWrcOcOXMwe/ZstGzZEps3bxZ7cAHA9OnTUVxcjIiICOTn56NXr16IjY0Ve3ABtxKqyMhI9O3bF3K5HMOGDUNMTEzlxCgUWLhwIU6dOgVBENC0aVNERkbi9ddfr/E5mrJsNVe4iKoj4tFmOHpFjd8OXcHEdan4NbIXPJ15/1giqp5q9+HSefnll+Ht7Y358+djyZIlePPNN9GzZ08kJydj6NChWLVqlbFiNTvm0IfrpdVJ2H4iGx8M9cOIbk2kDofIpN0oq8CwpQk4flWNjo2dsPGVIKisraQOi4gMTNI+XDorVqwQa5gmTpyIhg0bYt++fRg0aBBeeeUVgwRFtUfXZZ5F80T3V89GgRWjuuCZz/fg0KUCzNl8FB8+26HK250REd2uRkXzFRUVePfdd/V6VY0YMQIxMTGYNGkSbGxsDB4gGZe4S5GXFImqxbtBPXw+8lYR/Q8pl/BNAovoiej+apRwKRQKLFq0CBUVFcaKh2pRhUaL3GJd0TwTLqLq6tXSBbP6twEAvLMlHYlncyWOiIhMXY3bQvTt2xc7d+40RixUy/S7zHN1kqgmXu7ti8H+nqjQCnhtbSqu5N+UOiQiMmE1ruHq378/Zs6ciSNHjqBLly533OZn0KBBBguOjEtXv9XIXgk5u8wT1YhMJsMHQzvgdFYR0q+q8ep3KfieRfREdBc13qV4e4uGO95MJrtrF/a6yNR3KcalZ2H8N8no2NgJv0T2kjocIrN0Me8GBn2+B9dvlGNY58b46DkW0ROZO2N8ftf4kqJWq73rg8mWeRFXuFgwT/TAvBvUEzvR/5h6CWv2nZM6JCIyQQ9082qyDNmF7DJPZAg9W7hg9oB/i+h/P479LKInov9gwlWH6brMc4ci0cML7+WLUH9PaLQCJq5NxWUW0RPRbZhw1WFi01MHrnARPSyZTIbooR3Q1sMRucVlePXbFJSUs8yCiG5hwlWHVV5S5AoXkSHY2lhh+aguqF/PGkcuF2D2z0ce6B6zRGR5mHDVYWKXedZwERmMd4N6WPJCZ1jJZfgp9TJWs4ieiPAACZdara7yUVhYiLKyMmPESEZwe5d53taHyLB6tHDBrP6tAQDv/n4cCWdYRE9U19U44XJ2dkb9+vXveDg7O8PW1hZNmzbF/PnzxRtck2lil3ki49Irol/HInqiuq7GCdfq1avh6emJ2bNnY/Pmzdi8eTNmz54NLy8vLF26FBEREYiJicEHH3xgjHjJQNhlnsi4dEX07TwdkVdchle+TWYRPVEdVuNb+6xZswaLFy/G888/Lx575pln4Ofnh+XLlyM+Ph5NmjTBe++9h9mzZxs0WDKcLLElBOu3iIxFV0Q/6PO9OHpZjdk/HcHi5zuyEz1RHVTjFa59+/ahU6dOdxzv1KkTEhISAAC9evXChQsXHj46MhrdDkVX7lAkMqrG9evh8xc63SqiT7uMr/eekzokIpJAjRMub29vrFq16o7jq1atgre3NwAgNzcX9evXf/joyGiy2YOLqNb0aF7Zif69rcex78w1iSMiotpW40uKH330EZ577jls27YNXbt2BQAkJyfjxIkT+OGHHwAASUlJGD58uGEjJYPStYRgDy6i2vFSTx8cvVyAn9MuI3JdGn6N7InG9etJHRYR1ZIar3ANGjQIJ06cQP/+/ZGXl4e8vDz0798fJ06cwNNPPw0AmDBhAj7++GODB0uGk13IGi6i2nSriN4P7b1uFdG/+h070RPVJTVe4QIAX19f7kI0c2LTU/bgIqo1KmsrLB8VgGc+24Ojl9WY9dMRfMwieqI64YESrvz8fBw4cADZ2dl39NsaPXq0QQIj46osmucKF1Ft8nK2xZIXOuPFVYn4Oe0y2ns5IbyXr9RhEZGR1Tjh+u233xAWFoaioiI4Ojrq/ctMJpMx4TID5bd1mWcNF1HtC2reEP83oA0WbEnH+1uPo42HA3o0d5E6LCIyohrXcE2bNg0vvfQSioqKkJ+fj+vXr4uPvLw8Y8RIBnatqBSCACjkMjSoxy7zRFIY19MHQzt7QaMVELkuDZeu35A6JCIyohonXJcvX8bkyZNRrx5315ir7H/rtxo5sMs8kVRkMhneH+IHPy8nFtET1QE1TrhCQkKQnJxsjFiolmSxBxeRSVBZW2HZqC5oaGcjFtELgiB1WERkBDWu4Ro4cCDefPNNpKenw8/PD9bW1nrPDxo0yGDBkXFkscs8kcnwcrbF5yyiJ7J4NU64xo8fDwBYsGDBHc/JZDJoNFwSN3U5vI8ikUlhET2R5avxJUWtVnvXB5Mt88AeXESmh0X0RJatxgkXmb8sdpknMjn/LaJ/5dsU3CzjP2KJLEW1LinGxMQgIiICKpUKMTEx9xw7efJkgwRGxqPbpcgaLiLToiuiH/TZHhy7osasnw7jk+H+7ERPZAFkQjW2xPj6+iI5ORkNGzaEr+/dizllMhnOnj1r0ADNmVqthpOTEwoKCuDo6Ch1OKKAd+NwragMv0/uhXaeTlKHQ0T/kXAmFy+uSoRGK2DOwDZ4uXczqUMiqlOM8fldrRWujIyMKv9M5udWl/kyAOwyT2Sqgpo3xJyBbfD2b+mI3nYCbT0c0aMFi+iJzBlruOoYdpknMg9je9xWRL+eRfRE5q7GbSE0Gg1Wr16N+Pj4Km9evX37doMFR4aXxS7zRGZBV0R/OqsIRy4X4JVvU/DDqz1ga2MldWhE9ABqvMI1ZcoUTJkyBRqNBu3bt0fHjh31HmTasnVd5nk5kcjk3d6JXldEz070ROapxitcGzZswPfff48BAwYYIx4yMrHLPG/rQ2QWbu9Ev/ngFbT3cmIRPZEZqvEKl42NDVq0aGGMWKgWZLPLPJHZ0RXRA0D0thPY9881iSMiopqqccI1bdo0/O9//+OytpnS9eByY5d5IrNyexH9xHWpuJjHInoic1LjS4p79uzBjh07sG3bNrRr1+6Om1f/9NNPBguODE/XZd6VK1xEZqWqIvofJ7CInshc1HiFy9nZGUOGDEGfPn3g4uICJycnvQeZtix2mScyWyprKyz/t4g+/aoaM1lET2Q2arTCVVFRgccffxxPPfUU3N3djRUTGVGO7j6KvKRIZJY8nW2xJKwzXvwyEb8cvAI/FtETmYUarXApFAq8+uqrKC0tNVY8ZETlGi2uFd3qMs9LikTmq3szFtETmZsaX1Ls1q0b0tLSjBELGdm1oluJMrvME5m/MSyiJzIrNS6af+211zBt2jRcunQJXbp0gZ2dnd7zHTp0MFhwZFhi/Ra7zBOZPRbRE5mXGidcI0aMAABMnjxZPCaTySAIAmQyGTQajeGiI4PK+rcHVyMWzBNZBF0R/TOf7RGL6D8d7g+ZjP+gIjI1NU64MjIyjBEH1YLsQl0PLtZvEVkKFtETmYcaJ1xNmzY1RhxUCyrvo8iEi8iS6Iro3/otHe9vPY42Ho7o2cJF6rCI6DY1Trh00tPTceHCBZSVlekdHzRo0EMHRcahu6TIlhBElmdMDx8cuazGj6mXELkuFb9G9oJ3g3pSh0VE/6pxwnX27FkMGTIER44cEWu3AIg1A6zhMl3iJUXWcBFZHJlMhveGtMfp7EIcvsQieiJTU+O2EFOmTIGvry+ys7NRr149HDt2DLt27UJAQAD+/vtvI4RIhqLbpdiIlxSJLJLK2grLXqzsRD/jR3aiJzIVNU64EhISsGDBAri4uEAul0Mul6NXr16Ijo7W27lIpieblxSJLJ6nsy2+COsMhVyGXw9dwfJdZ6UOiYjwAAmXRqOBg4MDAMDFxQVXrlwBcKuY/uTJk4aNjgymXKNFbvGtejs3rnARWbTAZg0x/5m2AICFsSew40S2xBERUY0Trvbt2+PQoUMAgMDAQCxatAh79+7FggUL0KwZtyKbqpzCyi7z9dllnsjivdi9KUZ2awJBACavT8M/2UVSh0RUp9U44ZozZw60Wi0AYMGCBcjIyEDv3r2xdetWxMTEGDxAMgxdwTy7zBPVDTKZDG8PaoeuPvVRWFqBiG+SUXCzXOqwiOqsGu9SDAkJEf/cokULnDhxAnl5eahfvz67G5uwLLEHF+u3iOoKG4UcS1/sgkGf7cHZa8WYvD4NX43tCiv+o4uo1tV4hUvnn3/+wR9//IGbN2+iQYMGhoyJjEBsesou80R1iou9EitGB0BlLcfOUzlYFHtC6pCI6qQaJ1y5ubno27cvWrVqhQEDBuDq1asAgPDwcEybNs3gAZJhsAcXUd3V3ssJHz7bEQCwfNdZ/Jx2SeKIiOqeGidcr7/+OqytrXHhwgXUq1fZxXj48OGIjY19oCCWLFkCHx8fqFQqBAYG4sCBA/ccv2nTJrRu3RoqlQp+fn7YunWr3vOCIGDevHnw8PCAra0tgoODcfr0ab0xeXl5CAsLg6OjI5ydnREeHo6iosqi0r///huDBw+Gh4cH7Ozs4O/vj7Vr1z7Q+ZmCLK5wEdVpz3T0xMTHmwMAZvx4BIcu5ksbEFEdU+OE688//8TChQvRuHFjveMtW7bE+fPnaxzAxo0bERUVhfnz5yM1NRUdO3ZESEgIsrOr3sa8b98+jBw5EuHh4UhLS0NoaChCQ0Nx9OhRccyiRYsQExODZcuWITExEXZ2dggJCUFJSYk4JiwsDMeOHUNcXBy2bNmCXbt2ISIiQu/ndOjQAT/++CMOHz6McePGYfTo0diyZUuNz9EU6JqecoWLqO6a9uQj6NvaFWUVWkR8myyWGhBRLRBqyN7eXjh16pT45zNnzgiCIAhJSUlCgwYNavp2Qrdu3YSJEyeK32s0GsHT01OIjo6ucvzzzz8vDBw4UO9YYGCg8MorrwiCIAharVZwd3cXPvzwQ/H5/Px8QalUCuvXrxcEQRDS09MFAEJSUpI4Ztu2bYJMJhMuX75811gHDBggjBs3rtrnVlBQIAAQCgoKqv0aY+n36S6h6Ywtwo4TWVKHQkQSUt8sE/ou/ltoOmOLELpkj1BSXiF1SEQmxxif3zVe4erduze++eYb8XuZTAatVotFixbh8ccfr9F7lZWVISUlBcHBweIxuVyO4OBgJCQkVPmahIQEvfHArZ2TuvEZGRnIzMzUG+Pk5ITAwEBxTEJCApydnREQECCOCQ4OhlwuR2Ji4l3jLSgouOcGgdLSUqjVar2HqagsmucKF1Fd5qCyxsrRAXBUKZB2IR9zfj7K2/8Q1YIaJ1yLFi3CihUr0L9/f5SVlWH69Olo3749du3ahYULF9bova5duwaNRgM3Nze9425ubsjMzKzyNZmZmfccr/t6vzGurq56zysUCjRo0OCuP/f7779HUlISxo0bd9fziY6OhpOTk/jw9va+69jaVFbBLvNEVMnXxQ6fv9AZchmwKeUSvt57TuqQiCzeA3WaP3XqFHr16oXBgwejuLgYQ4cORVpaGpo3b26MGCW3Y8cOjBs3DitXrkS7du3uOm7WrFkoKCgQHxcvXqzFKO/uWtGt+i1rK3aZJ6JbHm3VCLMHtAEAvLf1OPacviZxRESWrcaNT4Fbl+j+7//+T+/YpUuXEBERgRUrVlT7fVxcXGBlZYWsrCy941lZWXB3d6/yNe7u7vccr/ualZUFDw8PvTH+/v7imP8W5VdUVCAvL++On7tz504888wz+OSTTzB69Oh7no9SqYRSaXorSLodio3s2WWeiCqF9/JF+lU1fkq9jInrUvHLxJ7wcbGTOiwii/TAjU//Kzc3F6tWrarRa2xsbNClSxfEx8eLx7RaLeLj4xEUFFTla4KCgvTGA0BcXJw43tfXF+7u7npj1Go1EhMTxTFBQUHIz89HSkqKOGb79u3QarUIDAwUj/39998YOHAgFi5cqLeD0dzodiiyyzwR3U4mk+H9IX7w93ZGwc1yjP8mGYUlvP0PkTEYLOF6UFFRUVi5ciXWrFmD48ePY8KECSguLhZrpUaPHo1Zs2aJ46dMmYLY2FgsXrwYJ06cwFtvvYXk5GRERkYCuPULZOrUqXj33Xfx66+/4siRIxg9ejQ8PT0RGhoKAGjTpg369euH8ePH48CBA9i7dy8iIyMxYsQIeHp6Arh1GXHgwIGYPHkyhg0bhszMTGRmZiIvL692J8gAcgpvrXCxfouI/ktlbYXlo7rA1UGJ09lFeH3jIWi1LKInMjTJE67hw4fjo48+wrx58+Dv74+DBw8iNjZWLHq/cOGC2M0eAHr06IF169ZhxYoV6NixI3744Qds3rwZ7du3F8dMnz4dkyZNQkREBLp27YqioiLExsZCpapc4Vm7di1at26Nvn37YsCAAejVq5fe5dA1a9bgxo0biI6OhoeHh/gYOnRoLcyKYYkrXNyhSERVcHNUYcXoANgo5PjreBY++vOk1CERWRyZYKD9wIcOHULnzp2h0WgM8XYWQa1Ww8nJCQUFBXB0dJQsjuk/HML3yZfwxlOtEPlES8niICLT9lPqJUR9fwgA8MnwjhjSqfF9XkFkmYzx+V3tovn7rezk5+c/bCxkJFzhIqLqGNq5MU5nF2Hp32cw48cjaNLADl2a1pc6LCKLUO2Ey8nJ6b7P328XH0lDvI8ia7iI6D7efOoRnMkuwp/pWXjl22RsntgTjevXu/8Lieieqp1wff3118aMg4wop5D3USSi6pHLZfhkuD+eXZaA41fVeHlNMn6c0AN2ygfqIkRE/5K8aJ6M6/Yu864OXOEiovuzUyrw5ZgAuNgrcSKzEFM3HuTORaKHxITLwuWwyzwRPQAvZ1usGN0FNgo54tKz8CF3LhI9FCZcFu72m1azyzwR1UTnJvWxaFgHAMDSv8/gx5RLEkdEZL6YcFk43Q7FRrycSEQPILSTFyIfbwEAmPXTEaScN7/mz0SmgAmXhctml3kiekhRT7ZCv3buKNNoEfFNCi5dvyF1SERmhwmXhctWc4ciET0cuVyGj4d3RDtPR+QWl+HlNckoKq2QOiwis8KEy8KJPbh4SZGIHkI9GwVWjg5AI4d/dy5uSIOGOxeJqo0Jl4XL/rcHlytXuIjoIXk622Ll6AAoFXL8dTwbH2w7LnVIRGaDCZeF4woXERmSv7czPnyuIwBg5e4MfLf/vMQREZkHJlwWLptd5onIwAZ19MS0J1sBAOb/egw7T+VIHBGR6WPCZcHKKrTI+7fLPBMuIjKkyCdaYFjnxtBoBUxcm4oTmWqpQyIyaUy4LJh+l3lriaMhIksik8kQPdQP3Zs1QFFpBV76OklstExEd2LCZcGybusyL5OxyzwRGZaNQo5lL3ZBMxc7XCkowcvfJONGGdtFEFWFCZcF0/XgcmXTUyIyEud6Nvh6XFc0sLPB4UsFmLrhINtFEFWBCZcF03WZ5w5FIjKmpg3tsGJUF9hYyfFnehbbRRBVgQmXBdNdUmTBPBEZW4BPA3z43K0bXbNdBNGdmHBZMN7Wh4hq02B/L712EX+fzJY4IiLTwYTLgmX924OrES8pElEtub1dROS6NLaLIPoXEy4Lls1LikRUy9gugqhqTLgsWGWXea5wEVHt+W+7iHGrk1BUynYRVLcx4bJQt3eZd3XgChcR1S5du4iGdjY4dkWN19amolyjlTosIskw4bJQ7DJPRFJr2tAOX43tCltrK+w6lYOZPx6BILBHF9VNTLgsFLvME5Ep6OjtjCVhnWAll+HH1Ev4OO6U1CERSYIJl4XSFamyyzwRSe2J1m54L7Q9AOCz7f9gbSJ7dFHdw4TLQokF86zfIiITMKJbE0zu2xIAMHfzUfyVniVxRES1iwmXhcriChcRmZjXg1viuS6NoRWAyPWpSLtwXeqQiGoNEy4LlcUu80RkYmQyGd4f6oc+rRqhpFyL8DXJyLhWLHVYRLWCCZeF0l1S5I2riciUWFvJ8UVYZ7T3ckRecRnGfn0A1/7dVU1kyZhwWajKonmucBGRabFTKvDV2K5oXN8W53NvIHx1Em6UsTEqWTYmXBaKXeaJyJS5Oqiw5qVuqF/PGocuFSByXRoq2BiVLBgTLgtUWqERu8xzlyIRmarmjezx5ZiuUCrk2H4iG//381E2RiWLxYTLAuX8u7plYyWHM7vME5EJ69K0PmJGdoJcBmxMvoiFsSelDonIKJhwWSDd5cRGDkp2mScikxfSzh3RQ/0AAMt2nsHynWckjojI8JhwWSB2mSciczO8axPM7N8aABC97QS+T7oocUREhsWEywKJPbhYv0VEZuTVPs3xyqPNAAAzfzqM2KOZEkdEZDhMuCxQduGtFS7uUCQiczOzf2s8H3CrG/3k9WnYd+aa1CERGQQTLgukW+FiDy4iMjcymQzvD/HDU23dUKbRYvyaZBy+lC91WEQPjQmXBRLvo8gu80RkhhRWcsSM7ISgZg1RXKbB2K+TcCanSOqwiB4KEy4LlFPI+ygSkXlTWVthxegu8PNyQl5xGUZ9mYgr+TelDovogTHhskBZ3KVIRBbAQWWN1eO6opmLHa4UlGDUqkSxqTORuWHCZWFKKzS4fqMcAHcpEpH5a2ivxLcvB8LDSYUzOcUY9/UBFJXyvotkfphwWRh2mSciS+PlbItvwyvvuxjxTTJKyjVSh0VUI0y4LIxuhyK7zBORJWnh6oCvx3WDnY0V9p3JxYTvUlBWwZtdk/lgwmVhctiDi4gslL+3M1aN7QqVtRw7TuZg8vo0VGiYdJF5YMJlYcQeXKzfIiIL1L1ZQ6wYFQAbKzlij2XijU2HoNEKUodFdF9MuCyMbociV7iIyFI92qoRloR1hkIuw+aDV/B/Px+BIDDpItPGhMvCZBeyyzwRWb4n27rhk+H+kMuADUkX8fZv6Uy6yKQx4bIw7DJPRHXFMx09sXBYBwDA6n3nsOiPk0y6yGQx4bIw2Wp2mSeiuuO5AG+8E9oeALD07zP4X/xpiSMiqhoTLguTLe5SZMJFRHXDqO5NMWdgGwDAp3+dRgyTLjJBTLgsyO1d5nlJkYjqkpd7N8PM/q0BAB/HncJnTLrIxDDhsiC6y4nsMk9EddGrfZpjRr9bSdfiuFNYsuMfiSMiqsSEy4JU7lBkl3kiqpsmPNYcb4Y8AgD48I+TTLrIZDDhsiDZ3KFIRISJj7fQS7qW/n1G4oiImHBZFN0KFwvmiaium/h4C7zxVCsAwMLYE1i2k0kXSUvyhGvJkiXw8fGBSqVCYGAgDhw4cM/xmzZtQuvWraFSqeDn54etW7fqPS8IAubNmwcPDw/Y2toiODgYp0/rF0/m5eUhLCwMjo6OcHZ2Rnh4OIqKisTnS0pKMHbsWPj5+UGhUCA0NNRg52tM7MFFRFQp8omWiHryVtL1wbYTvLxIkpI04dq4cSOioqIwf/58pKamomPHjggJCUF2dnaV4/ft24eRI0ciPDwcaWlpCA0NRWhoKI4ePSqOWbRoEWJiYrBs2TIkJibCzs4OISEhKCkpEceEhYXh2LFjiIuLw5YtW7Br1y5ERESIz2s0Gtja2mLy5MkIDg423gQYmHgfRa5wEREBACb3bYlp/yZdH/5xEh//yeaoJA2ZIOHfvMDAQHTt2hWff/45AECr1cLb2xuTJk3CzJkz7xg/fPhwFBcXY8uWLeKx7t27w9/fH8uWLYMgCPD09MS0adPwxhtvAAAKCgrg5uaG1atXY8SIETh+/Djatm2LpKQkBAQEAABiY2MxYMAAXLp0CZ6enno/c+zYscjPz8fmzZvvez6lpaUoLS0Vv1er1fD29kZBQQEcHR1rPD81NWpVInafvoaPnuuIZ7s0NvrPIyIyF8t3nkH0thMAgIhHm2FW/9bcXER3pVar4eTkZNDPb8lWuMrKypCSkqK3giSXyxEcHIyEhIQqX5OQkHDHilNISIg4PiMjA5mZmXpjnJycEBgYKI5JSEiAs7OzmGwBQHBwMORyORITEx/qnKKjo+Hk5CQ+vL29H+r9akrXFoKXFImI9L3SpzneeqYtAGDFrrN469dj0Gq50kW1R7KE69q1a9BoNHBzc9M77ubmhszMzCpfk5mZec/xuq/3G+Pq6qr3vEKhQIMGDe76c6tr1qxZKCgoEB8XL158qPerqSx2mSciuquxPX0RPdQPMhmwJuE8Zv98BBomXVRLFFIHYEmUSiWUSmlWl0orNMj/t8u8myNXuIiIqjKyWxMoFXK8sekQNiRdRFmFFoue7QCFleR7yMjCSfY3zMXFBVZWVsjKytI7npWVBXd39ypf4+7ufs/xuq/3G/PfovyKigrk5eXd9eeaA7HLvEIOJ1t2mSciupuhnRsjZmQnKOQy/JR2GVM2HERZhVbqsMjCSZZw2djYoEuXLoiPjxePabVaxMfHIygoqMrXBAUF6Y0HgLi4OHG8r68v3N3d9cao1WokJiaKY4KCgpCfn4+UlBRxzPbt26HVahEYGGiw86ttuptWuzqwyzwR0f083cETX4R1ho2VHL8fuYrwNUm4UVYhdVhkwSRdQ42KisLKlSuxZs0aHD9+HBMmTEBxcTHGjRsHABg9ejRmzZoljp8yZQpiY2OxePFinDhxAm+99RaSk5MRGRkJAJDJZJg6dSreffdd/Prrrzhy5AhGjx4NT09PsZdWmzZt0K9fP4wfPx4HDhzA3r17ERkZiREjRujtUExPT8fBgweRl5eHgoICHDx4EAcPHqy1uakp3QoX67eIiKrnqXbuWDU2APVsrLD79DWEfZmI/BtlUodFFkrSGq7hw4cjJycH8+bNQ2ZmJvz9/REbGysWvV+4cAFyeWVO2KNHD6xbtw5z5szB7Nmz0bJlS2zevBnt27cXx0yfPh3FxcWIiIhAfn4+evXqhdjYWKhUlYnI2rVrERkZib59+0Iul2PYsGGIiYnRi23AgAE4f/68+H2nTp0AwGT7t7DpKRFRzfVu2QjfvRyIcV8nIe1CPoYv349vwrvxH69kcJL24bJ0xujjcTeLYk/gi7/PYGwPH7w1qJ1RfxYRkaU5lVWIUasSkaUuReP6tvguPBA+LnZSh0USsag+XGRYui7zjbjCRURUY63cHPDDqz3g07AeLl2/iWeXJSD9ilrqsMiCMOGyENnswUVE9FC8G9TDpld7oI2HI64VlWL4igQcyMiTOiyyEEy4LERl0TxXuIiIHlQjByU2RHRHV5/6KCypwIurErH1yFWpwyILwITLQmSJbSG4wkVE9DCcbK3xzUuBCG7jhrIKLSauS8WqPRlSh0VmjgmXBSgpZ5d5IiJDsrWxwvJRXfBi9yYQBOCdLel4Z0s6779ID4wJlwXIKWSXeSIiQ7OSy/DO4PaY0a81AGDVngxMWp+GknKNxJGROWLCZQHYZZ6IyDhkMhkmPNYc/xvhD2srGX4/chWjVrFBKtUcEy4LkMUu80RERjXY3wtrXuoGB5UCSeeuY9jSfbiQe0PqsMiMMOGyANlqXUsI1m8RERlLj+Yu+OHVHvBwUuFMTjFCv9jLthFUbUy4LEDWvzVc3KFIRGRcj7g74OfXesLPywl5xWUI+3I/vk++KHVYZAaYcFkAXQ8uV65wEREZnbuTCt+/EoQBfu4o1wiY/sNhvL/1ODTcwUj3wITLAmSzBxcRUa2ytbHC5yM7Y3LflgCAFbvOIuKbZBSVVkgcGZkqJlwWIIs1XEREtU4ulyHqyVaIGdkJSoUc8SeyMeyLfbiYx2J6uhMTLguQXchdikREUhnU0RMbXwlCIwclTmYVInTJXiScyZU6LDIxTLjM3O1d5l0duMJFRCQFf29n/BrZE+08HZFbXIYXVyXiy91nIQis66JbmHCZOXaZJyIyDR5Otvjh1R4Y2skLGq2Ad38/jknr01DMui4CEy6zpyuYd3Nkl3kiIqnZ2lhh8fMdsWBwOyjkMmw5fBVDvtiLjGvFUodGEmPCZeZ0Xea5Q5GIyDTIZDKMDvLBhojucHVQ4lRWEQZ9tgd/pWdJHRpJiAmXmeMORSIi0xTg0wBbJvVCV5/6KCytwMvfJOOjP06iQqOVOjSSABMuM5fNLvNERCbL1VGFdeO7Y2wPHwDA5zv+wQsrE3G14Ka0gVGtY8Jl5nQrXOwyT0Rkmqyt5HhrUDvEjOwEe6UCB87lof//dvMSYx3DhMvM6W7r48YVLiIikzaooye2TOoFPy8n5N8ox8vfJGPBb+kordBIHRrVAiZcZk68rQ9XuIiITJ6Pix1+nNAD4b18AQBf7c3As0sTcI67GC0eEy4zp9ulyC7zRETmwUYhx9yn22LVmAA417PGkcsFGBizGz+mXGKjVAvGhMuMlZRrUHDzVpd5XlIkIjIvfdu4YduU3ujm0wDFZRpM23QIr36XgtyiUqlDIyNgwmXGbu8y72irkDgaIiKqKQ8nW6wbH4g3Qx6BQi7DH8eyEPLpLvx5LFPq0MjAmHCZsdt7cLHLPBGReVJYyTHx8RbYPLEnHnFzwLWiMkR8m4I3Nh2CuqRc6vDIQJhwmTFdDy5eTiQiMn/tvZzw66SeeKVPM8hkwA8pl9D/093Yd+aa1KGRATDhMmPswUVEZFmUCivM6t8G378ShCYN6uFy/k28sDIRs346ItbsknliwmXGeB9FIiLL1NWnAbZN6Y2wwCYAgPUHLiD44534/fBV7mQ0U0y4zJiuBxdbQhARWR47pQLvDfHDxojuaNbIDjmFpZi4LhUvr0nG5XzeGsjcMOEyY9niChcvKRIRWarAZg2xbUpvTOnbEtZWMsSfyMaTH+/Eqj0Z0Gi52mUumHCZscpdilzhIiKyZEqFFV5/shW2TemNrj71caNMg3e2pGPwkj1IOZ8ndXhUDUy4zJi4S5FF80REdUILVwdsjAjC+0P84KBS4OhlNYYtTcDk9Wm4wsuMJo0Jl5m6vcs8i+aJiOoOuVyGFwKbYPu0xzCiqzdkMuDXQ1fwxOK/8b+/TuNmGW+GbYqYcJkpXZd5JbvMExHVSY0clPhgWAf8FtkLXX3qo6Rci0/+OoW+i//Gb4eucDejiWHCZaZu78HFLvNERHVXey8nfP9KED5/oRO8nG1xpaAEk9anYejSfdj3D5ummgomXGZK14OLXeaJiEgmk+HpDp6In9YHUU+2gq21FdIu5OOFLxPxwsr9SL1wXeoQ6zwmXGaKPbiIiOi/VNZWmNy3JXa++RjGBDWFtZUM+87kYugX+xC+OgnpV9RSh1hnMeEyU7oVrkbswUVERP/h6qjC24PbY8cbj+H5gMaQy4D4E9kYELMbE9el4mRmodQh1jlMuMxUNntwERHRfTSuXw+Lnu2Iv6L64JmOngCA3w9fRcinuzDu6wPYfzaXxfW1hAmXmWIPLiIiqq5mjezx2chO2DalNwb4uUMuA3aczMGIFfsR+sU+bDtylV3rjYz9BMyUuEuRRfNERFRNbTwc8UVYF5y7VoyVu89iU8olHLqYjwlrU+HTsB5e7t0MQzp5wU7J9MDQuMJlpipv68MVLiIiqhkfFzu8N8QPe2c8gUlPtICTrTXO5d7AnM1HEfh+POZsPsICewOTCbx4azRqtRpOTk4oKCiAo6Ojwd63pFyD1nNjAQCH5j8FJ1trg703ERHVPcWlFdiQdBHfJpzDudwb4nF/b2e8ENgEz3TwhK2NlYQR1i5jfH5zzdAMZatv6zKv4n9CIiJ6OHZKBcJ7+WJcDx8knM3FusQL+ONYJg5ezMfBi/m3bpTt74lBHb0Q0LQ+5HI23K4pflqbodt7cLHLPBERGYpcLkPPFi7o2cIFOYWl2JRyEesPXMDFvJv4bv8FfLf/AtwdVRjYwQPPdPREx8ZO/ByqJiZcZkjXg8uVPbiIiMhIGjko8dpjLfDqo82x98w1/HLwCv44molMdQlW7cnAqj0Z8G5gi6c7eOKptm7o0NgZVlz5uismXGYoiz24iIiolsjlMvRu2Qi9WzbCu6HtsetUDn47fBV/pWfhYt5NLP37DJb+fQbO9azRu2Uj9GnVCI+2cuEu+v9gwmWGdD24XLlDkYiIapHK2gpPtXPHU+3ccaOsAttPZGPrkavYfeoa8m+U47dDV/DboSsAgHaejni0VSN09akPf+/6aGBnI3H00mLCZYay2YOLiIgkVs9Ggac7eOLpDp4o12hx8GI+dp7Mwa7TOTh8qQDHrqhx7IoaS/8d79OwHjo1qY9OTZzRuUl9POLuAGurutOdigmXGcoqZA8uIiIyHdZWcnT1aYCuPg3wRsgjuFZUij2nr2HvP9eQdjEf/2QX4VzuDZzLvYGf0y4DAGys5PBxqYfmjexvPVzt0KKRA5o1srPIxquWd0Z1gK4tBGu4iIjIFLnYKxHayQuhnbwAAAU3ynHwUj7SLlxH2oVbX9UlFTiVVYRTWUV3vL6BnQ0a2Svh6qhEI3slGjlUPhxUCtSzUcDORgE7pRWaNrQzi2J9JlxmqPK2PlzhIiIi0+dUzxp9Wt0qqAcArVbAlYKb+Ce7CGdyinEmpwj/ZBfhbE4RrhWVIa/41uNkVuF93/v4gn5m0ZSVCZeZKavQwkZx65q3K1e4iIjIDMnlMjSuXw+N69fDY4/oP5d/owxXC0qQU1h661FUWvnnwlIUl1WguLQCN8o0uFGmgcraPOrAeGsfIzLWrX0AoFyjhUIuY8M5IiIiA+OtfUhUl3Z2EBERmTuT+NResmQJfHx8oFKpEBgYiAMHDtxz/KZNm9C6dWuoVCr4+flh69ates8LgoB58+bBw8MDtra2CA4OxunTp/XG5OXlISwsDI6OjnB2dkZ4eDiKivQL9w4fPozevXtDpVLB29sbixYtMswJExERUZ0iecK1ceNGREVFYf78+UhNTUXHjh0REhKC7OzsKsfv27cPI0eORHh4ONLS0hAaGorQ0FAcPXpUHLNo0SLExMRg2bJlSExMhJ2dHUJCQlBSUiKOCQsLw7FjxxAXF4ctW7Zg165diIiIEJ9Xq9V46qmn0LRpU6SkpODDDz/EW2+9hRUrVhhvMoiIiMgyCRLr1q2bMHHiRPF7jUYjeHp6CtHR0VWOf/7554WBAwfqHQsMDBReeeUVQRAEQavVCu7u7sKHH34oPp+fny8olUph/fr1giAIQnp6ugBASEpKEsds27ZNkMlkwuXLlwVBEIQvvvhCqF+/vlBaWiqOmTFjhvDII49U+9wKCgoEAEJBQUG1X0NERETSMsbnt6QrXGVlZUhJSUFwcLB4TC6XIzg4GAkJCVW+JiEhQW88AISEhIjjMzIykJmZqTfGyckJgYGB4piEhAQ4OzsjICBAHBMcHAy5XI7ExERxzKOPPgobGxu9n3Py5Elcv369ythKS0uhVqv1HkRERESSJlzXrl2DRqOBm5ub3nE3NzdkZmZW+ZrMzMx7jtd9vd8YV1dXvecVCgUaNGigN6aq97j9Z/xXdHQ0nJycxIe3t3fVJ05ERER1iuQ1XJZk1qxZKCgoEB8XL16UOiQiIiIyAZImXC4uLrCyskJWVpbe8aysLLi7u1f5Gnd393uO132935j/FuVXVFQgLy9Pb0xV73H7z/gvpVIJR0dHvQcRERGRpAmXjY0NunTpgvj4ePGYVqtFfHw8goKCqnxNUFCQ3ngAiIuLE8f7+vrC3d1db4xarUZiYqI4JigoCPn5+UhJSRHHbN++HVqtFoGBgeKYXbt2oby8XO/nPPLII6hfv/5DnjkRERHVKQYrv39AGzZsEJRKpbB69WohPT1diIiIEJydnYXMzExBEARh1KhRwsyZM8Xxe/fuFRQKhfDRRx8Jx48fF+bPny9YW1sLR44cEcd88MEHgrOzs/DLL78Ihw8fFgYPHiz4+voKN2/eFMf069dP6NSpk5CYmCjs2bNHaNmypTBy5Ejx+fz8fMHNzU0YNWqUcPToUWHDhg1CvXr1hOXLl1f73LhLkYiIyPwY4/Nb8oRLEAThs88+E5o0aSLY2NgI3bp1E/bv3y8+16dPH2HMmDF647///nuhVatWgo2NjdCuXTvh999/13teq9UKc+fOFdzc3ASlUin07dtXOHnypN6Y3NxcYeTIkYK9vb3g6OgojBs3TigsLNQbc+jQIaFXr16CUqkUvLy8hA8++KBG58WEi4iIyPwY4/Ob91I0ImPeS5GIiIiMwxif39ylSERERGRkTLiIiIiIjEwhdQCWTHe1lh3niYiIzIfuc9uQVVdMuIyosLAQANhxnoiIyAwVFhbCycnJIO/Fonkj0mq1uHLlChwcHCCTyQz2vmq1Gt7e3rh48WKdL8bnXOjjfFTiXFTiXOjjfFTiXFS6fS4cHBxQWFgIT09PyOWGqb7iCpcRyeVyNG7c2Gjvz272lTgX+jgflTgXlTgX+jgflTgXlXRzYaiVLR0WzRMREREZGRMuIiIiIiNjwmWGlEol5s+fD6VSKXUokuNc6ON8VOJcVOJc6ON8VOJcVDL2XLBonoiIiMjIuMJFREREZGRMuIiIiIiMjAkXERERkZEx4SIiIiIyMiZcZmjJkiXw8fGBSqVCYGAgDhw4IHVIRhcdHY2uXbvCwcEBrq6uCA0NxcmTJ/XGlJSUYOLEiWjYsCHs7e0xbNgwZGVlSRRx7fnggw8gk8kwdepU8VhdmovLly/jxRdfRMOGDWFraws/Pz8kJyeLzwuCgHnz5sHDwwO2trYIDg7G6dOnJYzYODQaDebOnQtfX1/Y2tqiefPmeOedd/TuBWfJc7Fr1y4888wz8PT0hEwmw+bNm/Wer8655+XlISwsDI6OjnB2dkZ4eDiKiopq8SwM415zUV5ejhkzZsDPzw92dnbw9PTE6NGjceXKFb33sJS5AO7/d+N2r776KmQyGT799FO944aYDyZcZmbjxo2IiorC/PnzkZqaio4dOyIkJATZ2dlSh2ZUO3fuxMSJE7F//37ExcWhvLwcTz31FIqLi8Uxr7/+On777Tds2rQJO3fuxJUrVzB06FAJoza+pKQkLF++HB06dNA7Xlfm4vr16+jZsyesra2xbds2pKenY/Hixahfv744ZtGiRYiJicGyZcuQmJgIOzs7hISEoKSkRMLIDW/hwoVYunQpPv/8cxw/fhwLFy7EokWL8Nlnn4ljLHkuiouL0bFjRyxZsqTK56tz7mFhYTh27Bji4uKwZcsW7Nq1CxEREbV1CgZzr7m4ceMGUlNTMXfuXKSmpuKnn37CyZMnMWjQIL1xljIXwP3/buj8/PPP2L9/Pzw9Pe94ziDzIZBZ6datmzBx4kTxe41GI3h6egrR0dESRlX7srOzBQDCzp07BUEQhPz8fMHa2lrYtGmTOOb48eMCACEhIUGqMI2qsLBQaNmypRAXFyf06dNHmDJliiAIdWsuZsyYIfTq1euuz2u1WsHd3V348MMPxWP5+fmCUqkU1q9fXxsh1pqBAwcKL730kt6xoUOHCmFhYYIg1K25ACD8/PPP4vfVOff09HQBgJCUlCSO2bZtmyCTyYTLly/XWuyG9t+5qMqBAwcEAML58+cFQbDcuRCEu8/HpUuXBC8vL+Ho0aNC06ZNhU8++UR8zlDzwRUuM1JWVoaUlBQEBweLx+RyOYKDg5GQkCBhZLWvoKAAANCgQQMAQEpKCsrLy/XmpnXr1mjSpInFzs3EiRMxcOBAvXMG6tZc/PrrrwgICMBzzz0HV1dXdOrUCStXrhSfz8jIQGZmpt5cODk5ITAw0OLmokePHoiPj8epU6cAAIcOHcKePXvQv39/AHVrLv6rOueekJAAZ2dnBAQEiGOCg4Mhl8uRmJhY6zHXpoKCAshkMjg7OwOoe3Oh1WoxatQovPnmm2jXrt0dzxtqPnjzajNy7do1aDQauLm56R13c3PDiRMnJIqq9mm1WkydOhU9e/ZE+/btAQCZmZmwsbERf2HouLm5ITMzU4IojWvDhg1ITU1FUlLSHc/Vpbk4e/Ysli5diqioKMyePRtJSUmYPHkybGxsMGbMGPF8q/p/xtLmYubMmVCr1WjdujWsrKyg0Wjw3nvvISwsDADq1Fz8V3XOPTMzE66urnrPKxQKNGjQwKLnp6SkBDNmzMDIkSPFm1fXtblYuHAhFAoFJk+eXOXzhpoPJlxkdiZOnIijR49iz549UociiYsXL2LKlCmIi4uDSqWSOhxJabVaBAQE4P333wcAdOrUCUePHsWyZcswZswYiaOrXd9//z3Wrl2LdevWoV27djh48CCmTp0KT0/POjcXVD3l5eV4/vnnIQgCli5dKnU4kkhJScH//vc/pKamQiaTGfVn8ZKiGXFxcYGVldUdu82ysrLg7u4uUVS1KzIyElu2bMGOHTvQuHFj8bi7uzvKysqQn5+vN94S5yYlJQXZ2dno3LkzFAoFFAoFdu7ciZiYGCgUCri5udWZufDw8EDbtm31jrVp0wYXLlwAAPF868L/M2+++SZmzpyJESNGwM/PD6NGjcLrr7+O6OhoAHVrLv6rOufu7u5+x+ajiooK5OXlWeT86JKt8+fPIy4uTlzdAurWXOzevRvZ2dlo0qSJ+Pv0/PnzmDZtGnx8fAAYbj6YcJkRGxsbdOnSBfHx8eIxrVaL+Ph4BAUFSRiZ8QmCgMjISPz888/Yvn07fH199Z7v0qULrK2t9ebm5MmTuHDhgsXNTd++fXHkyBEcPHhQfAQEBCAsLEz8c12Zi549e97RHuTUqVNo2rQpAMDX1xfu7u56c6FWq5GYmGhxc3Hjxg3I5fq/0q2srKDVagHUrbn4r+qce1BQEPLz85GSkiKO2b59O7RaLQIDA2s9ZmPSJVunT5/GX3/9hYYNG+o9X5fmYtSoUTh8+LDe71NPT0+8+eab+OOPPwAYcD4evNafpLBhwwZBqVQKq1evFtLT04WIiAjB2dlZyMzMlDo0o5owYYLg5OQk/P3338LVq1fFx40bN8Qxr776qtCkSRNh+/btQnJyshAUFCQEBQVJGHXtuX2XoiDUnbk4cOCAoFAohPfee084ffq0sHbtWqFevXrCd999J4754IMPBGdnZ+GXX34RDh8+LAwePFjw9fUVbt68KWHkhjdmzBjBy8tL2LJli5CRkSH89NNPgouLizB9+nRxjCXPRWFhoZCWliakpaUJAISPP/5YSEtLE3feVefc+/XrJ3Tq1ElITEwU9uzZI7Rs2VIYOXKkVKf0wO41F2VlZcKgQYOExo0bCwcPHtT7fVpaWiq+h6XMhSDc/+/Gf/13l6IgGGY+mHCZoc8++0xo0qSJYGNjI3Tr1k3Yv3+/1CEZHYAqH19//bU45ubNm8Jrr70m1K9fX6hXr54wZMgQ4erVq9IFXYv+m3DVpbn47bffhPbt2wtKpVJo3bq1sGLFCr3ntVqtMHfuXMHNzU1QKpVC3759hZMnT0oUrfGo1WphypQpQpMmTQSVSiU0a9ZM+L//+z+9D1FLnosdO3ZU+TtizJgxgiBU79xzc3OFkSNHCvb29oKjo6Mwbtw4obCwUIKzeTj3mouMjIy7/j7dsWOH+B6WMheCcP+/G/9VVcJliPmQCcJtbYiJiIiIyOBYw0VERERkZEy4iIiIiIyMCRcRERGRkTHhIiIiIjIyJlxERERERsaEi4iIiMjImHARERERGRkTLiIiIiIjY8JFREREZGRMuIiozsnJycGECRPQpEkTKJVKuLu7IyQkBHv37gUAyGQybN68WdogiciiKKQOgIiotg0bNgxlZWVYs2YNmjVrhqysLMTHxyM3N1fq0IjIQnGFi4jqlPz8fOzevRsLFy7E448/jqZNm6Jbt26YNWsWBg0aBB8fHwDAkCFDIJPJxO8B4JdffkHnzp2hUqnQrFkzvP3226ioqBCfl8lkWLp0Kfr37w9bW1s0a9YMP/zwg/h8WVkZIiMj4eHhAZVKhaZNmyI6Orq2Tp2IJMSEi4jqFHt7e9jb22Pz5s0oLS294/mkpCQAwNdff42rV6+K3+/evRujR4/GlClTkJ6ejuXLl2P16tV477339F4/d+5cDBs2DIcOHUJYWBhGjBiB48ePAwBiYmLw66+/4vvvv8fJkyexdu1avYSOiCyXTBAEQeogiIhq048//ojx48fj5s2b6Ny5M/r06YMRI0agQ4cOAG6tVP38888IDQ0VXxMcHIy+ffti1qxZ4rHvvvsO06dPx5UrV8TXvfrqq1i6dKk4pnv37ujcuTO++OILTJ48GceOHcNff/0FmUxWOydLRCaBK1xEVOcMGzYMV65cwa+//op+/frh77//RufOnbF69eq7vubQoUNYsGCBuEJmb2+P8ePH4+rVq7hx44Y4LigoSO91QUFB4grX2LFjcfDgQTzyyCOYPHky/vzzT6OcHxGZHiZcRFQnqVQqPPnkk5g7dy727duHsWPHYv78+XcdX1RUhLfffhsHDx4UH0eOHMHp06ehUqmq9TM7d+6MjIwMvPPOO7h58yaef/55PPvss4Y6JSIyYUy4iIgAtG3bFsXFxQAAa2traDQavec7d+6MkydPokWLFnc85PLKX6X79+/Xe93+/fvRpk0b8XtHR0cMHz4cK1euxMaNG/Hjjz8iLy/PiGdGRKaAbSGIqE7Jzc3Fc889h5deegkdOnSAg4MDkpOTsWjRIgwePBgA4OPjg/j4ePTs2RNKpRL169fHvHnz8PTTT6NJkyZ49tlnIZfLcejQIRw9ehTvvvuu+P6bNm1CQEAAevXqhbVr1+LAgQNYtWoVAODjjz+Gh4cHOnXqBLlcjk2bNsHd3R3Ozs5STAUR1SImXERUp9jb2yMwMBCffPIJzpw5g/Lycnh7e2P8+PGYPXs2AGDx4sWIiorCypUr4eXlhXPnziEkJARbtmzBggULsHDhQlhbW6N169Z4+eWX9d7/7bffxoYNG/Daa6/Bw8MD69evR9u2bQEADg4OWLRoEU6fPg0rKyt07doVW7du1VshIyLLxF2KREQGUtXuRiIigDVcREREREbHhIuIiIjIyFjDRURkIKzQIKK74QoXERERkZEx4SIiIiIyMiZcREREREbGhIuIiIjIyJhwERERERkZEy4iIiIiI2PCRURERGRkTLiIiIiIjOz/AbwH5M1xnvSWAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(range(len(lrs)), lrs)\n",
    "plt.ylabel(\"Learning rate\")\n",
    "plt.xlabel(\"Steps\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a2f85b01-859b-4454-a3a3-c7ef593735a6",
   "metadata": {},
   "source": [
    "- And a quick look at the loss curves"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "445d8155-6eae-4b50-a381-d0820ebc27cc",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAnUAAAHWCAYAAAARl3+JAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAA9hAAAPYQGoP6dpAAB0m0lEQVR4nO3dd3wT9f8H8NcladKku6WTTqDQwZ5CkS1DRAERRH4MURQEAQfiAsGFICriAOWr4EBRlKWyyoayV6G0lN0W6KCU7p3c74+jaUMLFGh7afp6Ph73aPK59c4Z2xd39/mcIIqiCCIiIiKq1RRyF0BERERED46hjoiIiMgCMNQRERERWQCGOiIiIiILwFBHREREZAEY6oiIiIgsAEMdERERkQVgqCMiIiKyAAx1RERERBaAoY6Iap1Lly5BEAQcP35c7lKIiMwGQx0RyUIQhDtOs2bNkrtEIqJaRSV3AURUNyUmJhpf//HHH5g5cyZiY2ONbba2tnKURURUa/FMHRHJwsPDwzg5ODhAEATjezc3N3z++efw9vaGRqNBy5YtsXHjxttuS6/XY+zYsQgKCkJ8fDwAYO3atWjdujWsra3RoEEDzJ49G8XFxcZ1BEHA//73PwwaNAg6nQ6BgYFYt26dcf6NGzcwYsQIuLq6QqvVIjAwEEuXLr1tDX/99ReaNWsGrVYLFxcX9OrVCzk5Ocb5//vf/xAcHAxra2sEBQXh22+/NVk/ISEBQ4cOhaOjI5ydnfHEE0/g0qVLxvljxozBwIEDMX/+fHh6esLFxQUTJ05EUVFRpY85EVk2hjoiMjtffvklPvvsM8yfPx8nTpxAnz598Pjjj+Ps2bPlli0oKMBTTz2F48ePY/fu3fD19cXu3bsxatQoTJkyBdHR0fjuu++wbNkyfPTRRybrzp49G0OHDsWJEyfw6KOPYsSIEUhLSwMAzJgxA9HR0diwYQNiYmKwaNEi1KtXr8J6ExMTMXz4cIwdOxYxMTHYsWMHBg8eDFEUAQDLly/HzJkz8dFHHyEmJgYff/wxZsyYgZ9++gkAUFRUhD59+sDOzg67d+9GREQEbG1t0bdvXxQWFhr3s337dpw/fx7bt2/HTz/9hGXLlmHZsmVVcciJyBKIREQyW7p0qejg4GB87+XlJX700Ucmy7Rr10586aWXRFEUxYsXL4oAxN27d4s9e/YUO3fuLKanpxuX7dmzp/jxxx+brP/LL7+Inp6exvcAxHfffdf4Pjs7WwQgbtiwQRRFURwwYID47LPPVqr+I0eOiADES5cuVTi/YcOG4m+//WbS9sEHH4gdO3Y01takSRPRYDAY5xcUFIharVbctGmTKIqiOHr0aNHPz08sLi42LvPUU0+Jw4YNq1SNRGT5eE8dEZmVzMxMXL16FWFhYSbtYWFhiIyMNGkbPnw4vL29sW3bNmi1WmN7ZGQkIiIiTM7M6fV65OfnIzc3FzqdDgDQvHlz43wbGxvY29sjJSUFADBhwgQ8+eSTOHr0KHr37o2BAweiU6dOFdbcokUL9OzZE82aNUOfPn3Qu3dvDBkyBE5OTsjJycH58+fx3HPPYdy4ccZ1iouL4eDgYKz33LlzsLOzM9lufn4+zp8/b3wfGhoKpVJpfO/p6YmTJ0/e4WgSUV3CUEdEtdajjz6KX3/9Ffv27UOPHj2M7dnZ2Zg9ezYGDx5cbh1ra2vjaysrK5N5giDAYDAAAPr164e4uDisX78e4eHh6NmzJyZOnIj58+eX26ZSqUR4eDj27t2LzZs346uvvsI777yDAwcOGAPkkiVL0KFDh3LrldTbpk0bLF++vNy2XV1dK1UvERFDHRGZFXt7e3h5eSEiIgJdu3Y1tkdERKB9+/Ymy06YMAFNmzbF448/jv/++8+4fOvWrREbG4tGjRo9UC2urq4YPXo0Ro8ejYcffhjTpk2rMNQBUsAKCwtDWFgYZs6cCT8/P6xevRqvvvoqvLy8cOHCBYwYMaLCdVu3bo0//vgDbm5usLe3f6CaiajuYqgjIrMzbdo0vPfee2jYsCFatmyJpUuX4vjx4xWeyXr55Zeh1+vx2GOPYcOGDejcuTNmzpyJxx57DL6+vhgyZAgUCgUiIyMRFRWFDz/8sFI1zJw5E23atEFoaCgKCgrw77//Ijg4uMJlDxw4gK1bt6J3795wc3PDgQMHcO3aNePys2fPxuTJk+Hg4IC+ffuioKAAhw8fxo0bN/Dqq69ixIgR+PTTT/HEE0/g/fffh7e3N+Li4rBq1Sq88cYb8Pb2vv+DSUR1BkMdEZmdyZMnIyMjA6+99hpSUlIQEhKCdevWITAwsMLlp06dCoPBgEcffRQbN25Enz598O+//+L999/H3LlzYWVlhaCgIDz//POVrkGtVuOtt97CpUuXoNVq8fDDD2PFihUVLmtvb49du3ZhwYIFyMzMhJ+fHz777DP069cPAPD8889Dp9Ph008/xbRp02BjY4NmzZph6tSpAACdToddu3Zh+vTpGDx4MLKyslC/fn307NmTZ+6IqNIEUbzZ556IiIiIai2OU0dERERkARjqiIiIiCwAQx0RERGRBWCoIyIiIrIADHVEREREFoChjoiIiMgCMNTdo2+++Qb+/v6wtrZGhw4dcPDgQblLqlK7du3CgAED4OXlBUEQsGbNGpP5oihi5syZ8PT0hFarRa9evXD27FmTZdLS0jBixAjY29vD0dERzz33HLKzs02WOXHiBB5++GFYW1vDx8cH8+bNK1fLypUrERQUBGtrazRr1gzr16+v8s97v+bMmYN27drBzs4Obm5uGDhwIGJjY02Wyc/Px8SJE+Hi4gJbW1s8+eSTSE5ONlkmPj4e/fv3h06ng5ubG6ZNm4bi4mKTZXbs2IHWrVtDo9GgUaNGWLZsWbl6zPV7uWjRIjRv3hz29vawt7dHx44dsWHDBuN8HqOKffLJJxAEwTiOHcBjBQCzZs2CIAgmU1BQkHE+j5GpK1eu4P/+7//g4uICrVaLZs2a4fDhw8b5/H0O+Pv7l/tOCYKAiRMnAqiF3ymRKm3FihWiWq0Wf/zxR/HUqVPiuHHjREdHRzE5OVnu0qrM+vXrxXfeeUdctWqVCEBcvXq1yfxPPvlEdHBwENesWSNGRkaKjz/+uBgQECDm5eUZl+nbt6/YokULcf/+/eLu3bvFRo0aicOHDzfOz8jIEN3d3cURI0aIUVFR4u+//y5qtVrxu+++My4TEREhKpVKcd68eWJ0dLT47rvvilZWVuLJkyer/RhURp8+fcSlS5eKUVFR4vHjx8VHH31U9PX1FbOzs43LjB8/XvTx8RG3bt0qHj58WHzooYfETp06GecXFxeLTZs2FXv16iUeO3ZMXL9+vVivXj3xrbfeMi5z4cIFUafTia+++qoYHR0tfvXVV6JSqRQ3btxoXMacv5fr1q0T//vvP/HMmTNibGys+Pbbb4tWVlZiVFSUKIo8RhU5ePCg6O/vLzZv3lycMmWKsZ3HShTfe+89MTQ0VExMTDRO165dM87nMSqVlpYm+vn5iWPGjBEPHDggXrhwQdy0aZN47tw54zL8fS6KKSkpJt+n8PBwEYC4fft2URRr33eKoe4etG/fXpw4caLxvV6vF728vMQ5c+bIWFX1uTXUGQwG0cPDQ/z000+Nbenp6aJGoxF///13URRFMTo6WgQgHjp0yLjMhg0bREEQxCtXroiiKIrffvut6OTkJBYUFBiXmT59utikSRPj+6FDh4r9+/c3qadDhw7iiy++WKWfsaqkpKSIAMSdO3eKoigdFysrK3HlypXGZWJiYkQA4r59+0RRlAK0QqEQk5KSjMssWrRItLe3Nx6bN954QwwNDTXZ17Bhw8Q+ffoY39e276WTk5P4v//9j8eoAllZWWJgYKAYHh4udu3a1RjqeKwk7733ntiiRYsK5/EYmZo+fbrYuXPn287n7/OKTZkyRWzYsKFoMBhq5XeKl18rqbCwEEeOHEGvXr2MbQqFAr169cK+fftkrKzmXLx4EUlJSSbHwMHBAR06dDAeg3379sHR0RFt27Y1LtOrVy8oFAocOHDAuEyXLl2gVquNy/Tp0wexsbG4ceOGcZmy+ylZxlyPdUZGBgDA2dkZAHDkyBEUFRWZfIagoCD4+vqaHKtmzZrB3d3duEyfPn2QmZmJU6dOGZe503GoTd9LvV6PFStWICcnBx07duQxqsDEiRPRv3//cp+Hx6rU2bNn4eXlhQYNGmDEiBGIj48HwGN0q3Xr1qFt27Z46qmn4ObmhlatWmHJkiXG+fx9Xl5hYSF+/fVXjB07FoIg1MrvFENdJaWmpkKv15v8hwMAd3d3JCUlyVRVzSr5nHc6BklJSXBzczOZr1Kp4OzsbLJMRdsou4/bLWOOx9pgMGDq1KkICwtD06ZNAUj1q9VqODo6mix767G63+OQmZmJvLy8WvG9PHnyJGxtbaHRaDB+/HisXr0aISEhPEa3WLFiBY4ePYo5c+aUm8djJenQoQOWLVuGjRs3YtGiRbh48SIefvhhZGVl8Rjd4sKFC1i0aBECAwOxadMmTJgwAZMnT8ZPP/0EgL/PK7JmzRqkp6djzJgxAGrn/3eqe1qaiMqZOHEioqKisGfPHrlLMUtNmjTB8ePHkZGRgb/++gujR4/Gzp075S7LrCQkJGDKlCkIDw+HtbW13OWYrX79+hlfN2/eHB06dICfnx/+/PNPaLVaGSszPwaDAW3btsXHH38MAGjVqhWioqKwePFijB49WubqzNMPP/yAfv36wcvLS+5S7hvP1FVSvXr1oFQqy/V6SU5OhoeHh0xV1aySz3mnY+Dh4YGUlBST+cXFxUhLSzNZpqJtlN3H7ZYxt2M9adIk/Pvvv9i+fTu8vb2N7R4eHigsLER6errJ8rceq/s9Dvb29tBqtbXie6lWq9GoUSO0adMGc+bMQYsWLfDll1/yGJVx5MgRpKSkoHXr1lCpVFCpVNi5cycWLlwIlUoFd3d3HqsKODo6onHjxjh37hy/T7fw9PRESEiISVtwcLDxcjV/n5uKi4vDli1b8PzzzxvbauN3iqGuktRqNdq0aYOtW7ca2wwGA7Zu3YqOHTvKWFnNCQgIgIeHh8kxyMzMxIEDB4zHoGPHjkhPT8eRI0eMy2zbtg0GgwEdOnQwLrNr1y4UFRUZlwkPD0eTJk3g5ORkXKbsfkqWMZdjLYoiJk2ahNWrV2Pbtm0ICAgwmd+mTRtYWVmZfIbY2FjEx8ebHKuTJ0+a/NIMDw+Hvb298Zfx3Y5DbfxeGgwGFBQU8BiV0bNnT5w8eRLHjx83Tm3btsWIESOMr3msysvOzsb58+fh6enJ79MtwsLCyg2zdObMGfj5+QHg7/NbLV26FG5ubujfv7+xrVZ+p+6pW0Udt2LFClGj0YjLli0To6OjxRdeeEF0dHQ06fVS22VlZYnHjh0Tjx07JgIQP//8c/HYsWNiXFycKIpSF3hHR0dx7dq14okTJ8Qnnniiwi7wrVq1Eg8cOCDu2bNHDAwMNOkCn56eLrq7u4sjR44Uo6KixBUrVog6na5cF3iVSiXOnz9fjImJEd977z2z6QIviqI4YcIE0cHBQdyxY4dJd/jc3FzjMuPHjxd9fX3Fbdu2iYcPHxY7duwoduzY0Ti/pCt87969xePHj4sbN24UXV1dK+wKP23aNDEmJkb85ptvKuwKb67fyzfffFPcuXOnePHiRfHEiRPim2++KQqCIG7evFkURR6jOynb+1UUeaxEURRfe+01cceOHeLFixfFiIgIsVevXmK9evXElJQUURR5jMo6ePCgqFKpxI8++kg8e/asuHz5clGn04m//vqrcRn+Ppfo9XrR19dXnD59erl5te07xVB3j7766ivR19dXVKvVYvv27cX9+/fLXVKV2r59uwig3DR69GhRFKVu8DNmzBDd3d1FjUYj9uzZU4yNjTXZxvXr18Xhw4eLtra2or29vfjss8+KWVlZJstERkaKnTt3FjUajVi/fn3xk08+KVfLn3/+KTZu3FhUq9ViaGio+N9//1Xb575XFR0jAOLSpUuNy+Tl5YkvvfSS6OTkJOp0OnHQoEFiYmKiyXYuXbok9uvXT9RqtWK9evXE1157TSwqKjJZZvv27WLLli1FtVotNmjQwGQfJcz1ezl27FjRz89PVKvVoqurq9izZ09joBNFHqM7uTXU8VhJw0B4enqKarVarF+/vjhs2DCTcdd4jEz9888/YtOmTUWNRiMGBQWJ33//vcl8/j6XbNq0SQRQ7rOLYu37TgmiKIr3dm6PiIiIiMwN76kjIiIisgAMdUREREQWgKGOiIiIyAIw1BERERFZAIY6IiIiIgvAUEdERERkARjq7kNBQQFmzZqFgoICuUsxazxOlcPjVHk8VpXD41Q5PE6Vx2NVOXIfJ45Tdx8yMzPh4OCAjIwM2Nvby12O2eJxqhwep8rjsaocHqfK4XGqPB6rypH7OPFMHREREZEFYKgjIiIisgAquQuobsXFxTh27Bjc3d2hUFRNhs3KygIAXLlyBZmZmVWyTUvE41Q5PE6Vx2NVOTxOlcPjVHk8VpVT9jilp6cjOTkZrVq1gkpVM3HL4u+pO3ToENq3by93GURERFQHHTx4EO3atauRfVn8mTp3d3cA0kH19PSUuRoiIiKqCxITE9G+fXtjDqkJFh/qSi65enp6wtvbW+ZqiIiIqC6pqlu/KrWvGtsTEREREVUbhjoiIiIiC8BQR0RERGQBLP6eOiIiorL0ej2KiorkLoNqOSsrKyiVSrnLMMFQR0REdYIoikhKSkJ6errcpZCFcHR0hIeHBwRBkLsUAAx1RERUR5QEOjc3N+h0OrP5Q0y1jyiKyM3NRUpKCgCYzZBpDHVERGTx9Hq9MdC5uLjIXQ5ZAK1WCwBISUmBm5ubWVyKZUcJIiKyeCX30Ol0OpkrIUtS8n0yl3s0GeqIiKjO4CVXqkrm9n1iqCMiIiKyAAx1REREdYi/vz8WLFhQ6eV37NgBQRCqvdfwsmXL4OjoWK37sHQMdURERGZIEIQ7TrNmzbqv7R46dAgvvPBCpZfv1KkTEhMT4eDgcF/7o5rD3q9ERERmKDEx0fj6jz/+wMyZMxEbG2tss7W1Nb4WRRF6vR4q1d3/rLu6ut5THWq1Gh4eHve0DsmDZ+oeUH6RHjsiz+Hs4S1yl0JERBbEw8PDODk4OEAQBOP706dPw87ODhs2bECbNm2g0WiwZ88enD9/Hk888QTc3d1ha2uLdu3aYcsW079Pt15+FQQB//vf/zBo0CDodDoEBgZi3bp1xvm3Xn4tuUy6adMmBAcHw9bWFn379jUJocXFxZg8eTIcHR3h4uKC6dOnY/To0Rg4cOA9HYNFixahYcOGUKvVaNKkCX755RfjPFEUMWvWLPj6+kKj0cDLywuTJ082zv/2228RGBgIa2truLu7Y8iQIfe079qIoe4BLV/7Hx5a9RA8N4wFigvlLoeIiCpJFEXkFhbX+CSKYpV9hjfffBOffPIJYmJi0Lx5c2RnZ+PRRx/F1q1bcezYMfTt2xcDBgxAfHz8Hbcze/ZsDB06FCdOnMCjjz6KESNGIC0t7bbL5+bmYv78+fjll1+wa9cuxMfH4/XXXzfOnzt3LpYvX46lS5ciIiICmZmZWLNmzT19ttWrV2PKlCl47bXXEBUVhRdffBHPPvsstm/fDgD4+++/8cUXX+C7777D2bNnsWbNGjRr1gwAcPjwYUyePBnvv/8+YmNjsXHjRnTp0uWe9l8b8fLrA2rasgMyTtrAXZ8O/dnNUAY/JndJRERUCXlFeoTM3FTj+41+vw906qr58/v+++/jkUceMb53dnZGixYtjO8/+OADrF69GuvWrcOkSZNuu50xY8Zg+PDhAICPP/4YCxcuxMGDB9G3b98Kly8qKsLixYvRsGFDAMCkSZPw/vvvG+d/9dVXeOuttzBo0CAAwNdff43169ff02ebP38+xowZg5deegkA8Oqrr2L//v2YP38+unfvjvj4eHh4eKBXr16wsrKCr68v2rdvDwCIj4+HjY0NHnvsMdjZ2cHPzw+tWrW6p/3XRjxT94DaBLhig/AwACBz/y93WZqIiKjqtG3b1uR9dnY2Xn/9dQQHB8PR0RG2traIiYm565m65s2bG1/b2NjA3t7e+Aisiuh0OmOgA6THZJUsn5GRgeTkZGPAAgClUok2bdrc02eLiYlBWFiYSVtYWBhiYmIAAE899RTy8vLQoEEDjBs3DqtXr0ZxcTEA4JFHHoGfnx8aNGiAkSNHYvny5cjNzb2n/ddGPFP3gFRKBRL9BwGX/oF9/BYgNw3QOctdFhER3YXWSono9/vIst+qYmNjY/L+9ddfR3h4OObPn49GjRpBq9ViyJAhKCy88+1BVlZWJu8FQYDBYLin5avysnJl+Pj4IDY2Flu2bEF4eDheeuklfPrpp9i5cyfs7Oxw9OhR7NixA5s3b8bMmTMxa9YsHDp0yKKHTeGZuioQ0vIhnDL4QSkWA6dWyV0OERFVgiAI0KlVNT5V51MIIiIiMGbMGAwaNAjNmjWDh4cHLl26VG37q4iDgwPc3d1x6NAhY5ter8fRo0fvaTvBwcGIiIgwaYuIiEBISIjxvVarxYABA7Bw4ULs2LED+/btw8mTJwEAKpUKvXr1wrx583DixAlcunQJ27Zte4BPZv54pq4KdG3siq8NXRCq+AUFR36Dpt3zcpdERER1UGBgIFatWoUBAwZAEATMmDHjjmfcqsvLL7+MOXPmoFGjRggKCsJXX32FGzdu3FOgnTZtGoYOHYpWrVqhV69e+Oeff7Bq1Spjb95ly5ZBr9ejQ4cO0Ol0+PXXX6HVauHn54d///0XFy5cQJcuXeDk5IT169fDYDCgSZMm1fWRzQLP1FUBR50acV79UCwqoEk6AqSek7skIiKqgz7//HM4OTmhU6dOGDBgAPr06YPWrVvXeB3Tp0/H8OHDMWrUKHTs2BG2trbo06cPrK2tK72NgQMH4ssvv8T8+fMRGhqK7777DkuXLkW3bt0AAI6OjliyZAnCwsLQvHlzbNmyBf/88w9cXFzg6OiIVatWoUePHggODsbixYvx+++/IzQ0tJo+sXkQxJq+CF7DLl++DB8fHyQkJMDb27va9vPdzvMI3DIWPZTHgS7TgB7vVtu+iIjo3uTn5+PixYsICAi4p2BBVcNgMCA4OBhDhw7FBx98IHc5VeZO36uayh9l8UxdFekZ7IZVeqkXrCFyBSDD6W4iIiJzEBcXhyVLluDMmTM4efIkJkyYgIsXL+KZZ56RuzSLxlBXRRq62uK0Q2dkilooMhKA+L1yl0RERCQLhUKBZcuWoV27dggLC8PJkyexZcsWBAcHy12aRWNHiSoiCAI6B/vgv4MPYbhqOxD5O+DfWe6yiIiIapyPj0+5nqtU/XimrgqVvQQrnloLFFr+QIdERERkHhjqqlD7AGfEWAXjL30XJIR9DCiqboBJIiIiojthqKtCGpUSnQPd8XrReKwqeghQaeQuiYiIiOoIhroq1iPYDQCw7fTtn5lHREREVNUY6qpY9yZSqEu+fBHZW+YBkX/IXBERERHVBQx1VczVToMWPo7oqTwG2z0fAXsXyl0SERER1QEMddWgZ5Ab/tV3QIy2NdBhPAciJiIi2XTr1g1Tp041vvf398eCBQvuuI4gCFizZs0D77uqtnMns2bNQsuWLat1H7UFQ1016BHkhkzY4smc6chv9gyg4GEmIqJ7M2DAAPTt27fCebt374YgCDhx4sQ9b/fQoUN44YUXHrQ8E7cLVomJiejXr1+V7otuj2mjGoR62cPdXoPcQj0OXEyTuxwiIqqFnnvuOYSHh+Py5cvl5i1duhRt27ZF8+bN73m7rq6u0Ol0VVHiXXl4eECj4UgQNYWhrhoIgoAeQe4AgAMnTgP7FwHx+2WuioiIapPHHnsMrq6uWLZsmUl7dnY2Vq5cieeeew7Xr1/H8OHDUb9+feh0OjRr1gy///77Hbd76+XXs2fPokuXLrC2tkZISAjCw8PLrTN9+nQ0btwYOp0ODRo0wIwZM1BUVAQAWLZsGWbPno3IyEgIggBBEIw133r59eTJk+jRowe0Wi1cXFzwwgsvIDs72zh/zJgxGDhwIObPnw9PT0+4uLhg4sSJxn1VhsFgwPvvvw9vb29oNBq0bNkSGzduNM4vLCzEpEmT4OnpCWtra/j5+WHOnDkAAFEUMWvWLPj6+kKj0cDLywuTJ0+u9L7lxseEVZOeQW74/WA8AmIWAVH/AU2fBHwfkrssIiK6VWHOva+j1ADKm39C9cWAvgAQFICV9s7bVdtUehcqlQqjRo3CsmXL8M4770AQBADAypUrodfrMXz4cGRnZ6NNmzaYPn067O3t8d9//2HkyJFo2LAh2rdvf9d9GAwGDB48GO7u7jhw4AAyMjJM7r8rYWdnh2XLlsHLywsnT57EuHHjYGdnhzfeeAPDhg1DVFQUNm7ciC1btgAAHBwcym0jJycHffr0QceOHXHo0CGkpKTg+eefx6RJk0yC6/bt2+Hp6Ynt27fj3LlzGDZsGFq2bIlx48ZV6rh9+eWX+Oyzz/Ddd9+hVatW+PHHH/H444/j1KlTCAwMxMKFC7Fu3Tr8+eef8PX1RUJCAhISEgAAf//9N7744gusWLECoaGhSEpKQmRkZKX2aw4Y6qpJWKN60KgU+Cm3I57S/Aec/g/IzwCsy3/RiYhIRh973fs6Ty0DQgdJr0//A6wcA/h1Bp79r3SZBc2A3Oum683KuKfdjB07Fp9++il27tyJbt26AZAuvT755JNwcHCAg4MDXn/9dePyL7/8MjZt2oQ///yzUqFuy5YtOH36NDZt2gQvL+k4fPzxx+Xug3v33XeNr/39/fH6669jxYoVeOONN6DVamFrawuVSgUPD4/b7uu3335Dfn4+fv75Z9jYSOH266+/xoABAzB37ly4u0tXuJycnPD1119DqVQiKCgI/fv3x9atWysd6ubPn4/p06fj6aefBgDMnTsX27dvx4IFC/DNN98gPj4egYGB6Ny5MwRBgJ+fn3Hd+Ph4eHh4oFevXrCysoKvr2+ljqO54OXXaqJVK9GpoQuixACk6RoAxflA9Fq5yyIiolokKCgInTp1wo8//ggAOHfuHHbv3o3nnnsOAKDX6/HBBx+gWbNmcHZ2hq2tLTZt2oT4+PhKbT8mJgY+Pj7GQAcAHTt2LLfcH3/8gbCwMHh4eMDW1hbvvvtupfdRdl8tWrQwBjoACAsLg8FgQGxsrLEtNDQUSmXpYzY9PT2RklK5Af0zMzNx9epVhIWFmbSHhYUhJiYGgHSJ9/jx42jSpAkmT56MzZs3G5d76qmnkJeXhwYNGmDcuHFYvXo1iouL7+lzyoln6qpRj2B3bI+9hvWKrvg/XAAiVwCtR8ldFhERlfX21XtfR1nm5v+gAdI2hFvOk0w9+WB13fTcc8/h5ZdfxjfffIOlS5eiYcOG6Nq1KwDg008/xZdffokFCxagWbNmsLGxwdSpU1FYWFgl+waAffv2YcSIEZg9ezb69OkDBwcHrFixAp999lmV7aMsKysrk/eCIMBQhUODtW7dGhcvXsSGDRuwZcsWDB06FL169cJff/0FHx8fxMbGYsuWLQgPD8dLL71kPFN6a13mSNYzdbt27cKAAQPg5eVV4Vg2oihi5syZ8PT0hFarRa9evXD27Fl5ir0PPYKkp0t8e701RAhAXARw45K8RRERkSm1zb1PyjLnRJQqqa3s/XS32+59GDp0KBQKBX777Tf8/PPPGDt2rPH+uoiICDzxxBP4v//7P7Ro0QINGjTAmTNnKr3t4OBgJCQkIDEx0di2f79px769e/fCz88P77zzDtq2bYvAwEDExcWZflS1Gnq9/q77ioyMRE5O6b2GERERUCgUaNKkSaVrvhN7e3t4eXkhIiLCpD0iIgIhISEmyw0bNgxLlizBH3/8gb///htpadJoFVqtFgMGDMDChQuxY8cO7Nu3DydPVk1Ar26yhrqcnBy0aNEC33zzTYXz582bh4ULF2Lx4sU4cOAAbGxs0KdPH+Tn59dwpfenvqMWQR52uCq64JrrzU4SJ/6UtygiIqpVbG1tMWzYMLz11ltITEzEmDFjjPMCAwMRHh6OvXv3IiYmBi+++CKSk5Mrve1evXqhcePGGD16NCIjI7F792688847JssEBgYiPj4eK1aswPnz57Fw4UKsXr3aZBl/f39cvHgRx48fR2pqKgoKCsrta8SIEbC2tsbo0aMRFRWF7du34+WXX8bIkSON99NVhWnTpmHu3Ln4448/EBsbizfffBPHjx/HlClTAACff/45fv/9d5w+fRpnzpzBypUr4eHhAUdHRyxbtgw//PADoqKicOHCBfz666/QarUm992ZM1lDXb9+/fDhhx9i0KBB5eaJoogFCxbg3XffxRNPPIHmzZvj559/xtWrV6t9dOqq1DNYOlu3QdlNaoj8HRBF+QoiIqJa57nnnsONGzfQp08fk/vf3n33XbRu3Rp9+vRBt27d4OHhgYEDB1Z6uwqFAqtXr0ZeXh7at2+P559/Hh999JHJMo8//jheeeUVTJo0CS1btsTevXsxY8YMk2WefPJJ9O3bF927d4erq2uFw6rodDps2rQJaWlpaNeuHYYMGYKePXvi66+/vreDcReTJ0/Gq6++itdeew3NmjXDxo0bsW7dOgQGBgKQevLOmzcPbdu2Rbt27XDp0iWsX78eCoUCjo6OWLJkCcLCwtC8eXNs2bIF//zzD1xcXKq0xuoiiKJ5JAxBELB69Wrjl/HChQto2LAhjh07ZjJKddeuXdGyZUt8+eWXldru5cuX4ePjg4SEBHh7e1dD5Xd2JO4Gnly0F27WxThgNQFCUQ7wXDjgU3t60xAR1Xb5+fm4ePEiAgICYG1tLXc5ZCHu9L2SI3+Ybe/XpKQkACh3Stbd3d04ryIFBQXIzMw0TllZWdVa59209HGEs40aKfkqXPPpIzVG3nlgSCIiIqJ7Zbah7n7NmTPHOHaPg4ODyY2RclAqBHRr4goACLfqJjVG/Q0Ul7/fgIiIiOh+mW2oKxnA8NYbPpOTk+84uOFbb72FjIwM4xQdHV2tdVZGz5uPDFt6xRuwry8NQnxm413WIiIiIqo8sw11AQEB8PDwwNatW41tmZmZOHDgQIUDI5bQaDSwt7c3TnZ2djVR7h11aVwPKoWAc6n5SG90s1NIzD/yFkVEREQWRdbBh7Ozs3Hu3Dnj+5Lu0M7OzvD19cXUqVPx4YcfIjAwEAEBAZgxYwa8vLzuqWePObCztkKHBs6IOHcdm7T9MGx4Z6BRL7nLIiIiIgsia6g7fPgwunfvbnz/6quvAgBGjx6NZcuW4Y033kBOTg5eeOEFpKeno3Pnzti4cWOt7LnUI8gdEeeuY12cCsMe6Xf3FYiIqMpV5ZMJiMzt+2Q2Q5pUF7mHNClxKTUH3ebvgEoh4NjMR2BnbSWNV3dzVHAiIqo+BoMBZ8+ehVKphKurK9RqtfGpDET3ShRFFBYW4tq1a9Dr9QgMDIRCYXpHmxz5g89+rSH+9WzQwNUGF67lYPfZVDx6/Sfg2HJgxErALUju8oiILJpCoUBAQAASExNx9ep9POuVqAI6nQ6+vr7lAp1cGOpqUM8gN1y4dhFbY1LwqD4SyIgHTq4Ees64+8pERPRA1Go1fH19UVxcfNfnlBLdjVKphEqlMqszvgx1NahHkDuW7L6IHbEp0I98GcpmTwFNeH8dEVFNEQQBVlZWsLKykrsUoipnHucL64i2/k6ws1bhek4hIhXBQNPBgJVW7rKIiIjIAjDU1SArpQJdG0tPl9gWkyJzNURERGRJGOpqWM9gNwDA1tMp0qPCdn0KfNcFKMiWuTIiIiKqzRjqaljXxm5QCEBMYiauZumB478BiZHA6X/lLo2IiIhqMYa6GuZso0ZrXycAwLbYa0CL4dKMyN9lrIqIiIhqO4Y6GfS4eQl22+kUoPlQqfHCTiDjioxVERERUW3GUCeDnkHuAICIc6nIs/EB/MIAiMDJP+UtjIiIiGothjoZNHa3RX1HLQqKDdh7PhVo8bQ0I3KF9OgwIiIionvEUCcDQRBMe8GGPAGorIFrp4HE4/IWR0RERLUSQ51MegTdvK8uJgWixh4I6i/NiFwhY1VERERUWzHUyeShBi7QWimRlJmP6MTM0l6wJ1cC+iJ5iyMiIqJah6FOJtZWSnQOrAcA2BqTAjToDti4AbnXgXNbZK6OiIiIahuGOhn1DCpzX51SVTq8yfHlMlZFREREtRFDnYy63wx1kQnpuJZVALR8RpoRuwHIviZjZURERFTbMNTJyN3eGs3qOwAAtsemAO6hQP02gK0HkHZe5uqIiIioNmGok1nZXrAAgGHLgaknAN+HZKyKiIiIahuGOpmVjFe3++w1FBTrAXtPQKGUuSoiIiKqbRjqZNbUywGudhrkFOpx8GJa6Qx9EXD1mHyFERERUa3CUCczhUJAjyY3e8GWXILNvAp8HgL82A/IS5evOCIiIqo1GOrMQA/jI8OSIYoiYOcJ6FwAjR2Qelbm6oiIiKg2UMldAAGdG9WDWqlAQloezl/LRiM3O2D474CDN6C0krs8IiIiqgV4ps4M2GhU6NjQBUCZS7DOAQx0REREVGkMdWaiZ3CZp0uUZdADKadlqIiIiIhqE4Y6M9H9ZmeJI3E3kJ5bKDWmxwMLmgM/PAIU5shYHREREZk7hjoz4eOsQxN3O+gNInaeufmIMHtvacy6gkwgeq28BRIREZFZY6gzIyW9YDdHJ0sNCgXQepT0+ujPMlVFREREtQFDnRl5tKknACA8Orn0EmzLEYCgAOL3AdfOyFgdERERmTOGOjPStL49gj3tUVhswJpjV6RGe08gsI/0+hjP1hEREVHFGOrMiCAIGNbWGwDwx+HL0kDEANBmtPTz+O9AcaFM1REREZE5Y6gzMwNb1YdapUBMYiairmRKjY0eAWw9gNxUIHa9vAUSERGRWWKoMzOOOjX6hnoAAP44HC81KlVAqxHSa3aYICIiogow1JmhYe18AABrj11FXqFeamw1Uvp5fps0fh0RERFRGQx1ZqhjAxf4OGuRVVCMDVGJUqNzABDQFYAIHFsua31ERERkfhjqzJBCIeCpNtLZuj8OJZTOKBmz7tiv0uPDiIiIiG5iqDNTQ9p4QxCAAxfTcCn15iPCgh4DtE5A5mXpMiwRERHRTQx1ZsrLUYsuga4AgD8P3zxbZ2UNhE0Fes4EPFvIVxwRERGZHYY6M/b0zQ4Tfx25jGK9QWrsPBV4+DXA1k2+woiIiMjsMNSZsZ7B7nCxUSMlqwA7Yq/JXQ4RERGZMYY6M6ZWKTCoVX0AwB+Hy3SYKC4Eov4GVk8ASp46QURERHUaQ52ZKxmzbtvpFKRk5UuN+kJg3WQg8jcgbq+M1REREZG5YKgzc4Hudmjl6wi9QcSqo1ekRo0t8NAEoOt0afw6IiIiqvMY6mqBYW2ls3V/HkqAWHK5tce7QPe3AXsvGSsjIiIic8FQVws81sILOrUSF1JzcOjSDbnLISIiIjPEUFcL2GpUeKy5J4BbnjChLwZOrwc2TGeHCSIiojqOoa6WKOkwsf5kIrLyi6TGgkxg5RjgwGIg8bhstREREZH8GOpqida+TmjoaoO8Ij3+iUyUGnXOQPAA6fWRn+QrjoiIiGTHUFdLCIJgPFtnMmZdm9HSz5N/AYU5MlRGRERE5oChrhYZ3NobKoWAyIR0nE7KlBr9OgNOAUBhFnBqjaz1ERERkXwY6mqRerYa9Ap2B1Cmw4RCAbQeKb0+ykuwREREdRVDXS1Tcgl29bErKCjWS40tRwCCEkg4AKSclrE6IiIikotZhzq9Xo8ZM2YgICAAWq0WDRs2xAcffFA6AG8d1KWxKzzsrZGeW4Tw6GSp0c4DaNxXen3sF/mKIyIiItmYdaibO3cuFi1ahK+//hoxMTGYO3cu5s2bh6+++kru0mSjVAgY0sYbwC1j1rUeJf2M/B0oLpChMiIiIpKTWYe6vXv34oknnkD//v3h7++PIUOGoHfv3jh48KDcpclq6M3Hhu05l4rLN3Klxka9ADtPIPc6cPo/GasjIiIiOZh1qOvUqRO2bt2KM2fOAAAiIyOxZ88e9OvXT+bK5OXrokPHBi4QReCvI5elRqUKaPV/0uujP8tXHBEREcnCrEPdm2++iaeffhpBQUGwsrJCq1atMHXqVIwYMeK26xQUFCAzM9M4ZWVl1WDFNefp9tLZupWHL0NvuHmPYUmou7AduHFJnsKIiIhIFmYd6v78808sX74cv/32G44ePYqffvoJ8+fPx08/3X7ojjlz5sDBwcE4hYSE1GDFNadPqAfsrVW4kp6HiHOpUqOTP9CgG6CwAi4flrM8IiIiqmGCaMZdSX18fPDmm29i4sSJxrYPP/wQv/76K06frnjojoKCAhQUlHYUuHLlCkJCQpCQkABvb+9qr7kmzVwbhZ/3xaF/c09880xrqfHaGenxYTb15C2OiIioDrt8+TJ8fHxqNH+Y9Zm63NxcKBSmJSqVShgMhtuuo9FoYG9vb5zs7Oyqu0zZlHSYCD+VjBs5hVKja2MGOiIiojrIrEPdgAED8NFHH+G///7DpUuXsHr1anz++ecYNGiQ3KWZhab1HRDqZY9CvQGrj10pv0BmYs0XRURERLIw61D31VdfYciQIXjppZcQHByM119/HS+++CI++OADuUszGyVPmPjjUELpoMyFucCP/YAFTYGsZBmrIyIioppi1qHOzs4OCxYsQFxcHPLy8nD+/Hl8+OGHUKvVcpdmNp5oUR8alQKxyVmIvJwhNap1gKEYEA1AXIS8BRIREVGNMOtQR3fnoLNCv6YeAG55wkT/z4BXTgFNB8tUGREREdUkhjoLMPTmJdh/Iq8it7BYavRsDth7yVgVERER1SSGOgvwUIALfJ11yC4oxvqTSeUXSDgIXDlS84URERFRjWGoswAKhYChbaUxcP4sewkWACL/AH54BFg7CdAXyVAdERER1QSGOgsxpI0PFAJw8FIazl/LLp0R+AigcwFSooH938pXIBEREVUrhjoL4eFgjW5N3AAAfx4uc7ZO5ww8cnMImB2fAOkJFaxNREREtR1DnQUpecLE30euoEhf5qkbLYYDvh2Bolxg45syVUdERETViaHOgvQMdkM9WzVSswuw/XRK6QyFAuj/OaBQAaf/BWI3ylckERERVQuGOgtipVRgcOubHSYO33KZ1T0EeOgl6fWGadJTJ4iIiMhiMNRZmJJLsNtjryE5M990ZtfpgL03kB4P7J4vQ3VERERUXRjqLEwjN1u08XOC3iDiryOXTWdqbIF+c6XXEQuBa7E1XyARERFVC4Y6CzTs5hMmVh5OgCiKpjOD+gOBfQBDEfDfa8Ct84mIiKhWYqizQP2becJGrcSl67k4cDHNdKYgAI/OA1Ra4NJu4MSf8hRJREREVYqhzgLZaFQY0EJ67usv++PKL+DkD3R5XXq9dTafNEFERGQBGOos1KiO/gCA/04k4uTljPILdJoMtPo/YORqQGlVs8URERFRlWOos1AhXvYY1Ko+AODj9THl761TqYEnvgFcm8hQHREREVU1hjoL9lrvxlArFdh34Tp2xF6788LJpwCDvmYKIyIioirHUGfBvJ10GBPmDwCYsyEGesNterpu/QBY3Bk4/GPNFUdERERViqHOwk3s1ggOWiucSc7G37eOW1fCzgMQDUBKdM0WR0RERFWGoc7COeis8HKPRgCAz8JjkVtYXH6htmOBsZuBx76o4eqIiIioqjDU1QEjO/rB20mL5MwC/LjnYvkFFErAt0PNF0ZERERVhqGuDtColJjWR+rlunjnBaRmF9x+4YwrwIbpQPEdliEiIiKzw1BXRwxo7oVm9R2QXVCMhVvPVryQwQD89BhwYLH0bFgiIiKqNRjq6giFQsBbjwYBAH47EI8L17IrWgjo9rb0evd8IK2CS7VERERklhjq6pBODeuhR5Abig0i5m2MrXihZkOAgK5AcT6wfhpw66DFREREZJYY6uqYN/sFQSEAG08l4UhcWvkFBAHo/xmgVAPnwoGYf2q+SCIiIrpnDHV1TGN3Owxt6wMA+Hj96fKPDwOAeoFA2BTp9cY3gYIKLtUSERGRWWGoq4NeeaQxtFZKHIm7gU2nkipe6OHXAEc/IPMKsGNOzRZIRERE94yhrg5yt7fGuIcDAABzN8aiSG8ov5CVFnh0vvR6/yIgKaoGKyQiIqJ7xVBXR73QtSFcbNS4mJqDFQfjK16ocW8geAAg6oH/XpWGPCEiIiKzxFBXR9lqVJjaKxAAsGDLWWTlF1W8YN9PACsbIOEAsOltIO1CDVZJRERElcVQV4c93d4XDerZ4HpOIb7fdZuw5uANdL85dt2BRcBvT9dcgURERFRpDHV1mJVSgTf6SgMSL9l9AUkZ+RUv2HEiMOh7oEF3oPnQ0vbCXODngcDBJYD+Nmf6iIiIqEYw1NVxfULd0dbPCflFBnwRfqbihQQBaDEMGLVG6hVb4uxm4MJ26ZFiClVpe3FhtdZMRERE5THU1XGCIOCtR4MBACuPJCA2KetuK5S+9n0I6P0R0HlKaXtxIbCgqXSZ9sRKoOAu2yMiIqIqwVBHaOPnhH5NPWAQgU82xFR+RTsPoNMkoN3zpW0J+4HsZODMBmDV88CnjYA/RwHRa4GivKovnoiIiAAw1NFNb/QNgkohYHvsNew9l3r/GwroArx0AOg6HXBuKD1DNnqtFOw+bQT8PQ6I3QBcPw+kJwDZ1/jECiIioiogiBU+J8pyXL58GT4+PkhISIC3t7fc5Zi199ZG4ad9cWhW3wFrJ4ZBoRDuvtKdiCKQdAKI+huIWgVkJFS8nHd74Pnw0vdftwPybgCj1gLuoVLbseXA0Z8BlUaalGpAZS29ttICahtp6BW1DrDSAWpbwMYFaNSrdLsZVwCFEtC5AEqrB/tsREREdyBH/lDdfRGqKyb3DMTfR6/g5JUM/HPiKp5oWf/BNigIgGcLaeo1G7h8SAp4seuBvHTpLJ6+UApmZeWkAnlpgKAsbUuPky7t3ot6jYFJZULd8iFASrQUFht0k9qO/w5s/1gKg2obwM4TcPIHnAMApwDAuQHg4AMo+b8KERGZN/6lIiMXWw0mdGuITzfFYt7GWPRt6gGNSnn3FStDEACf9tLUb25pu8EAGIpNl31+ixT4nPxL25oOAdybAsUFgL5Aml9cKP0sygOKcoHCbGmYlaJcoDAHsK8olArSGb0SualAxm2eqFFCoZKCXUnQcwsG2o+71yNARERUrRjqyMTYsAD8si8OV9Lz8PPeOIzr0qB6d6hQAAq1aZtLw/LLuTaWpgfx0j7pknBZzZ8GfDsBRTnSvX0Zl4EbF4G0i6U/9QXS6xsXb9ZyS6j7ZbAUJB/9FPBoJrXlpkmXeq0dHqxmIiKiSmKoIxNatRKv9m6MN/46ga+2ncVTbb3hqFPffcXaQrjlPkFbV2m6HYMByEo0DXplg5ooAgkHgcIs6T6/Eod/ALZ9KJ3hcwuRzu65h0qv6zUGVBZ0TImIyCww1FE5T7b2xg+7LyI2OQvf7jiPt2+OY1cnKRSAQ31p8u9c8TKj10qBr+zl4pzr0s+MBGk6u6nMNlWASyMp4LmHAG6hUuhz9JP2R0REdB/Y+5UqtD02Bc8uPQS1UoGtr3WFj7NO7pJqn7wbQEoMkHxK6qCRHC29L8ioePnG/YBnVpS+j9sL1Gsi9eIlIqJahb1fyWx0a+yKsEYuiDh3HZ9tjsWCp1vJXVLto3UC/DpJUwlRBDKv3Ax4p0qDXmqs6b2EuWnA0n7S67cuAxo76XX0WiksOvoCDr6AgzdgZV1zn4mIiMwWQx1VSBAEvNUvGI99tQdrjl/F8w83QNP6vOn/gQmCFMQcvIHGvUvb9UWmT9zISiq9nFsS6ADgwPdA3B7Tbdq6S/fuOfrcDHs+0qVcRx/ptca22j4OEZGRKALizRENDMXSmKEl9zEXZEmjF1jppCGkAOn3Xm4aALF0fenFLe/LtAHS77ySsUbzM6TRDtQ2pfc7G/TS0FhAmfuohfLvtY5ShzYLwlBHt9W0vgMGtaqP1ceu4OP1MVj+fAcIt3Y0oKqhtDIdENk9BJgSKQ3bUpZ/mDTYckYCkB4v9brNTpamK4cr3nbYVOCR2dLr/ExprECPZkD9NuU7jhBR1cq+Jv1/auteelY94wqQdl76/1tfcHOopkLTnyXjeJa0qW2BHu+UbnfjW9JZ/h7vAt5tpbbYjcDWm/+vo8z/2+WCDUrfK62AcdtKl930DhAXAXR5Awh6VGq7tAdYO7F0CCrjpJd+ivrStrLeulL6j8qNbwLHfgV6vgc8/KrUlnwK+L7rvR/TycekMUQBYPfnQMQC4KGJQN+PpbasROCL0Ltv55Vo6X5pC8JQR3f0Wu/G+O9EIvaev45Np5LRt6mH3CXVLbf2ku3+dulrUZT+lZsRLwW89ITSsJd+82dBBmDrVrrO9XPAv1Olp2pMO1/aHrVK+peue1PA3othjyyXvlgawsigB3TOpe1nNgMFmUDjvqVBJHYDcDb85hiYOaU/i/LLhLGim68LpfDlFgS8uKt0uz/0Am5cAp4Ll8bpBIBTq4HNZQJaZdh5moa6y4eBywdNn72dnyHdv3svlLf8jrl+Hrh6DMhJKW0rLpA+w726NeQBMDnjBgBCSeewCs6kmbwv02YSWBVS57Nbf2cJiptn+iy620A5DHV0R95OOowJ88f3uy5g/K9H8FhzT7zWuwkC6tncfWWqXoIgdaKwcQG8bnPPY34GTH4BKq2Ahj2k+/3K/hLcPAPIvCy91joDHk0B92Y3fzYFXIM4DAuVJ4rSGaWCLOkPa0lI0hdJoQMAfB8q/a6lxADZKQDE0j+4xp+45f3Ns0I2boBvB2l9gwGI+EIKUGFTSi/jHVsOnNtSeoarop8lA5QX50vrNOgmPV2mxKpxQH46MPFQ6ZiYV45KwxPdi1vPrqu00lQ24Ni4Sp2gVOrSRx4q1bc8AlENKDWlr60dTbfbZZp0f61Xy9K2Bt2AUetw98uZYumPW//91vUNoO1YqUd+ifptgLGbpf/GCuXNn6o7vL/ZZlWmg93jXwNPfGO6L6+WwHs3KjyMldbrPWkqy8H79tsVxdLjYIH/eGWoo7ua2isQ17IKsPrYFfx7IhEbopIwtK0PpvQMhIcDb9I3a7cOfuzRDBi52rRNXyx15kg6CaSekR7RdnGXNJVQqKQ/Qh5NpW24NwXqt+bgyvfLoL/lEtutQaQAcPKT7pEEpEt4p/+V/sC3HF66ncgV0qV3kz+qZSZl2fdW0h9bRz+gXiNp/eJC4NppKUCVDQdRq6SzugWZUmArN5VpLwkr7Z4H+n8mvS7IApb2lV7PSC29tWDXp9Ll/3sR9Bjgu1x6LQjS+I+iAWj7bGmoSzoBnFp1b9stew8rIIXPwhzTe6wCHpbO+KhtpEljJ/0see60UiN9tpIwplSbBhkAmFjB4w1bDJOmB1H2ntwSdu7S9CDqty7fpnUsDdb3y1wClCCYTy3VgKGO7kqnVuGLYS0x7uEGmL85FttOp+D3g/FYdfQyxnTyx/iuDeFkw7M4tZZSBTy5RHpdlA9ciwGSooDkqJs/T968rHNKmk78IS07bDkQ/Jj0OuZf4MhS6Sxgx4ml2z79n3Tmz6aedMnX2tE8xuIrygNyr0vPF7b3LG0/t+Xm5bU8aSoJWSWX2PSFpZfZ9DfbW42U/vgDQMIhYPO70v0+gxaVbndJT2ng6pLAZii6e429ZgOdp0qvM+Kly+YOPqah7uD3wJUj9/bZO04C+nwkvc5OBr57GFDbAW9fLl3m2K/A+a33tl19mTNUChXg3LD8H097L+msL0r+sJb9ifLtSjVQL7B0fUEA2oyR5pe9bBg8QDrmKo1p4DK+t5buRdXYSvemqW3KP3P6mT/Kf6aALtJEVEsw1FGlhXjZ48cx7XDoUhrmbTyNQ5du4LtdF/DbgXi80KUBxnYOgI2GX6lazcpaupRb9nKuKEr36hmD3knpZ8kj0QBpSJZzWwDbMvdcFuYCK54x3b6gkEKezuVm0Lv5WldPOuNhKAZajyo9Q3Vxt3T/kWcLoM1oqc2gB/4ae/PmbH0FN27rb4ax/NJgVpQPPPm/0rMbp9YAa8ZLIbTsmcs/x0hPB7kXvh1LQ11hFpCwX7rMV1beDSlEVkRQVBxEyp4F1bkATfqXf/pJ477SGVRDUekx0BffckxK2oukY2PnaboNWw/p8p4oloawwEek3tMaO0Bjf/MMle3N92XaSia1rWlYt7YHJh8t/1l7fyhND+KxL8q3+Xe+/eDgRHUI/wLTPWvn74w/X+yIHbHXMHfjaZxOysJn4Wfw075LmNS9EYZ38IVGZVndxOs0QZBClqNvaW+4WzV5VOrd5+hX2laUC3i3k8JMznWp04ZoAHJTpSk1tuJtNeheGupSYqR7mkIHlYY6CED0mnv/HGWDlpW1dDlSuOWsoXdbKQSWnNkpOctTcu9T2amkzbtd6fruzYChv5jegA8Aw3+XPntJYCsb3pSV+DXs5A8M/618e9c3Kv3xK+ToA7xewX+HhyY82HaJSBZm/0SJK1euYPr06diwYQNyc3PRqFEjLF26FG3btq3U+nyiRPUyGET8c+IqPtt8BvFpuQAAbyctXunVGANb1YdSYbn3LtA9Ki6U7tfLvS6NIZV7vXTKSZXOvClUQPsXS+/5unwEOLtZumk7dGDptg58V8EN2jdv0haUN0OZtemlNzvP0l6NZc9KERFVAznyh1mHuhs3bqBVq1bo3r07JkyYAFdXV5w9exYNGzZEw4YN774BMNTVlMJiA/44nICFW8/iWlYBAKCxuy1e790Ej4S4c3w7IiKqU2pNqEtISIAgCMYiDx48iN9++w0hISF44YUXqqy4N998ExEREdi9e/d9b4OhrmblFeqxbO8lLNpxDpn5Uq+4Vr6OeKNPEDo25DNMiYiobpAjf9xXN7RnnnkG27dvBwAkJSXhkUcewcGDB/HOO+/g/fffr7Li1q1bh7Zt2+Kpp56Cm5sbWrVqhSVLltxxnYKCAmRmZhqnrKx7vOmZHohWrcSEbg2x+40eeKlbQ1hbKXAsPh3Dl+zHyB8O4OTl2zzMnoiIiB7IfYW6qKgotG8vjYz9559/omnTpti7dy+WL1+OZcuWVVlxFy5cwKJFixAYGIhNmzZhwoQJmDx5Mn766afbrjNnzhw4ODgYp5CQkCqrhyrPQWeFN/oGYde07hj5kB9UCgG7z6ZiwNd7MHH5UZxLYdgmIiKqSvd1+dXW1hZRUVHw9/fH448/jrCwMEyfPh3x8fFo0qQJ8vLy7r6RSlCr1Wjbti327t1rbJs8eTIOHTqEffv2VbhOQUEBCgoKjO+vXLmCkJAQXn6VWdz1HHwRfgZrI68aB/PuHeKOl7o3QksfR1lrIyIiqmq15vJraGgoFi9ejN27dyM8PBx9+0ojh1+9ehUuLlV335Snp2e5M23BwcGIj4+/7ToajQb29vbGyc7Orsrqofvn52KDBU+3wvrJD6N3iDTi+eboZAz8JgIj/rcfe86mwoz77BAREZm9+wp1c+fOxXfffYdu3bph+PDhaNGiBQDpHriSy7JVISwsDLGxpmMonTlzBn5+frdZg8xdsKc9vh/VFuGvdMHg1tKQJxHnruP/fjiAgd9EYGNUIgwGhjsiIqJ7dd9Dmuj1emRmZsLJycnYdunSJeh0Ori5uVVJcYcOHUKnTp0we/ZsDB06FAcPHsS4cePw/fffY8SIEZXaBnu/mrfLN3KxZNcFrDiUgIJiAwCgoasNxndtiIGt6sNKaQaPlCIiIrpHtWZIk7y8PIiiCJ1OenBxXFwcVq9ejeDgYPTp06dKC/z333/x1ltv4ezZswgICMCrr76KcePGVXp9hrraITW7AEsjLuLnfXHIujkUipeDNcZ1aYCn2/lCq+YTKoiIqPaoNaGud+/eGDx4MMaPH4/09HQEBQXBysoKqamp+PzzzzFhgvk8YoahrnbJyi/Cr/vj8cOei0jNljq8ONuo8Wwnf4zq6A8HnZXMFRIREd1drekocfToUTz8sPQA67/++gvu7u6Ii4vDzz//jIULF1ZpgVS32FlbYUK3htgzvTs+HNgUPs5apOUU4rPwMwibuw1z1scgJTNf7jKJiIjMzn2FutzcXGOv0s2bN2Pw4MFQKBR46KGHEBcXV6UFUt1kbaXE/z3kh+2vdcOXT7dEkIcdsguK8d2uC+g8bzveXn0Scddz5C6TiIjIbNxXqGvUqBHWrFmDhIQEbNq0Cb179wYApKSkwN7evkoLpLpNpVTgiZb1sWHKw/hhdFu08XNCYbEBvx2IR/f5OzD592NYe/wKDl5MQ0JaLor0BrlLJiIikoXqflaaOXMmnnnmGbzyyivo0aMHOnbsCEA6a9eqVasqLZAIAARBQM9gd/QIcsPBi2n4dsd57DxzDesir2Jd5NUyywFudhp4Omjh5WgNLwctPB218HKwhpejFp6O1qhno4FCIcj4aYiIiKrefQ9pkpSUhMTERLRo0QIKhXTC7+DBg7C3t0dQUFCVFvkg2FHCcp26moFf98fjYmo2EjPykZiej8JKnKlTKxVwd9DAy0ErBT0Ha3g6auHtqMVDDVzY05aIiB5Yren9Wtbly5cBwGwDE0Nd3WEwiLieU4jEjDxcTc/D1fR86XVGPq6m5yExPR8pWfm409jGAfVs8PPY9vBx1tVc4UREZHHkyB/3dfnVYDDgww8/xGeffYbs7GwAgJ2dHV577TW88847xjN3RDVJoRDgaqeBq50Gzb0dK1ymSG9AcmY+EkuC3s2fV9PzcTzhBi6m5uDJRXvx83PtEeTB+0OJiKj2uK9Q98477+CHH37AJ598grCwMADAnj17MGvWLOTn5+Ojjz6q0iKJqoqVUgFvJx28ncqfiUvKyMfoHw8iNjkLQxfvww9j2qGdv7MMVRIREd27+7r86uXlhcWLF+Pxxx83aV+7di1eeuklXLlypcoKfFC8/Er3IiO3CM/9dAiH425Ao1Lgm2dao1eIu9xlERFRLVNrBh9OS0ursDNEUFAQ0tLSHrgoIrk46Kzwy3Md0DPIDQXFBrz46xH8eThB7rKIiIju6r5CXYsWLfD111+Xa//666/RvHnzBy6KSE5atRKLR7bBk629oTeIeOOvE1i88zwesE8RERFRtbqve+rmzZuH/v37Y8uWLcYx6vbt24eEhASsX7++SgskkoOVUoH5TzVHPVs1vtt1AZ9sOI3r2QV4q18wx7gjIiKzdF9n6rp27YozZ85g0KBBSE9PR3p6OgYPHoxTp07hl19+qeoaiWQhCALeejQYbz8q3WqwZPdFvL4ykk+tICIis/TA49SVFRkZidatW0Ov11fVJh8YO0pQVfj7yGW88fcJ6A0iujdxxTcjWkOnvq8T3UREVAfUmo4SRHXNk228sWRUG1hbKbA99hr+738HkJ5bKHdZRERERgx1RJXUI8gdy5/vAHtrFY7Gp+OpxfuQmJEnd1lEREQAGOqI7kkbP2esHN8J7vYanE3JxpPf7sW5lGy5yyIiIrq33q+DBw++4/z09PQHqYWoVmjiYYe/J3TCqB8P4sK1HDy1eC9+HNMOrXyd5C6NiIjqsHs6U+fg4HDHyc/PD6NGjaquWonMhreTDn+N74QW3g64kVuEZ5YcwM4z1+Qui4iI6rAq7f1qjtj7lapTTkExxv96BLvPpkKlEPDZ0BZ4omV9ucsiIiKZsfcrUS1jo1Hhh9HtMKCFF4oNIqasOI6lERflLouIiOoghjqiB6RWKfDlsJYY08kfADD7n2h8uuk0HytGREQ1iqGOqAooFALeGxCC13s3BgB8s/083lt3isGOiIhqDEMdURURBAGTegTi40HNIAjAz/vi8O2O83KXRUREdQRDHVEVe6aDL957LAQA8OmmWKw+dlnmioiIqC5gqCOqBmPCAjDu4QAAwBt/ncDec6kyV0RERJaOoY6omrzVLxj9m3uiSC/ixV+O4HRSptwlERGRBWOoI6omCoWAz55qgfb+zsgqKMaYHw/xWbFERFRtGOqIqpG1lRLfj2qDhq42SMrMx7NLDyEzv0jusoiIyAIx1BFVM0edGsuebQ9XOw1OJ2Vhwq9HUFhskLssIiKyMAx1RDXAx1mHpWPaQadWIuLcdbz59wmOYUdERFWKoY6ohjSt74BvR7SGUiFg1bEr+GzzGblLIiIiC8JQR1SDujVxw8eDmgIAvt5+Dr8diJe5IiIishQMdUQ1bFg7X0zuGQgAeHfNSWw7nSxzRUREZAkY6ohk8EqvQAxp4w2DCExcfgwnLqfLXRIREdVyDHVEMhAEAXMGN8PDgfWQV6TH2GWHEH89t1r3WVCsx/bTKbiazrHyiIgsEUMdkUyslAp8O6I1gj3tkZpdiDFLD+JGTmGV7ycpIx+fbY5F2Cfb8OyyQxiyaC+yC4qrfD9ERCQvhjoiGdlZW2HZs+3g5WCNC6k5GPfzYeQX6R94u6Io4tClNEz87Sg6z92Gr7adQ2q2FBiv3gx5RERkWRjqiGTmbm+NZWPbw85ahcNxN/DKH8dhMNzfGHb5RXr8cSge/RfuwVOL9+G/E4koNohoH+CMb0e0xtIx7QAAP+29xPv4iIgsjEruAogIaOxuh+9HtsXoHw9iQ1QSPvwvBjMHhFR6/cs3cvHL/jj8cSgB6bnSY8isrRQY2LI+RnX0R4iXvXHZJ1p6Ye3xq3jz75NYNykMKiX/bUdEZAkY6ojMRMeGLvj0qeaYsuI4foy4CC9Hazz/cIPbLi+KIvaev46f9l7ClphklJzc83bSYuRDfhjWzgeOOnW59WY8FoIdsdcQnZiJpRGXMK7L7fdBRES1B0MdkRl5omV9XE3Px9yNp/HR+hh4OWrxaDNPk2VyCoqx6tgV/Lz3Es6mZBvbOzeqh1Ed/dAz2B1KhXDbfdSz1eCdR4Pxxt8n8Hn4GfRt6gEfZ121fSYiIqoZDHVEZmZ81wa4mp6HX/bHYeofx+Fqp0E7f2dcSs3Bz/visPJIArLypd6rOrUST7b2xuhOfmjkZlfpfTzV1ht/H72MAxfTMGNtFJaOaQdBuH0QJCIi88dQR2RmBEHArMdDkZiRjy0xyXj+p8No5euIHbHXjMv4u+gwqqM/hrT1hr211X3t4+PBzdBvwW7siL2Gf08kYkALr6r8GEREVMN4hzSRGVIqBHw1vBVa+DgiI6/IGOi6NXHF0mfbYdtr3TC2c8B9BboSDV1tMbF7IwDA7H+ikXGzgwUREdVOPFNHZKa0aiV+GN0Ws9adgpudNUZ29ENAPZsq3cf4bg2wLvIKzl/LwScbYzBncPMq3T4REdUcnqkjMmP1bDX4+pnWmDkgpMoDHQBoVEpjkPv9YAIOXkyr8n0QEVHNYKgjquPaBzjj6XY+AIC3V59EQfGDP9GCiIhqHkMdEeGtfsGoZ6vGuZRsfLfzgtzlEBHRfWCoIyI46Kwwc0AoAODrbedw/lr2XdYgIiJzw1BHRACAAc090bWxKwr1Bryz+iRE8f6eP0tERPJgqCMiANLYdR8ObAprKwX2X0jDyiOX5S6JiIjuQa0KdZ988gkEQcDUqVPlLoXIIvk46/BKr8YAgI/XxyA1u0DmioiIqLJqTag7dOgQvvvuOzRvznG0iKrT2M4BCPa0R3puET78N1rucoiIqJJqRajLzs7GiBEjsGTJEjg5OcldDpFFs1Iq8MngZhAEYM3xq9h99trdVyIiItnVilA3ceJE9O/fH7169ZK7FKI6oYWPI0Z39AcAvLM6CnmFHLuOiMjcmX2oW7FiBY4ePYo5c+ZUavmCggJkZmYap6ysrGqukMgyvd6nCTwdrBGflouF287KXQ4REd2FWYe6hIQETJkyBcuXL4e1tXWl1pkzZw4cHByMU0hISDVXSWSZbDUqzH5cGrvu+10XEJOYKXNFRER0J4JoxoNRrVmzBoMGDYJSqTS26fV6CIIAhUKBgoICk3mAdKauoKC0x96VK1cQEhKChIQEeHt711jtRJZi/C9HsPFUElr6OOLvCZ2gVAhyl0REZPYuX74MHx+fGs0fqhrZy33q2bMnTp48adL27LPPIigoCNOnTy8X6ABAo9FAo9EY32dm8uwC0YOY9Xgo9pxLxfGEdCw/EIdRN++1IyIi82LWoc7Ozg5NmzY1abOxsYGLi0u5diKqHh4O1nijbxPMXHsK8zbGoneIBzwcKnc7BBER1RyzvqeOiMzDiA5+aOnjiOyCYry3LkrucoiIqAK1LtTt2LEDCxYskLsMojpFqRAwZ3AzqBQCNp1KxuZTSXKXREREt6h1oY6I5BHsaY9xXRoAAGauPYWs/CKZKyIiorIY6oio0qb0DISvsw5Jmfn4bPMZucshIqIyGOqIqNKsrZT4aJDUSemnfZdwPCFd3oKIiMiIoY6I7snDga4Y2NILogi8/PtRxF3PkbskIiICQx0R3YcZj4XAx1mLhLQ8DP52L05cTpe7JCKiOo+hjojumYutBn9P6IRQL3tczynE09/vx47YFLnLIiKq0xjqiOi+uNlZY8ULD6Fzo3rILdTj+Z8O468jl+Uui4iozmKoI6L7ZmdthR/HtMPAll4oNoh4fWUkvtl+Dmb8SGkiIovFUEdED0StUuDzoS3xYldpDLtPN8XivXWnoDcw2BER1SSGOiJ6YAqFgLf6BeO9ASEQBODnfXGYuPwo8ov0cpdGRFRnMNQRUZV5NiwAXw1vBbVSgY2nkjDqh4PIyOWTJ4iIagJDHRFVqceae+Gnse1hZ63CwUtpGLJ4L66m58ldFhGRxWOoI6Iq17GhC1aO7wh3ew3OpmRj8Ld7EZuUJXdZREQWjaGOiKpFkIc9Vr0UhkZutkjKzMeQxXux/8J1ucsiIrJYDHVEVG3qO2rx1/iOaOvnhKz8Yoz64SDWn0yUuywiIovEUEdE1cpRp8avz3dAn1B3FOoNmPjbUSyLuCh3WUREFoehjoiqnbWVEt+OaIORD/lBFIFZ/0Tjkw2nOUgxEVEVYqgjohqhVAh4/4lQTOvTBACweOd5vPZnJAqLDTJXRkRkGRjqiKjGCIKAid0b4dMhzaFUCFh17Aqe++kQsguK5S6NiKjWY6gjohr3VFsf/G90W2itlNh9NhVPf78P17IK5C6LiKhWY6gjIll0b+KGFS88BBcbNaKuZGLYd/twPZvBjojofjHUEZFsWvg44u8JnVDfUYsLqTkY+9Nh5PBSLBHRfWGoIyJZ+dezwU9j28NJZ4XIhHRMWH6UnSeIiO4DQx0Rya6Rmy1+HNMOWisldp25hjf+ioTBwOFOiIjuBUMdEZmFVr5OWPR/raFSCFhz/Co+Wh/DceyIiO4BQx0RmY1uTdwwb0hzAMAPey7i+10XZK6IiKj2YKgjIrMyuLU33nk0GAAwZ8Np/HXksswVERHVDgx1RGR2xnVpgBe7NAAATP/7BLadTpa5IiIi88dQR0RmaXrfIAxuXR96g4iXlh/FkbgbcpdERGTWGOqIyCwpFALmPtkc3Zq4Ir/IgLHLDuFscpbcZRERmS2GOiIyW1ZKBb4d0RotfRyRkVeEUT8exNX0PLnLIiIySwx1RGTWdGoVlo5ph4auNkjMyMeoHw8iPbdQ7rKIiMwOQx0RmT0nGzV+fq4DPOytcS4lG2OXHUJeoV7usoiIzApDHRHVCvUdtfj5ufawt1bhaHw6Jv52FEV6Pk6MiKgEQx0R1RqN3e3w45h20KgU2HY6BW+tOsmnThAR3cRQR0S1Slt/Z3zzTGsoFQL+OnIZczfGyl0SEZFZYKgjolqnV4g75gxqBgBYvPM8/rebjxMjImKoI6JaaWg7H7zRtwkA4MP/YrDm2BWZKyIikhdDHRHVWhO6NsSzYf4AgNdXRmLXmWvyFkREJCOGOiKqtQRBwIz+IXi8hReKDSLG/3oExxPS5S6LiEgWDHVEVKspFALmP9UCnRvVQ26hHmOXHcL5a9lyl0VEVOMY6oio1lOrFFg8sg2aezsgLacQT3wdgQVbziC7oFju0oiIagxDHRFZBFuN9DixFj6OyC4oxoItZ9Fl3nb8sOci8ov49AkisnwMdURkMVxsNVg9oRO+eaY1GtSzQVpOIT74Nxo9P9uJPw8noJhPoCAiC8ZQR0QWRaEQ0L+5Jza/0gWfDG4GD3trXEnPwxt/nUDfL3djY1Qin0JBRBaJoY6ILJJKqcDT7X2xY1o3vPNoMBx1VjiXko3xvx7FwG/3Yu+5VLlLJCKqUgx1RGTRrK2UGNelAXa90R2TezSCTq1EZEI6nvnfAfzf/w4gkkOgEJGFYKgjojrB3toKr/Zugp3TumNMJ39YKQXsOZeKJ76JwIRfj+BcCodBIaLajaGOiOoUVzsNZj0eim2vdcPg1vUhCMCGqCT0/mIn3vgrElfS8+QukYjovjDUEVGd5OOsw+dDW2LjlC54JMQdBhH48/BldJ+/Ax/8G43r2QVyl0hEdE8Y6oioTmviYYclo9pi1Uud8FADZxQWG/DDnovoMm87Fmw5g6z8IrlLJCKqFEG08L79ly9fho+PDxISEuDt7S13OURkxkRRxO6zqZi36TSirmQCANRKBTo1csEjIe7oFewOd3trmaskotpAjvzBUEdEdAuDQcSGqCQs2HIGZ2/pQNHCxxGPBLvhkRAPNHa3hSAIMlVJROaMoe4Wc+bMwapVq3D69GlotVp06tQJc+fORZMmTSq9DYY6IrpfoijiXEo2NkcnIzw6GcdvGf7E11lnPIPXzt8JKiXvaCEiCUPdLfr27Yunn34a7dq1Q3FxMd5++21ERUUhOjoaNjY2ldoGQx0RVZWUrHxsjUlBeHQy9pxLRWFx6WPHHHVW6NHEDY+EuOPhxq6w1ahkrJSI5MZQdxfXrl2Dm5sbdu7ciS5dulRqHYY6IqoOOQXF2H02FeHRydh2Ohk3cks7VPA+PCKSI3/Uqn9KZmRkAACcnZ1lroSI6jobjQp9m3qgb1MPFOsNOBJ3A+HRyQiPSUbc9VzsiL2GHbHX8M7qKLTwdsAjIe7oGeyOIA873odHRNWi1pypMxgMePzxx5Geno49e/bcdrmCggIUFJSOL3XlyhWEhITwTB0R1Yi73Yfn5WCNbkFu6NHEDZ0auUCnrlX/tiaiSuLl1zuYMGECNmzYgD179tzx4MyaNQuzZ88u185QR0RyKHsfXsS5VBSUuQ9PrVKgYwMX9AhyQ/cmbvB10clYKRFVJYa625g0aRLWrl2LXbt2ISAg4I7L8kwdEZmr/CI99p2/jm2nU7DtdEq5R5I1dLWRAl6QG9r5O8OKvWmJai2GuluIooiXX34Zq1evxo4dOxAYGHjP22BHCSIyRyWXaUsC3uG4G9AbSn8d22lU6BxYD92D3NCtiSvc7NjZgqg2YUeJW0ycOBG//fYb1q5dCzs7OyQlJQEAHBwcoNVqZa6OiOj+CYKAQHc7BLrb4cWuDZGRV4TdZ69h++lr2BGbgus5hdgQlYQNUdLvvebeDujexA09gtzQrL4DFAp2tiAiU2Z9pu52PcSWLl2KMWPGVGobPFNHRLWNwSDixJUMbDudgu2nU3DySobJ/Hq2arT2dYKfiw5+LjbST2cbeDlacwBkIjPBM3W3MOO8SURUbRQKAS19HNHSxxGvPtIYKVn52BF7DdtPp2D32VSkZhdic3RyufVUCgH1nbTwddbB/2bY83WWgp+vsw5atVKGT0NENcWsQx0REQFudtYY2tYHQ9v6oLBYGhPvTHIW4q7nIj4tB3HXcxGXlovCYoP0+noudp9NrWA7Gvi72MDXRQc/Z53008UGQR52sLZi4COq7RjqiIhqEbVKgY4NXdCxoYtJu8EgIjkrXwp613MRl5aDSyWvr+cgM78YKVkFSMkqwMFLaSbrejpY4+tnWqGNHwd2J6rNGOqIiCyAQiHA00ELTwctHmrgUm5+em4hLt0MePE3z+zFX8/FmZQsJGbkY9h3+/FmvyA81zmAT7wgqqUY6oiI6gBHnRotdWq09HE0ac8uKMabf5/AvycS8eF/MTh86QbmPdUc9tZW8hRKRPeN3aSIiOowW40KXw1vhfefCIWVUsDGU0l4/Ks9OHU14+4rE5FZYagjIqrjBEHAqI7+WDm+E+o7anHpei4Gf7sXfx5KkLs0IroHDHVERAQAaOnjiH9f7ozuTVxRUGzAG3+fwLSVkcgr1MtdGhFVAkMdEREZOdmo8cPodpjWpwkUArDyyGUM+jYCF1Nz5C6NiO6CoY6IiEwoFAImdm+EX5/vgHq2apxOysKAr/Zg/clEuUsjojtgqCMiogp1algP/01+GO39nZFdUIyXlh/F7H9OobDYIHdpRFQBhjoiIrotd3tr/DauA17s2gAAsDTiEoZ9vw9X0/NkroyIbsVQR0REd6RSKvBWv2AsGdUWdtYqHItPR/+Fu7HzzDW5SyOiMhjqiIioUh4Jccd/Lz+MpvXtcSO3CGOWHsQX4WegN4hyl0ZEYKgjIqJ74Ouiw1/jO+GZDr4QReDLrWcx+seDuJ5dIHdpRHUeQx0REd0TayslPh7UDF8MawGtlRJ7zqWi/8I9OHwpTe7SiOo0PvuViIjuy6BW3gj1csD4X4/gwrUcPP39fgxuXR8uthrYalTQqZWw0ahgo1bBRqO82aaSft58r1EpIAiC3B+FyCIw1BER0X1r7G6HdZM6461VJ/FP5FX8efjyPa2vVAjQqZXGEFgS/Gw0KrTzd8JznQOgUvKiElFlMNQREdEDsdWosPDplujfzAOnrmYiu6AYuQV6ZBcWI7egGDkFeuQUFiOnoBjZBXrkFhYj9+ajx/QGEVn5xcjKLy633S0xydh2OgVfDW8FN3vrmv5YRLUOQx0RET0wQRDQt6kn+jb1rNTyBoOI3CI9cgqKb06lwS+nUI/E9Dx8te0cDlxMw6MLd2Ph063QqVG9av4URLUbQx0REdU4hUKArUa6v+52Hglxx0vLj+J0Uhb+74cDmNqrMSZ1bwSFgvfgEVWENyoQEZFZauBqizUTwzCsrQ8MIvB5+BmMWXaIw6cQ3QZDHRERmS1rKyXmDmmO+U+1gLWVArvOXEP/hXtwJI7DpxDdiqGOiIjM3pA23lg7sTMauNogKTMfw77bjyW7LkAU+TQLohIMdUREVCs08ZCGTxnQwgvFBhEfrY/BC78cQUZekdylEZkFhjoiIqo1SoZP+WBgU6iVCoRHJ+Oxr3bj5OUMuUsjkh1DHRER1SqCIGDkQ374e0In+DhrkZCWhycX7cUv++N4OZbqNIY6IiKqlZp5O+DfSQ/jkRB3FOoNmLEmClNWHEd2QfmBjInqAoY6IiKqtRx0Vvh+ZBu82z8YKoWAdZFX8fjXexCblCV3aUQ1joMPExFRrSYIAp5/uAFa+jhi0m/HcOFaDp74Zg8+HNgMQ9p4y12eibScQuw7fx17zqVi3/lUKBQCega54ZEQD7Txc4KSAyvTAxBEC78B4fLly/Dx8UFCQgK8vc3rf24iIqpa17ML8Mqfkdh15hoAYFhbH8x+IhTWVkpZ6skr1OPQpTREnEvFnnOpiE7MxO3+6rrYqNEjyA29Qz3wcGA92WqmqiFH/mCoIyIii2IwiPhm+zl8seUMDCIQ5GGHOYObobG7HWzu8FiyqlCsN+DklQxEnEtFxLnrOBJ3A4V6g8kyjd1tEdaoHsIa1kOR3oDw6GRsPZ1iMjSLtZUCXQJd0TvUAz2C3OBso67WuqnqMdRVA4Y6IqK6ae+5VExecQyp2YXGNkedFeo7aqXJSfrp7aSF1802Zxs1BKHyl0BFUcT5azk3Q1wq9l24jqx8044ang7WCGtUD50b1UOnhi5ws7cut50ivQGHLqZhc3QywqOTcSU9zzhPIQDt/J3xSIg7eod4wNdFdx9Hg2oaQ101YKgjIqq7UjLz8fbqKBy4WD5sVURrpYSXozXqO+nKBD5r1HfUob6TFu52GqTlFCLifCr2nL2OiHOpSMrMN9mGvbUKnRrWQ1gjF4Q1qoeAejb3HBSjEzOx+ZQU8KITM03mB3nYoXeIOx4J8UDT+vb3tG2qOQx11YChjoiIACAzvwhX0/Nw5UYertz8eTk9z9iWklVw120oFQL0BtM/m2qVAu38ndCpoXQ2rml9hyrt8JCQlostMcnYfCoZBy+lmezfy8EavW6ewevQwBlWSg5qYS4Y6qoBQx0REVVGQbEeien5xsB3JT3P5HViRh6K9CIEAWhW38F4X1xbf6ca69SQnluIbadTsPlUMnaeuYa8Ir1xnp21ytiTtmsTV9hW8/2DdGcMddWAoY6IiKqCwSDiWnYBrK2UcNBayV0O8ov0iDiXivDoZGyJSTa5d1CtVCCskQt6h3qgZ7Ab3OzK38dH1UuO/MEYT0REVAkKhQD3Cjo5yMXaSomewe7oGewOvUHEsfgbCI9OxqZTSbh0PRfbY69he+w1CALQ2tfpZkcLdzRwtZW7dKomPFNHRERkQURRxLmUbGyOTsbmU0mIvJxhMr+Rmy16h7ijd6gHmtd3gKIK7/8r1huQmJGPhLRcJNzIRUKa1Is32NMeoV728HXWVen+zBkvv1YDhjoiIqrLEjPysCU6GZujk7Hv/HUUl+lo4W6vQa9gKeB1bOACterOHS1EUboEnZCWh8s3cqXwlpaHhBu5iE/LRWJGfrmOJGXZalQI9rRDiKc9Qr0cEOJlj0B3W2hUljfQMkNdNWCoIyIikmTkFWFHbAo2Rydjx+kU5BSW6WihUaFbkBt6h7gjoJ4NLt8oE9xu5BnPvuUXGe6wB6k3sLeTFj5OOvg4a1GsFxGTmImYpCwUFpdfV6UQ0MjN1hjyQr3sEexpXyX3LYqiiOyCYtzIKUJabiHScgqQllNk/DmpR6Nq61DCUFcNGOqIiIjKKyjWY+/568bx8FKz7z6kCwAIAuBpbw1vZx18nXXG8OZz87WbnabCS6zFegPOX8tBdGIGTl3JRHRiJk5dzTR5kkZZPs7a0jN6nvYIrW+PerYa3MgtRFpO6XQjpxDXb/5Myy0yCW43corKPdGjrG2vda22ewwZ6qoBQx0REdGdGQwijl9OvxnwknAjtwg+Tlp4lwltJQHOy1F718u0lSWKIq5m5OPUlQxjyIu+mmnyRI2qoLVSwtlGDWcbNZxs1HCxUcNJp8a4LgHwdNBW6b5KsPcrERER1TiFQkBrXye09nXCm/2Camy/giAYH9vWO9TD2J6eW4joRCngRV+Vwt65a9nQG6RxAp10ajjprOBio4GTjRWcbTRwvvWnTg1nWzWcdWpo1ZZ3z15FGOqIiIjIrDjq1OjUsB46NaxnbMsv0iO3UA8HrVWVPrHDkjDUERERkdmztlLW2JM7ais+JI6IiIjIAjDUEREREVkAhjoiIiIiC8BQR0RERGQBGOqIiIiILABDHREREZEFYKgjIiIisgAMdUREREQWoFaEum+++Qb+/v6wtrZGhw4dcPDgQblLIiIiIjIrZh/q/vjjD7z66qt47733cPToUbRo0QJ9+vRBSkqK3KURERERmQ2zD3Wff/45xo0bh2effRYhISFYvHgxdDodfvzxR7lLIyIiIjIbZh3qCgsLceTIEfTq1cvYplAo0KtXL+zbt6/CdQoKCpCZmWmcsrKyaqpcIiIiItmYdahLTU2FXq+Hu7u7Sbu7uzuSkpIqXGfOnDlwcHAwTiEhITVRKhEREZGsVHIXUNXeeustvPrqq8b3CQkJaNq0KRITE2WsioiIiOqSktxhMBhqbJ9mHerq1asHpVKJ5ORkk/bk5GR4eHhUuI5Go4FGozG+z83NBQC0b9+++golIiIiqkBycjJ8fX1rZF9mHerUajXatGmDrVu3YuDAgQCkxLt161ZMmjSpUtto1aoVDh48CHd3dygUZn21uUpkZWUhJCQE0dHRsLOzk7scs8JjUzEel9vjsbk9HpuK8bjcXl07NgaDAcnJyWjVqlWN7dOsQx0AvPrqqxg9ejTatm2L9u3bY8GCBcjJycGzzz5bqfVVKhXatWtXzVWaj8zMTABA/fr1YW9vL3M15oXHpmI8LrfHY3N7PDYV43G5vbp4bGrqDF0Jsw91w4YNw7Vr1zBz5kwkJSWhZcuW2LhxY7nOE0RERER1mdmHOgCYNGlSpS+3EhEREdVFln+TWR2j0Wjw3nvvmXQWIQmPTcV4XG6Px+b2eGwqxuNyezw21U8QRVGUuwgiIiIiejA8U0dERERkARjqiIiIiCwAQx0RERGRBWCosxBz5sxBu3btYGdnBzc3NwwcOBCxsbFyl2V2PvnkEwiCgKlTp8pdilm4cuUK/u///g8uLi7QarVo1qwZDh8+LHdZstLr9ZgxYwYCAgKg1WrRsGFDfPDBB6iLtx/v2rULAwYMgJeXFwRBwJo1a0zmi6KImTNnwtPTE1qtFr169cLZs2flKbaG3enYFBUVYfr06WjWrBlsbGzg5eWFUaNG4erVq/IVXIPu9r0pa/z48RAEAQsWLKix+iwZQ52F2LlzJyZOnIj9+/cjPDwcRUVF6N27N3JycuQuzWwcOnQI3333HZo3by53KWbhxo0bCAsLg5WVFTZs2IDo6Gh89tlncHJykrs0Wc2dOxeLFi3C119/jZiYGMydOxfz5s3DV199JXdpNS4nJwctWrTAN998U+H8efPmYeHChVi8eDEOHDgAGxsb9OnTB/n5+TVcac2707HJzc3F0aNHMWPGDBw9ehSrVq1CbGwsHn/8cRkqrXl3+96UWL16Nfbv3w8vL68aqqwOEMkipaSkiADEnTt3yl2KWcjKyhIDAwPF8PBwsWvXruKUKVPkLkl206dPFzt37ix3GWanf//+4tixY03aBg8eLI4YMUKmiswDAHH16tXG9waDQfTw8BA//fRTY1t6erqo0WjE33//XYYK5XPrsanIwYMHRQBiXFxczRRlJm53bC5fvizWr19fjIqKEv38/MQvvviixmuzRDxTZ6EyMjIAAM7OzjJXYh4mTpyI/v37o1evXnKXYjbWrVuHtm3b4qmnnoKbmxtatWqFJUuWyF2W7Dp16oStW7fizJkzAIDIyEjs2bMH/fr1k7ky83Lx4kUkJSWZ/D/l4OCADh06YN++fTJWZp4yMjIgCAIcHR3lLkV2BoMBI0eOxLRp0xAaGip3ORalVjxRgu6NwWDA1KlTERYWhqZNm8pdjuxWrFiBo0eP4tChQ3KXYlYuXLiARYsW4dVXX8Xbb7+NQ4cOYfLkyVCr1Rg9erTc5cnmzTffRGZmJoKCgqBUKqHX6/HRRx9hxIgRcpdmVpKSkgCg3CMb3d3djfNIkp+fj+nTp2P48OF15pmndzJ37lyoVCpMnjxZ7lIsDkOdBZo4cSKioqKwZ88euUuRXUJCAqZMmYLw8HBYW1vLXY5ZMRgMaNu2LT7++GMAQKtWrRAVFYXFixfX6VD3559/Yvny5fjtt98QGhqK48ePY+rUqfDy8qrTx4XuT1FREYYOHQpRFLFo0SK5y5HdkSNH8OWXX+Lo0aMQBEHuciwOL79amEmTJuHff//F9u3b4e3tLXc5sjty5AhSUlLQunVrqFQqqFQq7Ny5EwsXLoRKpYJer5e7RNl4enoiJCTEpC04OBjx8fEyVWQepk2bhjfffBNPP/00mjVrhpEjR+KVV17BnDlz5C7NrHh4eAAAkpOTTdqTk5ON8+q6kkAXFxeH8PBwnqUDsHv3bqSkpMDX19f4OzkuLg6vvfYa/P395S6v1uOZOgshiiJefvllrF69Gjt27EBAQIDcJZmFnj174uTJkyZtzz77LIKCgjB9+nQolUqZKpNfWFhYuWFvzpw5Az8/P5kqMg+5ublQKEz/vatUKmEwGGSqyDwFBATAw8MDW7duRcuWLQEAmZmZOHDgACZMmCBvcWagJNCdPXsW27dvh4uLi9wlmYWRI0eWu7e5T58+GDlyJJ599lmZqrIcDHUWYuLEifjtt9+wdu1a2NnZGe9pcXBwgFarlbk6+djZ2ZW7r9DGxgYuLi51/n7DV155BZ06dcLHH3+MoUOH4uDBg/j+++/x/fffy12arAYMGICPPvoIvr6+CA0NxbFjx/D5559j7NixcpdW47Kzs3Hu3Dnj+4sXL+L48eNwdnaGr68vpk6dig8//BCBgYEICAjAjBkz4OXlhYEDB8pXdA2507Hx9PTEkCFDcPToUfz777/Q6/XG38nOzs5Qq9VylV0j7va9uTXgWllZwcPDA02aNKnpUi2P3N1vqWoAqHBaunSp3KWZHQ5pUuqff/4RmzZtKmo0GjEoKEj8/vvv5S5JdpmZmeKUKVNEX19f0draWmzQoIH4zjvviAUFBXKXVuO2b99e4e+V0aNHi6IoDWsyY8YM0d3dXdRoNGLPnj3F2NhYeYuuIXc6NhcvXrzt7+Tt27fLXXq1u9v35lYc0qTqCKJYB4dJJyIiIrIw7ChBREREZAEY6oiIiIgsAEMdERERkQVgqCMiIiKyAAx1RERERBaAoY6IiIjIAjDUEREREVkAhjoiIiIiC8BQR0T0gARBwJo1a+Qug4jqOIY6IqrVxowZA0EQyk19+/aVuzQiohqlkrsAIqIH1bdvXyxdutSkTaPRyFQNEZE8eKaOiGo9jUYDDw8Pk8nJyQmAdGl00aJF6NevH7RaLRo0aIC//vrLZP2TJ0+iR48e0Gq1cHFxwQsvvIDs7GyTZX788UeEhoZCo9HA09MTkyZNMpmfmpqKQYMGQafTITAwEOvWrTPOu3HjBkaMGAFXV1dotVoEBgaWC6FERA+KoY6ILN6MGTPw5JNPIjIyEiNGjMDTTz+NmJgYAEBOTg769OkDJycnHDp0CCtXrsSWLVtMQtuiRYswceJEvPDCCzh58iTWrVuHRo0amexj9uzZGDp0KE6cOIFHH30UI0aMQFpamnH/0dHR2LBhA2JiYrBo0SLUq1ev5g4AEdUNIhFRLTZ69GhRqVSKNjY2JtNHH30kiqIoAhDHjx9vsk6HDh3ECRMmiKIoit9//73o5OQkZmdnG+f/999/okKhEJOSkkRRFEUvLy/xnXfeuW0NAMR3333X+D47O1sEIG7YsEEURVEcMGCA+Oyzz1bNByYiug3eU0dEtV737t2xaNEikzZnZ2fj644dO5rM69ixI44fPw4AiImJQYsWLWBjY2OcHxYWBoPBgNjYWAiCgKtXr6Jnz553rKF58+bG1zY2NrC3t0dKSgoAYMKECXjyySdx9OhR9O7dGwMHDkSnTp3u67MSEd0OQx0R1Xo2NjblLodWFa1WW6nlrKysTN4LggCDwQAA6NevH+Li4rB+/XqEh4ejZ8+emDhxIubPn1/l9RJR3cV76ojI4u3fv7/c++DgYABAcHAwIiMjkZOTY5wfEREBhUKBJk2awM7ODv7+/ti6desD1eDq6orRo0fj119/xYIFC/D9998/0PaIiG7FM3VEVOsVFBQgKSnJpE2lUhk7I6xcuRJt27ZF586dsXz5chw8eBA//PADAGDEiBF47733MHr0aMyaNQvXrl3Dyy+/jJEjR8Ld3R0AMGvWLIwfPx5ubm7o168fsrKyEBERgZdffrlS9c2cORNt2rRBaGgoCgoK8O+//xpDJRFRVWGoI6Jab+PGjfD09DRpa9KkCU6fPg1A6pm6YsUKvPTSS/D09MTvv/+OkJAQAIBOp8OmTZswZcoUtGvXDjqdDk8++SQ+//xz47ZGjx6N/Px8fPHFF3j99ddRr149DBkypNL1qdVqvPXWW7h06RK0Wi0efvhhrFixogo+ORFRKUEURVHuIoiIqosgCFi9ejUGDhwodylERNWK99QRERERWQCGOiIiIiILwHvqiMii8Q4TIqoreKaOiIiIyAIw1BERERFZAIY6IiIiIgvAUEdERERkARjqiIiIiCwAQx0RERGRBWCoIyIiIrIADHVEREREFoChjoiIiMgC/D8YPsYDaCYvWQAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from previous_chapters import plot_losses\n",
    "\n",
    "epochs_tensor = torch.linspace(1, n_epochs, len(train_losses))\n",
    "plot_losses(epochs_tensor, tokens_seen, train_losses, val_losses)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c16fa614-67e1-4254-8b7e-c3e2f690c29c",
   "metadata": {},
   "source": [
    "- Note that the model is overfitting here because the dataset is kept very small for educational purposes (so that the code can be executed on a laptop computer)\n",
    "- For a longer pretraining run on a much larger dataset, see [../../ch05/03_bonus_pretraining_on_gutenberg](../../ch05/03_bonus_pretraining_on_gutenberg)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}