LLMs-from-scratch/ch03/02_bonus_efficient-multihead-attention/mha-implementations.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "e2e65c03-36d4-413f-9b23-5cdd816729ab",
   "metadata": {},
   "source": [
    "<font size=\"1\">\n",
    "Supplementary code for \"Build a Large Language Model From Scratch\": <a href=\"https://www.manning.com/books/build-a-large-language-model-from-scratch\">https://www.manning.com/books/build-a-large-language-model-from-scratch</a> by <a href=\"https://sebastianraschka.com\">Sebastian Raschka</a><br>\n",
    "Code repository: <a href=\"https://github.com/rasbt/LLMs-from-scratch\">https://github.com/rasbt/LLMs-from-scratch</a>\n",
    "</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6f678e62-7bcb-4405-86ae-dce94f494303",
   "metadata": {
    "id": "6f678e62-7bcb-4405-86ae-dce94f494303"
   },
   "source": [
    "# Comparing Efficient Multi-Head Attention Implementations"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b742938a-4bfc-4527-a1f1-d5963508967d",
   "metadata": {
    "id": "b742938a-4bfc-4527-a1f1-d5963508967d"
   },
   "source": [
    "This code notebook compares different ways to implement causal multi-head attention used in decoder-style LLMs like GPT, Llama, etc."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "7898551e-f582-48ac-9f66-3632abe2a93f",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "7898551e-f582-48ac-9f66-3632abe2a93f",
    "outputId": "7d088260-3fa1-44f2-bd65-2a46e289f9d4"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "PyTorch version: 2.2.1+cu121\n",
      "Running on cuda\n"
     ]
    }
   ],
   "source": [
    "import torch\n",
    "\n",
    "torch.manual_seed(123)\n",
    "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
    "print(f\"PyTorch version: {torch.__version__}\")\n",
    "print(f\"Running on {device}\")\n",
    "\n",
    "batch_size = 8\n",
    "context_len = 1024\n",
    "embed_dim = 768\n",
    "embeddings = torch.randn((batch_size, context_len, embed_dim), device=device)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2f9bb1b6-a1e5-4e0a-884d-0f31b374a8d6",
   "metadata": {
    "id": "2f9bb1b6-a1e5-4e0a-884d-0f31b374a8d6"
   },
   "source": [
    "<br>\n",
    "&nbsp;\n",
    "\n",
    "## 1) CausalAttention MHA wrapper class from chapter 3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "297c93ed-aec0-4896-bb89-42c4b294d3d1",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "297c93ed-aec0-4896-bb89-42c4b294d3d1",
    "outputId": "f8a33752-2cd6-4101-8feb-9d1699984719"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([8, 1024, 768])\n"
     ]
    }
   ],
   "source": [
    "from ch03 import MultiHeadAttentionWrapper as Ch03_MHA_Wrapper\n",
    "\n",
    "mha_ch03_wrapper = Ch03_MHA_Wrapper(\n",
    "    d_in=embed_dim,\n",
    "    d_out=embed_dim//12,\n",
    "    block_size=context_len,\n",
    "    dropout=0.0,\n",
    "    num_heads=12,\n",
    "    qkv_bias=False\n",
    ").to(device)\n",
    "\n",
    "out = mha_ch03_wrapper(embeddings)\n",
    "print(out.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "21930804-b327-40b1-8e63-94dcad39ce7b",
   "metadata": {
    "id": "21930804-b327-40b1-8e63-94dcad39ce7b"
   },
   "source": [
    "<br>\n",
    "&nbsp;\n",
    "\n",
    "## 2) The multi-head attention class from chapter 3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "4ee6a61b-d25c-4a0c-8a59-f285544e3710",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "4ee6a61b-d25c-4a0c-8a59-f285544e3710",
    "outputId": "b704a040-3547-422c-ecda-df9982a2da35"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([8, 1024, 768])\n"
     ]
    }
   ],
   "source": [
    "from ch03 import MultiHeadAttention as Ch03_MHA\n",
    "\n",
    "mha_ch03 = Ch03_MHA(\n",
    "    d_in=embed_dim,\n",
    "    d_out=embed_dim,\n",
    "    block_size=context_len,\n",
    "    dropout=0.0,\n",
    "    num_heads=12,\n",
    "    qkv_bias=False\n",
    ").to(device)\n",
    "\n",
    "out = mha_ch03(embeddings)\n",
    "print(out.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "73cd11da-ea3b-4081-b483-c4965dfefbc4",
   "metadata": {
    "id": "73cd11da-ea3b-4081-b483-c4965dfefbc4"
   },
   "source": [
    "<br>\n",
    "&nbsp;\n",
    "\n",
    "## 3) An alternative multi-head attention with combined weights"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1fa1a5ea-eaff-4d2d-aaf0-b34cdb6fd4dd",
   "metadata": {
    "id": "1fa1a5ea-eaff-4d2d-aaf0-b34cdb6fd4dd"
   },
   "source": [
    "- The code for the `MultiHeadAttentionAlt` class below is based on code that was kindly shared by [Rayed Bin Wahed](https://github.com/rasbt/LLMs-from-scratch/discussions/51)\n",
    "- The main difference between the `MultiHeadAttentionAlt` class and the `MultiHeadAttention` class used in chapter 3 is that `MultiHeadAttentionAlt` uses a single weight matrix, `self.qkv = nn.Linear(d_in, 3 * d_out, bias=qkv_bias)` instead of separate weight matrices:\n",
    "\n",
    "  - `self.W_query = nn.Linear(d_in, d_out, bias=qkv_bias)`\n",
    "  - `self.W_key = nn.Linear(d_in, d_out, bias=qkv_bias)`\n",
    "  - `self.W_value = nn.Linear(d_in, d_out, bias=qkv_bias)`\n",
    "\n",
    "- Here, `self.qkv` combines all three weight matrices `self.W_query`, `self.W_key`, and `self.W_value` to carry out the query, key, and value computation in a single step\n",
    "- Using `q, k, v = qkv.unbind(0)`, we obtain the individual query, key, and value tensors, which are then used similarly to the query, key, and value tensors in the `MultiHeadAttention` class in chapter 3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "9a6bd0a2-f27c-4602-afa0-c96cd295c1a6",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "9a6bd0a2-f27c-4602-afa0-c96cd295c1a6",
    "outputId": "5d948671-176f-4633-bede-97767e36becc"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([8, 1024, 768])\n"
     ]
    }
   ],
   "source": [
    "import torch.nn as nn\n",
    "\n",
    "\n",
    "class MultiHeadAttentionCombinedQKV(nn.Module):\n",
    "    def __init__(self, d_in, d_out, num_heads, block_size, dropout=0.0, qkv_bias=False):\n",
    "        super().__init__()\n",
    "\n",
    "        assert d_out % num_heads == 0, \"embed_dim is indivisible by num_heads\"\n",
    "\n",
    "        self.num_heads = num_heads\n",
    "        self.block_size = block_size\n",
    "        self.head_dim = d_out // num_heads\n",
    "\n",
    "        self.qkv = nn.Linear(d_in, 3 * d_out, bias=qkv_bias)\n",
    "        self.proj = nn.Linear(d_in, d_out)\n",
    "        self.dropout = nn.Dropout(dropout)\n",
    "\n",
    "        self.register_buffer(\n",
    "            \"mask\", torch.triu(torch.ones(block_size, block_size), diagonal=1)\n",
    "        )\n",
    "\n",
    "    def forward(self, x):\n",
    "        batch_size, num_tokens, embed_dim = x.shape\n",
    "\n",
    "        # (b, num_tokens, embed_dim) --> (b, num_tokens, 3 * embed_dim)\n",
    "        qkv = self.qkv(x)\n",
    "\n",
    "        # (b, num_tokens, 3 * embed_dim) --> (b, num_tokens, 3, num_heads, head_dim)\n",
    "        qkv = qkv.reshape(batch_size, num_tokens, 3, self.num_heads, self.head_dim)\n",
    "\n",
    "        # (b, num_tokens, 3, num_heads, head_dim) --> (3, b, num_heads, num_tokens, head_dim)\n",
    "        qkv = qkv.permute(2, 0, 3, 1, 4)\n",
    "\n",
    "        # (3, b, num_heads, num_tokens, head_dim) -> 3 times (b, num_head, num_tokens, head_dim)\n",
    "        queries, keys, values = qkv.unbind(0)\n",
    "\n",
    "        # (b, num_heads, num_tokens, head_dim) --> (b, num_heads, num_tokens, num_tokens)\n",
    "        attn_scores = queries @ keys.transpose(-2, -1)\n",
    "        attn_scores = attn_scores.masked_fill(\n",
    "            self.mask.bool()[:num_tokens, :num_tokens], -torch.inf\n",
    "        )\n",
    "\n",
    "        attn_weights = torch.softmax(attn_scores / keys.shape[-1]**-0.5, dim=-1)\n",
    "        attn_weights = self.dropout(attn_weights)\n",
    "\n",
    "        # (b, num_heads, num_tokens, num_tokens) --> (b, num_heads, num_tokens, head_dim)\n",
    "        context_vec = attn_weights @ values\n",
    "\n",
    "        # (b, num_heads, num_tokens, head_dim) --> (b, num_tokens, num_heads, head_dim)\n",
    "        context_vec = context_vec.transpose(1, 2)\n",
    "\n",
    "        # (b, num_tokens, num_heads, head_dim) --> (b, num_tokens, embed_dim)\n",
    "        context_vec = context_vec.reshape(batch_size, num_tokens, embed_dim)\n",
    "\n",
    "        context_vec = self.proj(context_vec)\n",
    "\n",
    "        return context_vec\n",
    "\n",
    "\n",
    "mha_combined_qkv = MultiHeadAttentionCombinedQKV(\n",
    "    d_in=embed_dim,\n",
    "    d_out=embed_dim,\n",
    "    block_size=context_len,\n",
    "    dropout=0.0,\n",
    "    num_heads=12,\n",
    "    qkv_bias=False\n",
    ").to(device)\n",
    "\n",
    "out = mha_combined_qkv(embeddings)\n",
    "print(out.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "48a042d3-ee78-4c29-bf63-d92fe6706632",
   "metadata": {
    "id": "48a042d3-ee78-4c29-bf63-d92fe6706632"
   },
   "source": [
    "<br>\n",
    "&nbsp;\n",
    "\n",
    "## 4) Multihead attention with PyTorch's scaled dot product attention"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f78e346f-3b85-44e6-9feb-f01131381148",
   "metadata": {
    "id": "f78e346f-3b85-44e6-9feb-f01131381148"
   },
   "source": [
    "- The implementation below uses PyTorch's [`scaled_dot_product_attention`](https://pytorch.org/docs/stable/generated/torch.nn.functional.scaled_dot_product_attention.html) function, which implements a memory-optimized version of self-attention calld [flash attention](https://arxiv.org/abs/2205.14135)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "1b8e5a0d-1f65-4a03-bf6e-723f0cc428f5",
   "metadata": {
    "id": "1b8e5a0d-1f65-4a03-bf6e-723f0cc428f5"
   },
   "outputs": [],
   "source": [
    "class MHAPyTorchScaledDotProduct(nn.Module):\n",
    "    def __init__(self, d_in, d_out, num_heads, block_size, dropout=0.0, qkv_bias=False):\n",
    "        super().__init__()\n",
    "\n",
    "        assert d_out % num_heads == 0, \"embed_dim is indivisible by num_heads\"\n",
    "\n",
    "        self.num_heads = num_heads\n",
    "        self.block_size = block_size\n",
    "        self.head_dim = d_out // num_heads\n",
    "        self.d_out = d_out\n",
    "\n",
    "        self.qkv = nn.Linear(d_in, 3 * d_out, bias=qkv_bias)\n",
    "        self.proj = nn.Linear(d_in, d_out)\n",
    "        self.dropout = dropout\n",
    "\n",
    "        self.register_buffer(\n",
    "            \"mask\", torch.triu(torch.ones(block_size, block_size), diagonal=1)\n",
    "        )\n",
    "\n",
    "    def forward(self, x):\n",
    "        batch_size, num_tokens, embed_dim = x.shape\n",
    "\n",
    "        # (b, num_tokens, embed_dim) --> (b, num_tokens, 3 * embed_dim)\n",
    "        qkv = self.qkv(x)\n",
    "\n",
    "        # (b, num_tokens, 3 * embed_dim) --> (b, num_tokens, 3, num_heads, head_dim)\n",
    "        qkv = qkv.reshape(batch_size, num_tokens, 3, self.num_heads, self.head_dim)\n",
    "\n",
    "        # (b, num_tokens, 3, num_heads, head_dim) --> (3, b, num_heads, num_tokens, head_dim)\n",
    "        qkv = qkv.permute(2, 0, 3, 1, 4)\n",
    "\n",
    "        # (3, b, num_heads, num_tokens, head_dim) -> 3 times (b, num_heads, num_tokens, head_dim)\n",
    "        queries, keys, values = qkv.unbind(0)\n",
    "\n",
    "        use_dropout = 0. if not self.training else self.dropout\n",
    "        context_vec = nn.functional.scaled_dot_product_attention(\n",
    "            queries, keys, values, attn_mask=None, dropout_p=use_dropout, is_causal=True)\n",
    "\n",
    "        # Combine heads, where self.d_out = self.num_heads * self.head_dim\n",
    "        context_vec = context_vec.transpose(1, 2).contiguous().view(batch_size, num_tokens, self.d_out)\n",
    "\n",
    "        return context_vec"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "fbc8ba92-3471-41cb-b1b2-4c0ef5be392b",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "fbc8ba92-3471-41cb-b1b2-4c0ef5be392b",
    "outputId": "af9e4855-7f20-4d61-8532-4827df8dfb30"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([8, 1024, 768])\n"
     ]
    }
   ],
   "source": [
    "mha_pytorch_scaled = MHAPyTorchScaledDotProduct(\n",
    "    d_in=embed_dim,\n",
    "    d_out=embed_dim,\n",
    "    block_size=context_len,\n",
    "    dropout=0.0,\n",
    "    num_heads=12,\n",
    "    qkv_bias=False\n",
    ").to(device)\n",
    "\n",
    "out = mha_pytorch_scaled(embeddings)\n",
    "print(out.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "351c318f-4835-4d74-8d58-a070222447c4",
   "metadata": {
    "id": "351c318f-4835-4d74-8d58-a070222447c4"
   },
   "source": [
    "<br>\n",
    "&nbsp;\n",
    "\n",
    "## 5) Using PyTorch's torch.nn.MultiheadAttention"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "74a6d060-6324-48fa-a35c-cb09f2a48965",
   "metadata": {
    "id": "74a6d060-6324-48fa-a35c-cb09f2a48965"
   },
   "source": [
    "- Below, we use PyTorch's [torch.nn.MultiheadAttention](https://pytorch.org/docs/stable/generated/torch.nn.MultiheadAttention.html) implementation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "3799c7ef-3155-42c6-a829-f95656453ae0",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "3799c7ef-3155-42c6-a829-f95656453ae0",
    "outputId": "2a085df8-0445-4818-9978-6dc74469f568"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([8, 1024, 768])\n"
     ]
    }
   ],
   "source": [
    "import torch.nn as nn\n",
    "\n",
    "\n",
    "class MHAPyTorchClass(nn.Module):\n",
    "    def __init__(self, d_in, d_out, num_heads, block_size, dropout=0.0, qkv_bias=False, need_weights=True):\n",
    "        super().__init__()\n",
    "\n",
    "        self.block_size = block_size\n",
    "        self.multihead_attn = nn.MultiheadAttention(\n",
    "            embed_dim=d_out,\n",
    "            num_heads=num_heads,\n",
    "            dropout=dropout,\n",
    "            bias=qkv_bias,\n",
    "            add_bias_kv=qkv_bias,\n",
    "            batch_first=True,\n",
    "        )\n",
    "\n",
    "        self.need_weights = need_weights\n",
    "        self.proj = nn.Linear(d_out, d_out)\n",
    "        self.register_buffer(\"mask\", torch.triu(torch.ones(block_size, block_size), diagonal=1).bool())\n",
    "\n",
    "    def forward(self, x):\n",
    "        batch_size, num_tokens, _ = x.shape\n",
    "\n",
    "        # Ensure attn_mask is compatible with expected shape and `batch_first=True`\n",
    "        # No need to manually adjust for num_heads; ensure it's right for the sequence\n",
    "        if self.block_size >= num_tokens:\n",
    "            attn_mask = self.mask[:num_tokens, :num_tokens]\n",
    "        else:\n",
    "            attn_mask = self.mask[:self.block_size, :self.block_size]\n",
    "\n",
    "        # attn_mask broadcasting will handle batch_size dimension implicitly\n",
    "        attn_output, _ = self.multihead_attn(\n",
    "            x, x, x, attn_mask=attn_mask, need_weights=self.need_weights\n",
    "        )\n",
    "\n",
    "        output = self.proj(attn_output)\n",
    "\n",
    "        return output\n",
    "\n",
    "\n",
    "mha_pytorch_class_default = MHAPyTorchClass(\n",
    "    d_in=embed_dim,\n",
    "    d_out=embed_dim,\n",
    "    block_size=context_len,\n",
    "    dropout=0.0,\n",
    "    num_heads=12,\n",
    "    qkv_bias=False\n",
    ").to(device)\n",
    "\n",
    "out = mha_pytorch_class_default(embeddings)\n",
    "print(out.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a3953bff-1056-4de2-bfd1-dfccf659eee4",
   "metadata": {
    "id": "a3953bff-1056-4de2-bfd1-dfccf659eee4"
   },
   "source": [
    "<br>\n",
    "&nbsp;\n",
    "\n",
    "## 6) Using PyTorch's torch.nn.MultiheadAttention with `scaled_dot_product_attention`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d2164859-31a0-4537-b4fb-27d57675ba77",
   "metadata": {
    "id": "d2164859-31a0-4537-b4fb-27d57675ba77"
   },
   "source": [
    "- Set `need_weights` (default `True`) to need_weights=False so that MultiheadAttention uses `scaled_dot_product_attention` [according to the documentation](https://github.com/pytorch/pytorch/blob/71d020262793542974cf13b30f2a9099773f015c/torch/nn/modules/activation.py#L1096)\n",
    "\n",
    ">  need_weights: If specified, returns ``attn_output_weights`` in addition to ``attn_outputs``.\n",
    "            Set ``need_weights=False`` to use the optimized ``scaled_dot_product_attention``\n",
    "            and achieve the best performance for MHA.\n",
    "            Default: ``True``."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "4a4c2afe-5e1f-4bd7-a118-67031176f147",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "4a4c2afe-5e1f-4bd7-a118-67031176f147",
    "outputId": "234771f4-8a53-4478-8a9b-cf19f79a5e07"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([8, 1024, 768])\n"
     ]
    }
   ],
   "source": [
    "mha_pytorch_class_noweights = MHAPyTorchClass(\n",
    "    d_in=embed_dim,\n",
    "    d_out=embed_dim,\n",
    "    block_size=context_len,\n",
    "    dropout=0.0,\n",
    "    num_heads=12,\n",
    "    qkv_bias=False,\n",
    "    need_weights=False # NEW!\n",
    ").to(device)\n",
    "\n",
    "out = mha_pytorch_class_noweights(embeddings)\n",
    "print(out.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8877de71-f84f-4f6d-bc87-7552013b6301",
   "metadata": {
    "id": "8877de71-f84f-4f6d-bc87-7552013b6301"
   },
   "source": [
    "<br>\n",
    "&nbsp;\n",
    "\n",
    "## Quick speed comparison (M3 Macbook Air CPU)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a97c0b2e-6593-49d8-98bc-2267b3aa610f",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "a97c0b2e-6593-49d8-98bc-2267b3aa610f",
    "outputId": "ebe635b2-5c03-4e9b-da3a-951d308acf7b"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200 ms ± 5.98 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)\n"
     ]
    }
   ],
   "source": [
    "## 1) CausalAttention MHA wrapper class from chapter 3\n",
    "%timeit mha_ch03_wrapper(embeddings)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "19db9c2c-8e75-431a-8eef-0b4d8284e6e6",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "19db9c2c-8e75-431a-8eef-0b4d8284e6e6",
    "outputId": "c6e7bcff-661c-45a6-da82-b1e3f89cf761"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "198 ms ± 6.66 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)\n"
     ]
    }
   ],
   "source": [
    "## 2) The multi-head attention class from chapter 3\n",
    "%timeit mha_ch03(embeddings)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "aa526ee0-7a88-4f34-a49a-f8f97da83779",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "aa526ee0-7a88-4f34-a49a-f8f97da83779",
    "outputId": "92b634f8-43f8-468f-87a1-bb774b64c212"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "236 ms ± 13.3 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)\n"
     ]
    }
   ],
   "source": [
    "## 3) An alternative multi-head attention with combined weights\n",
    "%timeit mha_combined_qkv(embeddings)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cc2b4256-16d8-4c34-9fd0-d4b4af0e60fa",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "cc2b4256-16d8-4c34-9fd0-d4b4af0e60fa",
    "outputId": "80c6e314-0771-470e-b090-628984ce2d85"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "71.6 ms ± 3.32 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)\n"
     ]
    }
   ],
   "source": [
    "## 4) Multihead attention with PyTorch's scaled dot product attention\n",
    "%timeit mha_pytorch_scaled(embeddings)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0f209e70-ebb6-4a1a-b608-1ff42e41c01d",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "0f209e70-ebb6-4a1a-b608-1ff42e41c01d",
    "outputId": "3cd37b53-04d4-4dd0-9450-6fc8ebaac083"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "217 ms ± 4.27 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)\n"
     ]
    }
   ],
   "source": [
    "## 5) Using PyTorch's torch.nn.MultiheadAttention\n",
    "%timeit mha_pytorch_class_default(embeddings)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3f4968c2-8d40-4ab9-8dba-052b4f77d756",
   "metadata": {
    "id": "3f4968c2-8d40-4ab9-8dba-052b4f77d756",
    "outputId": "2e86bdb4-7fa0-4051-b000-4a2b591060a2",
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "205 ms ± 3.9 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)\n"
     ]
    }
   ],
   "source": [
    "## 6) Using PyTorch's torch.nn.MultiheadAttention disabling `need_weights`\n",
    "%timeit mha_pytorch_class_noweights(embeddings)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a78ff594-6cc2-496d-a302-789fa104c3c9",
   "metadata": {
    "id": "a78ff594-6cc2-496d-a302-789fa104c3c9"
   },
   "source": [
    "<br>\n",
    "&nbsp;\n",
    "\n",
    "## Quick speed comparison (Nvidia A100 GPU)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "707a2a14-a089-48a8-88aa-d328e1e0a9d0",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "707a2a14-a089-48a8-88aa-d328e1e0a9d0",
    "outputId": "e99a17e9-8139-4b04-dac8-fa1dd5027735"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "8.35 ms ± 1.44 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)\n"
     ]
    }
   ],
   "source": [
    "## 1) CausalAttention MHA wrapper class from chapter 3\n",
    "%timeit mha_ch03_wrapper(embeddings)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "8686dd69-3655-40e4-a57b-a2c55532a010",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "8686dd69-3655-40e4-a57b-a2c55532a010",
    "outputId": "5553b42c-b709-41a4-8a8b-be36dae408ab"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "6.59 ms ± 231 ns per loop (mean ± std. dev. of 7 runs, 100 loops each)\n"
     ]
    }
   ],
   "source": [
    "## 2) The multi-head attention class from chapter 3\n",
    "%timeit mha_ch03(embeddings)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "2209d7df-e54b-4910-ae2b-c78cf684d9bf",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "2209d7df-e54b-4910-ae2b-c78cf684d9bf",
    "outputId": "01b0da88-510b-4b21-919a-0a7519a55ed8"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "7.21 ms ± 716 ns per loop (mean ± std. dev. of 7 runs, 100 loops each)\n"
     ]
    }
   ],
   "source": [
    "## 3) An alternative multi-head attention with combined weights\n",
    "%timeit mha_combined_qkv(embeddings)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "1075abe2-4839-4fd6-af3e-c09bb3651e26",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "1075abe2-4839-4fd6-af3e-c09bb3651e26",
    "outputId": "542706db-5041-45ca-f667-9e1bd1c2c7aa"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2.38 ms ± 362 ns per loop (mean ± std. dev. of 7 runs, 1000 loops each)\n"
     ]
    }
   ],
   "source": [
    "## 4) Multihead attention with PyTorch's scaled dot product attention\n",
    "%timeit mha_pytorch_scaled(embeddings)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "868e3670-8edc-47bc-9e06-eb505e44dc9d",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "868e3670-8edc-47bc-9e06-eb505e44dc9d",
    "outputId": "13cfc808-2b11-4041-fe67-e5a63abe4f28"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "6.67 ms ± 408 ns per loop (mean ± std. dev. of 7 runs, 100 loops each)\n"
     ]
    }
   ],
   "source": [
    "## 5) Using PyTorch's torch.nn.MultiheadAttention\n",
    "%timeit mha_pytorch_class_default(embeddings)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "944870e6-de54-4e3b-a455-b8f21f6f92c8",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "944870e6-de54-4e3b-a455-b8f21f6f92c8",
    "outputId": "c52858e7-999c-4782-adc9-731f8d69dfa6"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "4.54 ms ± 7.17 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)\n"
     ]
    }
   ],
   "source": [
    "## 6) Using PyTorch's torch.nn.MultiheadAttention disabling `need_weights`\n",
    "%timeit mha_pytorch_class_noweights(embeddings)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dabc6575-0316-4640-a729-e616d5c17b73",
   "metadata": {
    "id": "dabc6575-0316-4640-a729-e616d5c17b73"
   },
   "source": [
    "<br>\n",
    "&nbsp;\n",
    "\n",
    "\n",
    "## Speed comparison (Nvidia A100 GPU) with warmup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "29b63d3d-6d0b-43bb-9c68-d5514dc81000",
   "metadata": {
    "id": "29b63d3d-6d0b-43bb-9c68-d5514dc81000"
   },
   "outputs": [],
   "source": [
    "# CUDA benchmark code shared by Andrei Aksionov\n",
    "# and based on code from\n",
    "# https://github.com/cuda-mode/lectures/blob/main/lecture1/pytorch_square.py\n",
    "\n",
    "def time_pytorch_function(func, *input, num_repeats = 1_000):\n",
    "    # CUDA IS ASYNC so can't use python time module\n",
    "    start = torch.cuda.Event(enable_timing=True)\n",
    "    end = torch.cuda.Event(enable_timing=True)\n",
    "\n",
    "    # Warmup\n",
    "    for _ in range(5):\n",
    "        func(*input)\n",
    "    torch.cuda.synchronize()\n",
    "\n",
    "    start.record()\n",
    "    for _ in range(num_repeats):\n",
    "        func(*input)\n",
    "        torch.cuda.synchronize()\n",
    "    end.record()\n",
    "    torch.cuda.synchronize()\n",
    "    return start.elapsed_time(end) / num_repeats"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "CDJAPZaszaqx",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 489
    },
    "id": "CDJAPZaszaqx",
    "outputId": "f23e9b83-7fd6-4011-9434-0e6934cf762a"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAnAAAAHYCAYAAADNtNW9AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAADOM0lEQVR4nOzdd1gUydbA4d+QQVBERMGc1hxX16yYIwoiZgUTyoogCmYExYQYMAfMOec1rVl3zTlhTogYMKGICPL94UcvI6DuXkaC530en7v0dE/V1O3pOV1ddUplZmYWhxBCCCGESDe0UrsCQgghhBDi35EATgghhBAinZEATgghhBAinZEATgghhBAinZEATgghhBAinZEATgghhBAinZEATgghhBAinZEATgghhBAinZEATgghhBAinZEATgghhBAinUk3AVzVqlVZsWIFV65cITw8nKZNmybaZ/DgwVy5coWQkBA2btxIwYIFU6GmQgghhBCapZPaFfheRkZGXLlyhZUrV7J06dJEr7u5ueHs7EyfPn24f/8+Q4cOZd26dVSrVo0PHz58dzmWlpa8ffs2JasuhBBCCPHdjI2Nefz48Vf3UaXHxezDw8Pp3LkzO3bsULZduXKFWbNmMXPmTABMTEwIDg7G1dWVTZs2fdf7WlpacvnyZY3UWQghhBDie5UqVeqrQVy66YH7mnz58pEzZ04OHTqkbIuIiODMmTNUqlQp2QBOT08PfX39RNtLlSolvXBCCCGE+OGMjY25fPnyN+OQDBHAWVhYAPDs2TO17c+ePVNeS0q/fv0YNGhQou1v374lIiIiZSsphBBCCJFC0s0kBk0IDAwkf/78yr9SpUqldpWEEEIIIb4pQwRwT58+BSB79uxq27Nnz668lpTo6GgiIiKUf/LYVAghhBDpQYYI4O7fv09YWBi1atVStpmYmPDrr79y6tSpVKyZEEIIIUTKSzdj4DJlykSBAgWUv/PmzUupUqV4+fIljx49Yu7cuQwYMIA7d+4oaUTCwsLUZqoKIYQQQmQE6SaAK1euHFu3blX+HjNmDACrVq3C1dWVadOmYWRkxOTJk8mSJQsnTpygTZs2/yoHnBBCCCFEepAu88BpiomJCffu3SN//vzpahaqlpYWgwYNwsHBAQsLC8LCwli1ahWTJk36ruN/++03tm3bxrVr17C2tla2d+3ala5du5I3b14AgoODCQgIYN++fZr4GEIIIcRP73tjkXTTAyeS5+7uTteuXenTpw/BwcGUK1eOGTNmEBERwbx58756bObMmZk1axaHDx9ONAkkNDSUUaNGcefOHVQqFe3atWP58uVYW1tz/fp1TX4kIYQQQnyFBHAZQKVKldi5cyd//vknAA8fPsTe3p4KFSp889hJkyaxYcMGYmNjE60vu3v3brW/x4wZQ9euXalYsaIEcEIIIUQqyhCzUH92p06dolatWhQqVAiAkiVLUrlyZfbu3fvV4zp06ED+/PmZMGHCN8vQ0tLCzs4OIyMjTp8+nSL1FkIIIcR/Iz1wGUBgYCAmJiYcP36c2NhYtLW1GTNmDOvXr0/2mIIFC+Lt7U3z5s2JjY1Ndr/ixYuza9cuDAwMePfuHV26dJHeNyGEECKVSQCXAdja2tK6dWucnZ0JDg6mdOnSjBkzhrCwMFavXp1ofy0tLebNm4e/vz+3b9/+6nvfunULa2trMmfOTIsWLZg5cyYtWrSQIE4IIYRIRTILNYH0Ogv14sWLTJ06lQULFijbBgwYgIODA1WqVEm0f+bMmbl79y4xMTHKNi0tLbS0tIiJiaF169YcOXIkybI2btzI3bt3GTBgQMp/ECGEEOInJ7NQfyKGhoZ8+vRJbVtsbCwqlSrJ/SMiIqhevbratu7du1OzZk2cnJx48OBBsmVpaWmhr6//v1daCCGEEP+ZBHAZwO7du+nfvz8hISEEBwdTpkwZXFxcWLlypbKPt7c3lpaW/P7778TFxREcHKz2Hs+ePSMqKkptu7e3N3v37iUkJARjY2Nat25N9erVcXBw+GGfTQghhBCJSQCXAQwePJghQ4YQEBCAubk5YWFhLFmyhICAAGWfHDlykCtXrn/1vubm5syaNYscOXLw5s0brl69ioODAwcPHkzhTyCEEEKIf0PGwCWQXsfACSGEECJj+N5YRPLACSGEEEKkMxLACSGEEEKkMxLACSGEEEKkMxqdxJA3b16qVq1K7ty5MTIy4vnz51y6dIlTp07x4cMHTRYthBBCCJFhaSSAa926Nb169aJcuXI8ffqUsLAwoqKiyJo1K/nz5+fDhw+sX7+eqVOnEhISookqCCGEEEJkWCkewB04cICPHz+yatUqHB0dCQ0NVXtdT0+PSpUqYWdnx759+/Dy8mLr1q0pXQ0hhBBCiAwrxdOI1KlThwMHDnzXvlmzZiVv3rxcuHAhJavwn0kaESGEEEKkplRbSut7gzeAly9f8vLly5SughBCCCFEhqbRWahlypShePHiyt9NmjRh2bJlDB8+HF1dXU0WLYQQQgiRYWk0gJs8eTKFCxcGIF++fAQFBREZGUmLFi3w9fXVZNFCCCGEEBmWRgO4QoUKcenSJQBatmzJsWPH6NWrF66urtjY2GiyaCFEGnTu3DnCw8MT/ZswYUKS+3fu3Jnt27dz+/Ztbt++zcaNG6lQoYLaPtmzZ2fGjBlcuXKFhw8fsnbtWgoWLPgjPo4QQqQajeaBU6lUaGl9jhFr167N7t27AXj06BFmZmaaLDpNM+m5NLWrkKZEBHVJ7SqIH6R+/fpoa2srfxcvXpyNGzeyZcuWJPevXr06Gzdu5OTJk3z48AE3NzfWr19P9erVefz4MQDLli3j48ePdOrUiYiICFxcXNi4cSPVqlUjMjLyh3wuIYT40TTaA3f+/HkGDBhAmzZtqFatGn/++Sfw+XHqs2fPNFm0ECINCg8P5+nTp8q/hg0bcufOHf76668k9+/duzcLFy7k8uXL3Lx5E3d3d7S0tKhVqxbwuZe/UqVKeHp6cu7cOW7duoWnpycGBga0atXqR340IYT4oTQawA0dOpQyZcrg7+/P5MmTuXv3LgAtWrTg5MmTmixaCJHG6erq4uDgwMqVK7/7GCMjI3R0dJTZ63p6egBqK7vExcURHR1NlSpVUrbCQgiRhmj0EerVq1epWbNmou0+Pj7ExsZqsmghRBrXtGlTsmTJwqpVq777GB8fH8LCwjh06BAAN2/e5OHDh3h7e9O/f38iIyNxcXEhV65c5MiRQ1NVF0KIVPfDFrPPlCkTJiYmmJiYoKenh6GhYYq+v5aWFkOGDOHs2bOEhIRw+vRpBgwYkKJlCCFSTqdOndi7dy9hYWHftb+7uzt2dnZ06dJF6XGLiYnB0dGRQoUKcefOHUJCQqhRowZ//vknnz590mT1hRAiVWl8MXt/f3+qV6+OgYGBsl2lUhEXF4eFhUWKleXu7k7Xrl3p06cPwcHBlCtXjhkzZhAREcG8efNSrBwhxP8ud+7c1K5dG0dHx+/av0+fPri7u9OqVSuuXr2q9tqFCxewtrZWbg7Dw8PZs2cP58+f10DNhRAibdBoADdnzhxUKhVubm48e/aMuLgUXbVLTaVKldi5c6cyUeLhw4fY29snSjkghEh9HTp04NmzZ+zZs+eb+/bt25f+/fvj4ODw1aAsfsmZggULUq5cOcaOHZtS1RVCiDRHowFcyZIlqVevHrdu3dJkMQCcOnWKLl26UKhQIW7fvk3JkiWpXLky3t7eyR6jp6eHvr6+8rexsbHG6ynEz06lUtGhQwfWrFmTaCzsrFmzePz4MX5+fgC4ubkxePBgevXqxYMHD5Re+3fv3vHu3Tvg86So8PBwQkJCKFGiBGPHjmXHjh0cPHjwh34ukfFYWlri4+NDvXr1MDQ05O7du/Tt2/erNxKtW7emb9++FCxYkDdv3rBv3z58fHyUiTedO3embdu2yipFFy5cYPTo0Zw9e/ZHfCSRgWg0gDt37hy5cuX6IQFcYGAgJiYmHD9+nNjYWLS1tRkzZgzr169P9ph+/foxaNAgjddNCPGP2rVrkydPHlasWJHotVy5cqmNXevatSv6+vosXrxYbT9/f38l+W/OnDkZPXo02bNn58mTJ6xZs4aJEydq9DOIjC9Llizs2LGDo0eP0rZtW54/f07BggV59epVssf89ttvzJo1i+HDh7Nr1y4sLS2ZNGkSgYGBynCB78ltKMT3UJmZmWnsuWb+/PmZNGkS69at49q1a3z8+FHt9S/Hsvwv7OzsGDlyJD4+PgQHB1O6dGnGjBmDt7c3q1evTvKYpHrgLl++TP78+ZXHMZogiXzVSSJfIURaM2LECH777TeaN2/+3cf06dOHrl27UrFiRWVbz549cXNzo3Tp0kkeo6WlxZ07dxg0aBBr1qz5n+st0j8TExPu3bv3zVhEo7NQzc3NyZ8/P9OnT2fv3r0cOnSIgwcPKv+bkkaOHMnUqVPZtGkT165dY+3atcyZM4d+/fole0x0dDQRERHKv7dv36ZonUT6ZmlpyZw5c7h58yYhISEcOXKEcuXKffUYPT09hg0bxvnz5wkNDeXcuXN06NBBeX3Lli1JLiX1b1JpCCE0r3Hjxpw/f56FCxcSHBzMgQMH6Ny581ePOXXqFLly5aJ+/frA52XebGxslLHZSfkyt6EQ30ujj1CnTZvGpUuXcHZ25unTpxqdxGBoaJgobUBsbCwqlUpjZYqM6788PgFYuHAh2bNnx93dnTt37pAjRw5lOTkAR0dHJfksQNasWTl8+DBbt27V1EcRQvwH+fLlo2vXrsyePZspU6ZQvnx5xo0bx8ePH5N9qnPy5El69erFggUL0NfXR1dXl507dzJw4MBky/kyt6EQ30ujAVzu3Lnp2LGjsgKDJu3evZv+/fsTEhJCcHAwZcqUwcXF5V9leRcinru7O48ePaJv377KtgcPHnz1mLp161KtWjUqVKigBHoPHz5U2+fLANDOzo73798nuxaoECJ1aGlpcf78eUaPHg3ApUuXKF68OE5OTskGcEWLFmXcuHEEBASwf/9+cuTIwciRI5k0aRLu7u6J9o/PbdiiRQu11USE+B4afYR65MgRSpUqpckiFIMHD2br1q0EBARw7NgxRo4cyZIlSySVgPhP/svjkyZNmnD+/Hnc3Ny4fPkyJ06cYOTIkWo5EL/UqVMnNm7cKIuuC5HGPHnyhOvXr6ttu3HjBrlz5072mH79+nHixAlmzJjB1atXOXDgAF5eXnTq1CnRyiDxuQ1bt26douPBxc9Doz1wu3fvZvTo0RQvXjzJSQy7du1KsbLevn3LsGHDGDZsWIq9p/h5/ZfHJ/ny5aNy5cpERUXRpUsXzMzMCAgIwMzMTK0nL16FChUoUaJEknfmQojUdeLECQoXLqy2rVChQol61RMyNDQkJiZGbVt8qpyEw3m+N7ehEF+j0QBu0qRJAHh5eSV6LaVXYhAiJf2XxydaWlrExcXRq1cvZeaQt7c3ixYtwsvLi6ioKLX9O3bsyJUrVyT/kxBp0Jw5c9i5cyceHh5s3ryZChUq0KVLF/r376/s4+3tjaWlJb///jvwudNiypQpdO3aVXmEOnbsWM6cOaMsGfc9uQ2F+B4aDeCyZ8+uybcXQmOSe3xiY2Pz1WMeP36sNu37xo0baGlpYWVlxZ07d5TtRkZGtGrVinHjxqV85YUQ/7Nz587RpUsXvL298fT05MGDBwwbNkwtt2iOHDnIlSuX8veqVaswNjamR48ejBo1ijdv3nDkyBFGjhyp7PM9uQ2F+B4aDeCESK/+y+OTEydO0KJFCzJlyqTcSRcqVIjY2FhCQ0PV9m3ZsiV6enqsW7cu5SsvhEgRe/bs+epyb66urom2BQUFERQUlOwx5cuXT5G6CZHikxjs7Oy+e18rKyt+++23lK6CEP+zOXPmULFiRTw8PChQoAD29vZ06dKFBQsWKPt4e3sza9Ys5e8NGzbw8uVLpk+fTtGiRalatSq+vr6sWLEiycenO3bskNxPQggh/pMUD+C6du3KsWPH6Nu3L7/88kui101MTKhfvz5z587lwIEDmJmZpXQVhPifxT8+adWqFUePHsXT0/Obj0/evXuHvb09WbJkYe/evcydO5fdu3czZMgQtfcuXLgwVatWTXIpKSGEEOJ7aGQprcaNG9OzZ09q1qxJZGQkT58+5cOHD5iammJhYUF4eDirV69m9uzZPHv2LKWL/8++d/mK/7kcWUpLjSylJYQQQnz2vbGIRsbA7dq1i127dmFmZkaVKlXInTs3hoaGhIeHc+nSJS5evKjRVRmEEClDbjYSkxsOIURaoNFJDC9evGDHjh2aLEIIIYQQ4qej0ZUYhBBCCCFEypMATgghhBAinZEATgghhBAinZEATgghhBAinfkhAZyuri6FCxdGW1v7RxQnhBBCCJGhaTSAMzQ0ZOrUqYSEhPDXX3+RO3duAMaPH4+7u7smixZCCCGEyLA0mkbE29ubUqVK0aJFC9auXatsP3ToEAMHDmTq1KmaLF4IIcRPQnIWJiY5CzM2jQZwTZs2pUePHpw+fVpte3BwMAUKFNBk0UIIIYQQGZZGH6Fmy5YtyaWyjIyMZCUGIYQQQoj/SKM9cOfPn6dhw4YEBQUBKEFb586dOXXqlCaLFj8ZeXySmDw+EUKIjEujAdzo0aNZu3YtRYsWRVtbm169elG0aFEqVapEixYtNFm0EEIIIUSGpdFHqCdOnKB27dpoa2tz7do16tSpw/Pnz2ncuDEXLlzQZNFCCCGEEBmWRnvgAO7du4eHh4emixFCCCGE+GloPIADMDc3x9zcHC0t9Q6/q1ev/ojihRBCCCEyFI0GcGXLlmXmzJn88ssvqFQqtdfi4uKwsLDQZPFCCCGEEBmSRgO4adOmcfv2bdzd3Xn69KmkDhFCCCGESAEaDeDy58+Pk5MTd+/e1WQxCktLS3x8fKhXrx6GhobcvXuXvn37cv78+R9SvhBCCCHEj6DRAO7w4cOUKlXqhwRwWbJkYceOHRw9epS2bdvy/PlzChYsyKtXrzRethBCCCHEj6TRAM7d3Z2ZM2dSrFgxgoOD+fjxo9rru3btStGyHj16RN++fZVtDx48SLH3F0IIIYRIKzQawFWqVInKlStTv379RK+l9CSGxo0bs3//fhYuXEi1atV4/PgxCxcuZNmyZSlWhhBCCCFEWqDRRL7jx49n3bp1lChRguzZs6v9S+kZqPny5aNr167cuXMHBwcHFi1axLhx42jXrl2yx+jp6WFiYqL8MzY2TtE6CSGEEEJogkZ74MzMzJg9e3aSC9qnNC0tLc6fP8/o0aMBuHTpEsWLF8fJyYnVq1cneUy/fv0YNGiQxusmhBBCCJGSNNoDt337dmrUqKHJIhRPnjzh+vXrattu3LhB7ty5kz0mMDCQ/PnzK/9KlSql6WoKIYQQ4v+5u7sTHh7OmDFjvrpf5syZmTBhAleuXCE0NJQTJ06oDc8yNjZmzJgxnD9/npCQEHbu3En58uU1Xf1UpdEeuNu3b+Pt7U2VKlW4evUqMTExaq/Pmzcvxco6ceIEhQsXVttWqFAhHj58mOwx0dHRREdHp1gdhBBCCPF9ypcvj6OjI5cvX/7qfrq6umzcuJFnz57RtWtXHj9+TJ48eXj9+rWyT2BgIMWLF8fFxYWwsDAcHBzYuHGjMiY+I9JoANepUyfevXtHtWrVqFatmtprcXFxKRrAzZkzh507d+Lh4cHmzZupUKECXbp0oX///ilWhhBCCCH+d5kyZWLOnDl4eHh883e6Y8eOmJqa0rhxY6UjKGHnjIGBATY2NnTq1Iljx44BMGHCBBo1akTXrl0ZO3as5j5IKtJoAFehQgVNvr2ac+fO0aVLF7y9vfH09OTBgwcMGzaM9evX/7A6CCGEEOLbJkyYwJ9//smhQ4e+GcA1btyY06dPM2HCBJo0aUJ4eDgbNmxg6tSpfPr0CR0dHXR0dPjw4YPacVFRUVSuXFmTHyNV/ZDF7H+UPXv2sGfPntSuhhBCCCGSYWdnR5kyZZJMMZaU/PnzkydPHtavX0+7du0oUKAAAQEB6OjoEBAQwNu3bzl58iQDBgzgxo0bPH36FHt7eypVqvTDVoJKDSkewPn5+TFu3DgiIyPx8/P76r7e3t4pXbwQQggh0igrKyvGjh2Lvb19oh6z5KhUKp4/f46HhwefPn3iwoULWFpa4urqSkBAAAAuLi5MmzaNK1euEBMTw8WLF9m4cSNly5bV5MdJVSkewJUuXRodHR3lv4UQQgghAMqVK4eFhQUHDhxQtuno6FCtWjV69OiBpaUlnz59UjvmyZMnxMTEqG2/ceMGOXPmRFdXl48fP3Lv3j1atGiBkZERJiYmPHnyhPnz53Pv3r0f9dF+uBQP4GxtbZP8byGEEEL83A4fPkz16tXVts2YMYObN28qY9q+dPLkSezt7VGpVMTFxQGfs0yEhYUlWqIzMjKSyMhIsmTJQt26dfH19dXYZ0ltGs0DN23atCRXNzAyMmLatGmaLFoIIYQQaczbt28JDg5W+/fu3TtevHhBcHAwALNmzVIbYrVw4UKyZs3KuHHjKFSoEA0aNMDDw4MFCxYo+9SpU4e6deuSN29erK2t2bJlCzdv3mTlypU//DP+KBoN4Nq1a4eBgUGi7QYGBrRt21aTRQshhBAiHcqVKxc5cuRQ/g4NDaV169aUL1+ew4cPM27cOObNm0dgYKCyT3yi3+PHjzNz5kxOnDhB69atE+WfzUg0MgvVxMQE+Dzw0NjYWG2gopaWFg0aNOD58+eaKFoIIYQQ6UjLli2/+jfA6dOnadSoUbLvsWXLFrZs2ZLidUvLNBLA3blzh7i4OOLi4jh58mSi1+Pi4vD399dE0UIIIYQQGZ5GAriWLVuiUqnYvHkzTk5OvHz5UnktOjqakJAQwsLCNFG0EEIIIUSGp5EA7u+//wY+r3MWEhKiiSKEEEIIIX5aGp3EIMGbEEIIIUTK02gAJ4QQQgghUp4EcEIIIYQQ6YwEcEIIIYQQ6YwEcEIIIYQQ6YxGZqHGy549O6NGjaJWrVqYm5ujUqnUXrewsNBk8UIIIYQQGZJGA7gZM2aQO3duJk6cyJMnT5RFaIUQQgghxH+n0QCuSpUqNGvWjMuXL2uyGCGEEEKIn4pGA7hHjx4lemwqhBBCiPTBpOfS1K5CmhMR1CW1qwBoeBLD0KFDGTFiBHny5NFkMUIIIYQQPxWN9sAtWLAAQ0NDzpw5w/v37/n48aPa64ULF9Zk8UIIIYQQGZJGA7hhw4Zp8u2FEEIIIX5KGg3gVq9ercm3F0IIIYT4KWk0gAPQ0tKiWbNm/PLLLwAEBwezc+dOPn36pOmihRBCCCEyJI0GcAUKFGD16tVYWlpy69YtANzd3QkNDaVdu3bcu3dPk8ULIYQQQmRIGp2FOm7cOO7du0eZMmWoW7cudevWpWzZsty/f59x48ZpsmghhBBCiAxLowFctWrV8PX15dWrV8q2ly9fMmrUKKpVq6bJonF3dyc8PJwxY8ZotBwhhBBCiB9NowFcdHQ0xsbGibZnypQpUUqRlFS+fHkcHR1lBQghhBBCZEgaDeD27NnDlClT+PXXX5VtFStWZNKkSezatUsjZWbKlIk5c+bg4eGh1vMnhBBCCJFRaDSAGzx4MPfu3WPXrl2EhoYSGhrKjh07uHv3LkOGDNFImRMmTODPP//k0KFD39xXT08PExMT5V9SvYVCCCGEEGmNRmehvnnzhk6dOlGwYEGKFCkCwI0bN7h7965GyrOzs6NMmTLUr1//u/bv168fgwYN0khdhBBCCCE0ReN54ADu3LnDnTt3NFqGlZUVY8eOxd7eng8fPnzXMYGBgcyePVv529jYWMbNCSGEECLNS/EAzs/Pj3HjxhEZGYmfn99X9/X29k6xcsuVK4eFhQUHDhxQtuno6FCtWjV69OiBpaVlouTB0dHRREdHp1gdhBBCCCF+hBQP4EqXLo2Ojo7y3z/K4cOHqV69utq2GTNmcPPmTaZOnSorPwghhBAiw0jxAM7W1jbJ/9a0t2/fEhwcrLbt3bt3vHjxItF2IYQQQoj0TKOzUKdNm5bkzE4jIyOmTZumyaKFEEIIITIsjQZw7dq1w8DAINF2AwMD2rZtq8miAWjZsiXDhg3TeDlCCJGa+vXrx969e7l//z7BwcEsW7aMwoULf/fxdnZ2hIeHs2zZsmT3mThxIuHh4fTq1SslqiyE+B9pJICLz6umUqkwNjZWy7WWJUsWGjRowPPnzzVRtBBC/HSqVavGggULaNiwIfb29ujo6LB+/XqMjIy+eWyePHkYNWoUf//9d7L7NGvWjIoVK/L48eOUrLYQ4n+gkTQid+7cIS4ujri4OE6ePJno9bi4OPz9/TVRtBBC/HTatGmj9rerqys3btygbNmyHDt2LNnjtLS0mDt3LuPHj6dq1apkyZIl0T6WlpaMHz+e1q1bs3r16hSvuxDiv9FIANeyZUtUKhWbN2/GycmJly9fKq9FR0cTEhJCWFiYJooWQoifXubMmQHUrr1J8fLy4vnz56xYsYKqVasmel2lUjF79mymT5/O9evXNVJXIcR/o5EALr4rvnz58oSEhGiiCCGEEElQqVSMGTOG48ePf3UGfuXKlenUqRO1a9dOdh93d3diYmKYN2+eJqoqhPgfaHQlhjx58pAnT55kX/9a174QQoh/LyAggOLFi9OsWbNk9zE2Nmb27Nn069ePFy9eJLlP2bJlcXZ2pm7dupqqqhDif6DRAG7r1q2JtsXFxSn/bWFhocnihRDip+Lv70/Dhg1p3rw5oaGhye6XP39+8uXLx8qVK5VtWlqf57Q9efKEypUrU6VKFbJnz86FCxeUfXR0dPDz86N3796UL19ecx9ECPFNGg3gChYsqPa3rq4uZcqUYciQIYwZM0aTRQshxE/F39+fZs2a0aJFCx48ePDVfW/evJlo5Zphw4ZhbGzMkCFDePToEWvXruXQoUNq+6xfv561a9eqBX5CiNSh0QAuIiIi0baDBw8SHR2Nn58f9erV02TxQgjxUwgICMDe3p5OnTrx9u1b5enGmzdviIqKAmDWrFk8fvwYPz8/Pnz4kGh83OvXrwGU7S9fvkw0CeLjx488efKEW7duafojCSG+QaMBXHKePXv2r5JMCiGESF63bt0A2LZtm9p2V1dXVq1aBUCuXLlkTWghMhCNBnAlSpRQ+1ulUpEjRw7c3d25fPmyJosWQoifRrZs2b65T8uWLb/6uqur6zffQ8a9CZF2aDSAO3ToEHFxcahUKrXtp0+fxs3NTZNFCyGEEEJkWBoN4L68W/v06RPh4eF8+PBBk8UKIYQQQmRoGg3gJImvEEIIIUTK08hi9vHGjRuHs7Nzou09evSQNCJCCCGEEP+RRgM4GxsbTpw4kWj7yZMnadGihSaLFkIIIYTIsDQawGXNmpU3b94k2h4REYGZmZkmixZCCCGEyLA0GsDdvXs3yWS99evX5/79+5osWgghhBAiw9LoJIZZs2bh7+9PtmzZOHLkCAC1atXi999/Z9iwYZosWgghhBAiw9JoALdy5Ur09fXp378/np6eADx48AAvLy/WrFmjyaKFEEIIITIsjS+ltWjRIhYtWkS2bNmIiori3bt3mi5SCCGEECJD0+gYOABtbW1q165N8+bNlRUZcubMSaZMmTRdtBBCCCFEhqTRHrjcuXOzbt06cuXKhb6+PgcPHuTt27e4ubmhp6enPFYVQoifiUnPpaldhTQlIqhLaldBiHRH44l8z58/T6FChYiKilK2//HHH9SqVUuTRQshhBBCZFga7YGrUqUKTZo04ePHj2rbHzx4gKWlpSaLFkIIIYTIsDTaA6elpYW2tnai7VZWVrx9+1aTRQshhBBCZFgaDeAOHDhAr169lL/j4uLIlCkTgwcPZu/evSlaVr9+/di7dy/3798nODiYZcuWUbhw4RQtQwghhBAiLdBoADdixAgqV67M33//jb6+PvPmzePcuXNYWloycuTIFC2rWrVqLFiwgIYNG2Jvb4+Ojg7r16/HyMgoRcsRQgghhEhtGh0DFxoaSq1atbCzs6NkyZIYGxuzfPly1q9frzapISW0adNG7W9XV1du3LhB2bJlOXbsWIqWJYQQQgiRmjQawGXLlo3w8HDWr1/P+vXr1V4rXrw4165d01jZmTNnBuDly5fJ7qOnp4e+vr7yt7GxscbqI4QQQgiRUjT6CPXIkSM0aNAg0fY+ffrw559/aqxclUrFmDFjOH78OMHBwcnu169fP+7du6f8u3z5ssbqJIQQQgiRUjQawM2ePZvFixczceJEDAwMsLS0ZNOmTfTt21dtckNKCwgIoHjx4vTs2fOr+wUGBpI/f37lX6lSpTRWJyGEEEKIlKLRR6jTp0/n4MGDzJ49m8OHD5M1a1bOnDlDrVq1ePr0qUbK9Pf3p2HDhjRv3pzQ0NCv7hsdHU10dLRG6iGEEEIIoSkaXwv17t27XLt2jbx582JiYsLmzZs1Grw1a9YMW1tbHjx4oJEyhBBCCCFSm0YDuN9++43Dhw9TsGBBatWqhaenJ+PHj2f+/PlkyZIlRcsKCAjAwcEBZ2dn3r59i4WFBRYWFhgYGKRoOUIIIYQQqU2jAdzmzZvZvHkzjRo14saNGyxfvhxra2ty587N0aNHU7Ssbt26kSVLFrZt28a1a9eUf3Z2dilajhBCCCFEatPoGLjWrVvz999/q227d+8eTZo0oX///ilaVrZs2VL0/YQQQggh0iqN9sB9GbzFi4uLY9KkSZosWgghhBAiw9JIALd69WpMTEyUv93d3ZXEugBZs2ZNNrgTQgghhBBfp5EArm7dumorHHh4eJA1a1blbx0dHVloXgghhBDiP9JIAKdSqb76txBCCCGE+O80ngdOCCGEEEKkLI0EcHFxccTFxSXaJoQQQggh/ncaSSOiUqmYMWOGskyVvr4+kyZNIjIyEgA9PT1NFCuEEEII8VPQSAC3evVqtb/XrVuXaJ81a9ZoomghhBBCiAxPIwFc3759NfG2QgghhBACmcQghBBCCJHuSAAnhBBCCJHOSAAnhBBCCJHOSAAnhBBCCJHOSAAnhBBCCJHOSAAnhBBCCJHOSAAnhBBCCJHOSAAnhBBCCJHOSAAnhBBCCJHOSAAnhBBCCJHOSAAnhBBCCJHOSAAnhBBCCJHOSAAnhBBCCJHOSAAnhBBCCJHOSAAnhBBCCJHOZLgArnv37pw7d45Hjx6xZ88eKlSokNpVEkIIIYRIURkqgLO1tcXPz4+AgADq1q3L5cuXWbduHebm5qldNSGEEEKIFJOhArjff/+dZcuWsXLlSq5fv86AAQN4//49HTt2TO2qCSGEEEKkmAwTwOnq6lK2bFkOHTqkbIuLi+PQoUNUqlQpFWsmhBBCCJGydFK7AiklW7Zs6Ojo8PTpU7XtT58+pUiRIkkeo6enh76+vvK3sbGx2v9qirF+hmn2lGFi8j+/hbRpEqRdNUPaNeVJm2qGtKtmpEC7fs33xiA/9f8z/fr1Y9CgQYm2X758ORVq8xMbeS+1a5AxSbtqhrRrypM21QxpV834Qe1qbGxMREREsq9nmAAuPDycmJgYLCws1LZbWFgk6pWLFxgYyOzZs9W2Zc2alZcvX2qsnmmFsbExly9fplSpUrx9+za1q5NhSLumPGlTzZB21Qxp15T3M7apsbExjx8//uo+GSaA+/jxIxcuXKBWrVrs2LEDAJVKRa1atZg/f36Sx0RHRxMdHa227WvRbkb09u3bn+4z/wjSrilP2lQzpF01Q9o15f1Mbfo9nzPDBHAAs2bNYubMmZw/f56zZ8/Sq1cvjIyMWLlyZWpXTQghhBAixWSoAG7z5s2Ym5szePBgLCwsuHz5Mm3atOHZs2epXTUhhBBCiBSToQI4gPnz5yf7yFT848OHD/j7+/Phw4fUrkqGIu2a8qRNNUPaVTOkXVOetGnSVGZmZnGpXQkhhBBCCPH9MkwiXyGEEEKIn4UEcEIIIYQQ6YwEcEIIIYQQ6YwEcEIIIYQQ6YwEcBmISqVK7SoIIYQQ4geQAC6DUKlUxMV9nlBcr149ihYtira2dirXSoikyc1GypM21YyE7Wqi4UXMfxZyrqaMDJcH7mcVH7wNHz4cBwcHRo0aRWho6E+z7Iim2NjYULBgQbS1tdm2bRs3b95M7SqlewlvNrp06cLz58/Zv38/UVFRqVyz9Cthm1atWhUjIyOuXbvG48ePle3i30vYrh4eHhQsWJAJEybw8OHDVK5Z+pWwTVu2bImVlRX6+vocOHCACxcupHLt0hcJ4DKQAQMG0KFDB5ycnLh06RLv379P7SqlayNGjMDBwYHz589To0YNKlWqRKdOnYiNjU3tqqVr8RdvHx8f2rRpw9SpUzEwMJAA7n8Q36YjR47E3t4eExMTrl+/zvr161m4cCExMTGpXMP06ctzddy4cXz8+DGVa5W+fdmmhw4dolChQtjZ2bFq1SrmzJmTyjVMPySAyyCyZMlC7dq18ff35+TJk+TMmZMyZcrg4ODAjRs32Lx5M0+fPk3taqYbAwYMoG3btrRv356LFy9SrFgx9uzZQ86cOXn06FFqVy/dc3Z2pn379tjb23PlyhVA/c5c/HvVqlWjWrVqdO3alZcvX+Lq6oqdnR3GxsZMmzZNgrj/qHHjxrRp04YOHTooPUTGxsaYm5vz8uVLXr9+nco1TH9atGhBq1atlDa1t7dnxowZhISEpHbV0hUJ4NKpL3/stLW1MTMzw8zMDBsbG2xsbLCyssLIyIjy5cuTPXt2xo4dKz+Q36FEiRJUqlQJLy8vLl68CMDr16+5ceMGvXr1QktLi3PnzrFhw4ZUrmn68eX5WqpUKZYsWcKVK1fIly8f5cuXx9nZmRs3brBnzx527NiRirVNf5o1a0aDBg04evQop06dAmDYsGEMHz6chg0bEhcXx/Tp0yWI+w5fnqtmZmbcvHmTCxcuULJkSRo1akS7du3Q1dVl3759+Pv7y3rb/1KePHk4d+4cFy5coGXLlkycOJEhQ4awfft2DA0NyZs3L9evX0/taqZ5EsClQwkvMHXr1lXGumzatAknJydMTU2ZP38+S5cu5ejRowQFBWFqairB23d69OgRS5cu5fjx48Dn9t6wYQNxcXFoaWlRrFgxqlevDiBB3HfQ19dX1jCsU6cOBw4cwNzcnDJlyvDw4UNat25NdHQ0t2/f5pdffsHU1JR9+/bJuoffycjIiK5du1K5cmWOHDmibH/37h2jR49m2LBh1K9fHxMTE0aPHs2nT59SsbZpW/78+bl37x4Av//+O8eOHSMkJITq1aszZ84cqlevztGjR5k+fTqGhoa4u7sTFBQkAdxXJNWznilTJkJCQqhYsSLTpk3D19eXxYsXA5/HxVlYWPDw4UMiIyNTocbph8xCTYcSTliYMmUKtWvXRk9Pj0mTJtGmTRvq1KmDn58fR48eBSBr1qy8ffs2Naucrrx+/Zo///yTFy9eANCuXTuePHmCjY0Nw4cPp3Xr1sTExFC7du1Urmna16RJE+XCPHr0aCZOnIienh5ubm68e/cOV1dXDh06xLhx4+jbty/Lly/HzMwMLS25NH2vyMhIevfuzR9//EGhQoVwdHRUXnv37h1jxozh/v37ZMmSRYK3ryhZsiSnTp2iZcuWjBkzhv79+xMREcHhw4fp0aMHHz9+xM/Pj5EjR7Js2TLWr1/PkydPMDY2Tu2qp1kJg7dq1aphZmYGwJEjR+jVqxc7d+6kb9++LFq0CABDQ0NatWqFpaWlBG/fQRazT6e8vLzo1q0bXbp0ITg4ONFs0yxZslCoUCE8PT3JkycP1tbWMvj+K8qXL0/WrFl59OiRMntXS0uLT58+oaurS1xcHDExMcq2mTNn8ubNG4YMGZLaVU/TSpcuzbZt2wgJCSFXrlw0bdqUa9euAZ8v7qamprx8+RL4PAxg5cqVvHjxAhcXl9SsdpqV8AcxZ86cvHv3Dl1dXV68eIGFhQX+/v6Ym5uzevVqVqxYoRxnYGDAhw8fpBc+CZaWljx+/BgANzc3vLy8iI2NpWnTply9elVp8/jvvpaWFvr6+ixevBgjIyNatGgh7foNw4YNo0GDBixZsoSVK1fy4cMH+vTpw9ChQxk+fDgHDx4kc+bMDBs2jOzZs1O/fn35vfoO8gg1HcqaNSu1a9fGx8eHU6dOYWFhwS+//ELr1q05d+4cBw8eJH/+/IwZM4YXL15Qp04dYmNjlQuQUOfj44OtrS1GRka8evWKsLAw+vfvz+3bt1GpVGqzzj59+oSVlRXFihVj2bJlqVjr9OHSpUvs378fGxsbjhw5ojauJS4ujpcvX2JsbEzDhg1p3bo1VlZWdOzYMRVrnLbFBwpeXl40bNiQLFmy8ObNGwICAti9ezeDBg3C39+fdu3aERcXx8qVKwGUGb4yUUTdtGnTKFeuHN27d+fmzZuEhYVhYGDAp0+fKFasGFevXlXa69OnT+jp6dGtWzeaNWuGkZERjRo1Ii4uTtr1K4YMGYKjoyNdunTh6tWrytCIJUuWoK+vj6+vL2/fvuXZs2c8f/6cBg0ayO/Vd5IeuHQoR44cHDx4kMmTJ3P79m0cHBwoWLAgxsbGaGlpsXDhQoKCgqhYsSJnzpwhLi4ObW1tuaNJgr29PePHj6dz587cvXuX3377jY4dO/Lbb7/RqlUrzp8/r1xI4meeLVu2jFu3btG1a9fUrn66YGdnh7a2Nn5+fpw+fZo+ffrw5s0b5fXcuXPTt29fcuTIQffu3YmNjZXz9Ss8PT3p1asXXl5emJqaUrp0aTp37kz//v1Zvnw5OXPmZMyYMZQoUQIfHx/27NmT2lVOs3Lnzs3u3bu5fv067u7uPHz4EAsLC9q3b8/QoUPx8PBQgmAAPT09qlSpQp06dRg9erScq1+oU6cOZ8+eVWbmFipUiKCgIHx8fDhy5AjZsmXDysqKpk2bcujQIY4fP06BAgXIli0bERER3LhxQ36v/gUJ4NK45O7sBg0ahLOzMzo6OixcuJCDBw9y6NAhVqxYwaNHjxg4cOA330N8fmRSsWJFunTpomzLkycPvr6+1K1bl4YNG3Lz5k309fVxc3OjcePG3L17lx49egDStl9KeOE1NTXl7du3yszH8uXLs3r1ak6ePMnvv/+uPPa3s7Nj7969yt9y5/2PL/PjZc6cmVWrVrFq1SqWL18OfD4HPTw8GDJkCM2aNVPSCHXr1o3x48dLWyZDR0eHmJgYLC0t2b9/P7du3aJfv37cvn0b+HyN7d+/P25ubqxZswaAMWPGsH79es6dOwfIuZpQly5d8PPzw8fHhw0bNhAREYGVlRV79uxh7NixXLx4kZ49e/Lrr78CULRoUdq0acOBAwfU3keuqd9PArg0LOGJXKpUKYyNjXn+/Dm3bt0CoFy5ckRGRnLjxg3lmA0bNnDy5En8/f1Tpc7pjaenJ05OTpQtW1btji9XrlxMmDCB7Nmz07ZtW16+fEnevHmpXLky69atA+RCk1C9evU4d+6cMvFjwIABVK9eHVNTUwIDAzl+/DhPnz6lXLlyrF69mvPnzzNr1iz69OlD1qxZlUdR4h8bNmzg7NmzjBkzRtmWM2dO/vrrL4YMGcLatWuV7QYGBixbtozr16/j6+urli5EgozEvvzuWllZsW/fPoKDgxk4cKCy4srAgQPx8vJi8eLFlCpVClNTU2rUqCG9Q8kICAigVq1azJkzh40bNyozoZs0aYK5uTlLlizh8OHD7Nq1iy1btnD69Gn8/PxSu9rplkz1SsMSzjadO3cuK1asYOLEiYwePRqVSsX58+e5ceMGxsbGlCtXjhUrVmBhYcHEiRNTuebpx4EDB3j27Bm9e/fGwMBA2f7o0SMWLlyIsbExBQsWBODBgwcSvCWhU6dOLFq0CDs7O3R1denSpQu9e/dm3759hIaGMmrUKLp164aVlRXnz5+nZcuWFCtWjNGjR5M5c2aaNm0qbZmEkSNHEhAQAHzuLQIICwvj0KFD2NraYmFhoewbFRXFu3fvMDU1TZTrTYK3xOLPt8qVK5M7d25CQ0OpV68exYoVIyAggCJFigAwYcIEBg4cSO7cubl16xY1a9ZUxmeJf8S3h5eXF0eOHKFPnz60atUKlUrFmDFjcHJyonnz5gwdOpRdu3aho6ODrq6uMnlE/DdyFqZxHh4edOjQgUGDBlG2bFnu3r1L586dmTp1qrIgcOXKlfHz80NPT09twoJI7MtFlOOTSbZq1YrmzZujr6+vvHbixAmyZs2qXMwTkoDjH8uXL2fFihW4uLjQtm1bihcvjouLCzNnzqRTp04sWbKEli1b4uTkRK5cubh+/TpVq1alZ8+eNG3alJiYGLS1tVP7Y6QpKpWKixcvEh0djaurK4sXL1bSVezduxczMzNcXFwwNTUFPo/Nypo1K0+ePEnFWqcvVatWZdGiRXTo0AErKysliCtatCgBAQH88ssvACxatIiePXvSt29f5VyVoFhd/Oxc+PxU4+DBg7i6utKpUydUKhXnzp3j3LlzGBoaUqxYMZYuXYqhoaGSPkT8N/Irn4bUqlVL7e+iRYvSqFEj+vTpw9GjR6lYsSKtWrVi165dVK5cmUmTJqFSqdi3bx+jR4+mbdu2coH5Ch0dHSXwMjY2xszMjE+fPuHl5cWTJ0/o06cPjo6OyoXI1NSU169fS5LOr4gPvIYMGcLBgwfx8PDAxsZGbZ8pU6awdu1abGxs6Ny5M/nz5+f9+/dcv35dmcEnj6TUxZ+nWlpanD9/ntq1azNhwgS0tLRYuXIlu3fvpkaNGuzevZs5c+awfft2zM3NGTt2bCrXPP04duwYK1euxMbGhvbt26sFcb/88gtjx46lRIkSAGp5NOVc/UfCG+KEvznxQVyfPn2wt7cnc+bMADRv3hxvb28yZcqkNttU/DcyBi6NaNmyJfPnz8fd3V1t1lOHDh3Ys2cPhQsXZsGCBYwbN47ly5ezZMkSGjVqxN69e+ncubNywZdHe+qMjIyoXbs2O3fuVLZNmzaN4sWLo6ury6JFi1iyZAm6urpMnjyZUqVKoa+vz4kTJ6hRowY3btyQtBbJSOpc8/HxUXrfpk+fzqtXr5TX3N3d6du3r5IIVSRWrVo1VCoVf/31F35+fjx8+JB58+ZRpUoVVq1axZ49e+jduzdxcXHUrFmT6tWrkzt3bh49esSECRMk/cJ30NXVVUsNNGTIEGxsbNiwYQMrV67k8ePHWFlZceHCBebPny+5HpOR8Ptfq1YtLCwsCAsLIzg4mOfPnwMwadIkatWqxaxZs1ixYgVmZmaULFmSAwcO8OnTJ5lt+j+SAC4NGTBgAJ6ennh6eqol4QQYN24curq6DBkyhI8fPzJ06FAqVapEcHAwQ4cOlaAtGV26dGHSpElKYBwQEECVKlVYtmwZhQoVolu3bkyePJlx48ahpaVFrVq1qFevHtra2jx9+pTAwEBAAuOviX8MGp+uYty4cTRq1IiZM2eybt06tZQhbdq0Yf369RJgJCFHjhzMmjULgPDwcGxsbKhXrx5Xr14FUAviXF1d1YKQePKD+HVOTk7o6uqyatUqtV61oUOH0rFjRxYuXKgEcebm5rx48ULO1W8YMWIEbdu2JSwsjFy5crFr1y5Wr16tLEU4ceJEatasydKlS5k/f76SB06uqf87SeSbhkyaNAktLS0mT54MoBbE5cmTB319fT5+/IhKpaJw4cJs2rSJpUuXAvJlSM6GDRuwsLAgMDCQuLg4QkNDcXFx4fLlywBcuHBBae9x48Zx8OBBDh48qPYe0rbqErZHmTJl8PHx4ebNm0RERHDs2DGGDBmCjo6OsppCwiAufuak9BIl9uTJE/z9/ZkzZw41atTAw8NDCd5UKhXHjx+nffv2rFy5ksDAQAYPHpxoBRYJ3r6uVq1alC1blsjISLZs2aIEcWPHjqV48eJ06tQJExMTZs6cqQydkHNVXb58+bh//z4Affr0wcHBAScnJ06dOoWnpyf9+vXD1NQUbW1t/vrrLzw9PZk/fz6//vorM2fOVN5Hrqn/OwngUtmXwUFAQAAqlSpRELd//346d+7Mli1b0NXVJUuWLEouMpAvQ3IiIyOZOHEi2traTJ06lXfv3qkFaPGPqydOnEhsbCwTJkxI9B7Sturi22Pw4MGYm5sDn9OI6Onpoaury+HDh/Hy8mLChAk4OztjZGTEwoULeffunfIe8oOYtLdv3/Lo0SMeP35MixYtePjwIUeOHFGWcjp+/DgdOnRg27Zt3L17V2ac/0vdunVj6tSpuLm5oaWlxebNm5Ug+N69exQqVIhs2bKpjXuVc/Ufjo6OdOjQAUdHR2JiYihdujTjxo3j1KlTNG3aFBcXF1auXEm9evVwdXUlLi6Ov//+mx49eiSaQCb+d/IINRUlDN4cHBzQ0dFhzZo1fPr0CU9PT7y8vBgwYADLly8na9as2NraUqFCBSIjIxk6dKiMd/mKVq1aUaRIEbJmzcqIESPQ09OjS5cu+Pj44O3tzbx589T279ChA1OnTsXJyYk//vgjlWqdfjg7OzNkyBDatWvH48ePKVGiBMOGDePu3bvMmzePo0ePAjBnzhx0dXXp3r17Ktc4bfryBi7+71q1auHi4oKuri6BgYFKe8YrUaIE169flx63ZCRs1yxZsqClpcXr16+Va+WMGTOoVKkSs2bNYteuXTx58oQ5c+awfPnyRG0tPosfjuLo6MiOHTvQ1tamYsWK3Lhxgzx58rBkyRJmz57NvHnz+P333/Hy8uLcuXP4+fkpiY/laUbKkgAuDfD19cXOzo6ZM2eyfft2QkNDAZQgztPTM8lB3zLeJWne3t40bNiQbdu2cf78eWVsVqZMmejduzeDBw9ONFkEPqcVOHbsWGpUOU3r0KFDoraaN28eMTEx/P7778o2a2trpk+fzrVr15g2bZryQyg3Gd9ma2tL5syZefv2LRs3bgSgQYMGdOvWDZVKxaxZszh8+DDLly9n165dyioMcg1ILGGbDBgwgBo1alCiRAlWrVrFoUOHlMz/kydPplKlSujo6PD+/XsMDQ2pXr06nz59kkDjC506dWLixIl07dpVbUJYpkyZePfuHf369aNq1ao4OjoSFRWFs7MzTZs25dq1azJGW4PkEWoqa9++PW3atKFz586cOXNG7bX4xyP+/v4YGhom6jWSC3diHh4edOrUiXbt2nHhwgW1wOHdu3fMmjULlUrF1KlTAdQCk/jgTS7e/4hf+mbVqlVqM52joqIwMTFR/o6Li+PgwYPMnj2bwYMH8/r1a6Kjozl58qT8IH7DqFGjaN++Pc+fP8fIyAg7Ozs6d+7Mn3/+CXzu+Zg2bRqvXr0ic+bMODk5KcfKNeAfZcuW5cKFC0qbDB06FEdHR4YPH86nT59wdnamYsWKZMqUie3bt9O/f3/s7e3JkyePMgs9Pp+Z3HD8w9ramilTptCnTx+14G3hwoVs376djRs3YmRkhLGxMXny5OHmzZvUrFmTtWvXKtdX+f5rhgRwqaxChQrs2bNHLXhLeLJPnDiRLFmyYGNjkyiAE+ry5ctH8+bN8fHxUbrsv/T+/Xtlpl9gYCBGRkbMnz9fbR+50Pxj9erVLFiwgLi4OKpUqcLx48eJi4vj5MmTTJ48mdq1a3Po0CFl/4iICI4fP06hQoWwtbXl5MmTgLRpcszMzChatCg2Nja8ePGCcuXKMXnyZDZs2IC9vT1//vkn4eHh/PLLL+TKlYvAwEBZQD0JW7du5fLly1y6dIlPnz5Rp04dWrRoQceOHTl9+jRVqlShTJkyXL16lT59+hAdHc2ePXvYsGGD2vtI8JbY48ePefHiBc2aNWP79u1ERkYyf/58ypYty4gRIwA4ffo0dnZ2zJ8/HwMDA2JjY5X1Y0G+/5oiAVwqMzMzS5QOIC4uDl1dXapXr87hw4fx9vZOpdqlL5aWluTPnz9RT+aXPnz4wMSJE8mcOTPly5f/QbVLn+IHeMcnkt28ebOSi7BixYosXboUFxcXzp8/z5s3b2jUqBHr1q1DW1ub6dOnM2fOHB48eJDKnyJtcnZ2xsHBgQcPHvDw4UPevXvHvn37cHV1ZebMmaxfv57WrVtz9uxZzp49qxynpaUlwVsCzs7OFChQAHt7ez59+oSuri4hISGsXbuW06dPU79+fWbPns2AAQMIDg5m1apV9O/fH2NjY+VxdTwJ3hK7fv06LVq0YOPGjcybN4+4uDjy5ctHixYtePToEQB79uwhNjaWwoULY2BgwIwZM2SM9g8gAVwqu3v3Lh07diRXrlzKlwEga9asdOjQgY8fP/LXX3+lYg3TDyMjo29m9S5dujSdOnVi2LBhjB49mujo6B9Uu/TtypUr7N27l1q1aimzdfv160dUVBRz5sxRZu19/PiRTZs2Ua5cOe7cuaPkfBLqdHR0iIqKwszMDGNjY2WGbmxsLEeOHOH3339n2rRp7Nu3j3r16qkdKz+I6kxMTLh//z4fP37Ez8+PO3fusGzZMoKCgjAwMKBXr17MmTNHeZwXHBxMzpw5+fXXXxMFcCIxlUrF9evXsbe3Z8mSJRQsWJA6deoov1fxQdq+ffvYt2+fcpwEb5ona1ikMn9/f0JCQlizZg0lSpQgR44c5MyZk+nTp5M7d24ZVP8vvHz5EmNjY6pVq5bsPr/99hsxMTHExMRI8JaML6f76+np8fz5cwICAjhx4gQNGzbE09MT+JxKpFOnTowaNYqxY8dSrVo1YmJisLW1JSIigqioqNT4CGnOl20aExPDpk2bGD9+PJaWlsyYMUN5LT6I8/Ly4vHjx5J+4Ru2b99O+fLl2bt3L7179+b48ePExMQQERGBnp4eefPmVVYEyZw5M48fP2b8+PEMHz48dSuexsXfDMc//gwODqZLly48fvyYoUOHYmZmBiR/QyHBm+bJLFQNSzieLUeOHEkuNp0nTx6mT59O8eLFiY6O5vnz58TGxtK4cWNiYmJkAOi/MHv2bJo3b067du0S9VxaWFiwYMECdu3apZZQUvwj4bnWvXt3SpUqRaFChVi9ejVbtmxBpVLh5eVF1apV2bt3b6K8eUWKFMHV1ZWmTZvSsmVLJRHtzyxhm1asWJHs2bMTFhbG7du3efPmDW3btsXHx4c///wTd3f3JI+Ta0DS4ttl5cqVNGjQgD/++IPu3bsrj5izZ8/O1KlT+fjxI8eOHaNu3boYGxvTrFkzZR1eadfE4nvPSpcuzciRI2nTpg0xMTEAFCtWjPXr13Pp0iV+//13Xr58mcq1/XlJD5yGxV8cvL298fX1VRb1Tejhw4fY2tri5uaGj48PAQEBNGzYUFmYXi4w32/+/PlcunSJ1atX06ZNG6ysrDA1NcXa2pqNGzfy8uVLCd6+Iv5c8/HxwcPDg/DwcPbt28fUqVMZPnw4b9++JTAwkGPHjlGnTh1GjRqlHGtkZESuXLnInDmzBG8JxLfpiBEjmDdvHoMGDWLWrFnMnTuX8uXLs2HDBnx9falXr56ydFvC4778b/GPuLg4JfGul5cX9evXZ9KkSZiamgLw7Nkzli1bhq6uLh06dODTp0+0aNFCgreviA/eihUrxurVq3nw4IESvMHnnjh7e3tKlCjBmjVrlNno4seTHrgfoEaNGowdOxY3NzfOnz+f6PXkLiQyhuC/qVSpEr169aJly5aEh4ejo6NDaGgoFy9exNXVFZAeja+pUqUKM2fOpEePHpw7d47SpUuzf/9+fv/9d9atWweAqakpvr6+xMXF4eHhoRyrra2Nrq6uPDr9gpOTE15eXnTr1o0TJ04wbNgwnJ2dcXJy4sCBAxgaGtK8eXNmzZrF2LFjmTJlSmpXOc1K6rsbf620trZmxYoVrF+/nlGjRhEeHg58fnQaFxenTMqRWbxJi2/H4sWLs3nzZtasWcOIESPQ0tJi5syZuLm5KZPuSpUqxeDBg+ncubNcS1OJBHAa1qZNGypUqAB8Hi8kQdn/7nuDrxo1apArVy5iY2O5efMmFy5c+FfH/6zq1KmDu7s7tra22NraMnXqVHx9fVm0aBEmJiYUKVKEs2fPYmJiovwgSpsmLb5dpk2bxpMnTxgzZgxNmzZl5syZ+Pr6smTJEgwNDdHR0SE6Oppq1apx6NAhuUYkI+F55uTkRKFChcibNy9r1qzh7NmzhIWFUbNmTVatWqUEcS9evEj2PcQ/vgze1q5di7e3NyqVit27d6OlpUWLFi2IjIxMdKy0aeqQR6gaZm9vr4wl0tPTkwvz/6BEiRIYGRl994Xi6NGjrFmzhvXr1yvBG8jjqG8xMDDA0tKSNm3aMHnyZCV4A6hevTru7u7kypVLgrfvYGBgAIChoSHnzp2jSpUqzJ49WwnetLW1cXBwoE6dOnz48IEDBw4oyWRFYgkf8Q8aNIi3b9/y7t07fH198fT0xMjIiCNHjtC2bVvs7OyYPHlyokd8cq4mplKplOBt48aNasHb3r17efnyJba2tkkGbyBtmlrkKpGCkpot1rZtW1auXEmhQoVo3749RkZGqVCz9K9Pnz7s27ePnTt3Uq9ePQoXLqz2uszU+9906NCBzZs3A3DgwAHu3LnDzJkzmTlzphK86evr06lTJ6KiotRS3sjF+x81a9ZU/nvAgAG0a9cOgJCQEObMmcO6devo378/S5YsAT6nwLCzsyN//vxq7yM3esmrXbs2NjY2tGvXDn9/f9asWUPevHn5+++/iYyMREtLi7/++otu3bphamrK27dvU7vKaV5cXByFChVi+/btbNiwQS14Cw8Pp3v37tKOaZDkgUshCXshSpYsSVxcHAYGBpw9exZ3d3eCgoJwdnbm/fv3bNu2jffv36dyjdOP+OBszZo13L17FxcXF4yMjNi/fz+rV68mJCREgoj/UUREBDo6OjRp0oSdO3eyZs0aTE1NqV27NhcvXsTMzAx7e3ssLS2xtrYGpOftSzly5GDixIm8ePGCc+fO4ejoSIMGDYDP624WLFiQypUrc+jQIUxMTMiUKRNTp07FyMhIJtYko3v37pw+fVqtBz1z5sw8efKECxcuYGtrS2BgIEOGDFGWdCpdujQXLlxQy0sm56q6pNojfi3jwMBAVCqVsgpIt27dJHhLo2QMXAobOnQoTZo0QU9PD0NDQ/744w+GDBkCfJ4hWaxYMaZOncoff/yRbHe0SKxSpUosXboUGxsbnj9/To0aNXB1dSUqKopbt24xdepUwsPDpU3/JVNTU169ekWWLFmYNm0a2tradOrUCQA7OzuaN29OvXr1uHLlCo8ePeL3338nJiZGxnImQUtLiwoVKrB+/Xq0tLRo3rw5Fy9eREdHh5iYGCpVqoSvry8lSpTgyZMnvH37ltjYWJo1ayZtmoSqVasyZ84cDh48yJw5c7h27Rrwef1oBwcHpkyZwrJlyxg1ahQLFy4EwMbGhqpVqzJlyhQlubRQlzB4a9SoEaGhoVy6dEnt9QMHDvDs2TO6du0qwVsaJgFcCnJzc8PV1ZWOHTty5coVvLy8cHV1pWHDhsranEFBQdSqVQsXFxf279+fyjVO+xJebHx8fLCwsGDYsGG8evWK8uXLs3v3bp4+fcr79+85c+YMu3btUh4Fiq/r378/bdu2ZcCAARw9ehQrKyuOHDnC7NmzmThxorKflZUVz549U2afyQw+dQnP0aJFi7J06VJ0dHQICQmhdevWakvlqVQqWrVqhZ6eHi9evODPP//k06dP0qbJaN26NS4uLly+fJmgoCAuX75M5syZ+euvv8iZMycuLi6sX78e+PyIf/Hixbx48YI+ffqkcs3TvhEjRtC0aVOWLFnCihUriIiIIC4ujjZt2lCjRg2GDRumjHMVaZMEcClES0uLefPmsWfPHtauXUuzZs2YNm0ao0aNYsmSJRgZGSm9Q0OGDMHf31/utr+icuXK3Lhxg5cvXyo9E40bN8bLy4uGDRuSNWtWDh8+zK5du+jfvz8dO3akSZMmRERE4OLiktrVTxfmzp1Lq1atePToEStXruTIkSNYWFjQt29fRo0axeHDhwF5/PQ1VatWBeDYsWNMmTKFqKgoxo0bR4kSJQgICOD169fY2tqq5dH6MliTnrfEdHV1lcC3a9eutG/fnuDgYGbNmkVwcDCNGjVi8uTJnDhxgnnz5mFmZoaTkxM5c+akTp06Egx/w4ABA+jVqxcdOnTgwoULidbjlhuK9EECuBSSKVMmjh07hpeXF2/fvmXlypX4+PiwePFidHR08PLy4tixYxw8eFA5Ri7cSatZsyaBgYGsW7eO2bNn8/r1a+W1devWoauryy+//MKBAwcYOHCgso5kpkyZlP8W35YjRw4GDx6Mnp4eL1++pGDBgujp6fH69Wvu3LmDv7+/WuAh1JmYmLBv3z7u3r3L69evqV+/Pi1atODy5ctoa2tTs2ZNRo0axatXr2jVqhUxMTFMnjyZU6dOsWrVqtSufrrQr18/cubMSdOmTcmZMycbNmxg8uTJ3Lx5k7p16+Ln54eJiQnPnj3j/v37ODs7y+PoJCS8CTM3N2fJkiXMnj2b7du3Y2VlRcGCBXFwcODKlSssWrQoUUAn0iYJ4P6D5HokfH19KVq0KNWrV2fo0KEsX74c+LyE0/Tp09m+fTvLli370dVNl0aNGkXVqlXZs2cPQUFBylqGderUYcGCBWzbtg0vLy9Zz/Rf6t+/P9HR0ezatYtbt27Rp08fcuTIwdKlS8mcOTP+/v6UK1cO+DzbT1ZT+Lps2bJx+PBhsmXLRv/+/ZUF0wG1IC5r1qzcvXuXvHnz8uuvv0rvxnfo06cPnp6edO3alfDwcGrUqIGTkxOnTp0iMDCQW7duoa2tTb58+Xj16pWS7016j5JnZWXFixcv+OOPPzh79iwbNmzA2dmZPHny8OrVK6ytrRk9ejRTp05N7aqK7yCzUP+lhMGbpaUlWlpaSkqFEydO4ODgwN9//62MbzM3N2fq1KkYGxuzYsWKVKt3ehF/8R0xYgSDBg2icePGxMXFERQUxOvXr7l69SqPHz/m5cuXErz9Bx8/fsTR0ZEqVaqwdetWFi1axJ49e3j8+DGzZ8+mUaNGeHh4ULx4cYKDg1O7ummajo4OWbNm5enTp7x7946mTZty//59ZQ3e2NhYDh8+TPfu3enQoQOxsbHY2dkRGxsrPUTfoK2tTY0aNVixYoXy1OLSpUu8fv0aHx8ftLS0mDFjBlevXuXOnTvKcSqVSoK3BOrVq0eFChUICAhg3LhxmJmZ4eXlxZo1a+jYsSMdOnRg3rx5LF68mMOHDxMQEEChQoVSu9riO0kP3H80bNgw7OzsyJQpE6GhoUybNo0tW7bQoUMH+vXrR1RUFK9fv0ZPTw8dHR0aNWokXfvfSV9fnw8fPgBw69YtXrx4wbp161iwYAEvXrygVatWjBs3jvbt23P27NlUrm36U65cOZo1a4aTkxPr1q3j/v37uLm50b17d44fP662r/RmqEuu993Kyoq1a9cSEhLCtGnT+Pvvv5N9D7kGfJ9Fixbx8uVL+vfvr9Zm48ePp3Xr1hw9epRRo0apBXDiH4aGhvTv3x9bW1tCQ0MpW7YsjRo14vr16xgZGWFkZISpqSm3bt1Sjtm6dSvHjx9n7NixqVhz8b0kke93Spgotm3btnTp0oXx48fTq1cv7t+/j5eXF71792blypX07duXBQsWcOLECRYuXEiDBg2Uhenlwp1Yv379cHd3Bz7/uH348AE9PT12797NmTNn2Lt3L40bN6ZHjx6Ymppy9OhR4uLiEiXzFf+oXbu2kq/tS+fPn2fSpEm0bNmSGjVq0KJFC4yMjGjbti2ZM2dW21eCt38kDN6KFClCpUqVyJQpEwYGBoSGhtK1a1dy5cpFnz59qF27NgDbtm3D09NT7X3kGvB9Ll++TMuWLSlSpIhamz179oy7d+/y5MkT7t69m4o1TNvev3/PtGnTeP78OdWqVWP16tVcv34dgA8fPvD8+XNu3bqFoaEhFStWZM2aNWTJkgV/f/9Urrn4XtID9y81bdoUc3NzAJYuXapsHz16NI0aNcLFxYXTp08nOk7uupM3YMAABg8ezPDhw5k7d66SAfzFixe0adOGuLg4fHx8qFmzJjt37mTy5MnUqlWLQ4cOpXbV0xyVSoWxsTHHjh1j/fr1+Pr6fnV/Q0NDunbtiqOjI48ePaJVq1Y/pqLp2NChQ2nZsiVmZmaEhISwatUqNm3axLNnzyhSpAhz5sxBW1sbfX19YmNjqVOnjgwK/4/Wrl1LkSJFcHJy4uHDh0RERLBgwQL++OMP1qxZA8gs6a/JmjUrnp6eGBgYUKVKFTZt2qSkCIrPT9isWTNsbGwwNzenXbt28qQoHZEA7l/IlSsXx48fx8DAgAkTJhAQEKD2iOnPP//k3r179OzZM5Vrmj4kvPD26tULPz8/RowYgZ2dHa9evUq0fIuPjw9t2rShf//+7N69O9F7iH/06dMHV1dXWrZsyY0bN5LcJ/4irVKpyJUrF48ePZK2/IYBAwbQrVs33Nzc2LdvH8uXL6dEiRLKI/6nT5+SN29eateujaGhIQsWLCA2NlYeRf9HFhYWTJ48mWrVqvHkyRNUKhUqlYpq1aoRGxsr3/8vJNceFhYWdO/enZYtW7Ju3TomTZqkvPbbb78RGxvL2bNniYuLk3M1HZEA7l/Q1tamWrVq+Pv78+LFC+zt7fnw4YPypRk3bhw5cuSgW7duqV3VNM/b2xszMzMGDhyo9E64uLgwatQoHj58SO3atZUkkgkvKLa2tpKo9zuUKFGC2bNns3z5coKCgr77jlp+EJNXtGhRJk+ezLRp09i9ezfW1tYsXryYM2fOUKhQIVatWsXChQsTrQAgvRmJJTzP8uXLR2ho6Fd7KW1sbDA1NUVfX59FixbJRJAvZMmSRS3dUs+ePSlcuDAqlYqAgACePXuGlZUVXbp0wcbGhu3btxMQEMDq1au5ceMGQ4cOBeT7n95IAJeMhCeylpaW2uymGjVqMH/+fM6cOYOrqyvv37/n48eP7Ny5k2vXrinjuUTSihYtytGjR4HPj6EHDhyotK2TkxMBAQEMGTKE+fPnK8d8eVcoF5pvmz59Or/99huVK1dO7apkCJkzZ8ba2pq9e/dSpkwZFi5cyPjx41m6dCnr16+nUKFC7Nq1i/Hjx6v9mAp1Cb+7Xl5elChRgqVLl3Lw4MFE3+nkvucSvP1j2LBh9OrVi0qVKvHkyROGDx9Oly5dOH78OIULF8bc3Jw2bdpw/vx5rKysaNu2Lb169SIyMpK3b99St25dyfeYTkkakWTEXzTc3NwoV64cuXLlYtmyZZw4cYKjR4/So0cPZeWFe/fu8fLlSzJlysSAAQNSueZp3/Xr11m1ahUmJia0aNECU1NTevbsyadPn1i8eDGGhoaMHTtWWd0CEg+ml+DtH/ny5eP+/fvK3/FZ7AMDA1mzZg3dunVT1ooU/92bN2/Yv38/kZGRtGvXjj/++EPJ9fjgwQOsrKzQ1taW4O0b4r+7I0aMoGPHjvTr14+LFy+qfafjA7TkvucSvP1jzZo1VKlShe3bt2NnZ4eJiYkSsGXLlo1Jkyaxbt062rZty9mzZ1mwYAE7duygWLFibNu2TZZyS8dkFuoXEs429fT0xM3NjQcPHnD//n3c3d0ZOnQov/76K0ePHsXZ2ZkPHz5QrFgxJk6cSPXq1ZXZpuLrQkJCyJYtG+3bt6dKlSrMnTsXLa3Pp+Ps2bMZMWIEfn5+eHh4pHJN07aSJUty+vRpli1bRvfu3QGUR1FPnz7l5s2b1KlTJzWrmKHEj8nMmjUrhoaG6Oh8vgc2MTFh+PDhDBw4MDWrl27UqlULOzs7HBwc2LlzJ2/evCFnzpzUrVuXLFmy8OnTJ+V6IL7u1q1b9O3bl+fPn7Nnzx5+/fVX5TwNDw+nb9++HD16lNWrV1O+fHnevHnD9evX2bJli9LOErylT/IN+UL8HZ+VlRWWlpZ069YNX19fnJ2dGTFiBMbGxvTs2RNzc3OOHTvG4MGD0dHRYfjw4cp7yJfh2yZNmkTmzJkpV64cPXr0oG7dusyePVu5aM+ZM4eAgABJFfIVNjY2VK9enU6dOqGjo4Obmxt//fUXPXr0oFChQkRERDBp0iRq165N8+bNU7u6Gcrdu3cpU6YM8+bNY9euXZQsWVJJOJvwJlAkLS4ujnfv3vHmzRuKFi3K4MGD2bFjB5MnT2b//v2YmZlJL9s3JDzP7t27x++//86ZM2coVaoU+vr6yj4RERG4ublx+PBh9uzZQ5EiRdTeR9o5/ZIALgk2NjZcuHCBxo0bq40N2LlzJ0uXLqVevXrkyZOH2NhY/vrrL3r27En58uXZtGlTKtY67fL19WXx4sW0atWKrFmzAp+D3M2bN1O8eHGOHTuGk5MTDRo0YNasWUoQN3HiRPr06ZOaVU+TVCoVWbNmZdy4cTx69Ijdu3fj7OxMq1atuHTpEp06dWLPnj0MGDAAMzMzNm/eTM2aNaVn+Bv+TeDl6+vLn3/+yZs3bwgODqZmzZpKb4Y83leXsF2trKzQ1dUlIiKCjx8/MmPGDP744w+yZcvGxIkTlQlgVapUSa3qphvx59lvv/0GfL6p8PHx4dSpUyxbtowcOXIo+0RERDBgwACmTJnC7du3U63OImXJJIYk6OnpMWHCBDp27IiHh4cyziXeyZMnWbFihdp6cdbW1vj7+2NnZ0doaOiPrnKalXDCwu7duylevDgBAQGcPHmSt2/fcuLECbp27crBgwepXr06ixcv5ty5c7Rp0yaVa5626erqcubMGdzc3JSen3jFihWjQYMGdOrUiffv31OyZEmioqL47bffePz4cepUOI1LOAboW+OBkhtAL+OIEvtywkKRIkWYN28ep0+fpm7duhQqVIgHDx7w999/ExERgampKZs2bWLkyJGJzmvxWcI2LV68OIcPH2bYsGHKeOH8+fMza9YscuTIQdOmTZX0KwlvLORczRh++h64pO66o6OjGTRoEJs3b8bX15datWop+5mamhIXF0d4eLjaMQcPHsTa2lqCty9cv34dNzc3YmJiuHbtGkFBQdja2rJkyRK6dOnC0aNHadWqFYaGhkpv5oMHD1K72mle/KzoyMhItW0AwcHBTJ8+nU6dOjFx4kQuXrzInTt3ePLkSWpVN01r0KABv/76KwBjx45l9uzZX90/Pnj78tohP4iJJZyw0K1bN7Zv3658v/fv309QUBC7d+8mKioKc3NzZs2aRVRUFIcPH07Naqdp8W3at29fGjduzMePH/H19cXV1RX453FqWFgYW7duxcrKKlGvsJyrGcNP3QOX8K6kYsWK6Onp8e7dOy5cuAB8vktZsGAB1tbWrF69mnv37lGzZk3y5s2LtbW1fAn+hW7dujF+/Hj69evHgQMHyJ8/P4MGDaJUqVJcvnyZ1q1by1T2b4hfLP3KlSvkzp2bgwcPYmtry+XLl9X2Syr1Qvw2Sb+Q2P79+8mePTvHjx+nTp06NG/enODg4O8+vmDBgjx58oR3795psJbpl7W1NdOmTaNjx45cunQJLS0tzM3NyZcvH/fu3ePZs2e4urpSq1YtTE1Nadq0qawG8A1eXl706NEDNzc3MmXKROnSpenTpw9jxoxRngzly5ePtWvXcuXKFclNmkH91GlE4n/khg0bhoODA+/fv6dAgQJMnDiR5cuXExYWRvfu3ZkxYwbdu3dnw4YN7N+/n8WLF0t29X9p4cKFaGtrExgYiJ+fH9OnT6dNmzYUK1aMBw8eSPD2Dfr6+jg6OlKjRg3q1q1LSEiIki7kS0mNwYqLi0OlUskPYhLq1q3LlStXaN68Of379/9XwVvPnj1p3749nTp1kgAuGdra2jx58oTnz59TtGhR7O3tlRu2ly9f0qZNGy5evEhMTAzz5s2TtBZfyJs3r9pTiUyZMlG3bl0loTTAxo0bCQsLw8/Pj+joaObMmcP9+/extbWVnvcM7KcO4AA8PDzo0KED3bt35/jx43h7ezNo0CCyZs3KtGnTePLkCW5ubqhUKmrWrMnChQuVLOBygfl3goKC+PTpE+PHj0dLS4upU6dy8eJFQBLzfsuHDx/w8PBg7NixbN26ld69e3Pt2jWaNGlC9uzZMTY2VgaHq1QqihUrxpYtW9Qe6Uv7/iP+fItfO/bJkye8ePECDw8P7t27x/Hjx5XX49vty3PU0dGRwYMH4+npKUMnviImJoacOXMSGBhI+fLl2bVrFxMnTuTZs2eMHTuWUqVKcfjwYeWxqVxb/7F06VLevHmjPB6Fz2O0c+fOrXYzplKpWLBgATVr1mTkyJHExsYyb948Zcyr9GZmTD/dI9Qvl3AZPXo0q1ev5o8//qBZs2ZMnTqVLVu20LlzZ4KCgpg5cyahoaFoa2szf/58fvvtN5ydnfnrr79S+ZOkHf82+OrevTtjxoxh9OjRzJgxQ4M1y3hy5szJxIkTadSoEQB37twhU6ZM6Orqoq2tzfv371GpVDx58oQGDRrIRTsJCc9XGxsbrl27xq1bt4DPM82zZ89O3759OXHihNJ+X/Z2Ojo64uvrS9++fdm+ffuP/xBpUMJ2NTY2Jjo6mujoaACaN29O4cKFuXXrFkePHuXVq1dkyZKFLVu24OvrKxMWkpElSxYiIyP5+PEjZmZmvHjxAgB/f3+qV6+Ok5OTcu4C+Pn5Ubp0aapXr07Xrl3l3MzgfroALl782Ct7e3t27txJiRIlWLBgATNmzCAoKAg/Pz969uzJ2rVr8fX15cWLF+jo6LB69Wry5s1LrVq1iIqKSu2Pkep0dHRYs2YNwcHB3L9/n/nz5ys/el+76+vWrRv+/v60b9+evXv3/sgqpxsFCxbE0tKS7Nmz8/z5c2U2r7m5OcOHD6d9+/bY2tpy/PhxMmfOrPQMq1QqZTUA6dlMno+PD82bN2fNmjUsXbqUp0+fAp+DuPh1es+ePcvs2bN58uSJsspKfPDm5ubGtm3bUvMjpBkJzzMXFxcaNGiAlpYWISEhSu+Rjo4OMTEx6OjokClTJubMmUOWLFlo3ry53GgkIeH109nZmS5duuDs7MzVq1epUqUKXl5evHv3jhEjRnDv3j309fWZP38+S5cupVGjRpQrV45WrVrx5s2bVP4kQlN+mgAu4QVm1KhR9O7dm0KFChETE8P79+/x8fEhT548uLq6EhUVxcCBA6lYsSJGRkbY2Ngox2pra5MjRw55ZJKAjY0NmTNnZujQoVy5coUjR44wd+5coqOjvxrE1apVS2abJaNdu3b07dsXHR0dzM3NyZw5M4cPH1aWwTE3N2fq1KlUqFCBNm3acOnSJYCvPvIT/3B2dsbT05O2bdty+fJlPn78qDbuasuWLRQsWJB3797x8eNH6tSpQ0xMDC1atGDWrFn07t1bejeS4O3tTbt27ZgxYwavX79m2LBhXLt2jc6dO/P+/XsMDQ35/fffqVatGpkzZ6ZJkyYyYeE7ZM+enUOHDnH79m369evH7du3sbGxwcnJibJly3L69Gny5MnDp0+fqFmzJv3796dx48Y0bNgwtasuNOinSSMS/0NWuHBhjIyMaNmyJREREcojp8KFC6OlpUVMTAwqlYoyZcowdepUmjdvroyFgc/TryV4U7dt2zZWrFhBjRo1CA4OpkmTJsqapl9bEic+eJPM9eratGnDxIkTmTFjBg4ODtSqVQtHR0eKFy+Oj48PzZo14/nz53h4eHDq1CnWrVtH6dKlAfVxbhK8JU1PT49KlSoxZ84czp07p0ygSRhAtGzZkjFjxjBlyhSsra2VfS5fvkzHjh0leONzUJFQw4YNadSoEY6OjsyePZvw8HCMjY359ddf2bp1K4aGhrx//56LFy/y999/K4nStbW1JXhLIKnr4bNnz6hduzYFChRg5syZFCxYkG3btjFgwADGjh3LgwcP2Lx5M3Xr1gUgd+7chISEYGBg8KOrL36gn6YHDsDW1hYfHx/evHlDmzZtePr0qfIj17p1a2bPns2RI0ewsLAAoHbt2jKY9jvF30Hr6+vTqFEj+vbty4cPH2jdujVRUVHSG/SdcufOzZIlS1i8eDHLli1Te+2XX35hw4YNhIaG0q5dO16+fEnOnDmZN28e79+/p23btqlU67Tty3NPR0eH3bt38/fff+Pt7a22r76+PoULF+bKlStq2+ODDDmHP5syZQpaWlpMnjyZ+/fvA5/T3BQrVozJkydTv359Zs2axfjx47l8+TIbNmxQVlxJmLtQet7UJTxXmzdvTsGCBfn48SNnz57lxIkTZM+enX379vHo0SPc3Ny4efOm2vHm5ua4u7vToUMHmjZtyvXr11PjY4gf5KfpgQN4//499+7dI3/+/JiYmBAXF6csL7R+/Xq6d+/OnTt32L17txK8yYLKSatZsyYtW7bE3t4ePT095SL84cMHtm/fztixY9HV1WX06NFoa2vLD993Mjc3J0eOHEouwnhaWlrcuHEDJycnKlSooDwaCQsLw9HRkXbt2qVGddOF+HMvfhk3PT09Hj58SJEiRciaNataj0eePHlwd3fnl19+UXuP2NhYOYcTuHz5MnXq1KFbt24UKFAAgB07drBmzRqMjIzw8PBg3rx5LFy4kDt37nDv3j3q1KlDQECA2vtI8KYu/hzz8fHBz8+PqlWrUq5cObZv306LFi149uwZ1tbWWFpaMnnyZMqUKaMcmy1bNpycnChfvjwtW7aU4O0n8FNEJ/b29tjY2LB7925mzJjBjRs3mDt3LgUKFFAL0rZu3Yqnpyd+fn5Knje5wCQ2fPhwpkyZQv/+/Zk9ezZz585FR+dzRpr4O+ojR46wYcMGfvnlFypWrAjIo9LvkTNnTgwMDIiIiABQzs343Fhnzpzh7NmzFCpUSDnm5cuXao/5RWItW7bk6NGjFCtWjMjISKZPn07NmjUZPnw4VlZWaGtrkzVrVkaNGkXWrFkT9WwIdQsWLGDs2LG0atWKbt26UbBgQQAePXpEzpw5sbS0ZP/+/cDnoOTKlSs0aNCAvn37pma10wUbGxscHBzo0aMH7du3Z8+ePQAYGRkB8OLFC+rVq0elSpXo0qWLclx4eDjLly+nS5cuiZJ7i4wpwwdwBgYGdO7cmQ4dOgCwb98+Jk2axKtXr5g2bRr58+dXfhy/JI9PE+vbty8dOnSgZ8+eNG/enMqVK1OvXj06duwI/HNHHRMTw/Lly9HW1qZ9+/aAjMn6Hjdv3sTY2BhbW1vgc3smHH8JnwPhV69eJTpW2jd54eHhXL58mXnz5lG8eHHOnDlDx44dadWqFUuXLuXQoUOsXr0aKysr2rdvLwFxMhK2yerVq5kwYQK2trZ07dqV/PnzAyi5xzw9PalVqxZBQUHkzJmTCxcufHVMrPisQIEC7N+/nzNnztC8eXMmT55M//79Wb16NSYmJhQpUoTw8HB++eUXBg4cqHZsWFiYkmpEZHwZ7puU8AKjo6NDVFQU/fr1o3r16srd365duwgKCiIqKoqpU6dSuHBhCda+Q9GiRWnQoAFDhgzh3LlzREZGcvfuXXbv3k2RIkXU9tXS0iIyMpIhQ4ZQoUKFRI+kRNIePXrEpk2b6NWrF3Z2doB6YJY1a1b09PSoXLkynp6elCtXDn19/dSqbpqUVOB19OhRJk2aREhICAsWLKBYsWIcPHiQ+vXrs2jRIrZu3crSpUupV6+eMrBeAmJ1CcdnValSBYBly5Yxbtw4bG1t6d69OwULFuT9+/d4eXlRuHBh/P390dLSonXr1rIayDfEB7YJcxTOnDkTHx8fZTxsgwYN6NixI6amprx580YC4p9chvt/Pv7kd3Z2pmfPnhQoUIB79+4xevRoWrZsSbVq1YB/gjgTExN69eqVmlVON8LDw4mMjOT27dvAPz1Cz58/Vx6hJHzkF//agwcP0NPTS4Uap21JBRpRUVEsW7aMR48e4evrqzwiMTQ0xMLCghkzZpA9e3Zy5MhBtmzZyJ49Ox8+fPjRVU/T4q8B9vb2WFlZKdtPnjzJ1KlTuXfvHgsWLKBo0aLcvn2bFStWMGHCBFasWCGrrHxFfLsOHTqUadOm0b17dwBWrlzJuHHjsLOzo1u3blhZWbFv3z5q1apFx44dsbW1laD4G1q3bk2nTp2Az4vRV6lShZkzZzJ69GgWL14MfE6O3KZNG7S0tNR64CUg/nllyFmoOXLkYN++fRgZGXH37l38/PwICQlhxIgRXL58mWnTpilJeH/77TdOnTolF5bvZGxszNu3b4F/xrsNHTqUQoUKKRd0IyMj8ufPz9WrV4HPi1kHBwcTFhaWavVOS7y9vVm/fj3Xrl1LdnautbU1ffr0wdramhs3bqCjo8Pz58/R1dWV3E5JWLRoEffu3WPkyJHA5xm78+bN482bNzg7O6ude9bW1kyfPp2nT5/i6urKtWvXUqva6Y6npyc9e/akc+fOPHjwQK1d27dvz9ChQ9m0aRPLli1TG0cos9CTp62tzZIlS8iSJQs2NjYATJgwgY4dOzJo0CDOnTuHlpYW3t7emJub06BBA7nBEEAG7IEDeP36NUFBQRw7doxt27Yxb948GjVqxIcPH3B0dFTGasDnu3IZ7/L94oO3hBIuRG9qasrhw4dp0qSJsu3gwYMSvP2/0qVLU6tWLSZNmkSRIkWSPfcOHjyIm5sbDg4O7Nmzhw0bNjB37lwaN24MkOSYzZ9V/OQOFxcXPDw8ALhx4wZTpkwhJiaG2bNnY2lpqex/5MgR7t27h6WlJf369UulWqc/2bJlo3bt2nh7e3Py5EnlOx1/Lq5atYpx48bRq1cvrK2t1Y6V4O0fCb/venp6xMbG4urqStGiRZUbkIEDB7J582Z69uzJ/v37CQgIUG7eJDuCiJeheuAcHBy4ceMGFy5cIGfOnGzevJkpU6Zw4sQJevfujbGxMW3btuXKlSvY2NgoM/3E/2b48OEULVqUPn36sHPnTp48eUKrVq1Su1ppVp06dejduzempqb07duXGzdu/KseCsmdlZi2tjZdunRh7NixTJw4kUmTJgGfxxH16NGD2NhYevTowYsXLzAxMcHf359Nmzaxd+9eCS6+U758+Thy5Ai9e/dmx44daq/FJ+kFaNSoEX/++aeco9/g4uKCsbExf/zxB1evXqVdu3b07NmTCRMmsHv3bgDy589Pzpw5efr0KXfv3lVSX0kPnIAM1AOXO3du7Ozs2LVrF87Ozrx9+5bevXvj4eFBtmzZGDVqFEuXLiU4OJj3798n2ZMk/pt3796RJUsWtm3bRlhYmBK8Sa+muvhUKwcOHGDlypVEREQwefJkChQo8K96geWH8R/xPRGxsbFcuHCBRYsWMXjwYFxcXIDPq4TMnz8fbW1t9u3bh5eXF6tXryZXrlxK8Cbn6fd58+YNwcHBFC1aVJk4E9921tbWDB06FIDdu3fL4PpvyJYtG46Ojri6ujJ37lzs7Ow4duwY9+/fp0qVKhgbGwOfx8MdP36cO3fuKOeqBG8iXob5hoWEhNCtWzeGDh1Kr169mD17Nr/99htLliyhSZMmaGtrc/LkSWrWrEnTpk3lwp2CdHV1qVq1KteuXcPe3h6QMS9JiX/U7O7ujp2dHRYWFlSuXJkZM2Z89XGqSF58MDtixAgCAwPJkiULd+/eZdSoUcri89u2bWPUqFEcPHiQ2rVr8/DhQ+zt7ZX2lvP0+7x8+ZJLly7RvXt36tSpg46ODnFxcRgYGNCxY8dEM9HlRiN5r1+/ZsGCBRw7dowlS5YwZswY2rRpQ2RkJF26dKF48eJA4ptgOVdFQhnqEWq8ihUr0rRpU1q0aIGpqSlPnjyhb9++nD17VtlHLtwpp2jRori6uippWqRtk+fs7MywYcNwdHTk/v371KlTBzs7O3R0dOjbty+3bt2S9vuXGjZsSFBQEK1bt+bUqVPkzJkTBwcHvL29GT9+PJMnT1b2NTExUYZOyKOo75fwnFyyZAmlSpXi8uXLPH36lJIlS2JiYkKdOnXUxsOKxNq3b09oaCiHDh3CxMSErVu3snbtWtatW4ebmxsmJiZ06tSJhw8f0rBhQ54/f57aVRZpWIbpgUvo9OnTTJ48mW7dunH16lV++eUXnJ2d1faRH8iUc/36dQnevoOOjg4VK1Zk1apVHDx4kLt377Jw4UJmz55NpkyZmDJlCvnz55f2+5eyZ8/O/fv3OXXqFPA5memiRYsIDAxkyJAhyuxoQG3cqwRv3y8uLk55JOro6MjcuXN58eIFlpaWnDhxAmtrayVViEiapaUldevWZf369Xh5eaGvr0/37t1p3749pUqVws/Pj4ULF3L8+HGePn1KeHh4aldZpHHpqgcuPjj4N0GCjo4ODg4OrF27Vi7YItVNmzYNS0tL2rZtq/aIyc/Pj969e3Pnzh0cHBx48OBBKtYyfbG2tmbp0qXY2NiorSFbpUoVtmzZgpaWFm5ubqxatSoVa5m2lSlThuDgYACio6OTvcZ+OYEm4X7So/ltOjo6NGvWjKFDh3L37l1OnTpFREQEVlZWTJ06lZcvXwL/7bdO/HzSTQ+cra0tU6ZMoUCBAhgYGHzXMVpaWsTExLBq1SplbVMhfoTkxrKdO3eOPHnyYG1trZbcODg4mH379rFu3TpCQkJ+VDXTleTa9OrVq5w+fRoXFxdKlCihbH/+/Dlr1qyhS5curF279kdVM92pW7cu+/btY+zYsYwfP558+fIlGzR8Oa4t4X4SvH1bTEwMW7ZswcnJiTNnzmBvb8/w4cOxsbGhbNmyyn4SvInvkS564ExMTDh48CDGxsaEhYVx9uxZ/v77b9atW6fsI6kVRFqR8MLbqFEjzMzM0NPTY/Pmzbx+/ZqVK1eSP39+/P39OXHiBO/evWPWrFlcvHiRgIAAQM7nLyVs07Zt25InTx7MzMzYsGEDZ8+epUGDBnh4ePD69WtWr17No0eP8PT0JCYmRlmnV3qIklatWjVWrlzJ1KlTyZ49Oy1btmTt2rWcPXuWbdu2KfvJOZmyDAwMyJUrFyNHjqRRo0bs2LEDR0fH1K6WSEfSRQCnpaXFsGHDuHfvHpcuXaJmzZp4eHiwe/durl27xowZM+TCItIcHx8fHBwcuHDhAkWLFuX169eMHj2aAwcOsHz5cvLmzUvOnDl59uwZ2traVK9eXQKMbxg5ciTt27fnr7/+omTJksTFxbFjxw7Gjh1L9erVadu2La1ateL27dtERETQrFkzGVj/DVpaWowePZo7d+4wf/58WrduTc6cORkwYAC7du3i77//Zvny5dIb9A0tWrTgyJEjymPQf6Nly5Zs27ZNfsfEv5IuAjj4PNMsPhP99evXMTIywt3dnf79+3PhwgU2bdrEvn37lHEcQqSmtm3b4u3tTfv27bl06RIODg7MmjWLzp07s2vXLuDzbOkiRYoQGxvLhg0blAzrchFPWt26dQkMDKRTp05cvHgRgAEDBlCvXj327dunJO/NnTs32traPHjwQBKffkXCXk13d3ccHByoW7cu0dHR6OjocOHCBV69ekVUVBRGRkasWrWKdevW8fjx41SuedrTrl07hgwZwtKlSwkKCuLNmzffddyX33c5V8W/kWbHwMUnioyf+RS/nFDXrl0BiIyMpHnz5uzcuZNjx45Rp04djhw5Qtu2bVOtzkLEK1CgAHv27OHSpUvY2dnh7+/PwIED2bVrF8bGxmTLlo3Tp0+zatUqZYKNBG/qvkwEa2JiQnR0NKGhocq2yZMnc+LECdq0aYORkRHwOSfk/fv3JfFpMqpVqwagBLcAU6dOVXKQAezbt4/r16/j4OBA586duXjxIuXLl5cl8ZKxevVqNm3aRNOmTenVqxdZsmT5ruPiv+/x4zvlXBX/RpoM4GrWrMn06dOxtLRUy+h94cIFJefQ/v37efXqFb///jve3t64ubnRu3dv1q9fn8q1Fz+z+HM1V65chIWFUbp0aQIDAxk1ahSLFi1CpVLRrl07WrRooazMEE+CN3Xx7dG7d28qVKiArq4u2trays1dfCLZwMBAcufOrQQmCcljP3WmpqYsXLhQWQorNjZWOQ//+OMPatSowalTp3j9+jW9e/cmNDSU0NBQevXqRdeuXSXZdBLiJyP5+voqnQk9evTAxMTku9+jUKFCmqqeyMDSZABXokQJChUqxODBg8mZM6dyIV+2bBlGRkbcuXOHiIgIOnbsqCyJFRoaqjyGktmm4kf58scs/lzdtWsXbm5u7N+/Hw8PDxYvXgx8XjOyUaNG5M2bV8ZmJSNhm3bq1IlRo0bx5s0b/vjjDwD8/f1RqVRK+2XLlo179+79p7FHP5tXr17RpUsXcuTIwaZNm4B/VgjZunUr1apVIzo6Gjs7O54+fQok7gmVoFhddHQ08DlJb0xMDIULF8bFxYWePXuSOXPmbx7fvXt3jh07Ru7cuTVdVZHBpMkAbu7cuaxatYqCBQvi7e2NhYWF8lpQUBDBwcF4e3vz6tWrJI+Xbmjxo8T/mNWrV4927dpRrFgxjIyM2LVrF8uWLePp06d8/PiRTJky8csvv7Bo0SKyZcvG6NGjU7nmaVd8m1pbWxMXF0efPn24desW79+/p1u3bpQvX56NGzdiY2ND7dq1GTNmDO/evePcuXOpXPP04cyZM3Tp0oW8efOydOlSJUC7e/cuU6dOJSwsjBw5cij7S8/wt3l5eTF69GguXbpEv379+Ouvv3BwcKBnz55f7YlzdHRk0KBB9OjRQ9IHiX8tzQVw8ReT+fPns3r1aqpXr87w4cPJmTMnAH/99RfZsmWjZs2aqVlNIRQjR45kxowZjBgxgqVLl+Lu7o6RkRGzZs1i586dzJs3j+PHjxMUFISBgQENGzZUxryJpBUuXJh169YRGBio1otx7tw5WrRogaGhISNGjMDf3x9dXV2aNm0qC6gnI35hdPj82Dk2NpYrV67w8OFDmjRpwrp165Rez2vXrlG4cGFlLU7xbVmzZqVJkyaMGTOGDRs2KOlADh8+jJOTk9rj1ITnp6OjI76+vgwYMIAtW7akVvVFOpYmrnYlS5YkW7ZsgHr3fL169dDX16dYsWJ4e3tjZWXFw4cPmTFjBu7u7vzyyy+pVWXxE0v4iK9ixYqULVuWTp06UblyZTZu3EjdunUZNGgQr1+/xtPTkwYNGuDp6YmHhwe2trbKkkPSs5G8+/fv4+joSFhYGDVq1FC2a2lpcfv2bZo2bUrLli1xcHDAwcFB2jQZ1atXZ968eRQtWhT453HpwoULyZo1K05OThQoUEB5nLp//35CQkLo3LlzqtU5vYmMjCQ2NlYJlOOH8AwaNIhHjx7RqVMnPD09MTExUc5PJycnZex2wlx7QvwbqRrAqVQqrKysOHjwIF5eXlhYWCgB3OLFiylUqBD16tVTEp8OHTqU7Nmz8/fff7N//35u3ryZmtUXP5lSpUoB/9xk2Nra0r17d27fvs2ZM2eIiIhg/Pjx7Nixg8qVKzNw4EBy5MjB5cuX2b17N2fPnpWZkUlIalD8x48f2bFjB0OHDqVevXpKipBPnz4pgVpoaCgPHz6UNv2KfPnykS1bNgYNGkSePHmAz9fWIkWK0L59e/744w+cnZ3JmzcvmzdvBsDV1ZVu3bqlYq3TrqTO1ejoaJ4+fUqjRo2UHs74nrYbN24QExODgYGBsg5v7dq1CQgIwMPDQ4I38T9JE3ng7O3tmTFjBrNmzcLf3585c+ZQpEgROnfuzL179wDo0aMHLVu25MWLF/Ts2VMZOCqpF8SPMGrUKIyMjPD09FTyZ82ePZvGjRtz69YtGjdurBZAeHh40LBhQ27fvs3QoUO/Oy/Uz8zFxYWSJUtiYWHBsmXLOHPmDKGhodjY2DBr1izWrFmDp6dnalczXbCwsFAmITg4ONCxY0eePXtGtmzZMDU1xcnJSW293V9//ZVt27axYMECvL29Abm2filh3rzy5cujUqnQ1tbm1KlTWFlZsWfPHk6fPk3fvn2Jiori48ePBAUFsWnTJnbu3KncaMQH1WfOnEnlTyTSu1QL4CpUqMCbN2+4ffs2cXFx2NraEhQUxOPHj3n16hXt2rUjNDRULbGhm5sb+fLlw9PTU2ZCiR+qUqVKnDt3jpiYGPLkycPDhw9RqVSMGDECGxsbVq1axbx585S7bIDhw4djZmbGgAED5HxNQsIfxEGDBtGrVy/Wr19PwYIFyZcvH6dOnSIwMJAbN25gY2PDtGnT2Lt3Lz179kzlmqdttra2uLq6EhgYyPbt24HPiaWdnJwoVqwYXbt25eDBg4nW2ixatCg3b96UoO0bhg0bRosWLYiOjsbS0pKtW7cSEBBA7ty5Wbx4Ma9eveLZs2eYmJhgbGxM1apVlfGZ0rYiJaVKAGdjY8PChQvZtm0bfn5+3L17F4AmTZqwdOlSVqxYgbe3t/JjmNSJLwv9itRgZ2eHi4sLY8aM4dChQ6hUKsaPH0+FChX4448/mD9/vpLaJiE5X5OXI0cOfH19Wbp0KceOHQM+Z7Zv164dt2/fxtfXl8jISOzs7OjQoQP29vbSlskwNzdn69at5MqViwMHDrBx40a2bt0KfH7S0aVLF8LDw/H39+f69etJvocEGslzcXGhX79+dOjQgTNnzuDp6cmgQYNo0KAB58+fx9jYGBcXFzJlykRsbCxjx46VJN1CY1JlDJyuri4AzZo1Y9y4ceTLlw+AnTt30r17dzp27IiXl5cyseHTp0+Jxh7IBVykhvfv3/P69WtcXFyoVasWcXFxDB48mHPnztGsWTO6deuWZNoAOV+T1rZtW86fP8+vv/5KZGSksj0+s32TJk0wNzdXlhtr1aqVJJP9iufPn/PXX38RFxfHp0+faNu2Lc2bNwdgw4YNrFy5EjMzMwYPHpzsJDAJNJJXunRpAgICOHPmDDY2Nri4uDBw4EDOnz+PoaEhb9++JSAgAF9fX/z8/JS8pNKmQhNSJYD7+++/WblyJSNGjKBw4cJMnz6dvHnzAp+TSfbo0QMXFxdcXV0xNzcH5AdQ/HhJBQm7du1izpw5wOdH+vFB3KBBgzhz5gxdu3alSZMmP7qq6dbOnTvZt28fBQoUUK4B8e2+ZMkS4uLiqFu3LqB+DZDrQWLxKyoEBgZy6NAhgoOD0dbWpmvXrjRr1gyANWvWsGrVKrJkycKECROUiQ3i2wwMDKhYsSJPnjyhUqVKzJgxAz8/PxYtWoSOjg6DBg1KMr2VTK4RmpIqAVxYWBifPn2iZs2aNGnShFy5cjFt2jTlAr5lyxa6d++Oq6srtra2qVFFIZQgoVGjRrRs2RI7Ozvg8zqRc+bMITY2lr59+1KzZk3i4uIYMmQIQUFBspzbv/DmzRtcXFw4cuQIfn5+lC1bVml3c3NzIiMjefHiRSrXMm2Lf1IRnyLk3bt3fPr0idevX+Pl5UVMTEyiIG7r1q0EBwdL8thkJHXzFhUVxfr16+nbty+bN29myJAhygorxsbGlC5dmpIlS/7gmoqf2Q8ZA/frr78SERHBo0ePePfuHfB5YeotW7YwatQobt68ye7du7lx4wb9+vVTZkfVrFmTv//+W+5gxA8zbtw4YmNjGT58OABjx46lTZs2vHr1CiMjI549e8bvv//OlStXqFOnDs7OzmhpaREUFMTevXuV95ExL/+OiYkJy5cvp3DhwqxYsYIHDx7QpEkT8ubNi7W1tVwDkmFra4uHhwf79+9n9uzZREZG8vbtW6ytrZk3bx5NmjTBwMAAb29vtLS0WLx4sbIOajwZn6kuYXsULVqUbNmyERISQlhYGKVKlWLy5Mm8efMGNzc37t27R/bs2Zk2bRpZsmShefPm8r0XP4zGA7iWLVsyf/58Lly4wMuXLxk7diwPHz7k+fPnTJo0iQ8fPjB06FDy5cvHtm3buHHjBgMHDuTOnTvKeySciSqEpmTOnBlPT0/q1avHpk2bWLt2LfPnz6d///48e/YMHR0d5s+fT/bs2bG1tSUkJIT69eszcOBATp48qQR94r8xMTEhKCiIevXqsWrVKm7fvs2MGTOUcURyDVCXM2dO1q5dS4ECBYiLi2Pfvn1ER0czd+5crl69ysiRI7l+/ToLFy6kcuXKuLm5kTNnToYPH65MFhHJ8/b2plGjRpiZmXHr1i2ePXuGq6srNjY2dO3aldy5c/PkyRMl4GvcuDExMTFy8yZ+GI0HcNbW1qxbt47z589z9+5dSpcuzaVLl/jzzz+5d+8ea9asoXXr1pw9e5Y8efJw6tQpFi5cyNChQzVZLSGSlCNHDjp37kyLFi148OABnz59onv37nz8+FHZ5+DBgzx//pzWrVsDn3uY45P0CnX/9scsc+bMBAUFkS9fPhwdHbl+/br8IH5Fq1atlNU9rl69ikqlokePHqxdu5b69esTFRVFo0aNiI6OpkqVKjRp0gRfX185V7+hd+/e9OvXj65du3Ls2DECAgLo0KEDrVq14sSJE5QqVYqSJUtiaWnJvXv32Lp1q5JkWm40xI+i0QAu/s6kTp06rF27lvHjx3Pt2jVMTU3x9vbmxIkTNGvWjLFjxzJ9+nRiYmKwsLDg+fPncsEWqSZnzpx07tyZNm3aEBkZSe3atQHQ19fnw4cPNGvWjFGjRmFvb68kmgZ5FPWlhO2RI0cOnjx58l3HmZiYsGLFCiwsLOjevTtXrlzRZDXTpYRt6+DggL29PR8/fqR///5YWlpSo0YNnJycyJEjB9WrV0801k3O1eTp6+szd+5cDh8+zMKFC6lfvz4LFixg+PDhLFu2DF1dXbS1tYmKilI7Tm40xI+m0UkM8ReIAwcO4OjoyODBg6latSrr1q2jatWq7Nmzh61bt7Jnzx5iYmJQqVQ8ffpUFqUWP9SXA5bDwsJYsWKF8njKx8cHgA8fPgAoF+4v77TlB/EftWvXZtCgQQBMmDABf39/9PT0vnmcSqUiIiKC9u3b8/btW2bPnq2kHRL/SHiurVu3jrVr15I5c2YmT57M27dvmTVrFvXr16dmzZqEhIRIGqZ/4cOHDxgYGHD79m0lePPx8WHZsmXo6OjQtm1batWqleg4Cd7Ej/ZDE/k2btyYZcuWsXjxYkaOHJlkwlMhfqSEPRHFixfnw4cPPHv2jIiICCwtLenUqRPt2rVj9+7dTJ8+ncyZM+Pn54ehoSEtWrSQH8Ik6Onp4evry2+//UZkZCQlS5akcePG/2rt4ooVKyq9RmFhYZqqaoZiZ2f3f+3de1yO9//A8VdnOgxlIWdJzmZOI4ecN0qRU2dJci6HORVpZSrHiJpTchjm/EXW0JizzWlzqhkW5ZxD6UCH3x/9uibnbdZdvJ9/cd2f6/Z2Pa77vt735/D+4OrqSkpKCiEhIfz222+A9La9zsuujbq6OlFRUVStWhUTExMCAgKIiooCoEKFCoSFhbF161ZWrVqlipCFUBT6Tgz5SdyyZcuYNWsWd+/eLcx/XoiXmjJlCk5OTjx69IiUlBScnZ1JTExUkjgvLy8yMjLYtWsXurq6eHp6Kr3G8nB8kaamJv/73/9o1qwZkZGRjB8/Hni7ZMLNzY1x48bRv39/JQkRb6dnz544Ozvz6NEjZs+eLdfvNZ69Fxs2bEhKSgqZmZkkJSVRsWJFNm/eTFpaGl27dkVDQ4MSJUoQHh6OgYEB1tbW0uMmVE7zXbxJmzZtSElJ4fTp08qxV31Rf//99zg5ObFixQo++ugjJk2axMOHD99FGEL8I61atcLKygoPDw/KlCmDo6MjsbGx9OjRg7i4OFauXElOTg6DBg3i/PnzhIeHA7I6+lU0NDQoVaoUv/76K1euXKF+/fqMHz+ekJAQcnNzX7huz35XuLq6MnXqVEaNGiXJx//7Oz8StmzZQm5uLqNHj8bW1lau4WvkX1M/Pz969+6Nmpoa8fHxhIeHs3v3biZOnMjSpUuJjY0lOzub1NRUSpYsSZcuXWRvU1Ek/OseOAsLC8aPH4+JiQlnzpxh27ZtfP/99zx9+vS1DzgbGxsGDx6MlZWV9GCIQvX8A7FFixa0bNmSefPmAWBiYsKsWbNo0qSJksRVrlyZNm3asG7dOvnSfolXJRkGBgaMHz+ezz77jN27dxMSEqK8VrlyZRITE5Xr6erqyrRp0xg5cqSyCbv4S/369SlRogSnT59Wiva+Stu2bTl48KDcq2/QvHlzIiIiGDZsGFWrVqV169a0adOGL7/8kt27d2NkZET//v1RV1fnxo0bbN68WVabiiLjnQyhamtrY2xsTEBAAIaGhmRkZODm5kZaWtpbbUQvw1BCFUaMGIGZmRkNGzbk7NmzeHt7K1/KJiYmzJw5k8aNG9OnT58CKyHll3dBz6+IrFWrFurq6uzZs4cjR45QqlQpxowZQ/PmzTl06BChoaFERUVx7do1vLy8AHB3d2fy5Ml4e3uzfft2Vf53ioT8/XVjYmIA8Pf3x9bWFiMjI06ePElERAQxMTEvJBHPf5fKvfpq/fv3p169ety/f585c+YAULt2bTw9PenQoQOTJ09m586dL5wn11QUFe9kqefTp0+5fv06w4YNIzw8nDJlyrB//36MjIxeuqL0+WRNkjdRGJ5diTdy5EhGjx6Njo4O6enpWFlZ0bJlS+X1pKQkxo0bR0JCwgsFeuXLu6D8z6+/vz/+/v58+umnfPbZZ/zvf/9j9OjRPHz4kLlz53Lo0CGsrKw4dOgQpUqVYty4cUBeD+jo0aMZM2aMJG9AqVKl6NevH0OHDqVt27Z06dKFjh074uXlhZWVFVlZWYwcORI7Ozs0NDQKnPv8d6ncqy9XsWJFZb6ggYGBcvzixYtERESwd+9eAgICXrqVo1xTUVT8J4sYzM3NmTt3LqVLl6ZDhw4v1MsRQpWqVavG8OHD2bRpE0ePHkVXV5cFCxbQunVrXF1dOXr0qNLWyMiI5ORk+ZHxBu3bt2fRokX079+fM2fOADBgwACCg4Px8fFh6dKlGBgYUKVKFapUqUJMTIzyIKxevTq6urpS7+0Z5cuXJyoqijt37nD+/HkeP35MaGgokDcsvWjRIj7++GOWLVvG5s2bZTjvH2jVqhXDhg2jWbNmODs7c/z4ceU1c3NzJkyYgIaGBq6uriqMUohX+0c9cJ999hlNmzZ96Ya/AHFxcYwfP56UlBQCAwNf+JUohKp069aNn3/+mS5duij3b1paGp6enhw8eJAVK1bQokULpf29e/fIzc195b3+IRo7diy1atUqcKxMmTLcunWLuLg45VqtWLGCgIAAfH19qVGjBikpKZw7d45du3Yp84gArly5IsnbM9TU1Lh58yZubm5UqFCB0aNHY2ZmpryekpLCsGHDuHPnDgMGDMDR0VHqZv4Dhw8fJjQ0lKNHjxIUFESzZs2U1+Li4pg2bRoDBgxQXYBCvMHf/tT37NmT7du3M3PmTBo2bPjKdhcuXGDjxo3UqFGDihUr/qsghXhXoqOjiYqKwsTEhHr16qGjowNAVlYWnp6eHDhwgB07dlC3bt0C50kPXJ6PP/6YiRMn8tVXX1G9enXleHZ2Nubm5hgaGpKbm4umZt4C9x9++IGUlBSMjY1feC/pNSooP/HNzc3FyMiIpKQkHBwcOH78OI0aNaJjx45K2/wkLjc3l0aNGsmw3j/0888/ExERQUJCAsHBwTRt2lR5LSEhQX68iSLtbyVwtWvXZsSIEcyaNQtNTU0WLFjAJ5988tK22dnZrF27lnLlyuHm5vYuYhXib3nVF++4cePYtGkTvr6+dOrUSdkhICsri6FDhzJnzhwuXrxYmKEWC2pqaty5c4cmTZrwySefEBwcrPQM7d+/n2PHjhEcHEylSpWUVZJpaWnKYibxas8uPvD29iYsLIyaNWty69YtBg4cSEZGBiNHjsTS0lI5JyUlhT59+ihzCcU/c+TIEb755huuXLlCZGQktWvXLvC6/HgTRdXf+lbV19fn6NGjrF69mrZt26KhoUFoaOgrk7jU1FT8/f2pWrVqgYmiQvzXnn0gWltbM3r0aAYOHKjsazpkyBB2795NWFgYnTt3LpDEzZgxo8AQn8iTnxAnJCRgY2NDixYtGDt2LDVr1uTBgwdERUVhYGBAeHg4lpaWWFpaMnPmTB4+fFhgXqF4Uf69OnXqVDw8PNi+fbvSq3br1i2cnZ3R19fH29tbuYcB0tPTpZfoJZ7dfq106dIFXnvZtTpy5AirVq1i3bp1xMfH/9fhCfFO/K1FDCVKlMDY2JiEhAQgb9Pf2NhYsrKy8PLyUgr5lixZkvT0dADq1q1L//79mTlzJikpKe/+fyDEa0ybNg17e3t+++03qlatSnZ2Nrt27cLf3x+Ab775hg4dOjBx4kS2bdv2xvpaIi/J0NTUpFu3blStWpXY2FjGjBlDYmIiXbt2xcHBgS5duhAfH8+9e/fo27cvWVlZUn7hDZo1a0ZERATe3t4cOHBAOZ5fcyx/YYOuri6jRo3i1KlTKoy2aOrVqxdbt25V7rPRo0fz+eefk5qaSkxMDFFRUWRmZr7xXpR7VRQH/3gVqpaWFk+fPkVLS4t9+/YpS9tv3brFtGnT+PHHH/nuu+8AqFSpkrKvoRCFpXPnzsybNw83NzeOHz9O+fLl6dWrFx4eHqxdu1YpKrt27Vo0NTXp06ePiiMu+jw9PRk3bhwODg48efIEQ0NDlixZwunTp/H29lY+56ampqSkpHDnzp2X7r4gXqzZ9sUXXxAQEED79u1f+LGb/31bqVIlxo0bx5gxYyTBeE7fvn2ZMGECGzZsICgoiP79+xMQEMDMmTNp27YtRkZGXLx4kUmTJpGRkSFJmij2/lUZkfwvZS0tLWJjY5XJyxoaGrRq1Uq+sIVKeXh44ODgQMeOHZUvaiMjIzw9PWnVqhUeHh7cuHEDkGLSb2vhwoVkZ2czatQo5Zi5uTnR0dEcOnSI6dOnExcXV+AcubavN2DAAOLj4ylZsiRz5szBwcFBWZWbf+0cHBw4ffo058+fV86TBKSgUqVK4eXlhYWFBT/99BPq6uqcPHmSnTt3oqGhgYeHBz179uTixYtMmDBBkjhR7P2rmcXZ2dmoq6vz9OlT+vXrR506dXjw4AEWFhbKa0IUhmfnteT/+ebNm+jq6hYowXDv3j1iY2Np1qwZ5cuXV47LPKK3U6ZMmQLzWbW1tYmLi2PBggV88cUXBAcHY2JiUuAcSd4KevY+8/DwYOLEiSQnJ5OUlISamhr29vZUqFABQOm97Nu3Lz179izwPpJ4/EVTU5OHDx8SGhrKwYMHad26NX369CE5ORnIe1ZFRkayZcsWzM3NmTFjBiVLlpRrKIq1f51h5eTkYGRkRFRUFPHx8VhbW5OVlYWGhoZ8OESheLaHx8bGBgsLC0qWLMmlS5fQ0tKiX79+lCtXTml/+/ZtLl68+MJ8N0k03mzt2rV07NhRqVD/5MkTAO7fv8+mTZvIyMhQejXFy+XfZ/Xr16d8+fJMnjyZixcvcuHCBQICAnB2dmbixIk4ODjQuXNnNmzYQOnSpQkKClJx5EWTpqam8lkuX748X3/9NQcPHqREiRL069dPSZgzMzNZvnw5mzdvpk2bNnh6eqoybCH+Nc138SalS5cmPj4eLy8vsrOzZb6LKFT5D0Q/Pz/69u1LUFAQFy5c4MKFCwQFBRESEoKBgQEHDhzgypUrTJ06lYyMDM6ePaviyIufI0eO8O233+Lj44OWlhabN2/mo48+4vPPP2f79u18++23gAybvknTpk3ZtWsXWVlZeHt7K8c3bNhAeno6Li4u+Pv7k5CQwK1bt+jUqZMyqiE/jP9ibW1NmzZtGD9+PIGBgXTs2JE2bdowf/581NTUsLS0ZOLEicyYMQPI+8ERFRXFrVu3ZNs2Uey98620JHkTquDq6sqECROU+UNPnz5VXuvZsycDBgygfv36JCUlcf/+fXr16kVWVpYkGv9ArVq1sLe3Z8iQIVy/fh1NTU1SUlLo0KGDrOL9GwYMGMDMmTNZuXIl06dPV4b7AHR1ddHX1wfyeoxBvluflf+57dq1K6tXr+bUqVOYmZnRvXt3ZZ6ggYEBo0ePpnXr1vz4448EBQW98FmXhFgUZ//JXqhCFLbQ0FCePn1aoKjpsw88AwMDDA0NKVGiBPHx8bIy8jmdOnXixIkT3L9//63aa2pqUqtWLRo3bkxmZiZbtmyR3vdXeN01GTp0KF999RVfffUVK1aseGWpJfmh8ZeVK1cyd+5cpYxK/pDopk2bGDJkSIG2+Ulcy5YtOX36NJMmTVJFyEL8J97JEKoQqqSlpUWDBg1eKBabnZ2NtrY2tWvX5tKlS/z555/Ka2pqapJo/D8HBweCg4Px9/dnw4YNPHz48I3nZGdnc/78+RdWRco1fVH+NXFwcKBOnTqoqalx5swZNmzYQHh4OFpaWkydOpXc3FyioqJemsRJ8vaX5OTkAtMfvv/+e/bs2YOPjw8PHjxg6tSpPHnyBA0NDVJSUpg7dy66urro6uqqMGoh3j1J4ESx9/TpU3bv3k3fvn1Zs2ZNgY3RK1eujLu7O4sWLSpQ3kIeiH/59ttvqVu3LkOGDEFNTY2NGze+sSfuZddPrmlBPXr0QFdXl3Xr1jFt2jQcHR35/vvvqVu3LpaWllhbW+Pi4sL8+fPJycnB19cXfX19QkNDlULo4i/5w535cwaHDBnC+fPnWbx4MQBXr15lyZIlAPj6+irD+ebm5kycOFElMQvxX5I6H+K9sHfvXq5cucKUKVOoX78+AGXLluWrr77C1NRUtsd5hfwtxHx9fdmzZw9ubm707t2bjz766K3fI3/vSEng/jJgwACWLVtGQkICTZo0oWfPnjg5OTFy5Ei6du3KnDlzqFKlChEREQCEhYUxa9Ys2rRpI8nbK+TfX/mrSu3t7YmIiMDCwgINDQ2io6MZNGgQTk5OhISE8Mknn7BmzRqmTZumwqiF+O9IAifeC8ePH2f58uVkZWURHR3NwYMH2bZtG+XLl8fW1lbqvL1CfhkQBwcHbty4QZUqVRg3bhz9+vV7qyRuwIABbNu2jWrVqv3HkRYfDg4OfP311wwcOJDDhw9TuXJlNDQ0lB7grKwsdu3aRVRUFLVq1aJWrVoAzJo1i+7du6sy9CItP4HL3wu2Xbt2xMXFER4eTsuWLdHQ0GDXrl04OTlhZ2dHWFiY8vkX4n0kixhEkfeyCdyvmtRdoUIFGjVqRNWqVbl9+zbbtm1TNqaX+Vkv9+WXXzJkyBDGjh2LhoYGX3zxBW3atGHWrFmsX7+eR48evfQ8V1dX/Pz88PLykpIM/69Pnz4sWrSIoKAgZs+eDeSVDFm0aBFjx44tsMdp5cqVOXbsGIMGDSI6OlpVIRcr1atX5/jx44wfP57IyEgAtm7dSo0aNRg2bBhHjx4lKyuL8uXLU758ec6cOSMLlsR7S+bAiSItfw9IADMzM7Kysrh27dorS4DcuHHjhUKyMrn+1UqXLo2VlRUzZsxg69atAGzatInZs2czefJkcnNzlYUNz15vV1dXpk2bxqhRoyR5+3+urq7MnDmTX375heHDh3P48GGOHDlCYmIi6enpDBgwgBs3bnDp0iUgr/czLi6O1NRUFUdedD1f5uP27duEhYXRpk0bDh8+TFxcHLa2tmzdupWwsDBGjBjBsWPHuHnzJjdv3gRkwZJ4f8kQqiiSpk+fTpkyZZTkbcqUKWzdupXNmzcTExND+fLl33rOldR5erX8id75DzgdHR0Axo4dy/nz5/Hw8MDNzQ0DAwPleru5uTFlyhRJ3p7h5uZGcHAwAwYMwNraml27dvHdd9/RqlUrbty4gZeXF61atcLf35+hQ4fSvn17wsLCyMnJ4eDBg6oOv8jK/+x27doVNTU1Hj9+zM6dO6lduzatWrVS2tna2vLHH3+wYcMG6tSpU+A9ZG6meF9JAieKHBMTE3r06MG2bdswMDBQ9jX09vbGz8+PlJQUdu/ejbm5uapDLVZeNgcwNTWVpKQkHBwcgLzthjQ18zrmr127RsmSJalTp45S2qJdu3Z8/fXXjB49WpK3/6erq4udnR0eHh5ER0eTlZXFlClT+N///sf69euxsLDg9OnT9O3bl6dPn+Lu7o6fnx/Z2dl07dqVnJwc2Tf6NaytrVm9ejUbN26kS5cunD59mrCwMAIDA6lRo4bSzs7OjhUrVsgOK+KDIXPgRJFUq1YtwsPD0dTUJCIiAn19faVEgJGREYsWLaJevXrY2dkVKA8iXu7Z4c9GjRqhpqaGjo4Ox44dw9TUlE2bNnHhwgXs7e2VYaslS5awdOlSjh8/XuBcTU1NTpw4ocr/TpHxukr+hoaGBAQE0KNHD/r378+hQ4fQ1dVFS0sLPT09kpKSANlh4XnPT42oVKkSu3btQltbm+3bt/PRRx+xcuVKbG1tMTQ0xMvL64XaebLDgvgQSAInipRnv7zNzMxYuHAhjRs3Zu7cuXz99ddKO0NDQxYtWkTt2rVxdHQsUPtNvJqPjw/dunVDU1OTkiVLsnfvXqZNm8ann37KrFmzyMnJIT4+ngoVKqCnp0fLli2VHiJ5IL6as7MzAKtWrSqQkOUncdbW1vTt2/eFYtOyw8KrVaxYkYcPH5KamoqVlRV9+/Zl37596OrqMm7cOC5cuECpUqXw9/cnJiZG1eEKUeik314UGZUrV1YeZra2tly/fp2RI0dy6NAhbG1tMTIyUtomJyczdOhQ7ty5I0U639KIESNwdXXFy8uL1q1bs2bNGpycnKhcuTI//vgjnTt3Zvv27Vy5coWffvqJVq1aSfL2lnr27EmvXr0ACvSmJScnK/M3t2/fTt26dQucJ8nby/Xo0YMffvgBLy8vqlevzo8//si9e/fIyckhLCwMV1dXUlNTMTMzo1OnTqoOVwiVkB44USS0bNkSX19fQkNDadu2LZ6enjRq1IikpCRlOFVHR4du3boVKGthYGBAamqqPAifo6mp+cJK3YiICH766Se+/fZbrKysCA0N5auvviIqKooSJUqQkZHxwvvI8N7r5Se3devWZdWqVUybNu2lcwPLli2Lq6sr8+bNk+v5lsaMGcMnn3xCvXr18Pb2pmbNmnh6etKnTx+uXbtGxYoV+eSTT9i1a5f8wBAfJOmBEyplaGgIQGJiIikpKYSEhGBvb0+bNm2UOULx8fEMHTqUJ0+esHPnzgIFZlNSUqRI73OCgoI4duwYOjo6yrUpUaIETZs2JS0tDQsLCxYuXEhAQABRUVFoamri7e390p4MSTZeLz9xuHXrFvHx8Xz22WfAiwtG7t69y+zZs8nOzkZDQ6PQ4yxO8hd0zJkzB39/f7Zu3crq1aspV64curq6TJ8+HT09PRITE9m5c6dS51GID40kcEJlZs2axZAhQ1BXVychIYHjx49TtmxZLl++XGB1GeQlcUOGDCEzM5Pjx4+jp6dX4HXpgfvL+vXryczMZNu2bUoSl5GRwaZNm3B0dGTt2rX4+PiwYsUKIK8W3CeffEKVKlVUG3gx4uLiwuTJkzEwMEBTU5N79+6xfv16BgwYQMOGDV97P0pS/HrP9qb98ccfBAQE4O7ujqmpKRkZGXzxxRcFSoiAXFPxYZIhVKEytra27Nixg6ysLLS1talZsyZGRkZ4enqir6/PypUr2bx5c4FzateuzYgRIxg1apQMm7xG/fr1Wbp0KQ8fPqRHjx5kZmbSrVs3AgMDuXLlCuPGjePKlSsYGxsTGhpKqVKlsLKykmv6Cubm5hgZGaGmpsbFixcZNmwYjo6OXLp0iYsXLzJ79mxSUlKYNWsWCQkJBAcHk5OTIz8s3iETExOaNm1Kjx49GDx4sNyr4oMnCZxQOQcHB7p06cLkyZNJSkrC1NSUwMBASpYsSWRkJNu2bQPAw8ODlStXkpmZCUipgDfJT+JSUlLo3r07T548wcnJieHDh5OTk0N6eroy/NS1a1eysrLkmr6Evb0948ePR0dHh48//piIiAjmzZtHZmYmAwYMoEOHDtStW5d169ZhYWFBeno6/fv3Jy0tTdWhF1nNmzfn9u3b3L59+x9fJ7lXxYdOEjhR6J4vnTB48GB69+5NfHw8M2bMIDExkRo1ahAYGIi+vj4///wztWvXplmzZtSuXVu+tF/iVfvF1q9fnyVLlpCamsoXX3zB06dPadGiBVWrVqVatWr8/vvvsl/sazg7OzNz5kyGDRvGtWvXqFWrFrNmzWLu3LmEhIQUaGdubk7fvn0pU6YMISEhzJw5U4WRF13NmjUjOjqadevWUbFiRfz8/Lh27RoPHjxQdWhCFCuSwAmVsbOz4+LFi5w7dw53d3d69erFn3/+yfTp00lMTKRatWqMGDECU1NT0tLScHV1feUeqB+yZ69HzZo1ycrKIj09nVu3bqGmpka9evVYunQpjx8/plu3bkoP5rOkN+NFNjY2LF26FFdX1wKbza9YsYJKlSphY2PD48ePlePq6uqYmZkxefJkSpQoQf/+/eU+fYlGjRoRHR3NpEmTKF++PLa2tpw/f56DBw8q8zJB7kkh3kQWMQiVKFmyJL6+vowcORKAZcuWsXXrVqpWrYqPjw8VK1bk6tWr+Pn54ejoiKOjI1lZWWhoaMhD8Tn51+PLL79k1apVfPfdd8TGxmJpaUlubi5nz57F3d0dXV1dtm3bRsmSJV94D3lQvkhfXx8AY2NjZXsxgPT0dO7fv6/sI5svNzeXuLg4vvrqK9q2bUv79u0LNd7i4syZM4SFhVGzZk1CQkKYNGkSu3btws/Pj40bNzJp0iRKlCgh96QQbyAJnCgUz5ZVUFNTIz09HQ8PD7p06aLsw7lkyRI2b95MlSpVmDRpEpUqVeLx48cF5sjIEN/LjR8/Xtlk3tbWllOnThEZGUnfvn0BOHfuHIMGDaJmzZoFdrQQr7ZmzRrGjx+vDKECdOvWjV69ehEeHv5CT2Zubi7q6ur88ccfnDx5klKlSqki7GLh999/p02bNpQtW5b9+/ezadMm7t+/j4GBAZ07d+bnn38mNDT0hdXoQoi/SAInCkV+L5GrqytffPEFxsbG/PLLL0RFRdGtWzdq164N5PXEbdq0iaZNm9KvXz9VhlxsNGzYEAsLC4YNG8aePXto2LAhn332GadOnWL+/Pn06dMHyEviOnfuzNixY1UccfERGRnJxIkT8fX15ZtvviE0NJSxY8cSGxv70tqDOTk5ODo60rx5c06dOqWCiIseS0vLF0rUbNy4kbS0NKUHfv/+/SQkJODs7EzHjh3ZuHEjubm5XL16VQURC1E8yBw4UWjMzMzYv38/t2/f5sSJEyxYsIDU1FQWL17M8uXLWb16tdLWysqK6OhoGUZ5iefnAJqamtKxY0cWL15M69at+eabb5g7dy5Lly5ly5YtNGjQgK+++oqVK1cq58j8or/H1dWVWbNmERMTg5OT02vb6unpUalSJeLi4gopuqJLW1ubAwcOkJubS+/evbl+/bpy/1pbW+Pk5ETdunW5evUq7u7u3L59+4X3kDmvQryc9MCJQnPz5k2+/fZbbt68yS+//MKOHTto1KgR8fHx+Pr6YmJiorTdsWOHsg+n+MuzD7NmzZoBecVO169fD4CjoyPR0dEsX74cgKSkJO7evav0wuWT5O3viYqKYuzYsXTt2pXhw4e/sp2GhgaPHz+W5O3/PXnyBBsbG9LS0li1alWB/Y5PnDhBtWrVSEtLw9raWknenu/ZlORNiJeTp6P4z3Xp0gUzMzNSUlKYP38+1apV49q1a/To0QM7OzuysrIwMjIiODgYXV3dAudKolFQ/sNs8uTJLFy4kAEDBgDw8OFDdHV1qV27Nrdu3VLKgujp6TF8+HCsra1VGHXR1aBBgwI/HODFBCLfypUrmTBhAr6+vkycOPGlbWSO5otu3ryJjY0NOTk5LFiwgKpVqwJ5Py5mzpxJdnY2derUUdpLwibE25EETvyn6tSpw8iRI9m6dSs2NjYkJCQwZswY3N3duXPnDmPHjmX//v3cuXOHUqVKSfHTtzBu3DhcXV0ZOXIkMTExyvG0tDQOHjzIqFGjCAwM5Pvvv6datWrKXCzZL7agLl26sHTpUlasWMGcOXNo0KCBssr5VT2/y5cv5+uvv6Z169aFHG3x8ezijfzVuykpKdy8eRMLCwuWL1+uzIk7f/48mZmZtGzZUiWxClGcyRw48Z+rUaMGdnZ2jBgxgg0bNhAfH4+xsTGJiYlERkYCeWVFMjMzpcftDYyMjIiKiiIqKooNGzYox/OL8Orq6jJ+/Hjq1KnDnTt38Pb2lh0WXsPY2JgKFSowZ84cUlJSuHTpEr6+vmRkZMg1+weaN2/O3Llz8fLy4pdfflGOR0ZGUr16dby9vZk7dy5qamo4OTlx/fp1Fi9eTJkyZV4Y5hdCvJ4kcKLQdOrUid69e2NqakqNGjW4fv06jo6OXL9+XWkjD83Xq169Ovv378fDw6NA7xvkTRh/8uQJkDeRPr/IrOyw8Gb6+vo4ODhgZ2dHRkYG/fv3Jz09Xe7Hv8nS0pKhQ4diaGjIyJEjuXjxIitWrMDU1BQHBweuXbuGsbGxMmdzwIABPHjwgEePHsnQqRB/kwyhikKzZ88eAgICmDt3LgkJCdStW5ehQ4cWaCMPy788O+SZP6T34MED4uLiqF27Njo6OgXade7cmQkTJgAU2CFAkreC+vXrV2BOoJqaGqmpqSxfvpyZM2eiq6tLVFQU2tracj++pYoVKwKwb98+wsLCuHHjBgsWLGD79u1UrlwZR0dHrl27BsDt27fp27cvxsbGjB07locPH5KbmytD/EL8TZLAiX8t/4v3bVaMJiYmEh0djbW1NYGBgUydOvW/Dq9Yena1qaenJ4MGDcLAwID79+8rRXnbtWunzNkqUaIE9vb2mJubqzjyos3FxYWwsDDS09OVY/lz3rKysoiNjWXevHno6+vj6empwkiLj549e7J3716cnZ0BOHDgAMuWLePGjRt88sknzJ49m4SEhAIJ2p07d7CwsMDb21s5Jj1wQvw9MoQq/pVu3bpRt25dVqxYwd27d9/qnOeHpWSI79X8/Pzo27cvoaGhbNu2jVu3bgGwevVq6taty5kzZ7h9+zYNGjTAwMCA9u3bv7DFk8jj6upKUFAQQ4cOZevWra9sp6Ojw5QpU6hbty729vYv3TtW5DEwMGD58uW0atWKX3/9lc2bN7NkyRIA2rZty6BBg6hYsSJffvklJ0+efGlNNxmmFuKfkQRO/GPly5dn3759pKamoqamxrp16zh58iR79+5V2siX8z/n5OSEr68vvXr14vz580DeYo/83iNXV1eaNWtG6dKl+f333wkMDCQ7O1sS4pfo1KkTa9euVTamr1mzJj179sTc3Jw///yT6OhoTpw4obT/6KOPOHToEAsXLiQiIkKFkRd9Y8eOZejQoaxdu5amTZuyYcMGpQ5hu3btcHd3p2LFiowbN052pxDiHdJ8cxMhXi4tLY1Dhw6xfft2bt26Rffu3Vm8eDGbN2/m8OHDbNmyRZK3f6Fq1ars3LmT8+fPU7NmTSwsLBg0aBD37t1j48aNymrUZ3s1JHl7kYaGBnXq1OHatWvUqVOH+Ph4Vq5cSVJSEmlpafTq1YvmzZuzbNkytm7dioaGBo8ePSI0NBQzMzNVh19kaWpqkpWVxaJFi2jdujW5ubmcO3cOZ2dncnJyWLFiBfv37yc3N5eBAwcSFRVF7969iY+PV3XoQrwXZA6c+McePXpETEwMQUFB/Pnnn/j6+mJhYYGBgQFhYWHs2LEDKyurF/ZBFG9HW1ubPn364O3tzeLFi+nUqRPff/899+7dY8CAAZQuXRooOHdIkrcXZWdnExUVRUREBL179+ann34iJiaGAQMG4OLiQufOncnKylLmcOVfwwsXLqCtra0sFhF58gsf5w/V5+TkcPr0aZ4+fcqcOXM4efIkrq6uSpHpn376iTVr1rBhwwYuXbqkqrCFeO/IEKr4W/J/decPjWpoaLBw4UJOnjzJ4sWLAThy5AhxcXFkZmZSvXp16tati6ura4GhVfF28nuB/ve///Hjjz8SFxfHZ599RmBgIE5OTty8eVPVIRYbH330EU5OTlSuXJmFCxcW2JfTwsKCrVu30rp16wLbYFWvXp0rV66oMOqixdbWltmzZ7N9+3YiIyNJSEjg/v37NGrUiK1bt9KvXz/+/PNPvvzySxo3bszKlSuJiooq8B4yrUKId0OGUMVbs7S0pFWrVoSHh3P//n0gr7ciISEBa2trFi9eTGxsLMnJyYwYMYLU1FSaNGlCkyZN2Ldvn2qDL2byEwsvL68CNd00NTUZPXo0N2/elOTtb3r06BGrVq3CxMREqT2Y33tpaGjImTNnXrimkrz9pUyZMvTt2xdtbW2sra1RV1endu3azJw5k6NHj7JgwQK6d++On58fS5YsYdCgQYwdO5Y7d+4QHR2tvI8kb0K8G9IDJ95aYGAgHTp0YNOmTSxbtowHDx4AeXOMfvzxR+rUqcPRo0dxdXUlOTn5hfNlftY/p6urS58+fejWrRvly5enY8eOZGVlvXRVn/h7tLW1Wb58Oenp6Xh4eKg6nCKtTZs22NnZ0aBBA9auXUtOTg6enp6cPXuW2rVrk5ubi5WVFQ8ePKB27dp06tSJRYsWSdImxH9AeuDEW/P19cXPz49u3bqhrq7O4sWLefjwIWpqauzcuRNNTU08PDxemryBzM96lpaWFk+fPgUK7poAvDQp09HRwdjYmDt37uDg4CCrTd8BPT092rVrh7OzM5UqVcLS0hJ4+fUXeQ4cOEBubi56eno4OTnh7OxMdHQ0zZs3p3bt2lSoUAFDQ0MePHjAxYsXuXjxIiDDpkL8F6QHTryVZ5OFSZMmYWtry4YNG1i2bBn379/H3NycvXv3MnbsWGWbHPGidu3aceDAAeVhNnz4cNq2bUtKSgobNmxg7969r+xZezbpkwfiv/fxxx8zc+ZMNDQ0cHNzIysrS5Lit9SyZUuGDh1K5cqVGTduHCdOnEBPTw8DAwNu3rwpSbAQhUBWoYpXqlGjhvLnZ7+MzczMKFeuHN27d8fDwwMjIyPi4uJYunQpnp6eyio1UdDw4cMJDg7G3t4eQJkjdPLkSWrUqMHo0aMZNWoUWlpaL91aKD95A5lH9C7cuXOH0aNH4+LiIsnb33TkyBHCw8P5888/mTVrFi1btuTx48eSvAlRiCSBEy9lamrKsWPHGD58OBoaGkrCEBUVRY0aNWjVqhWxsbF07dqVgQMHoqenxy+//MKNGzdISkpScfRF04YNGzh79iz29va4urpibm7OwIEDCQ4OpkuXLpw6dYouXbq8NokT79b9+/eV6yzJ299z5MgRvvnmG65cuUJgYCAWFhaAbIklRGGRIVTxSl5eXowfPx4fHx9WrFhBZGQkNWvWxNnZmatXrwJ5Wz21adOGAwcO4O/vr5wrv8ILyh/yNDIyYtasWZQtWxZjY2MGDhzIuXPngLw5WT4+PjRu3Jjdu3cTFhbGkydPVBy5+ND83c/uZ599xsSJE7l+/TojRoz4DyMTQjxLEjhRQL169fj999+VxGHYsGFMmzaNy5cvk56ejrOzM9evXy8w3DR79mx0dHTky/sVnn8gfvzxxwQEBNCtWzfmzZvHnDlzlNd0dXWZPHkyXbt2Zc6cOaxdu1YVIQtBq1at0NPT49y5c9y8eZOcnJxXJnf16tXj/Pnz8qNNiEIkCZxQ2NnZERERQVRUFBMnTlQqrbu5uRESEsLs2bMJCgpS2stE+jd79oHXu3dvEhMTOXLkCGXKlCE4OJjKlSuzZs0aVq9erZyTv8JvyZIlcn1FofDx8eHu3bt88803AAQEBNCzZ08MDAyIj49n06ZNLF++nCdPnry2h0563oUoPFJGRCgMDQ0BcHFxQU9Pj+HDh5OTk0NkZCTa2toEBASQnJys7Ljwul/kIk/+tfHz86N3794sW7aMCxcucP/+fSZNmkRISIiyqCE/iXv8+LHyIJUkWfzXDAwMaNKkCVpaWqSmpnLlyhU+++wzBgwYoBTltrW1RU9PjwULFrw2iZPvAiEKj/TACUWDBg3w9fVl3759DBs2jOPHj+Ph4aEkEEOHDmXatGn4+vqyZMkSFUdbfAwcOJAJEybQp08fLl68WOABaGRkRHBwMMbGxuzYsUNJjoUoTIaGhgQHB1OqVCmuXLlCeno606ZNA/KG9X18fGjatCk//PCDksQJIVRLVqEKxW+//UZmZibNmzfHxcUFCwsLIiIiUFfPu03Cw8Px8/Pj66+/pkePHiqOtvho1KgRa9eu5ddff1WGpfPdu3eP8ePHk5WVRa1atVQUofiQqampkZyczKRJk0hNTaVfv37UrVtXeT0tLY3AwEB+/vlnOnbsyKRJk9DUlMEbIVRNErgPWMOGDdHT00NbW1s5Nn36dAwNDcnNzcXd3Z1OnToRHh6uJHEREREMHjyYnTt3qirsYqNx48ZA3gRvIyMj4K/6bbm5uWhra1OrVi2Sk5Nxc3Pjyy+/VFms4sOTX6ImNzcXExMT7t69y5gxY9i1axdVq1bFzc1NaZOens706dP5448/KFWq1As/RIQQhU8SuA+UjY0Ne/fuZeXKlQQFBWFqagpAQkICT58+pWPHjhw6dAhXV1c6duzIokWLlCRuy5YtylZO4uV8fX2ZMWMGJiYmxMbGYmpqyieffFKgTbVq1fD19cXc3JyHDx9K3TdRaJ6dwzZ27FjCwsJo3LgxDx48wMfHh99++w07OzucnJyUc9LT0xk3bhxjx45VVdhCiGdIAveB0tXVBaBMmTJoaWmxc+dO/P39adasGSEhITg5OWFqasqBAwdwcXHBzs6OcePGFXgPKXz6cg0bNqRJkyb4+vqSlJTEjz/+SNmyZXF1daVFixYAlC9fnqlTp1K6dGl+//135VyZBC4KQ/595uvri7u7O6tWreL27dsAJCcnM2HCBG7dukW/fv1wdHRUzsvMzJQfGkIUEbKI4QNmb2/P/Pnz8fLy4v79+9SvX5/Bgwdz+vRpLCws8Pb25rvvvgPyFjicO3dOVkS+gbu7O23btkVLSwt3d3fS09MB6Nq1K2PGjFGGUlNTU8nNzaVz586v3PtUiP9SvXr1WL58OZMnT2bv3r3K8fwaj4aGhgQFBdGwYUP8/PyIiYlRYbRCiOfJTNQP2Nq1a9HX12fevHn4+Pgwc+ZM1qxZg5ubGxoaGvz2229K2/w/y36Rr5ednY2lpSWPHz+mZs2aynWLiYnh8uXLVKhQgcaNG3P16lW2b99OTk6OXFNRKJ7/kaCvr4++vj6//vprgXbZ2dloa2uTnJyMj48P7u7u7N69u7DDFUK8gfTACQYNGsSMGTOYPn068+bNQ11dHU1NTSkV8Aav6jXr1asXX3/9NTt27GDhwoVcuXLlle8hdd5EYRs5ciTXrl3j999/Z/PmzQwdOpTY2Fjgr/vR2tqa5ORkDh06pJwn96oQRYv0wAmWLl1Kbm4uQUFBZGdnS52nt5SfvNWvX5+SJUvy6NEj4uLi2Lx5M7q6ukycOJH09HSWLVum7B37PHkgiv/asz807O3t8fT0xMnJiQcPHnD58mX69u3L3bt3+fXXX8nJyUFdXR03NzcuXLhQIIGTe1WIokUSuPdYgwYNuHfvHklJScqxV/UaLVu2jNzcXKZPn46uri7BwcGFGWqx0qhRI86cOQPA1KlT6d69O8bGxiQmJpKYmEi/fv1YvXo16urqjBs3jpycHFauXMkff/yh4sjFhyj/8960aVPq1atHcHAwp0+fBmDu3LlMmzaNSZMmcfToUW7dukX//v0xNDRk6tSpKoxaCPEmksC9p7p06UJAQAAPHz7k7NmzREZGcv78ebKzs185FLJ8+XL09PTo2rWrJHCv4Orqyvjx4+ncuTNWVlY4Ozvj6urKw4cPqVmzJhMmTGDv3r107NiRlStXkpWVxdy5c7l+/bokcEJl6tWrx7Zt21BXVycwMFA5/sMPP5CWlkavXr0YPHgwV69e5datW/Tu3fu13xVCCNWTOXDvMWNjYypUqMCcOXNISUnh0qVL+Pr6kpGRIV/M/4CLiwuzZs3Czc2NnTt3smjRIhITE5k+fTqQ17vZqFEjIiIi+Omnnxg/fjwAnTp1IjY2Vq63UKlevXoxY8YMTp48iZ+fH/Hx8QVe19PTA/L24gVZsCREUSd14N5jt2/f5syZM9jY2BAdHU2DBg1Yv349JUuWVOa6iLdjY2PD7NmzcXFxUXahKF++fIEth3Jzczl9+jS7du2iVq1alChRAoA9e/bI9RaF5tkC28/ec5s3b2batGk0bNgQFxcXqlWrVqDd48ePleQNpM6jEEWdPFHeMx999BHGxsYFjqWmprJ8+XJmzpyJrq4uUVFRaGtrS4/QW3J1dWXp0qUvHP/+++8pW7Ys7du3L3D86tWr6OnpoaWlVeC4XG9RGPITr4EDBxIWFkZERARjxowB8koHzZgxAxsbG9zd3ZUkTu5NIYofSeDeI7169SIqKorY2FhWrVpFo0aNgLyhvaysLGJjY5k3bx76+vp4enqqONriYcCAAcrOFEFBQURGRtK7d28gL4HLzs7G3d0dKysr1NTUKFOmDNbW1ly5coWUlBQVRy8+JP3798fb2xsAPz8/JkyYwL1799DX16dv377s2bMHNTU1Vq9eTVBQENbW1owePZoKFSqoNnAhxD8ic+DeE/b29syYMYOZM2dy/fp1vvrqK2JjYxk9enSBdjo6OkyZMoW6detib29PZmamiiIu+tq2bcuqVasYNmyYMmw6ZcoUhg0bhpeXF9999x21atVS9jwtVaoUN2/eRENDg44dO8qG36LQuLq6MnPmTOzt7bl27Rrr1q3Dy8uLAwcOAHkrUOfOncv9+/fp0aMHkNdD1759e1xcXGQXECGKIUng3gOtW7cmPDycKVOmsHXrVgDc3NyoUqUKy5Yt4969e8qWTpA3zHro0CEWLlxIRESEiqIu+kqXLk2VKlX49ddfC0zonjJlCsOHD8fLy4v169fz8ccfU6lSJVq0aMGNGzdkhwVRqOzt7Zk7dy4DBw4kOjoaS0tLli5dSps2bbhx4waQN8etbdu2zJgxg0mTJrFv374C7yFbuQlR/EgZkWJOXV2dypUr88033/DDDz8ox21sbDAxMcHFxYVff/2VgwcPMnv2bAAePXpEaGgoZmZmqgq7WHjw4AFNmjTBwMCgwHBoQEAAAKGhoeTk5LBhwwbu3LnDqVOnlDbq6uqSvIn/XN++fZk/fz7Lli0jOjoagN9//52HDx9iaWnJ2rVrgbw5bufOnaN06dKYmJi88D6SvAlR/MgcuGIuJyeHHTt2sG3bNtLS0gBYuXIl1atXZ8KECfTu3Zu4uDi++OILatWqpZx34cIFtLW10dHRUVXoRV6TJk0ICgrC1NQUKLiiLyAggIULFzJ37lycnJxeOFcmhYv/mqurK/Pnz+eHH37AwcGBvn37Ank/0M6fP4+NjQ3t2rVT2mdkZJCUlKR8TwghijcZQn3PaGlp0atXLw4fPsy1a9cAMDU15ejRozg5ORETE6O0rV69+mv36fzQaWhosH//fk6dOsXIkSNf2iYkJITatWsr84qEKAyOjo7MmzcPV1dXoqOjlbmZ3t7erF+/npo1axIWFsaTJ084c+YMp0+fxsnJCUNDQ9q3by8/MIR4D8gQ6nvm6dOnrF+/vsCxEiVK8PPPP5OQkFDguCRvf3l+DpCmpiZZWVkEBATg4+PDp59+ysmTJ184L79YrxCFpUSJErRp0wYXFxd27doF5PUI5+bmEhoaipqaGuvWrWPo0KG4urrSqVMnmjVrxq1bt+jTp49Sk1CSOCGKN+mBe89pa2uzfPlyNDU1sbe3l7kub9C8eXOOHz+u/L1mzZpERUURFRXF4sWLVRiZEG9ebODr68uIESPw9vZm3bp1QF5Psp6eHo8ePVL+LvMzhSj+pAfuPaWrq0vbtm1xdnamSpUqtG/fntzcXFlt9pwSJUqgpaVFSkoKzZo1Y9u2bRw/fpydO3eyfv16Ll26xLJlyxg7dix79uzh8uXLqg5ZfMDyP7sODg6YmZnh7+9f4DOdv8/pnDlzyMrKYuPGjWRnZyvJG8gOC0K8L2QRw3tKV1cXW1tb0tPTsbS0JCsrCw0NDUnenmFtbc2SJUvYs2cP/v7+aGtr07RpUy5fvkyPHj04cuQIAwcOJCUlhcOHD9O8eXMA2RJLqFzz5s1p27Yt8OIK0sDAQMLCwggPD8fS0lIF0QkhCoMMob7HSpUqxcOHDwFkzstzXF1d8ff3Z+PGjWhra9OrVy+OHDlCv379UFdXR09PjyFDhtC4cWPMzMyoUqUKR44ckcUKQqXye9uMjIzYv38/oaGhLFmy5KVtXV1dWb16tfS4CfGekgTuAyDDpgU5OjoSEhLCwIEDlVW57dq1Y+PGjQwaNIht27YpbU1MTKhatSrDhw+ncePGBAYGKrW1hFCVEiVKEBAQgIGBAUOGDHltW5nzJsT7ScaCPgCSvP2lbNmyzJs3j59//lmpRq+mpsapU6e4du0aurq6yjGApKQkjhw5wogRI/j5559p0aKFqkIXH7DBgwcTGhqKubk5WlpaZGRksGPHDmxtbd84TCrJmxDvJ0ngxAfl7t27uLi40Lx5c/z9/SlXrhy5ubm0a9eOihUrcvr0aaBg0quurs6DBw/YsGED7dq1w9jYWEXRiw9Fw4YN6d69O927d6dSpUqkp6fz2WefMX/+fFavXk39+vU5duwY4eHhODg4YGBgoOqQhRCFTFahig/Orl27cHd3Z+XKlTx8+JCrV68SFBTEqFGjuHDhwgvt8+cONm/enJSUlAL7ygrxrjk4OODj48OTJ0+oVKkS33//PX5+fqxZs4Zu3brRp08f1qxZw6lTpyhdujQ6OjrKdm8yXUKID4fMgRMfrG7duhEVFQXA1KlTCQ8Pf2VbDQ0NVqxYwezZs5VeOiHeNUdHR2bPno2Hhwdnz56lSpUqfPvtt3z33XeMHj1aadelSxfq1auHp6cnRkZGrFmzBm9vb9UFLoQodJLAiQ+apaUlGzZsICIigtDQUO7evavqkMQHysbGhqVLlzJy5EjWrVun9KbNmDGDDh060LVrVx48eFDgHBMTEzw8PPj000/x8PDg9u3bqgleCFHoZA6c+KDt27cPFxcXPD098fb2ply5cqoOSXygUlJSADAzM6N8+fLKUKimpiYPHjx4YTGCmpoaSUlJLFmyhEaNGknNNyE+MJLAifdS/irSNx2DvDlx+Umcra3tfxyZEC9SU1MjNjYWJycnRo0apQyXfv755zg5OTFnzhwlwcuXv7NKUlISP//8M4aGhqoIXQihIjKEKt47WlpaPH36FMjrzcjKyuLatWtkZWW9dpJ3ixYt+OWXX6TsglCprl27snr1avbv30+jRo3w9/dn9erVryzGnT/02rJlSy5duqSCiIUQqiAJnHhvTJ8+nVmzZnH//n0ApkyZQv/+/Xny5AnJyck4Ojpy8+bNN76PFD4VqtapUyfWrl3LsWPHcHR0VHZUeZmSJUtSvnx5rly5UogRCiFUTYZQxXvBxMSEHj16sG3bNgwMDGjdujV9+vTB29sbPz8/UlJS2L17N+bm5m98L0nehKrt2bMHe3t7WrRowcSJEylbtuxL26mrq5Oeni7JmxAfIOmBE++NWrVqER4ejqamJhEREejr6yv7RBoZGbFo0SLq1auHnZ0dcXFxKo5WfIjatGlDSkpKgVI0rxvW79q1KytWrGDLli1MmjTptT1xQogPi/TAiWIvf3FCfHw8Q4YMITMzk/nz5/Pxxx8rbe7du8fQoUM5e/Ys69evp169eqoKV3ygLCwsGDduHEuWLGHp0qVYW1ujpaVFbm4uGhoaLz0nJiaGIUOGULVqVR49elTIEQshijLpgRPFWuXKlbl27RoAtra2xMTEUKVKFYKDgzExMeGLL77g3r17SvsyZcrw3XffcfPmTZydnVUVtvhA6ejo8PHHHxMQEIChoSEZGRm4ubmRlpb20kUKz/fOyU4LQoh8ksCJYqtly5b4+voSGhpK27Zt8fT0pFGjRiQlJSnDqTo6OnTr1q1A74WBgQGpqanyIBSFpkSJEmRkZCh/L1myJO3atWPMmDGUKVOGzz//nHv37r1ypakQQjxPEjhR7BgaGpKcnEyVKlUICQmhdu3aGBgY0L17dy5evKi0q1WrFhEREWhpadG9e/cXhqCkN0MUBhsbG6pXr863337L7du3C9x35ubmzJ07l9KlS9OhQ4cCSZ4QQryOzIETxcqsWbMYMmQI6urqJCQkcPz4ccqWLcvly5epUaNGgbbPzok7fvw4enp6BV6X5E381xwdHQkLC+PJkydKbcJn77u4uDjGjx9PSkoKgYGBr5wLJ4QQz5METhQrBw8eJCQkhJycHLS1tfn++++xt7fn1q1bDB48mF69ehVoHx8fz4gRI9izZw/p6ekqilp8iJo0acLEiRMZNWoUixYtIi0tDSMjI8qUKVOg3YULF9i4cSM1atSgYsWKKopWCFHcSAInipWtW7eSlZWFg4MDixcv5sGDBxw4cAA/Pz/S09NxcXHBxsZGae/h4cGVK1cYMWIEOTk5qKvLLS8KR9myZTl79ixbtmyhXr16REZGsnPnTtavX8+sWbOUdtnZ2axdu5Zy5crh5uamwoiFEMWJPM1EsfD8Pqb6+vqYmJgwefJkKlasyB9//IGPjw9paWm4u7szZcoU1qxZw5dffqkMXQEyQVwUmrp16/Lxxx9TsmRJIiIiuHLlCjNmzCA6OppmzZqxevVqpW1qair+/v5UrVoVAwMDFUYthCguJIETxUL+vCE7Ozvq1avH4sWLWb9+PdWrV8fHx4eKFSty+fJlJk+eTHx8PJ9++imQ9xDNycl55Ub2QvxXfvrpJ54+fcrw4cO5evUqISEhbNu2jQULFjBr1iwqVqyIhYWF0v769etcv35dhRELIYoTWYUqio2SJUty+PBhjh07xpAhQ4C8IVJbW1v+/PNPpk+fTmJiInp6euTm5pKWlgbI3qZCNYyNjYmKisLc3Jz4+Hg+//xz5bWyZcty4MABpk2bxvr165XjFStWJDExURXhCiGKGemBE0XWs71mampqpKen4+HhQZcuXXBwcABgyZIlbN68mSpVqjBp0iQqVarE48ePleQNZG9ToRq3b9/G29ubJ0+e0KRJE/r376+8lpaWxqVLl7h//z7w170uyZsQ4m1JD5wo8lxdXblz5w6//PILt2/fxs/PDzMzMwIDA5W6b25ubnh6erJhwwZmz56t4oiF+Iu5uTlr1qwhLS2NY8eOcfToURwdHSlVqhSdO3eWeZlCiH9EEjhRpJmZmbF//35u377NiRMnWLBgAampqSxevJjly5cXmAhuZWVFdHS0PBBFkVOtWjVcXV3p0KEDDx48IDk5GQ8PD7KysmT3BSHEPyIJnCjSDAwM8PPzo379+mzbtg0fHx+8vLzo3LkzlpaWdOjQgaSkpALnyANRFFWamppoa2vL/EwhxL8mc+BEkdSlSxfMzMxISUlh/vz5VKtWjWvXrtGjRw/s7OzIysrCyMiI4OBgdHV1C5wryZsoqrKysmR+phDinZAEThQ5derUYeTIkWzduhUbGxsSEhIYM2YM7u7u3Llzh7Fjx7J//37u3LlDqVKlCjwQhRBCiA+BDKGKIqlGjRrY2dkxYsQINmzYQHx8PMbGxiQmJhIZGQnklRXJzMyUHjchhBAfHEngRJHWqVMnevfujampKTVq1OD69es4OjoWKHgqc96EEEJ8aCSBE0VexYoVadSoEV9++SX169dn8eLF+Pj4qDosIYQQQmUkgRMqoaamRm5u7t/qPdPX18fd3Z2wsDCZ/C2EEOKDJgmcKHTdunWjbt26rFixgrt3777VOc8nelJ+QQghxIdMEjhRqMqXL8++fftITU1FTU2NdevWcfLkSfbu3au0kTltQgghxOtpqjoA8WFJS0vj0KFDbN++nVu3btG9e3cWL17M5s2bOXz4MFu2bJHkTQghhHgDqQMnCtWjR4+IiYkhKCiIP//8E19fXywsLDAwMCAsLIwdO3ZgZWVFlSpVVB2qEEIIUWRJAif+c5qaeR296up5t9umTZvYt28fVlZWANy8eZNGjRqxe/duEhMTGTVqFIcPH6Zjx44qi1kIIYQoymQIVfynLC0tadWqFeHh4dy/fx/I2z4oISEBa2trFi9eTGxsLMnJyYwYMYLU1FSaNGlCkyZN2Ldvn2qDF0IIIYoo6YET/6lOnTphZWXFwIEDKV26tHI8ODiYUqVKcefOHR4/foyzszOpqakAnDhxgsWLF5OdnY2GhoaKIhdCCCGKLkngxH/K19eXmJgYunXrhoeHB6VKlQLy6sDt3LmT33//HQ8PD5KTk196vpQKEUIIIV4kCZz4z+T3nvn7+7Nnzx569+6Nh4cHZcqUISsri61bt1KlShXatWun4kiFEEKI4kUSOPFO1ahRQ/lzbu5fJQbNzMwoV64c3bt3x8PDAyMjI+Li4li6dCmenp6YmJioIlwhhBCiWJIETrwzpqamHDt2jOHDh6OhoaHUc4uKiqJGjRq0atWK2NhYunbtysCBA9HT0+OXX37hxo0bJCUlqTh6IYQQoviQVajinfnjjz8IDAxk8uTJPH78mBUrVhAZGUmNGjVwdnYmKSmJgIAA1NXV6dq1K7q6uvj7+7Njxw7gr/1RhRBCCPF6spWW+Nfq1avH77//zpMnTwAYNmwY06ZN4/Lly6Snp+Ps7Mz169cL7F86e/ZsdHR0GDFihCpDF0IIIYolGUIV/4qdnR379u3j66+/Vgr2Llq0iAkTJmBqakpMTAzXr18H8laU5hfzHTt2rCRvQgghxD8kQ6jiXzE0NATAxcUFPT09hg8fTk5ODpGRkWhraxMQEEBycjKLFy8GICcnR4ZKhRBCiH9JEjjxrxw9epTY2Fj27dvHsGHDWLJkCR4eHuTk5PDNN9+grq5OQEAAubm5LFmyBECSNyGEEOJfkiFU8a/89ttvZGZm0rx5c1xcXLCwsCAiIkIZKg0PD8fPz4+vv/6aHj16qDhaIYQQ4v0gCZz4Wxo2bIienh7a2trKsenTp2NoaEhubi7u7u506tSJ8PBwJYmLiIhg8ODB7Ny5U1VhCyGEEO8VSeDEW7OxsWHv3r2sXLmSoKAgTE1NAUhISODp06d07NiRQ4cO4erqSseOHVm0aJGSxG3ZskX2NhVCCCHeEUngxFvT1dUFoEyZMmhpabFz5078/f1p1qwZISEhODk5YWpqyoEDB3BxccHOzo5x48YVeA/Z21QIIYT492QRg3hra9euBWD+/PksXbqU6Oho6tevz7Jlyzh9+jTlypWjSZMm/PHHHxw+fJgOHTpw7tw5FUcthBBCvH+kB078LWvXrmXy5MnMmzePSpUqMXPmTNq1a8fp06c5evQov/32m9L2t99+IycnR4ZNhRBCiHdMeuDE37ZkyRJyc3OZMWMGenp6zJs3jxkzZqCpqansxvAsGTYVQggh3i1J4MQ/snTpUnJzcwkKCiI7O5sFCxa8NHkTQgghxLsnCZwooEGDBty7d4+kpCTl2Kt2Tli2bBm5ublMnz4dXV1dgoODCzNUIYQQ4oMlm9kLRZcuXQgICODhw4ecPXuWyMhIzp8/r+xhmpOT89LzRo4cSdeuXbGysirkiIUQQogPkyRwogBjY2MqVKjAnDlzSElJ4dKlS/j6+pKRkfHaJE4IIYQQhUcSOFFA/nCpvr4+Dg4O2NnZkZGRQf/+/UlPT5ckTgghhCgCJIETNG3alIyMDM6ePQuAhoYG2dnZaGpqYmlpyYQJE7h//z5OTk6yUEEIIYQoAqQO3AeuZcuW7Nq1i5EjR/LJJ58AeWU/1NTUyMrKIjY2lnnz5qGvr4+np6dqgxVCCCEEIAncB8/Y2JinT59Srlw5Bg8eTMOGDQHIzc1FTU2NnJwc9uzZw8mTJ2nfvj06OjoqjlgIIYQQksB94E6cOMGWLVuIjIzE3NycYcOGUaVKFSBvPhxAZmYmISEhmJmZ4ebmpspwhRBCCIEkcB88DQ0NWrRoQWxsLPPnz6datWpMmjSJy5cv4+/vD4CmpiaPHj0iNDSU6tWrqzhiIYQQQkgh3w+Ympoaf/75JxcvXqRKlSps27YNDQ0N5s6dS0pKCnv37gUgKysLgAsXLlCvXj10dHTIzMxUZehCCCHEB00SuA9Y/u4KampqNGzYkHPnzjFy5EgSExPJyMigV69epKSkcOLECQAOHTpEUlKSJG9CCCGEiskQquCXX36hRo0axMTEkJKSQuvWrZk7dy5t2rTB0tKyQNsrV66oJkghhBBCKKQHTnD27Fm+/fZbDh06hIeHBzk5OWzfvp0HDx5w6NAhVYcnhBBCiOdIIV+Bjo4OHTt25Pjx49y9e/eF12X3BSGEEKJokQROCCGEEKKYkTlwQgghhBDFjCRwQgghhBDFjCRwQgghhBDFjCRwQgghhBDFjCRwQgghhBDFjCRwQgghhBDFjCRwQgghhBDFjCRwQgghhBDFjCRwQgghhBDFjCRwQgghhBDFjCRwQgghhBDFjCRwQgghhBDFjCRwQgghhBDFzP8BMsXu1QAzqwQAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "\n",
    "embeddings_cuda = embeddings.to(torch.device(\"cuda\"))\n",
    "\n",
    "functions = {\n",
    "    \"1) MHA wrapper class\": mha_ch03_wrapper,\n",
    "    \"2) MHA Ch03\": mha_ch03,\n",
    "    \"3) MHA with combined QKV weights\": mha_combined_qkv,\n",
    "    \"4) MHA with PyTorch scaled_dot_product_attention\": mha_pytorch_scaled,\n",
    "    \"5) PyTorch MHA class defaults\": mha_pytorch_class_default,\n",
    "    \"6) PyTorch MHA with need_weights=False\": mha_pytorch_class_noweights\n",
    "}\n",
    "execution_times = [time_pytorch_function(fn, embeddings_cuda) for name,fn in functions.items()]\n",
    "\n",
    "\n",
    "# Plotting\n",
    "\n",
    "# Customize further for dark mode aesthetics\n",
    "plt.rcParams['figure.facecolor'] = '#121212'  # Dark figure background\n",
    "plt.rcParams['axes.facecolor'] = '#121212'    # Dark axes background\n",
    "plt.rcParams['axes.edgecolor'] = 'white'      # White axes border\n",
    "plt.rcParams['axes.labelcolor'] = 'white'     # White labels\n",
    "plt.rcParams['text.color'] = 'white'          # White text\n",
    "plt.rcParams['xtick.color'] = 'white'         # White x ticks\n",
    "plt.rcParams['ytick.color'] = 'white'         # White y ticks\n",
    "plt.rcParams['grid.color'] = '#444444'        # Lighter grid lines for contrast\n",
    "plt.rcParams['lines.linewidth'] = 2           # Thicker plot lines for visibility\n",
    "plt.rcParams['lines.markersize'] = 8          # Larger markers for visibility\n",
    "\n",
    "fig, ax = plt.subplots()\n",
    "bars = plt.bar(functions.keys(), execution_times)\n",
    "\n",
    "plt.ylabel('Execution time (ms)')\n",
    "plt.xticks(rotation=45, ha=\"right\")\n",
    "\n",
    "# Calculate new ylim with a margin\n",
    "max_execution_time = max(execution_times)\n",
    "upper_ylim = max_execution_time + 0.2 * max_execution_time  # Adding a 20% margin\n",
    "\n",
    "plt.ylim(0, upper_ylim)  # Setting new ylim\n",
    "\n",
    "# Annotate bars with execution times\n",
    "for bar in bars:\n",
    "    yval = bar.get_height()\n",
    "    plt.text(bar.get_x() + bar.get_width()/2, yval + (0.05 * upper_ylim), round(yval, 2), ha='center', va='bottom')\n",
    "\n",
    "\n",
    "plt.tight_layout()\n",
    "plt.savefig(\"1.pdf\")\n",
    "plt.show()\n"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "gpuType": "A100",
   "machine_shape": "hm",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}