{
"cells": [
{
"cell_type": "markdown",
"id": "e2e65c03-36d4-413f-9b23-5cdd816729ab",
"metadata": {},
"source": [
"\n",
"Supplementary code for \"Build a Large Language Model From Scratch\": https://www.manning.com/books/build-a-large-language-model-from-scratch by Sebastian Raschka
\n",
"Code repository: https://github.com/rasbt/LLMs-from-scratch\n",
""
]
},
{
"cell_type": "markdown",
"id": "6f678e62-7bcb-4405-86ae-dce94f494303",
"metadata": {
"id": "6f678e62-7bcb-4405-86ae-dce94f494303"
},
"source": [
"# Comparing Efficient Multi-Head Attention Implementations"
]
},
{
"cell_type": "markdown",
"id": "b742938a-4bfc-4527-a1f1-d5963508967d",
"metadata": {
"id": "b742938a-4bfc-4527-a1f1-d5963508967d"
},
"source": [
"This code notebook compares different ways to implement causal multi-head attention used in decoder-style LLMs like GPT, Llama, etc."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "7898551e-f582-48ac-9f66-3632abe2a93f",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "7898551e-f582-48ac-9f66-3632abe2a93f",
"outputId": "7d088260-3fa1-44f2-bd65-2a46e289f9d4"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"PyTorch version: 2.2.2\n",
"Running on cpu\n"
]
}
],
"source": [
"import torch\n",
"\n",
"torch.manual_seed(123)\n",
"device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
"print(f\"PyTorch version: {torch.__version__}\")\n",
"print(f\"Running on {device}\")\n",
"\n",
"batch_size = 8\n",
"context_len = 1024\n",
"embed_dim = 768\n",
"embeddings = torch.randn((batch_size, context_len, embed_dim), device=device)"
]
},
{
"cell_type": "markdown",
"id": "2f9bb1b6-a1e5-4e0a-884d-0f31b374a8d6",
"metadata": {
"id": "2f9bb1b6-a1e5-4e0a-884d-0f31b374a8d6"
},
"source": [
"
\n",
" \n",
"\n",
"## 1) CausalAttention MHA wrapper class from chapter 3"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "297c93ed-aec0-4896-bb89-42c4b294d3d1",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "297c93ed-aec0-4896-bb89-42c4b294d3d1",
"outputId": "f8a33752-2cd6-4101-8feb-9d1699984719"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"torch.Size([8, 1024, 768])\n"
]
}
],
"source": [
"from ch03 import MultiHeadAttentionWrapper as Ch03_MHA_Wrapper\n",
"\n",
"mha_ch03_wrapper = Ch03_MHA_Wrapper(\n",
" d_in=embed_dim,\n",
" d_out=embed_dim//12,\n",
" context_length=context_len,\n",
" dropout=0.0,\n",
" num_heads=12,\n",
" qkv_bias=False\n",
").to(device)\n",
"\n",
"out = mha_ch03_wrapper(embeddings)\n",
"print(out.shape)"
]
},
{
"cell_type": "markdown",
"id": "21930804-b327-40b1-8e63-94dcad39ce7b",
"metadata": {
"id": "21930804-b327-40b1-8e63-94dcad39ce7b"
},
"source": [
"
\n",
" \n",
"\n",
"## 2) The multi-head attention class from chapter 3"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "4ee6a61b-d25c-4a0c-8a59-f285544e3710",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "4ee6a61b-d25c-4a0c-8a59-f285544e3710",
"outputId": "b704a040-3547-422c-ecda-df9982a2da35"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"torch.Size([8, 1024, 768])\n"
]
}
],
"source": [
"from ch03 import MultiHeadAttention as Ch03_MHA\n",
"\n",
"mha_ch03 = Ch03_MHA(\n",
" d_in=embed_dim,\n",
" d_out=embed_dim,\n",
" context_length=context_len,\n",
" dropout=0.0,\n",
" num_heads=12,\n",
" qkv_bias=False\n",
").to(device)\n",
"\n",
"out = mha_ch03(embeddings)\n",
"print(out.shape)"
]
},
{
"cell_type": "markdown",
"id": "73cd11da-ea3b-4081-b483-c4965dfefbc4",
"metadata": {
"id": "73cd11da-ea3b-4081-b483-c4965dfefbc4"
},
"source": [
"
\n",
" \n",
"\n",
"## 3) An alternative multi-head attention with combined weights"
]
},
{
"cell_type": "markdown",
"id": "1fa1a5ea-eaff-4d2d-aaf0-b34cdb6fd4dd",
"metadata": {
"id": "1fa1a5ea-eaff-4d2d-aaf0-b34cdb6fd4dd"
},
"source": [
"- The code for the `MultiHeadAttentionAlt` class below is based on code that was kindly shared by [Rayed Bin Wahed](https://github.com/rasbt/LLMs-from-scratch/discussions/51)\n",
"- The main difference between the `MultiHeadAttentionAlt` class and the `MultiHeadAttention` class used in chapter 3 is that `MultiHeadAttentionAlt` uses a single weight matrix, `self.qkv = nn.Linear(d_in, 3 * d_out, bias=qkv_bias)` instead of separate weight matrices:\n",
"\n",
" - `self.W_query = nn.Linear(d_in, d_out, bias=qkv_bias)`\n",
" - `self.W_key = nn.Linear(d_in, d_out, bias=qkv_bias)`\n",
" - `self.W_value = nn.Linear(d_in, d_out, bias=qkv_bias)`\n",
"\n",
"- Here, `self.qkv` combines all three weight matrices `self.W_query`, `self.W_key`, and `self.W_value` to carry out the query, key, and value computation in a single step\n",
"- Using `q, k, v = qkv.unbind(0)`, we obtain the individual query, key, and value tensors, which are then used similarly to the query, key, and value tensors in the `MultiHeadAttention` class in chapter 3"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "9a6bd0a2-f27c-4602-afa0-c96cd295c1a6",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "9a6bd0a2-f27c-4602-afa0-c96cd295c1a6",
"outputId": "5d948671-176f-4633-bede-97767e36becc"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"torch.Size([8, 1024, 768])\n"
]
}
],
"source": [
"import torch.nn as nn\n",
"\n",
"\n",
"class MultiHeadAttentionCombinedQKV(nn.Module):\n",
" def __init__(self, d_in, d_out, num_heads, context_length, dropout=0.0, qkv_bias=False):\n",
" super().__init__()\n",
"\n",
" assert d_out % num_heads == 0, \"embed_dim is indivisible by num_heads\"\n",
"\n",
" self.num_heads = num_heads\n",
" self.context_length = context_length\n",
" self.head_dim = d_out // num_heads\n",
"\n",
" self.qkv = nn.Linear(d_in, 3 * d_out, bias=qkv_bias)\n",
" self.proj = nn.Linear(d_in, d_out)\n",
" self.dropout = nn.Dropout(dropout)\n",
"\n",
" self.register_buffer(\n",
" \"mask\", torch.triu(torch.ones(context_length, context_length), diagonal=1)\n",
" )\n",
"\n",
" def forward(self, x):\n",
" batch_size, num_tokens, embed_dim = x.shape\n",
"\n",
" # (b, num_tokens, embed_dim) --> (b, num_tokens, 3 * embed_dim)\n",
" qkv = self.qkv(x)\n",
"\n",
" # (b, num_tokens, 3 * embed_dim) --> (b, num_tokens, 3, num_heads, head_dim)\n",
" qkv = qkv.view(batch_size, num_tokens, 3, self.num_heads, self.head_dim)\n",
"\n",
" # (b, num_tokens, 3, num_heads, head_dim) --> (3, b, num_heads, num_tokens, head_dim)\n",
" qkv = qkv.permute(2, 0, 3, 1, 4)\n",
"\n",
" # (3, b, num_heads, num_tokens, head_dim) -> 3 times (b, num_head, num_tokens, head_dim)\n",
" queries, keys, values = qkv.unbind(0)\n",
"\n",
" # (b, num_heads, num_tokens, head_dim) --> (b, num_heads, num_tokens, num_tokens)\n",
" attn_scores = queries @ keys.transpose(-2, -1)\n",
" attn_scores = attn_scores.masked_fill(\n",
" self.mask.bool()[:num_tokens, :num_tokens], -torch.inf\n",
" )\n",
"\n",
" attn_weights = torch.softmax(attn_scores / keys.shape[-1]**-0.5, dim=-1)\n",
" attn_weights = self.dropout(attn_weights)\n",
"\n",
" # (b, num_heads, num_tokens, num_tokens) --> (b, num_heads, num_tokens, head_dim)\n",
" context_vec = attn_weights @ values\n",
"\n",
" # (b, num_heads, num_tokens, head_dim) --> (b, num_tokens, num_heads, head_dim)\n",
" context_vec = context_vec.transpose(1, 2)\n",
"\n",
" # (b, num_tokens, num_heads, head_dim) --> (b, num_tokens, embed_dim)\n",
" context_vec = context_vec.contiguous().view(batch_size, num_tokens, embed_dim)\n",
"\n",
" context_vec = self.proj(context_vec)\n",
"\n",
" return context_vec\n",
"\n",
"\n",
"mha_combined_qkv = MultiHeadAttentionCombinedQKV(\n",
" d_in=embed_dim,\n",
" d_out=embed_dim,\n",
" context_length=context_len,\n",
" dropout=0.0,\n",
" num_heads=12,\n",
" qkv_bias=False\n",
").to(device)\n",
"\n",
"out = mha_combined_qkv(embeddings)\n",
"print(out.shape)"
]
},
{
"cell_type": "markdown",
"id": "48a042d3-ee78-4c29-bf63-d92fe6706632",
"metadata": {
"id": "48a042d3-ee78-4c29-bf63-d92fe6706632"
},
"source": [
"
\n",
" \n",
"\n",
"## 4) Multihead attention with PyTorch's scaled dot product attention"
]
},
{
"cell_type": "markdown",
"id": "f78e346f-3b85-44e6-9feb-f01131381148",
"metadata": {
"id": "f78e346f-3b85-44e6-9feb-f01131381148"
},
"source": [
"- The implementation below uses PyTorch's [`scaled_dot_product_attention`](https://pytorch.org/docs/stable/generated/torch.nn.functional.scaled_dot_product_attention.html) function, which implements a memory-optimized version of self-attention calld [flash attention](https://arxiv.org/abs/2205.14135)"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "1b8e5a0d-1f65-4a03-bf6e-723f0cc428f5",
"metadata": {
"id": "1b8e5a0d-1f65-4a03-bf6e-723f0cc428f5"
},
"outputs": [],
"source": [
"class MHAPyTorchScaledDotProduct(nn.Module):\n",
" def __init__(self, d_in, d_out, num_heads, context_length, dropout=0.0, qkv_bias=False):\n",
" super().__init__()\n",
"\n",
" assert d_out % num_heads == 0, \"embed_dim is indivisible by num_heads\"\n",
"\n",
" self.num_heads = num_heads\n",
" self.context_length = context_length\n",
" self.head_dim = d_out // num_heads\n",
" self.d_out = d_out\n",
"\n",
" self.qkv = nn.Linear(d_in, 3 * d_out, bias=qkv_bias)\n",
" self.proj = nn.Linear(d_in, d_out)\n",
" self.dropout = dropout\n",
"\n",
" def forward(self, x):\n",
" batch_size, num_tokens, embed_dim = x.shape\n",
"\n",
" # (b, num_tokens, embed_dim) --> (b, num_tokens, 3 * embed_dim)\n",
" qkv = self.qkv(x)\n",
"\n",
" # (b, num_tokens, 3 * embed_dim) --> (b, num_tokens, 3, num_heads, head_dim)\n",
" qkv = qkv.view(batch_size, num_tokens, 3, self.num_heads, self.head_dim)\n",
"\n",
" # (b, num_tokens, 3, num_heads, head_dim) --> (3, b, num_heads, num_tokens, head_dim)\n",
" qkv = qkv.permute(2, 0, 3, 1, 4)\n",
"\n",
" # (3, b, num_heads, num_tokens, head_dim) -> 3 times (b, num_heads, num_tokens, head_dim)\n",
" queries, keys, values = qkv\n",
"\n",
" use_dropout = 0. if not self.training else self.dropout\n",
" context_vec = nn.functional.scaled_dot_product_attention(\n",
" queries, keys, values, attn_mask=None, dropout_p=use_dropout, is_causal=True)\n",
"\n",
" # Combine heads, where self.d_out = self.num_heads * self.head_dim\n",
" context_vec = context_vec.transpose(1, 2).contiguous().view(batch_size, num_tokens, self.d_out)\n",
"\n",
" context_vec = self.proj(context_vec)\n",
"\n",
" return context_vec"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "fbc8ba92-3471-41cb-b1b2-4c0ef5be392b",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "fbc8ba92-3471-41cb-b1b2-4c0ef5be392b",
"outputId": "af9e4855-7f20-4d61-8532-4827df8dfb30"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"torch.Size([8, 1024, 768])\n"
]
}
],
"source": [
"mha_pytorch_scaled = MHAPyTorchScaledDotProduct(\n",
" d_in=embed_dim,\n",
" d_out=embed_dim,\n",
" context_length=context_len,\n",
" dropout=0.0,\n",
" num_heads=12,\n",
" qkv_bias=False\n",
").to(device)\n",
"\n",
"out = mha_pytorch_scaled(embeddings)\n",
"print(out.shape)"
]
},
{
"cell_type": "markdown",
"id": "351c318f-4835-4d74-8d58-a070222447c4",
"metadata": {
"id": "351c318f-4835-4d74-8d58-a070222447c4"
},
"source": [
"
\n",
" \n",
"\n",
"## 5) Using PyTorch's torch.nn.MultiheadAttention"
]
},
{
"cell_type": "markdown",
"id": "74a6d060-6324-48fa-a35c-cb09f2a48965",
"metadata": {
"id": "74a6d060-6324-48fa-a35c-cb09f2a48965"
},
"source": [
"- Below, we use PyTorch's [torch.nn.MultiheadAttention](https://pytorch.org/docs/stable/generated/torch.nn.MultiheadAttention.html) implementation"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "3799c7ef-3155-42c6-a829-f95656453ae0",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "3799c7ef-3155-42c6-a829-f95656453ae0",
"outputId": "2a085df8-0445-4818-9978-6dc74469f568"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"torch.Size([8, 1024, 768])\n"
]
}
],
"source": [
"import torch.nn as nn\n",
"\n",
"\n",
"class MHAPyTorchClass(nn.Module):\n",
" def __init__(self, d_in, d_out, num_heads, context_length, dropout=0.0, qkv_bias=False, need_weights=True):\n",
" super().__init__()\n",
"\n",
" self.context_length = context_length\n",
" self.multihead_attn = nn.MultiheadAttention(\n",
" embed_dim=d_out,\n",
" num_heads=num_heads,\n",
" dropout=dropout,\n",
" bias=qkv_bias,\n",
" add_bias_kv=qkv_bias,\n",
" batch_first=True,\n",
" )\n",
"\n",
" self.need_weights = need_weights\n",
" self.proj = nn.Linear(d_out, d_out)\n",
" self.register_buffer(\"mask\", torch.triu(torch.ones(context_length, context_length), diagonal=1).bool())\n",
"\n",
" def forward(self, x):\n",
" batch_size, num_tokens, _ = x.shape\n",
"\n",
" # Ensure attn_mask is compatible with expected shape and `batch_first=True`\n",
" # No need to manually adjust for num_heads; ensure it's right for the sequence\n",
" if self.context_length >= num_tokens:\n",
" attn_mask = self.mask[:num_tokens, :num_tokens]\n",
" else:\n",
" attn_mask = self.mask[:self.context_length, :self.context_length]\n",
"\n",
" # attn_mask broadcasting will handle batch_size dimension implicitly\n",
" attn_output, _ = self.multihead_attn(\n",
" x, x, x, attn_mask=attn_mask, need_weights=self.need_weights\n",
" )\n",
"\n",
" output = self.proj(attn_output)\n",
"\n",
" return output\n",
"\n",
"\n",
"mha_pytorch_class_default = MHAPyTorchClass(\n",
" d_in=embed_dim,\n",
" d_out=embed_dim,\n",
" context_length=context_len,\n",
" dropout=0.0,\n",
" num_heads=12,\n",
" qkv_bias=False\n",
").to(device)\n",
"\n",
"out = mha_pytorch_class_default(embeddings)\n",
"print(out.shape)"
]
},
{
"cell_type": "markdown",
"id": "a3953bff-1056-4de2-bfd1-dfccf659eee4",
"metadata": {
"id": "a3953bff-1056-4de2-bfd1-dfccf659eee4"
},
"source": [
"
\n",
" \n",
"\n",
"## 6) Using PyTorch's torch.nn.MultiheadAttention with `scaled_dot_product_attention`"
]
},
{
"cell_type": "markdown",
"id": "d2164859-31a0-4537-b4fb-27d57675ba77",
"metadata": {
"id": "d2164859-31a0-4537-b4fb-27d57675ba77"
},
"source": [
"- Set `need_weights` (default `True`) to need_weights=False so that MultiheadAttention uses `scaled_dot_product_attention` [according to the documentation](https://github.com/pytorch/pytorch/blob/71d020262793542974cf13b30f2a9099773f015c/torch/nn/modules/activation.py#L1096)\n",
"\n",
"> need_weights: If specified, returns ``attn_output_weights`` in addition to ``attn_outputs``.\n",
" Set ``need_weights=False`` to use the optimized ``scaled_dot_product_attention``\n",
" and achieve the best performance for MHA.\n",
" Default: ``True``."
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "4a4c2afe-5e1f-4bd7-a118-67031176f147",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "4a4c2afe-5e1f-4bd7-a118-67031176f147",
"outputId": "234771f4-8a53-4478-8a9b-cf19f79a5e07"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"torch.Size([8, 1024, 768])\n"
]
}
],
"source": [
"mha_pytorch_class_noweights = MHAPyTorchClass(\n",
" d_in=embed_dim,\n",
" d_out=embed_dim,\n",
" context_length=context_len,\n",
" dropout=0.0,\n",
" num_heads=12,\n",
" qkv_bias=False,\n",
" need_weights=False # NEW!\n",
").to(device)\n",
"\n",
"out = mha_pytorch_class_noweights(embeddings)\n",
"print(out.shape)"
]
},
{
"cell_type": "markdown",
"id": "8877de71-f84f-4f6d-bc87-7552013b6301",
"metadata": {
"id": "8877de71-f84f-4f6d-bc87-7552013b6301"
},
"source": [
"
\n",
" \n",
"\n",
"## Quick speed comparison (M3 Macbook Air CPU)"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "a97c0b2e-6593-49d8-98bc-2267b3aa610f",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "a97c0b2e-6593-49d8-98bc-2267b3aa610f",
"outputId": "ebe635b2-5c03-4e9b-da3a-951d308acf7b"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"191 ms ± 2.47 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)\n"
]
}
],
"source": [
"## 1) CausalAttention MHA wrapper class from chapter 3\n",
"%timeit mha_ch03_wrapper(embeddings)"
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "19db9c2c-8e75-431a-8eef-0b4d8284e6e6",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "19db9c2c-8e75-431a-8eef-0b4d8284e6e6",
"outputId": "c6e7bcff-661c-45a6-da82-b1e3f89cf761"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"186 ms ± 2.94 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)\n"
]
}
],
"source": [
"## 2) The multi-head attention class from chapter 3\n",
"%timeit mha_ch03(embeddings)"
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "aa526ee0-7a88-4f34-a49a-f8f97da83779",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "aa526ee0-7a88-4f34-a49a-f8f97da83779",
"outputId": "92b634f8-43f8-468f-87a1-bb774b64c212"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"207 ms ± 1.68 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)\n"
]
}
],
"source": [
"## 3) An alternative multi-head attention with combined weights\n",
"%timeit mha_combined_qkv(embeddings)"
]
},
{
"cell_type": "code",
"execution_count": 12,
"id": "cc2b4256-16d8-4c34-9fd0-d4b4af0e60fa",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "cc2b4256-16d8-4c34-9fd0-d4b4af0e60fa",
"outputId": "80c6e314-0771-470e-b090-628984ce2d85"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"73.3 ms ± 654 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)\n"
]
}
],
"source": [
"## 4) Multihead attention with PyTorch's scaled dot product attention\n",
"%timeit mha_pytorch_scaled(embeddings)"
]
},
{
"cell_type": "code",
"execution_count": 13,
"id": "0f209e70-ebb6-4a1a-b608-1ff42e41c01d",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "0f209e70-ebb6-4a1a-b608-1ff42e41c01d",
"outputId": "3cd37b53-04d4-4dd0-9450-6fc8ebaac083"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"210 ms ± 12.7 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)\n"
]
}
],
"source": [
"## 5) Using PyTorch's torch.nn.MultiheadAttention\n",
"%timeit mha_pytorch_class_default(embeddings)"
]
},
{
"cell_type": "code",
"execution_count": 14,
"id": "3f4968c2-8d40-4ab9-8dba-052b4f77d756",
"metadata": {
"id": "3f4968c2-8d40-4ab9-8dba-052b4f77d756",
"outputId": "2e86bdb4-7fa0-4051-b000-4a2b591060a2",
"tags": []
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"199 ms ± 6.48 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)\n"
]
}
],
"source": [
"## 6) Using PyTorch's torch.nn.MultiheadAttention disabling `need_weights`\n",
"%timeit mha_pytorch_class_noweights(embeddings)"
]
},
{
"cell_type": "markdown",
"id": "a78ff594-6cc2-496d-a302-789fa104c3c9",
"metadata": {
"id": "a78ff594-6cc2-496d-a302-789fa104c3c9"
},
"source": [
"
\n",
" \n",
"\n",
"## Quick speed comparison (Nvidia A100 GPU)"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "707a2a14-a089-48a8-88aa-d328e1e0a9d0",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "707a2a14-a089-48a8-88aa-d328e1e0a9d0",
"outputId": "e99a17e9-8139-4b04-dac8-fa1dd5027735"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"8 ms ± 1.42 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)\n"
]
}
],
"source": [
"## 1) CausalAttention MHA wrapper class from chapter 3\n",
"%timeit mha_ch03_wrapper(embeddings)"
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "8686dd69-3655-40e4-a57b-a2c55532a010",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "8686dd69-3655-40e4-a57b-a2c55532a010",
"outputId": "5553b42c-b709-41a4-8a8b-be36dae408ab"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"6.22 ms ± 490 ns per loop (mean ± std. dev. of 7 runs, 100 loops each)\n"
]
}
],
"source": [
"## 2) The multi-head attention class from chapter 3\n",
"%timeit mha_ch03(embeddings)"
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "2209d7df-e54b-4910-ae2b-c78cf684d9bf",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "2209d7df-e54b-4910-ae2b-c78cf684d9bf",
"outputId": "01b0da88-510b-4b21-919a-0a7519a55ed8"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"6.85 ms ± 824 ns per loop (mean ± std. dev. of 7 runs, 100 loops each)\n"
]
}
],
"source": [
"## 3) An alternative multi-head attention with combined weights\n",
"%timeit mha_combined_qkv(embeddings)"
]
},
{
"cell_type": "code",
"execution_count": 12,
"id": "1075abe2-4839-4fd6-af3e-c09bb3651e26",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "1075abe2-4839-4fd6-af3e-c09bb3651e26",
"outputId": "542706db-5041-45ca-f667-9e1bd1c2c7aa"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"2.95 ms ± 336 ns per loop (mean ± std. dev. of 7 runs, 1,000 loops each)\n"
]
}
],
"source": [
"## 4) Multihead attention with PyTorch's scaled dot product attention\n",
"%timeit mha_pytorch_scaled(embeddings)"
]
},
{
"cell_type": "code",
"execution_count": 13,
"id": "868e3670-8edc-47bc-9e06-eb505e44dc9d",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "868e3670-8edc-47bc-9e06-eb505e44dc9d",
"outputId": "13cfc808-2b11-4041-fe67-e5a63abe4f28"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"6.39 ms ± 672 ns per loop (mean ± std. dev. of 7 runs, 100 loops each)\n"
]
}
],
"source": [
"## 5) Using PyTorch's torch.nn.MultiheadAttention\n",
"%timeit mha_pytorch_class_default(embeddings)"
]
},
{
"cell_type": "code",
"execution_count": 14,
"id": "944870e6-de54-4e3b-a455-b8f21f6f92c8",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "944870e6-de54-4e3b-a455-b8f21f6f92c8",
"outputId": "c52858e7-999c-4782-adc9-731f8d69dfa6"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"4.49 ms ± 3 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)\n"
]
}
],
"source": [
"## 6) Using PyTorch's torch.nn.MultiheadAttention disabling `need_weights`\n",
"%timeit mha_pytorch_class_noweights(embeddings)"
]
},
{
"cell_type": "markdown",
"id": "dabc6575-0316-4640-a729-e616d5c17b73",
"metadata": {
"id": "dabc6575-0316-4640-a729-e616d5c17b73"
},
"source": [
"
\n",
" \n",
"\n",
"\n",
"## Speed comparison (Nvidia A100 GPU) with warmup"
]
},
{
"cell_type": "code",
"execution_count": 15,
"id": "29b63d3d-6d0b-43bb-9c68-d5514dc81000",
"metadata": {
"id": "29b63d3d-6d0b-43bb-9c68-d5514dc81000"
},
"outputs": [],
"source": [
"# CUDA benchmark code shared by Andrei Aksionov\n",
"# and based on code from\n",
"# https://github.com/cuda-mode/lectures/blob/main/lecture1/pytorch_square.py\n",
"\n",
"def time_pytorch_function(func, *input, num_repeats = 1_000):\n",
" # CUDA IS ASYNC so can't use python time module\n",
" start = torch.cuda.Event(enable_timing=True)\n",
" end = torch.cuda.Event(enable_timing=True)\n",
"\n",
" # Warmup\n",
" for _ in range(5):\n",
" func(*input)\n",
" torch.cuda.synchronize()\n",
"\n",
" start.record()\n",
" for _ in range(num_repeats):\n",
" func(*input)\n",
" torch.cuda.synchronize()\n",
" end.record()\n",
" torch.cuda.synchronize()\n",
" return start.elapsed_time(end) / num_repeats"
]
},
{
"cell_type": "code",
"execution_count": 16,
"id": "CDJAPZaszaqx",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 489
},
"id": "CDJAPZaszaqx",
"outputId": "f23e9b83-7fd6-4011-9434-0e6934cf762a"
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAnAAAAHWCAYAAAD3vrTNAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAADNB0lEQVR4nOzddVhU6dvA8e/QIiglCuYqdrt2d6MgtghYYIKo2FjYKIKiYucqdq7dunYnNio2JopIzfuHL+fniL3qgHt/rstrnTPnHO959swz93nOEyoLCws1QgghhBAi1dDRdgBCCCGEEOLbSAInhBBCCJHKSAInhBBCCJHKSAInhBBCCJHKSAInhBBCCJHKSAInhBBCCJHKSAInhBBCCJHKSAInhBBCCJHK6Gk7gJTGxsaGV69eaTsMIYQQQvxHmZiYcP/+/c/uIwnce2xsbDh//ry2wxBCCCHEf1yhQoU+m8RJAveepJa3QoUKSSucEEIIIX45ExMTzp8//8U8RBK4j3j16hVRUVHaDkMIIYQQ4qNkEIMQQgghRCojCZwQQgghRCojCZwQQgghRCojCZwQQgghRCojCZwQQgghRCojCVwqp6Ojw4ABAzh58iQREREcP36c3r17f/aYjBkzMmPGDI4cOcLjx48ZNWrUR/fz8PDgyJEjREREcPbsWUaOHImhoeHP+BhCCCGE+AYyjUgq5+XlRbt27ejWrRthYWEUK1aM4OBgoqKimDlz5kePMTAw4MmTJwQEBNC5c+eP7uPk5MSQIUPw9PTk6NGj5MqVi6lTp6JWq/H19f2ZH0kIIYQQXyAJXCpXqlQpNm/ezPbt2wG4c+cOTk5OlChR4pPH3Llzh4EDBwLQunXrj+5TunRpjh49yqpVq5RjVq1axZ9//vmDP4EQQgghvpU8Qk3ljh07RuXKlcmVKxcABQsWpEyZMuzYseNfnffo0aMULVpUSQSzZ89OrVq1/vV5hRBCCPHvSQtcKhcYGIipqSmHDx8mISEBXV1dRo0axcqVK//VeVetWoWlpSV///03KpUKfX195s2bx6RJk35Q5EIIIYT4XtICl8o5ODjQtGlT3N3dqVatGt26daNbt260bNnyX523QoUK9OzZEx8fH6pVq4aLiwu1atX64gAJIYQQQvx80gKXyg0fPpygoCDWrFkDwKVLl8iaNSs9e/YkNDT0u887YMAAli9fzuLFi5XzGhsbExAQQEBAAGq1+ofEL4QQQohvJy1wqVyaNGlITEzU2JaQkIBKpfrX5/0wSUtISAD41+cWQgghxL8jLXCp3NatW+nVqxcRERGEhYVRpEgRunTpwpIlS5R9fH19sbGxoWvXrsq2QoUKAWBiYoKlpSWFChUiLi6Oy5cvK+ft2rUrZ8+e5cSJE+TMmZMBAwawdevWZAmjEEIIIX4tSeBSuf79+zNgwAD8/f2xsrLiwYMHLFiwAH9/f2WfjBkzkjlzZo3j9u7dq/y9WLFiNGvWjNu3b1O8eHEAJk6ciFqtZuDAgdjY2PDkyRO2bt3KyJEjf80HE0IIIcQnqSwsLKQz0/8zNTUlPDycHDlyEBUVpe1whBBCCPEf87W5iPSBE0IIIYRIZSSBE0IIIYRIZSSBE0IIIYRIZSSBE0IIIYRIZbQ+CjVbtmyUK1eOLFmyYGxsTGRkJOfOnePYsWO8fftW2+EJIYQQQqQ4WkvgmjZtioeHB8WKFePRo0c8ePCAmJgYzM3NyZEjB2/fvmXlypUEBQURERGhrTCFEEIIIVIcrSRwu3fvJi4ujqVLl+Lq6sq9e/c03jcwMKBUqVI4Ojqyc+dOfHx8WL9+vTZCFUIIIYRIcbQyD1y1atXYvXv3V+1rbm5OtmzZOHPmzE+OSuaBE0IIIYR2fW0uorUWuK/17Nkznj179hOjEUIIIYRIXbQ+CrVIkSLkz59feV2vXj0WLVrE4MGD0dfX12JkQgghhBApk9YTuICAAOzs7ADInj07s2bNIjo6mkaNGjFs2LCvPo+Ojg4DBgzg5MmTREREcPz4cXr37v2TohZCCCGE0B6tTyOSK1cuzp07B0Djxo05dOgQHh4elC5dmtmzZzNo0KCvOo+Xlxft2rWjW7duhIWFUaxYMYKDg4mKimLmzJk/8yN8M9NOC7UdQooSNctF2yGIX8jGxoahQ4dSo0YN0qRJw82bN+nRowenT5/+5DEdOnSgY8eOZM2albt37xIQEMCyZcuU99etW0fFihWTHbdt2zZatWr1Mz6GEEJoldYTOJVKhY7Ou4bAKlWqsHXrVgDu3r2LhYXFV5+nVKlSbN68me3btwNw584dnJycKFGixI8PWgjxXdKnT8+mTZs4cOAALVq0IDIykpw5c/L8+fNPHtOuXTt8fX3p2bMnp06dokSJEgQGBvL8+XOlvnB1dcXAwEA5xtzcnH379snodSHEb0vrCdzp06fp3bs3e/fupXz58vTp0wd49zj18ePHX32eY8eO4eLiQq5cubh+/ToFCxakTJky+Pr6/qzQhRDfyMvLi7t379KjRw9l2+3btz97TPPmzZk/fz5r164F4NatWxQvXhxPT08lgfswAXR0dOTNmzesW7fuh8YvhBAphdYTuIEDBzJjxgzq169PQEAAN2/eBKBRo0YcPXr0q88TGBiIqakphw8fJiEhAV1dXUaNGsXKlSs/eYyBgQGGhobKaxMTk+//IEKIL6pbty67du1i7ty5lC9fnvv37zN37lwWLVr0yWMMDAySrcoSExNDiRIl0NPTIz4+Ptkxzs7OrF69mujo6B/+GYQQIiXQegJ38eJFKlWqlGz70KFDSUhI+OrzODg40LRpU9zd3QkLC6Nw4cKMGjWKBw8eEBoa+tFjevbsSb9+/b47diHEt8mePTvt2rVj+vTpTJo0ieLFizNmzBji4uI++T3dvXs3zs7ObNq0iTNnzlCsWDGcnZ0xMDDA0tKShw8fauxfokQJChQogJeX16/4SEIIoRVaT+DelzZtWqU/XJKvnVB3+PDhBAUFsWbNGgAuXbpE1qxZ6dmz5yd/GAIDA5k+fbry2sTEhPPnz39n9EKIL9HR0eH06dOMHDkSgHPnzpE/f37c3Nw++T2dMGEC1tbWbN26FZVKxePHj1m2bBmenp4kJiYm279NmzZcuHCBkydP/tTPIoQQ2qT1aUSyZcvG0qVLuX37Njdv3uT69etcv36dGzducP369a8+T5o0aZJV5gkJCahUqk8eExsbS1RUlPLn1atX3/05hBBf9vDhQy5fvqyx7cqVK2TJkuWTx8TExODp6UmWLFkoXrw4RYoU4fbt20RFRREZGamxr7GxMU2aNGHx4sU/JX4hhEgptN4CFxISgkqlwtPTk8ePH6NWf9/KXlu3bqVXr15EREQQFhZGkSJF6NKlC0uWLPnBEQshvteRI0eUeR+T5MqVizt37nzx2Pj4eGXdZEdHR7Zu3ZqsvmjcuDEGBgasWLHixwUt/rO+Z8obAwMDfHx8aNasGdbW1jx8+BB/f3/lt6hVq1YEBwdrHBMTE0PmzJl/5kcRvyGtJ3AFCxakRo0aXLt27V+dp3///gwYMAB/f3+srKx48OABCxYswN/f/wdFKoT4t0JCQti8eTPe3t6sXbuWEiVK4OLiQq9evZR9fH19sbGxoWvXrsC7BK9EiRKcOHECMzMzunTpQv78+enWrVuy87dp04ZNmzbJ8nviX/ueKW8A5s6dS4YMGfDy8uLGjRtkzJgxWdegly9fUqZMGeX19zZciP82rSdwp06dInPmzP86gXv16hWDBg366ol/hRC/3qlTp3BxccHX15c+ffpw+/ZtBg0apDFaPGPGjBqtEbq6unTt2hU7Ozvi4+M5cOAA9erVS9ZqZ2dnR7ly5XBycvpln0f8vr5nypvq1atTvnx5SpQooSR6H2tdVqvVPHr06IfGK/57VBYWFlpN/XPkyMHEiRNZsWIFly5dIi4uTuP9ixcv/rJYTE1NCQ8PJ0eOHF89eOK7/h1ZiUGDrMQghEhpDh48yK5du7C1tf3qKW/8/f3JlSsXp0+fpnnz5rx+/ZotW7YwZswYYmJigHePUAMDA7l//z46OjqcPXsWPz+/ZH1DxX/X1+YiWm+Bs7KyIkeOHEyZMkXZplarUalUqNVqrK2ttRidEEKI/6LvmfIme/bslClThpiYGFxcXLCwsMDf3x8LCwulJe/q1at4enpy4cIF0qVLR7du3diyZQsVKlRQ+ngK8TW0Pgp18uTJnDt3jrp161KiRAmKFy+u8V8htMXGxoaQkBCuXr1KREQE+/fvp1ixYp/cv0yZMmzatEnZ//Dhw3Tu3PmT+3t5efHkyRNGjRr1E6IXQvwbSa1jI0eO5Ny5cyxcuJBFixbh5ub22WPUajUeHh6cPHmSHTt24OvrS8uWLTEyMgLg+PHjLFu2jPPnz3Pw4EFcXV2JjIzE1dX1F30y8bvQegtclixZaNOmjbICgxApwfd0YI6Ojmb27NlcuHCB6OhoypYty8SJE4mOjmbhQs3H5sWLF8fV1VXmHRQihfrUlDf29vafPeb+/fsaj72uXLmCjo4Otra23LhxI9kx8fHxnDt3jj/++OPHBS/+E7TeArd//34KFSqk7TCE0PB+B+aTJ09y+/Zt9uzZQ3h4+CePOXfuHKtXr+by5cvcuXOHFStWsHv3bsqVK6exX9q0aQkJCcHb2/uLI9qEENrxPVPeHDlyhEyZMpE2bVqNYxISEj75eFRHR4cCBQokW1FEiC/RegK3detWRo4cSd++fbG3t6du3boaf4TQhrp163L69Gnmzp1LWFgYu3fvpm3btt90jsKFC1OqVCn++ecfje3jx49n+/bt7N2790eGLIT4gUJCQihZsiTe3t788ccfODk54eLiwpw5c5R9fH19mTZtmvJ61apVPHv2jClTppA3b17KlSvHsGHD+Ouvv5RBDH369KFq1apkz56dIkWKEBISQpYsWWTyafHNtP4IdeLEiQD4+Pgke08GMQht+Z4OzEnOnTuHpaUlenp6jBs3TqNidnR0pEiRItSsWfNnfwQhxL/wPVPevH79GicnJ8aOHcuOHTt49uwZa9euZfTo0co+ZmZmBAYGYm1tzfPnzzlz5gz16tWTUajim2l9GpGURKYR0Y6UOI3I/fv3OX36NPXq1VO2jRkzhuLFi3+xZThbtmykTZuWkiVLMmTIEPr168fq1auxtbVl586dODk5KdPjrFu3jvPnz8v8hUIIIYBUNI2IECnR93RgTpI02eelS5ewtrZWErhixYphbW3N7t27lX319PQoX748HTt2xMbG5qOLswshhBAf0koC5+joyJo1a75qX1tbW7JkycLRo0d/clRC/M+/WbPzfSqVCgMDAwD27dtHhQoVNN4PDg7m6tWrBAUFSfImhBDiq2llEEO7du04dOgQPXr0IE+ePMneNzU1pWbNmsyYMYPdu3djYWGhhSjFf9n3dGDu0KEDderUIWfOnOTMmZM2bdrQvXt3ZWH1V69eERYWpvHn9evXPH36lLCwsF/+GYUQQqReWmmBa9SoEXXr1qVTp074+voSHR3No0ePePv2LWZmZlhbW/PkyRNCQ0OpWLEijx8/1kaY4j/sezow6+jo4OvrS7Zs2UhISODmzZsMHz6c+fPna+ETCCGE+J1pfRCDhYUFZcuWJUuWLKRJk4YnT55w7tw5zp49i1r9a0OTQQzakRIHMQghhBDakGoGMTx9+pRNmzZpOwwhxEfIzUZycsMhhEgJtD6RrxBCCCGE+DaSwAkhhBBCpDKSwAkhhBBCpDKSwAkhhBBCpDIpJoHT19fHzs4OXV1dbYcihBBCCJGiaX0Uapo0aRg7diwtW7YEoHTp0ty6dYuxY8dy//59goKCtByhEEKIlE5GTCcnI6Z/b1pvgfP19aVQoUI0atSImJgYZfvevXtxcHDQXmBCCCGEECmU1lvg6tevT8eOHTl+/LjG9rCwMP744w8tRSWEEEIIkXJpvQXO0tLyo0tlGRsb//KVGIQQQgghUgOtJ3CnT5+mdu3ayuukpK1t27YcO3ZMW2EJIYQQQqRYWn+EOnLkSJYvX07evHnR1dXFw8ODvHnzUqpUKRo1aqTt8EQqIR2Yk5MOzEII8fvSegvckSNHqFKlCrq6uly6dIlq1aoRGRlJ3bp1OXPmjLbDE0IIIYRIcbTeAgcQHh6Ot7f3vz6PjY0NQ4cOpUaNGqRJk4abN2/So0cPTp8+/e+DFEIIIYRIIVJEAgdgZWWFlZUVOjqajYIXL178quPTp0/Ppk2bOHDgAC1atCAyMpKcOXPy/PnznxCtEEIIIYT2aD2BK1q0KFOnTiVPnjyoVCqN99RqNdbW1l91Hi8vL+7evUuPHj2Ubbdv3/6hsQohhBBCpARaT+AmT57M9evX8fLy4tGjR989dUjdunXZtWsXc+fOpXz58ty/f5+5c+eyaNGiHxyxEEIIIYR2aT2By5EjB25ubty8efNfnSd79uy0a9eO6dOnM2nSJIoXL86YMWOIi4sjNDT0o8cYGBhgaGiovDYxMflXMQghhBBC/ApaT+D27dtHoUKF/nUCp6Ojw+nTpxk5ciQA586dI3/+/Li5uX0ygevZsyf9+vX7V/+uEEIIIcSvpvVpRLy8vGjdujU+Pj7Y29tTt25djT9f6+HDh1y+fFlj25UrV8iSJcsnjwkMDCRHjhzKn0KFCn335xBCCCHEt/Hy8uLJkyeMGjXqk/usW7eOJ0+eJPuzdOnSj+4/YcIEnjx5goeHx88KO0XQegtcqVKlKFOmDDVr1kz23rcMYjhy5Ah2dnYa23LlysWdO3c+eUxsbCyxsbHfFrAQQggh/rXixYvj6urK+fPnP7ufq6srBgYGymtzc3P27dvH+vXrk+3boEEDSpYsyf379394vCmN1lvgxo4dy4oVKyhQoAAZMmTQ+PO1yRtASEgIJUuWxNvbmz/++AMnJydcXFyYM2fOT4xeCCGEEN8qbdq0hISE4O3t/cXpvp4/f86jR4+UP1WrVuXNmzesW7dOYz8bGxvGjh2Lh4cHcXFxPzH6lEHrCZyFhQXTp0//6IL23+LUqVO4uLjQpEkTDhw4QJ8+fRg0aBArV678QZEKIYQQ4kcYP34827dvZ+/evd98rLOzM6tXryY6OlrZplKpmD59OlOmTEnWnep3pfVHqBs3bqRixYqEh4f/63Nt27aNbdu2/fughBBCCPFTODo6UqRIkY92nfqSEiVKUKBAAby8vDS2e3l5ER8fz8yZM39UmCme1hO469ev4+vrS9myZbl48SLx8fEa7/+X/mcIIYQQvzNbW1tGjx6Nk5MTb9++/ebj27Rpw4ULFzh58qSyrWjRori7u1O9evUfGWqKp/UEztnZmdevX1O+fHnKly+v8Z5arZYETgghhPhNFCtWDGtra3bv3q1s09PTo3z58nTs2BEbGxsSExM/eqyxsTFNmjRhzJgxGtvLli1LhgwZOHPmjMY5/fz86Ny5M8WLF/85H0bLtJ7AlShRQtshCCGEEOIX2LdvHxUqVNDYFhwczNWrVwkKCvpk8gbQuHFjDAwMWLFihcb25cuXJ+tLt3LlSpYvX86SJUt+XPApjNYTOCGEEEL8N7x69YqwsDCNba9fv+bp06fK9mnTpnH//n38/Pw09mvTpg2bNm3i2bNnGtufPXuWbFtcXBwPHz7k2rVrP+FTpAxaSeD8/PwYM2YM0dHRyf4HfcjX1/cXRSWEEEIIbcucOXOyljg7OzvKlSuHk5OTlqJKebSSwBUuXBg9PT3l70IIIYT4b2rcuPFnXwNcu3YNS0vLrz7n79rv7X1aSeAcHBw++nchhBBCCPFlWp/Id/LkyZiYmCTbbmxszOTJk7UQkRBCCCFEyqb1BK5ly5YYGRkl225kZESLFi20EJEQQgghRMqmtVGopqamwLvlL0xMTDQm9NPR0aFWrVpERkZqKzwhhBBCiBRLawncjRs3UKvVqNVqjh49mux9tVrNuHHjtBCZEEIIIUTKprUErnHjxqhUKtauXYubm5vGHC6xsbFERETw4MEDbYUnhBBCCJFiaS2BO3jwIPBuqG9ERIS2whBCCCGESHW0PohBkjchhBBCiG+j9QROCCGEEEJ8G0nghBBCCCFSGUnghBBCCCFSGa0NYhBCCCFEymbaaaG2Q0hxoma5aDsEIAUkcBkyZGDEiBFUrlwZKysrVCqVxvvW1tZaikwIIYQQImXSegIXHBxMlixZmDBhAg8fPkStVms7JCGEEEKIFE3rCVzZsmVp0KAB58+f13YoQgghhBCpgtYHMdy9ezfZY1MhhBBCCPFpWk/gBg4cyJAhQ8iaNau2QxFCCCGESBW0/gh1zpw5pEmThhMnTvDmzRvi4uI03rezs9NSZEIIIYQQKZPWE7hBgwZpOwQhhBBCiFRF6wlcaGiotkMQQgghhEhVtN4HDkBHRwd7e3t69+5N7969adCgATo6/y40Ly8vnjx5wqhRo35QlEIIkTK1a9eOffv2ER4eTnh4OFu2bKFGjRqfPaZRo0YcPnyYu3fvsn//fmrWrKnxftq0aRk3bhznzp0jIiKCgwcP4ubm9hM/hRDiW2i9Be6PP/4gNDQUGxsbrl27BrxLvu7du0fLli0JDw//5nMWL14cV1dXmZpECPGfcO/ePUaMGMGNGzdQqVS0bNmSxYsXU7VqVS5fvpxs/1KlSjFr1iz8/PzYtm0bTk5OLFq0iGrVqhEWFgaAn58flSpVonPnzty+fZtq1arh7+/PgwcP2LJly6/+iEKID2i9BW7MmDGEh4dTpEgRqlevTvXq1SlatCi3bt1izJgx33y+tGnTEhISgre3N8+fP//xAQshRAqzdetWduzYwY0bN7h+/TqjRo3i9evXlCxZ8qP7e3h4sHPnToKDg7ly5Qpjxozh7NmzdOzYUdmndOnShIaG8s8//3Dnzh0WLlzI+fPnKVGixK/6WEKIz9B6Ale+fHmGDRumkWw9e/aMESNGUL58+W8+3/jx49m+fTt79+79gVEKIUTqoKOjg6OjI8bGxhw/fvyj+5QqVSpZHblr1y5KlSqlvD569Cj16tXDxsYGgIoVK2JnZ8fu3bt/XvBCiK+m9UeosbGxmJiYJNueNm3aZFOKfImjoyNFihRJ1pfjUwwMDDA0NFRefywOIYRIDfLnz8+WLVswMjLi9evXuLi4fPTxKbxbY/rx48ca2x4/fqyx9nT//v2ZNGkS58+fJy4ujsTERLy9vTl06NBP/RxCiK+j9QRu27ZtTJo0CS8vL06cOAFAyZIlmThx4jf1s7C1tWX06NE4OTnx9u3brzqmZ8+e9OvX77viFkKIlOTatWtUrVqVdOnS0ahRI6ZOnUqjRo0+mcR9SadOnShZsiStW7fmzp07lC9fnvHjx/PgwQN5wiFECqD1BK5///5MmzaNLVu2KC1uenp6bNmyhQEDBnz1eYoVK4a1tbVG876enh7ly5enY8eO2NjYkJiYqHFMYGAg06dPV16bmJjIwAchRKoUFxfHzZs3AThz5gzFixfH3d2d3r17J9v30aNHZMiQQWNbhgwZePToEQBGRkYMHjwYFxcXtm/fDsDFixcpVKgQ3bp1kwROiBRA6wncy5cvcXZ2JmfOnOTOnRuAK1euKBXR19q3bx8VKlTQ2BYcHMzVq1cJCgpKlrzBu8e3sbGx3x+8EEKkUDo6OhpdRN537NgxKleuzIwZM5RtVatW5dixYwDo6+tjYGCQrN5MSEj411M8CSF+DK0ncElu3LjBjRs3vvv4V69eKcPfk7x+/ZqnT58m2y6EEL8TX19fduzYQUREBCYmJjRt2pQKFSrQrFkzAKZNm8b9+/fx8/MDYMaMGWzYsIGuXbuyfft2HB0dKVasGN7e3gBERUVx4MABhg8fTkxMDHfu3KFChQq0aNECX19frX1OIcT/aCWB8/PzY8yYMURHRysVyqdIZSGEEJ9nZWXFtGnTyJgxIy9fvuTixYs0a9aMPXv2AJA5c2aN1rRjx47h7u7OoEGDGDx4MDdu3KBt27YaN7udOnXC19eXGTNmYGZmRkREBKNGjWLevHm/+uMJIT5CKwlc4cKF0dPTU/7+szRu3PinnVsIIVIKLy+vz77/sbpw/fr1rF+//pPHPHr0iB49evzr2IQQP4dWEjgHB4eP/l0IIYQQQnyZ1nujTp48+aPzrxkbGzN58mQtRCSEEEIIkbJpPYFr2bIlRkZGybYbGRnRokULLUQkhBBCCJGyaW0UqqmpKQAqlQoTExONyXd1dHSoVasWkZGR2gpPCCGEECLF0loCd+PGDdRqNWq1mqNHjyZ7X61WM27cOC1EJoQQQgiRsmktgWvcuDEqlYq1a9fi5ubGs2fPlPdiY2OJiIjgwYMH2gpPCCGEECLF0loCd/DgQQCKFy9ORESEtsIQQgghhEh1tL4SQ9asWcmaNesn3z906NAvjEYIIYQQIuXTegL3sYkk1Wq18ndra+tfGY4QQgghRIqn9QQuZ86cGq/19fUpUqQIAwYMYNSoUVqKSgghhBAi5dJ6AhcVFZVs2549e4iNjcXPz48aNWpoISohhBBCiJRL6wncpzx+/Bg7OztthyGEED+caaeF2g4hRYma5aLtEIRIdbSewBUoUEDjtUqlImPGjHh5eXH+/HktRSWEEEIIkXJpPYHbu3cvarUalUqlsf348eN4enpqKSohhBBCiJRL6wlc8eLFNV4nJiby5MkTjaW1hBBCCCHE/2g9gZNJfIUQQgghvo2OtgMYM2YM7u7uybZ37NhRphERQgghhPgIrSdw9vb2HDlyJNn2o0eP0qhRIy1EJIQQQgiRsmk9gTM3N+fly5fJtkdFRWFhYaGFiIQQQgghUjatJ3A3b9786GS9NWvW5NatW1qISAghhBAiZdP6IIZp06Yxbtw4LC0t2b9/PwCVK1ema9euDBo0SMvRCSGEEEKkPFpP4JYsWYKhoSG9evWiT58+ANy+fRsfHx+WLVum5eiEEEIIIVIerSdwAPPmzWPevHlYWloSExPD69evtR2SEEIIIUSKpfU+cAC6urpUqVKFhg0bKisyZMqUibRp02o5MiGEEEKIlEfrLXBZsmRhxYoVZM6cGUNDQ/bs2cOrV6/w9PTEwMBAeawqhBBCCCHe0XoL3JgxYzh9+jS5cuUiJiZG2f73339TuXJlLUYmhBBCCJEyaT2BK1u2LBMnTiQuLk5j++3bt7Gxsfnq8/Ts2ZMdO3Zw69YtwsLCWLRoEXZ2dj86XCGEEEIIrdN6Aqejo4Ourm6y7ba2trx69eqrz1O+fHnmzJlD7dq1cXJyQk9Pj5UrV2JsbPwjwxVCCCGE0DqtJ3C7d+/Gw8NDea1Wq0mbNi39+/dnx44dX32e5s2bs3TpUi5fvsyFCxfo3r07WbNmpWjRoj8jbCGEEEIIrdH6IIYhQ4awYsUKDh48iKGhITNnziRnzpw8ffqUTp06ffd506VLB8CzZ89+VKhCCCGEECmC1hO4e/fuUblyZRwdHSlYsCAmJiYsXryYlStXagxq+BYqlYpRo0Zx+PBhwsLCPrmfgYEBhoaGymsTE5Pv+veEEEIIIX4lrSdwlpaWPHnyhJUrV7Jy5UqN9/Lnz8+lS5e++Zz+/v7kz5+fBg0afHa/nj170q9fv28+vxBCCCGENmm9D9z+/fupVatWsu3dunVj+/bt33y+cePGUbt2bRo3bsy9e/c+u29gYCA5cuRQ/hQqVOib/z0hhBBCiF9N6y1w06dPZ/78+SxdupTBgwdjbm7OtGnTyJ8/v8bghq8xbtw4GjRoQKNGjbh9+/YX94+NjSU2NvZ7QxdCCCGE0AqtJ3BTpkxhz549TJ8+nX379mFubs6JEyeoXLkyjx49+urz+Pv74+TkhLOzM69evcLa2hqAly9ffndfOiGEEEKIlEjrj1ABbt68yaVLl8iWLRumpqasXbv2m5I3gPbt25M+fXo2bNjApUuXlD+Ojo4/KWohhBBCCO3Qegtc6dKlCQkJ4dmzZ1SuXJnSpUszduxYatasSe/evXnx4sVXncfS0vInRyqEEEIIkTJovQVu7dq1rF27ljp16nDlyhUWL15M1apVyZIlCwcOHNB2eEIIIYQQKY7WW+CaNm3KwYMHNbaFh4dTr149evXqpaWohBBCCCFSLq23wH2YvCVRq9VMnDjxF0cjhBBCCJHyaS2BCw0NxdTUVHnt5eWlLH8FYG5u/snkTgghhBDiv0xrCVz16tU1lrHy9vbG3Nxcea2np4ednZ02QhNCCCGESNG0lsCpVKrPvhZCCCGEEB+n9T5wQgghhBDi22gtgVOr1ajV6mTbhBBCCCHE52ltGhGVSkVwcLCyFqmhoSETJ04kOjoaAAMDA22FJoQQQgiRomktgQsNDdV4vWLFimT7LFu27FeFI4QQQgiRamgtgevRo4e2/mkhhBBCiFRNBjEIIYQQQqQyksAJIYQQQqQyksAJIYQQQqQyksAJIYQQQqQyksAJIYQQQqQyksAJIYQQQqQyksAJIYQQQqQyksAJIYQQQqQyksAJIYQQQqQyksAJIYQQQqQyksAJIYQQQqQyksAJIYQQQqQyksAJIYQQQqQyksAJIYQQQqQyksAJIYQQQqQyv10C16FDB06dOsXdu3fZtm0bJUqU0HZIQgghhBA/1G+VwDk4OODn54e/vz/Vq1fn/PnzrFixAisrK22HJoQQQgjxw/xWCVzXrl1ZtGgRS5Ys4fLly/Tu3Zs3b97Qpk0bbYcmhBBCCPHD/DYJnL6+PkWLFmXv3r3KNrVazd69eylVqpQWIxNCCCGE+LH0tB3Aj2JpaYmenh6PHj3S2P7o0SNy58790WMMDAwwNDRUXpuYmGj892cxMfxtiv3HMDX916eQMv0IKdefQ8r1x5My/TmkXH+OH1Cun/O1Och/+v9Mz5496devX7Lt58+f10I0/2HDw7Udwe9JyvXnkHL98aRMfw4p15/jF5WriYkJUVFRn3z/t0ngnjx5Qnx8PNbW1hrbra2tk7XKJQkMDGT69Oka28zNzXn27NlPizOlMDEx4fz58xQqVIhXr15pO5zfhpTrjydl+nNIuf4cUq4/3n+xTE1MTLh///5n9/ltEri4uDjOnDlD5cqV2bRpEwAqlYrKlSsze/bsjx4TGxtLbGysxrbPZbu/o1evXv3nPvOvIOX640mZ/hxSrj+HlOuP918q06/5nL9NAgcwbdo0pk6dyunTpzl58iQeHh4YGxuzZMkSbYcmhBBCCPHD/FYJ3Nq1a7GysqJ///5YW1tz/vx5mjdvzuPHj7UdmhBCCCHED/NbJXAAs2fP/uQjU/E/b9++Zdy4cbx9+1bbofxWpFx/PCnTn0PK9eeQcv3xpEw/TmVhYaHWdhBCCCGEEOLr/TYT+QohhBBC/FdIAieEEEIIkcpIAieEEEIIkcpIAieEEEIIkcpIAvcbUalU2g5BiK8i16oQQvw7v900Iv9VKpUKtfrdgOIaNWoQERHBtWvXSEhI0HJkqZu9vT05c+ZEV1eXDRs2cPXqVW2HlOq9f626uLgQGRnJrl27iImJ0XJkqdv75Sp+nPfL1dTU9D+zEsDPJNfqjyEJ3G8i6cswePBgmjVrxogRI7h3755UNv/CkCFDaNasGadPn6ZixYqUKlUKZ2dnSYr/paRrdejQoTRv3pygoCCMjIwkgfsX3v9BLFeuHMbGxly6dIn79+/LD+W/8H65ent7kzNnTsaPH8+dO3e0HFnq9X6ZNm7cGFtbWwwNDdm9ezdnzpzRcnSpiyRwv5HevXvTunVr3NzcOHfuHG/evNF2SKlW7969adGiBa1ateLs2bPky5ePbdu2kSlTJu7evavt8FI9d3d3WrVqhZOTExcuXADkrvzfSCq34cOH4+TkhKmpKZcvX2blypXMnTuX+Ph4LUeYOn14szFmzBji4uK0HFXq9mGZ7t27l1y5cuHo6MjSpUsJCQnRcoSphyRwv4n06dNTpUoVxo0bx9GjR8mUKRNFihShWbNmXLlyhbVr1/Lo0SNth5kqFChQgFKlSuHj48PZs2cBePHiBVeuXMHDwwMdHR1OnTrFqlWrtBxp6vFhclaoUCEWLFjAhQsXyJ49O8WLF8fd3Z0rV66wbds2Nm3apMVoU6fy5ctTvnx52rVrx7Nnz+jevTuOjo6YmJgwefJkSeK+U926dWnevDmtW7dWWohMTEywsrLi2bNnvHjxQssRpj6NGjWiSZMmSpk6OTkRHBxMRESEtkNLVSSBS6U+/EHU1dXFwsICCwsL7O3tsbe3x9bWFmNjY4oXL06GDBkYPXq0tHB8hbt377Jw4UIOHz4MvCvrVatWoVar0dHRIV++fFSoUAFAkrivYGhoqCyBU61aNXbv3o2VlRVFihThzp07NG3alNjYWK5fv06ePHkwMzNj586dsmzON2jQoAG1atXiwIEDHDt2DIBBgwYxePBgateujVqtZsqUKZLEfYUP61YLCwuuXr3KmTNnKFiwIHXq1KFly5bo6+uzc+dOxo0bJ+ttf6OsWbNy6tQpzpw5Q+PGjZkwYQIDBgxg48aNpEmThmzZsnH58mVth5niySjUVOj9CqZ69erY2Njw9OlT1qxZg5ubG8HBwdy5c4exY8dSvXp1bty4gZmZmSRvX+nFixds376dp0+fAtCyZUsePnyIvb09gwcPpmnTpsTHx1OlShUtR5ry1atXj/nz5wMwcuRIJkyYgIGBAZ6enrx+/Zru3buzd+9exowZQ48ePVi8eDEWFhbo6EjV9LWMjY1p164dTk5O5M2bV9n++vVrRo4cycmTJ6lZsyYDBw6Ucv2CHDlyKPVk165dKV68OBEREVSoUIGQkBBCQ0PJnTs3U6ZMYfr06dSrVw8LCwstR52yfWzEedq0aYmIiKBkyZJMnjyZESNGKPVE48aNqVOnDsbGxr840tRHWuBSoQ8HLIwZM4bVq1czceJENm7cSFxcHDdu3FD2Nzc3l6bpLyhevDjm5ubcvXtXGfyho6NDYmIiK1euZMWKFcTHxyvbrly5wuvXr7UddooXERFBuXLlOHDgAJkzZ6Z+/frExsYSGRlJw4YNMTMz49mzZ8C7VuTGjRtz9+5d6b/5DaKjo+ncuTMjR46kePHiuLq6smDBAuBdEjdq1Cj8/f1Jnz49iYmJWo425SpYsCB79uyhY8eOlC5dmhYtWlC3bl1OnTpFx44dqVmzJn5+fuzbt48HDx5gYWFBy5YtMTEx0XboKdb7jQ3ly5cnLCyMp0+fsn//ftavX4+HhwcdOnRg/fr1AKRJk4YmTZpw/fp1oqOjtRl6qiCL2adSPj4+tG/fHhcXF8LCwpKNNk2fPj25cuWiT58+ZM2alapVq8royU8YOnQoDg4OGBsb8/z5cx48eECvXr24fv36RzvW29rasmjRIhYtWqTcNYpPmzt3Lvb29uzfv5+mTZsmSyJMTEyoXbs2TZs2JWvWrFSrVk0e9X3C+9djpkyZeP36Nfr6+jx9+hRra2vGjRuHlZUVoaGh/PXXX8pxRkZGvH37VlrhP8LGxob79+8D4OnpiY+PDwkJCdSvX5+LFy8qZZ5086ajo4OhoSHz58/H2NiYRo0aSbl+waBBg6hVqxYLFixgyZIlvH37lm7dujFw4EAGDx7Mnj17SJcuHYMGDSJDhgzUrFlTfq++grTApULm5uZUqVKFoUOHcuzYMaytrcmTJw9Nmzbl1KlT7Nmzhxw5cjBq1CiePn1KtWrVSEhIUCog8T9OTk44OzvTtm1bbt68SenSpWnTpg3bt2+nSZMmnD59Wim3pI7LixYt4tq1a5K8faUNGzawadMm/Pz8WLBgAd26dePly5fK+2ZmZpQpU4bY2FjlRkNXV1cq8I9IShR8fHyoXbs26dOn5+XLl/j7+7N161b69evHuHHjaNmyJWq1miVLlgAoU7TISF9NkydPplixYnTo0IGrV6/y4MEDjIyMSExMJF++fFy8eFEpr8TERAwMDGjfvj0NGjTA2NiYOnXqoFarpVw/Y8CAAbi6uuLi4sLFixeVvq0LFizA0NCQYcOG8erVKx4/fkxkZCS1atWS36uvJC1wqVDGjBnZs2cPAQEBXL9+nWbNmpEzZ05MTEzQ0dFh7ty5zJo1i5IlS3LixAnUarX8IH6Cp6cnJUuWxMXFRdmWNWtWhg0bRvXq1alduzZXr17F0NAQT09P6taty82bN+nYsSMgP4gfev86MzMz49WrV0prWvHixQkNDeXo0aN07dpVaTV2dHRkx44dymupuD+vT58+eHh44OPjg5mZGYULF6Zt27b06tWLxYsXkylTJkaNGkWBAgUYOnQo27Zt03bIKVaWLFnYunUrly9fxsvLizt37mBtbU2rVq0YOHAg3t7eShIMYGBgQNmyZalWrRojR46Um40PVKtWjZMnTyojc3PlysWsWbMYOnQo+/fvx9LSEltbW+rXr8/evXs5fPgwf/zxB5aWlkRFRXHlyhX5vfoGksClcJ9KEPr164e7uzt6enrMnTuXPXv2sHfvXv766y/u3r1L3759v3gO8e7H0M3NjaJFi2pUGJkzZ2b8+PFkyJCBFi1a8OzZM7Jly0aZMmVYsWIFIOX6vho1anDq1Cll4Efv3r2pUKECZmZmBAYGcvjwYR49ekSxYsUIDQ3l9OnTTJs2jW7dumFubq60ZAhNH05wnC5dOpYuXcrSpUtZvHgx8O469Pb2ZsCAATRo0ECZRqh9+/aMHTtWkuFP0NPTIz4+HhsbG3bt2sW1a9fo2bMn169fB97Vsb169cLT05Nly5YBMGrUKFauXMmpU6cAudl4n4uLC35+fgwdOpRVq1YRFRWFra0t27ZtY/To0Zw9e5ZOnTrx559/ApA3b16aN2/O7t27Nc4j9erXkyFJKdj7F3KhQoUoW7YsdnZ2AIwbNw4nJydq1arF8OHD2bt3L/Cuwn/y5InGeeTL8Gm7d+/m8ePHdO7cGSMjI2X73bt3mTt3LiYmJuTMmROA27dvS/L2Ec7OzsybNw9HR0f09fVxcXGhc+fO7Ny5k3v37jFixAjat2+Pra0tp0+fpnHjxuTLl4+RI0eSLl066tevL2X5EatWraJ3794a24yNjcmXLx+xsbHKNrVazbRp09izZw+NGjVCT0+PBw8eMHr0aKXPltCkUqmUluH79+9To0YN7OzsmDBhArlz5wbe1bETJ04kODgYf39/Nm/eTPXq1ZW5IQFJ3t6zcOFCli9fTpcuXWjatCnp06fn0aNHbNy4kX79+rF161ZlZHTFihU5ePAgFStWTHYeqQu+nrTApQKDBw+mXr16ZMqUiXPnznH+/Hl8fX2VC93ExAQ7Ozt8fHzIli2bDFj4jA8TLx0dHQICAihcuDDTp09nw4YNSh8NExMTjh07xvDhwwkNDdVWyKnCmDFjqFWrFoGBgRQsWJCdO3eyY8cO4N0SRM2bN2fDhg0sWLCAu3fvKnM9ySOTTytSpAhhYWHExsYqrUXwblCIkZERPXv21Jice/78+bx69Yru3btrK+RUp0yZMty9e5eIiAhsbW3ZuXMnly9fxsfHR1n3uF27dtSuXZvIyEi8vb01RqOLd94vjwkTJlC1alWmTp3K4sWLMTIyUhoeklou9fT0WL9+PatXr2b27Nlaizu1k1uzFM7b25vWrVvTr18/ihYtys2bN2nbti1BQUHK/DplypTBz88PAwMDjQELQpOenp5G0mthYUFiYiI+Pj48fPiQbt264erqqpSdmZkZL168kEk6P0NXVxd411F5z549eHt7Y29vr7HPpEmTWL58Ofb29rRt25YcOXLw5s0bLl++rHQAl+RNk0ql4uzZs8TGxtK9e3fmz5+vTFexY8cOLCws6NKlC2ZmZsC7vlnm5uY8fPhQi1GnLuXKlWPevHm0bt0aW1tb7t27R40aNcibNy/+/v7kyZMHgHnz5tGpUyd69OhBfHw8urq6krx94P2W3j59+rBnzx66d++Os7MzKpWKU6dOcerUKdKkSUO+fPlYuHAhadKkYd68eVqOPHWTX/kUpHLlyhqv8+bNS506dejWrRsHDhygZMmSNGnShC1btlCmTBkmTpyISqVi586djBw5khYtWkgF8wFjY2Pq1asHoLRgTJ48mTVr1rB69WpcXV2Ji4vD1dWV8+fP06pVKw4cOMCkSZNYs2YN169fZ+fOndr8CCnWh4lXnz59WL9+PVZWVpQrV05JLuBdEhcaGkrHjh2pVKmSxnnkkUlySWWio6PD6dOnqVKlCuPHj0dHR4clS5awdetWKlasyNatWwkJCWHjxo1YWVkxevRoLUeeehw6dIglS5Zgb29Pq1atNJK4PHnyMHr0aAoUKADAq1evlOPkZuN/3p+k9/3fnKQkrlu3bjg5OZEuXToAGjZsiK+vL2nTptUYbSq+jzxCTSEaN27M7Nmz8fLy0hj11Lp1a7Zt24adnR1z5sxhzJgxLF68mAULFlCnTh127NhB27ZtlQpf+mZpcnFxYeLEiUq5+vv7U7ZsWRYtWkSuXLlo3749AQEBjBkzBh0dHSpXrkyNGjXQ1dXl0aNHBAYGAlKun1O/fn3i4+OV0Y5jxoyhTp06TJ06lRUrVmhMGdK8eXNWrlwpNxifUL58eVQqFf/88w9+fn7cuXOHmTNnUrZsWZYuXcq2bdvo3LkzarWaSpUqUaFCBbJkycLdu3cZP368TL/wFfT19TUWpB8wYAD29vasWrWKJUuWcP/+fWxtbTlz5gyzZ89mwIABWow25Xq/TqxcuTLW1tY8ePCAsLAwIiMjAZg4cSKVK1dm2rRp/PXXX1hYWFCwYEF2795NYmKidJ34l2QeuBRi3bp12NnZKa1qSZNwJiVzvXv3ZvPmzcpoqMuXL5MuXTru3LmjcR5JMjStWrUKa2trAgMDUavV3Lt3jy5dunD+/HkAzpw5Q0BAAPAu8dizZw979uzROIckb5reL48iRYowdOhQrl69SlRUFIcOHWLAgAHo6enRpUsXAI0kbvny5YCM3vuYjBkzKoMWXF1dsbe3p0aNGgAcPnyYVq1asXTpUkJCQujevTv79+9n//79GueQH8TPc3NzQ19fn6VLlyqtamPGjEGlUtG+fXvgXZ1779498ufPr4yqFskl1QFDhgyhRYsWPHjwgMyZM7NlyxZCQ0M5fPgwvXv3ZsKECXTu3BljY2Nmz56tPNGQrhP/niRwKcjEiROVTvWAxkzqWbNmxdDQkLi4OFQqFXZ2dqxZs4aFCxcCkmR8SnR0NBMmTEBXV5egoCBev36tkaAlJcgTJkwgISGB8ePHJzuHlKumpPLo378/VlZWwLtpRAwMDNDX12ffvn34+Pgwfvx43N3dMTY2Zu7cuRpLj0nyltzDhw8ZN24cISEhVKxYEW9vby5evAi8+34nJXFLliwhMDCQ/v37J1uBRX4QP69y5coULVqU6Oho1q1bpyRxo0ePJn/+/Dg7O2NqasrUqVOVvq9ys6Epe/bs3Lp1C4Bu3brRrFkz3NzcOHbsGH369KFnz56YmZmhq6vLP//8Q58+fZg9ezZ//vknU6dOVc4j9eq/Jwmcln2YePn7+6NSqZIlcbt27aJt27asW7cOfX190qdPr0wmC/Jl+FCTJk3InTs35ubmDBkyhKlTpxIVFcXQoUMpVaqUMhoK/pfEBQUFceHCBf7++29thZ1quLu74+HhQcuWLZk8eTIFChRg0KBBdOzYkcTERA4cOEDfvn0JCQmhWLFism7sV3r16hV3797l/v37NGrUiDt37rB//35lKafDhw/TunVrNmzYwM2bN5kwYYK2Q05V2rdvT1BQEJ6enujo6LB27VolCQ4PDydXrlxYWlpqDFyS5O1/XF1dad26Na6ursTHx1O4cGHGjBnDsWPHqF+/Pl26dGHJkiXUqFGD7t27o1arOXjwIB07dvzoovbi35EEToveT96aNWuGnp4ey5YtY/z48SQmJhIQEIBKpWLx4sWsWbMGtVpNiRIliI6OZuDAgcrIH6lgNPn6+lK7dm02bNjArl27iI2NJTY2lgULFpAmTRpGjRrFq1evNPoaLlmyhJs3b3Lo0CEtRp4ytW7dWqOsAEqWLMnmzZs5cuQI8G6OvJiYGKZMmYKnpycABw4coHPnztJJ+TM+vIG7dOkS9vb2VK5cmS5duuDl5YVarebAgQPK9/zw4cNUqVKFy5cvayvsFO/9ck2fPj06Ojq8ePGCxMREvLy8CA4Opnv37ujo6LBlyxYePnxIhgwZ6Nu3LwcOHNBy9CmTi4sLEyZMwNXVlQcPHqCrq8u8efO4cuUKRYoUYdSoUYwbN46ZM2fStWtXfHx8MDQ0xM/Pj1OnTsmSYz+BJHBalHQhDxs2DEdHR6ZOnUqmTJm4d++ecmed1Cdu0aJFzJs3T2PYtfR3Sc7b2xtnZ2datmzJmTNnNJLb169fM23aNFQqFUFBQQAaiUlS8iaVzP8kzZy+dOlSjYEyMTExmJqaKq/VajV79uxh+vTp9O/fnxcvXhAbG8vRo0dJTEyUMv2EpDJxcHAgXbp0vHr1itWrV7Nv3z4MDQ1p3769kmjs27ePxYsXs2XLFmUVBqkDknu/THr37k3FihUpUKAAS5cuZe/evezevZvu3bsTEBBAx44d6dy5M2/evCFNmjR07doVkDrgQ87OzowfPx4XFxc2b94MvHtcf/78eV6/fo2rqytXrlxRuvTEx8dz5swZLl++zOnTp5XzSJn+WJLAaVmrVq1o3rw5bdu25cSJExrvJSVx48aNI02aNMycOVPjfam4NWXPnp2GDRsydOhQjUek73vz5g3Tpk0DIDAwUOlY+z6pZP4nNDSUOXPmoFarKVu2LIcPH0atVnP06FECAgKoUqWKsgoIQFRUFIcPHyZXrlw4ODhw9OhRQMr0c0aMGEGrVq2IjIzE2NgYR0dH2rZty/bt24F3LR+TJ0/m+fPnpEuXDjc3N+VYqQP+p2jRopw5c0Ypk4EDB+Lq6srgwYNJTEzE3d2dkiVLkjZtWjZu3EivXr1wcnIia9as6OvrExAQIE81PqJq1apMmjSJbt26KckbvJtQeuPGjaxevRpjY2NMTEzImjUrV69epVKlSixfvly5QZaE+OeQBE7LSpQowbZt2zSSt/cv9gkTJpA+fXrs7e2TJXBCk42NDTly5EiWCH/o7du3TJgwgXTp0lG8ePFfFF3qlNQ/KGkesrVr1ypT2ZQsWZKFCxfSpUsXTp8+zcuXL6lTpw4rVqxAV1eXKVOmEBISwu3bt7X8KVIuCwsL8ubNi729PU+fPqVYsWIEBASwatUqnJyc2L59O0+ePCFPnjxkzpyZwMBAWUD9I9avX8/58+c5d+4ciYmJVKtWjUaNGtGmTRuOHz9O2bJlKVKkCBcvXqRbt27Exsaybds2Vq1apXEeSd6Su3//Pk+fPqVBgwZs3LiR6OhoZs+eTdGiRRkyZAgAx48fx9HRkdmzZ2NkZERCQoIyYwLIDdzPIgmclllYWGjMSQTvLnZ9fX0qVKjAvn378PX11VJ0qYuxsfEX+1sVLlwYZ2dnBg0axMiRIzXWlBSfduHCBXbs2EHlypWV0bo9e/YkJiaGkJAQpdN3XFwca9asoVixYty4cUNZlkwk5+7uTrNmzbh9+zZ37tzh9evX7Ny5k+7duzN16lRWrlxJ06ZNOXnyJCdPnlSO09HRkeTtPe7u7vzxxx84OTmRmJiIvr4+ERERLF++nOPHj1OzZk2mT59O7969CQsLY+nSpfTq1QsTExNWr16tcS5J3pK7fPkyjRo1YvXq1cycORO1Wk327Nlp1KgRd+/eBWDbtm0kJCRgZ2eHkZERwcHBMifhLyAJnJbdvHmTNm3akDlzZuXLAGBubk7r1q2Ji4vjn3/+0WKEqcezZ88wMTGhfPnyyjqGHypdujTx8fHKqgwiuQ8fdxgYGBAZGYm/vz89e/akdu3aJCYmMmHCBPr378+mTZswNzdHT0+PNWvWkJiYiIODA1FRUcTExGjxk6Rcenp6xMTEYGFhgYmJiTJKNyEhgf3799O1a1cmT57Mzp07lbngksgPoiZTU1Nu3bpFXFwcfn5+3Lhxg0WLFjFr1iyMjIzw8PAgJCREeZwXFhZGpkyZ+PPPP5MlcCI5lUrF5cuXcXJyYsGCBeTMmZNq1aopv1dJSdrOnTs1Vq2R5O3nk+FhWjZu3DgiIiJYtmwZBQoUIGPGjGTKlIkpU6aQJUsWGRX5DU6dOsXKlSsZOXIkFSpUSPa+tbU1Dg4O3Lt3TwvRpQ7vJ28dOnRg0qRJrFy5UrmZmDBhAv/88w+1a9emb9++AOzbt49169axatUqcuXKRVBQEC1atMDT05MXL15o8+OkGB9OoRAfH8+aNWsYO3YsNjY2BAcHK+8lJXE+Pj7cv39fpl/4go0bN1K8eHF27NhB586dOXz4MPHx8URFRWFgYEC2bNl4/vw5AOnSpeP+/fuMHTuWwYMHazfwFC7paUZSfRAWFoaLiwv3799n4MCBWFhYAJ++oZDk7eeTBO4ne7/yzZgxY7L34+LiaN++PZGRkaxZs4YdO3awdOlSLC0tadSokTKCT3yd2bNnc+7cOUJDQ2nevDm2traYmZlRtWpVVq9ezbNnzzQmkxSakirroUOH4u3tzZMnT9i5cydBQUEMHjyYV69eERgYyKFDh6hWrRojRoxQjjU2NiZz5sykS5eOxo0bK5PQ/te9nxSXLFmSevXqUbx4cVQqFStWrKBfv35Ur15dGRkN7378tm/fjrOzszL9gkguqXVo7969FC1alL///psrV64o7xsaGnL9+nUqV65M586dmT17NtmyZWPt2rVSrp+R1HpWuHBhVq9ejZ7eu4d1ly9fpnnz5hQpUoSpU6dibm6u5Uj/22Qt1F/E19cXW1tb+vXrp7E25Pvq1KlD2rRpiYmJYcuWLbJW3HcqVaoUHh4eNG7cmCdPnqCnp8e9e/c4e/Ys3bt3B2RU1OeULVuWqVOn0rFjR06dOkXhwoXZtWsXXbt2ZcWKFQCYmZkxbNgw1Go13t7eyrG6urro6+vLo9OPGDJkCA4ODrx8+RJDQ0PCw8MZP348586do0mTJgwZMoQdO3bQs2dPbYeaqlhaWjJkyBBOnTrFyJEjWblyJcOGDVNa3erVq0fbtm3JkiUL9+7dw9nZmfj4eKkDPiEpecuXLx+rVq1i+/btya7JvHnzsnz5ch4+fIiTk1OyFUHEryF94H6BihUrUqtWLTw9PT+avCVVJFu3btXYLp2VP+5LFe+xY8c4duwY8+fPJ3PmzCQkJHD16lXOnDnzVcf/16VJk4Y7d+5w6tQpHBwcCAoKom/fvqxYsQJTU1Ny587NyZMn8fX1VSrupDJNSEiQa/Yj3NzcaNGiBe3bt+fIkSMMGjQId3d3zMzMiI+PZ8OGDajVaqZNm8atW7eYNGmStkNOsT78/j558gRvb28SExMJDw/nr7/+QqVSMWLECJ48ecLmzZv5559/UKvVyvUqN8Yfl5S85c+fn7Vr17Js2TKGDBmCjo4OU6dOxdPTk7i4OC5fvkybNm3o37+/shyZ+PUkgfvJmjdvTokSJTh48CCnT5/+aMfOTyUT0odAU4ECBQgPDyc6Ovqr9v/UjOqSvH2ekZERNjY2NG/enLFjxzJs2DBlAukKFSrQqlUrBg4cqHRiloT405LKpkSJEixZsoQjR45Qv359OnbsyJAhQ9i9ezdp0qRBT0+P9evXExkZqTGvntD0/rXm5uZGrly5yJYtG8uWLePkyZPs2bOHli1bKhNPjxgxgqdPn2rcOMsi6h/3YfK2fPlyhgwZgkqlYsuWLejo6KCvr6/MmnD+/HmcnZ0BqQO0RfrA/WROTk506NCBQoUKYWBgIEnZd+rWrRs7d+5k8+bN1KhRAzs7O433pS/Lv9O6dWvWrl0LwO7du7lx4wZTp05l6tSpSvJmaGiIs7MzMTExGiOmpeL+NCMjI+Bdq+apU6coW7Ys06dPZ9iwYSxYsABdXV2aNWtGtWrVePv2Lbt371YmkxXJvd9Hs1+/frx69YrXr18zbNgw+vTpg7GxMfv376dFixY4OjoSEBCgrBjy4TnE/6hUKiV5W716NcuXL8fX1xeVSsWOHTt49uwZDg4On7x5ljLVDmmB+4E+dhfSokULgoKCqF27Nq1atWLFihVf3YIk3klKzpYtW8bNmzfp0qULxsbG7Nq1i9DQUCIiIqQC+ZeioqLQ09OjXr16bN68mWXLlmFmZkaVKlU4e/YsFhYWODk5YWNjQ9WqVQG56/6YSpUqsX//fuDdMk5Pnz5l3rx5REREEBISgkqlomfPnsoEsqampjg6OrJ7926N88iN3qdVqVIFe3t7Zbm8KlWq0KRJEw4ePEh0dDQ6Ojr8888/tG/fnh49esgjvq+gVqvJlSsXGzduZOnSpRrJ25MnT+jQoYOUYwokgxh+kPd/zAoWLIharcbIyEiZgHPWrFkUKFCAoKAgNmzYwJs3b7QZbqpTqlQpFi5ciL29PZGRkVSsWJHu3bsTExPDtWvXCAoK4smTJ5IcfyMzMzOeP39O+vTpmTx5Mrq6uspjEUdHRxo2bEiNGjW4cOECd+/epWvXrsTHx8scTx+RMWNG1q9fz9OnTzl16hSurq7UqlWLixcvYmpqSnBwMGXKlKFixYq8ffuWtGnTEhQUhJmZGfXr15fHeh/RoUMHjh8/rvRfBbC3t6dz5840aNAABwcHAgMDGT58OPPmzcPY2JjChQtz5swZjYE0crOh6WPl0aFDB0xNTQkMDESlUrF9+3aePn1K+/btJXlLoSSB+8EGDhxIvXr1MDAwIE2aNPz9998MGDAAeDfFRb58+QgKCuLvv/+WZOMrvF/RDB06FGtrawYNGsTz588pXrw4W7du5dGjR7x584YTJ06wZcsW5VGg+LxevXrRokULevfuzYEDB7C1tWX//v1Mnz5dWYcXwNbWlsePHyt9X6QD+Mfp6OhQokQJVq5ciY6ODg0bNuTs2bPo6ekRHx9PqVKlGDZsGAUKFODhw4e8evWKhIQEGjRoIEnxR5QrV46QkBD27NlDSEgIly5dAt6tH92sWTMmTZrEokWLGDFiBHPnzgXeJXflypVj0qRJyuogQtP7dWqdOnW4d+8e586d03h/9+7dPH78mHbt2knyloJJR4sfyNPTEzc3N3r16kW1atVYtWoVHTt2VNbb7NixI5cuXWLkyJGULVtWy9GmbGXKlMHc3By1Wq30Bzpy5Aj58uUjKioKKysr/vrrLxYvXkyhQoUIDAzExMSEOnXqaDny1CNv3rzkzJmT4OBg+vbtS/bs2enVqxd169alcuXKyn7379/XWO5NkjdNSY/4ExMTiYqK4uHDhzx+/Bg/Pz/09fWVVT+OHTtGw4YN6dOnD0FBQfj7+1OvXj3i4+PR1dWV5O0Dhw4dws/Pj0KFCtG5c2cKFSoEwN9//03u3LlZvXo1ffr0UZI3Q0NDWrduTfr06SV5+4yk5G3IkCEMHz6cihUrki5dOuU6btasGWfPnpWWt1RAWuB+EB0dHWbOnMm2bdtYvnw5DRo0YPLkyYwYMYIFCxZgbGystLgNGDCAcePGSYX9CZUqVSIwMJAVK1Ywffp0jdn8V6xYgb6+Pnny5GH37t307dtXWYYobdq0yt/Fl2XMmJH+/ftjYGDAs2fPyJkzJwYGBrx48YIbN24wbtw4WXLsC8qVKwe8SzYmTZpETEwMY8aMoUCBAvj7+/PixQscHBw0yvHDFkxpeUvu/dGO7dq1o1WrVoSFhTFt2jTCwsKoU6cOAQEBHDlyhJkzZ2JhYYGbmxuZMmWiWrVqcpPxBb1798bDw4PWrVtz5syZZOtxSyt76iAJ3A+SNm1aDh06hI+PD69evWLJkiUMHTqU+fPno6enh4+PD4cOHWLPnj3KMVJxf9qIESMoV64c27ZtY9asWcqknNWqVWPOnDls2LABHx8fWYz+G/Xq1YvY2Fi2bNnCtWvX6NatGxkzZmThwoWkS5eOcePGUaxYMeBdZ3FZTeHTTE1N2blzJzdv3uTFixfUrFmTRo0acf78eXR1dalUqRIjRozg+fPnNGnShPj4eAICAjh27BhLly7VdvipQs+ePcmUKRP169cnU6ZMrFq1ioCAAK5evUr16tXx8/PD1NSUx48fc+vWLdzd3eVx9Ee8/9jUysqKBQsWMH36dDZu3IitrS05c+akWbNmXLhwgXnz5iVL6ETKJKNQv8PHOoC+fv2a1atX4+bmRoUKFRg4cCCLFy8GwMLCgmLFihEREaFxjFQwySXd+Q0ZMoR+/fpRt25d1Go1s2bN4sWLF1y8eJH79+/z7NkzSd6+Q1xcHK6urpQtW5b169czb948tm3bxv3795k+fTp16tTB29ub/PnzExYWpu1wU7SoqCjq1avHvn37sLS0pFevXpw/fx7433qmQ4YMYcSIEZw6dYqbN2+SLVs2fHx8tBx56tCtWze8vLxo164df/31FxUrVsTNzQ0vLy8CAwPZtWsXe/fuJXv27Dx//pynT58C0nr0MUm/V7a2tjx9+hQjIyOqVKlCZGQk7u7uZM2alefPn9O6dWvSpEmjsaybSLmkBe4bvZ+82djYoKOjo8yJVa9ePSZMmMC5c+fo1asX9+7dw8rKiilTppAuXTrs7e0lafsKhoaGvH37FoBr167x9OlTVqxYwZw5c3j69ClNmjRhzJgxtGrVShnlK75esWLFaNCgAW5ubqxYsYJbt27h6elJhw4dOHz4sMa+8mP4aXp6euTIkYNZs2aRNm1arly5wvTp0/nnn3+UfXR0dMiVKxetW7cmISGBMWPGkJCQIC1EX6Crq8vixYu5fv26xqLzrVu3ZujQoezcuZPg4OBkLcQy2lRTjRo1KFGiBP7+/owZMwYLCwt8fHxo2bIlbdq0wc7OjpkzZ7J792727duHv78/hoaGeHp6ajt08RWkBe4bJVUOgwYNwtHRkbRp03Lv3j0mT57MunXrMDc3p2fPnoSGhvLixQsMDAzQ09OjTp06ygSdUnFr6tmzJ2q1mqCgIHR0dHj79i0GBgZs2LCBEydOcP36derWrYtKpWLmzJkcOHAAtVqNnZ2dJHCfUKVKFVQqlcYj+ySnT58mLCyMNWvWEBISQtGiRTE2NqZFixZcvHhRY9Z6Sd40vZ8gxMfHc+3aNapVq4atrS3Lly+nR48eqNVqDh48CLxrZb969SrDhw9XziF1wJclJCQQExODsbEx8L8yW7JkCUWKFKFp06YYGxszYsQIbty4oRwnydv/pEmThrJly+Lg4EDFihUpWrQoderU4eXLlyxevJjVq1djZmbGtWvXlGPy5s2b7CZOpFwyCvUrvT/Tf4sWLXBxcWHs2LF4eHhw69YtfHx86Ny5M0uWLKFHjx7MmTOHI0eOMHfuXGrVqiUjzT5DV1eXwYMH4+HhQWJiIiqVis2bN/Py5UtatmzJwIED2bdvH7Vr16ZDhw48fvwYDw8Pli9fru3QUxyVSoWpqSlTp05VJtz9mJiYGC5evEidOnX4+++/efToEdmzZ//oWr3infeTt9y5c1OqVCnSpk2LkZER9+7do127dmTOnJlu3bpRpUoVADZs2ECfPn00ziN1wNc5f/48jRs3Jnfu3Bpl9vjxY27evMnDhw+5efOmFiNM2d68ecPkyZOJjIykfPnyhIaGcvnyZQDevn1LZGQk165dI02aNJQsWZJly5aRPn16xo0bp+XIxdeSR6jfqH79+lhZWQGwcOFCZfvIkSOpU6cOXbp04fjx48mOk7vu5N7/QfTw8MDPz48hQ4bg6OjI8+fPk83+PXToUJo3b06vXr3YunVrsnOI/+nWrRvdu3encePGXLly5aP7JF2TKpWKzJkzc/fuXSnLrzBw4EAaN26MhYUFERERLF26lDVr1vD48WNy585NSEgIurq6GBoakpCQQLVq1aRT+Hdavnw5uXPnxs3NjTt37hAVFcWcOXP4+++/WbZsGSB1wOeYm5vTp08fjIyMKFu2LGvWrFHmeEyan7BBgwbY29tjZWVFy5YtZRBIKiIJ3DfInDkzhw8fxsjIiPHjx+Pv76/RR2j79u2Eh4fTqVMnLUea8vn6+mJhYUHfvn2VH7cuXbowYsQI7ty5Q5UqVYiKigI0+2E5ODjIRL1foUCBAkyfPp3Fixcza9asr66Q5cfw83r37k379u3x9PRk586dLF68mAIFCih9NB89ekS2bNmoUqUKadKkYc6cOSQkJEhfwu9kbW1NQEAA5cuX5+HDh6hUKlQqFeXLlychIUGu1w98qjysra3p0KEDjRs3ZsWKFUycOFF5r3Tp0iQkJHDy5EnUarVcq6mIJHDfQFdXl/LlyzNu3DiePn2Kk5MTb9++Vb40Y8aMIWPGjLRv317boaZoefPm5cCBA8C7Vsy+ffsqFYabmxv+/v4MGDCA2bNnK8d8WKlIxf1lU6ZMoXTp0pQpU0bbofwW8ubNS0BAAJMnT2br1q1UrVqV+fPnc+LECXLlysXSpUuZO3dusklkpTUjufe/v9mzZ+fevXufbaW0t7fHzMwMQ0ND5s2bJwNBPpA+fXqN+TI7deqEnZ0dKpUKf39/Hj9+jK2tLS4uLtjb27Nx40b8/f0JDQ3lypUrDBw4EJB6NbWRBO4T3r+QdXR0UKlUSgJRsWJFZs+ezYkTJ+jevTtv3rwhLi6OzZs3c+nSJby8vLQZeqowefJkTE1NqVixIvv27aNTp05KZdylSxeGDx/O4MGDmTlzppYjTfmyZ8/OrVu3lNdJk6DmypWLZcuWMW3aNGW2evH90qVLR9WqVdmxYwdFihRh7ty5jB07loULF7Jy5Upy5crFli1bGDt2rMaPqdD0ft3q4+NDgQIFWLhwIXv27EmWPHwqoZDk7X8GDRqEh4cHpUqV4uHDhwwePBgXFxcOHz6MnZ0dVlZWNG/enNOnT2Nra0uLFi3w8PAgOjqaV69eUb16dZmwO5WSQQyfkFRpeHp6Mnv2bDZt2oSzszO5c+fmwIEDyhJZ27ZtY9GiRUyfPp20adPSu3dvLUeeOkRERGBpaUmrVq0oW7YsM2bMUJbMmj59OkOGDMHPzw9vb28tR5qyFSxYkOPHj7No0SI6dOgAoLRkPHr0iKtXr1KtWjVthvjbePnyJbt27SI6OpqWLVvy999/K3M93r59mzdv3qCrqyvJ2xe8v5RTx44dWb58OWfPntVI1JLqgk+1Bkny9j/Lli3jzJkzbNy4kSxZsmBqakrz5s2V1raDBw+yYsUKSpQowb1795gzZw6NGzdm+PDhVK1aVRlgJ1IfSeA+8P5o0z59+uDp6cnt27e5desWXl5eDBw4kD///JMDBw7g7u7O27dvyZcvHxMmTKBChQryZfhKEydOJF26dBQrVoyOHTtSvXp1pk+frlTcISEh+Pv7Y2dnp+VIUy57e3sqVKiAs7Mzenp6eHp68s8//9CxY0dy5cpFVFQUEydOpEqVKjRs2FDb4f4WkgbVmJubkyZNGvT03s3EZGpqyuDBg+nbt682w0s1KleujKOjI82aNVNGnGfKlInq1auTPn16Zcol8WXXrl2jR48eREZGsm3bNv7880/lOn3y5Ak9evTgwIEDhIaGUrx4cV6+fMnly5dZt26dUs7S5y11km/IB96fsdrGxob27dszbNgw3N3dGTJkCCYmJnTq1AkrKysOHTpE//790dPT05hsUr4MmoYNG8b8+fNp0qQJ5ubmwLsyWrt2Lfnz5+fQoUO4ublRq1Ytpk2bplTcEyZMoFu3btoMPUVSqVSYm5szZswY7t69y9atW3F3d6dJkyacO3cOZ2dntm3bRu/evbGwsGDt2rVUqlRJbix+oJs3b1KkSBFmzpzJli1bKFiwoDLn3vs3geLj1Go1r1+/5uXLl+TNm5f+/fuzadMmAgIC2LVrFxYWFtLK9gXvX2fh4eF07dqVEydOUKhQIQwNDZV9oqKi8PT0ZN++fWzbto3cuXNrnEfKOfWSPnAfYW9vz9y5c3n06BGdOnVSJuVMei8gIIDmzZtz6tQpVCoVFSpUYPr06Vy7dg1HR0ctRp7yvD9gYevWreTPnx9/f3+OHj3Kq1evOHLkCO3atWPPnj1UqFCB+fPnc+rUKZo3b67lyFM2fX19Tpw4gaenZ7LJevPly0etWrVwdnbmzZs3FCxYkJiYGEqXLs39+/e1E3Aq8K0duH19fcmQIQOJiYn07t1bOtZ/wvvlamtry+PHjylYsCCTJk3i9evX5MuXj40bN3L06FHCwsKYNWsWvr6+bNq0ScuRpw6lS5fm6NGjAOTMmZOgoCAyZ85MvXr1ePjwobJf+vTp6datG2PHjpVr9DchCdxHGBgYMH78eNq0aYO3t7fSzyXJ0aNH+euvvzTWi6tatSrjxo3D0dGRe/fu/eqQU7RWrVoREBBAcHAwT548oWrVqmTOnJl169ZRtGhRnj17Rr9+/Xjz5g1Vq1alYcOGySY/FZoMDAw4cuQIHh4eSuX9YQKSO3du8ubNi7e3N3p6elSrVk0q7k94f5Tzl6ZR+FSSJtMvJPfhgIXcuXMzc+ZMjh8/TvXq1cmVKxe3b9/m4MGDREVFYWZmxpo1axg+fPhHVxERmmWaP39+9u3bx6BBg5QBXzly5GDatGlkzJiR+vXrK9OvvF83yLX6e/jPJ3Cfuus2NDQkODiYatWq0b59e/bv349arcbMzIytW7cyZcqUZIldmjRpePPmza8KPVVp3749Y8eOpWfPnuzevZscOXLQr18/ChUqxPnz52natKmMhPqC+vXrc+vWLS5cuECWLFnYs2cPDg4OygLqST52TSdtkxai5GrVqsWLFy84evQoo0ePxsrKCnd39y8eJ1MufL0hQ4bQqlUr+vXrx+HDh3n06JHG+/r6+qRPn57Jkydjbm5OgwYN5Dr9gh49eqCnp4ePjw8Ao0ePJjg4GHiXxE2dOhUrKytpVPiN/acTuPcr4JIlS2JgYMDr1685c+YM8O4uZc6cOVStWpXQ0FDCw8OpVKkS2bJlo2rVqnIH8406derEqFGj8PPzY8qUKRgYGJAvXz5u377N8+fPtR1eimZoaMjChQupWLEi1atXJyIigosXL1K7dm1leZwvkYTj43bt2kWGDBk4fPgw1apVo2HDhoSFhX318Tlz5uThw4e8fv36J0aZelWtWpXJkyfTpk0bzp07h46ODlZWVmTPnp3w8HAeP35M9+7dqVy5MmZmZtSvX19WA/gCHx8fOnbsiKenJ2nTpqVw4cJ069aNUaNGKU+GsmfPzvLly7lw4YLMTfqb+k8vZv/+wvTNmjXjzZs3/PHHH0yYMIHFixfz4MEDOnToQHBwMB06dGDVqlXs2rWL+fPny+zq32HWrFkkJiYyduxYdHR0CAoK4uzZs4AkF1/y9u1bvL29GT16NOvXr6dz585cunSJevXqkSFDBkxMTNDX1ycqKgqVSkW+fPlYt26dxp23lO/HVa9enQsXLtCwYUN69er1Tclbp06daNWqFc7OzpLAfYKuri4PHz4kMjKSvHnz4uTkpLS4P3v2jObNm3P27Fni4+OZOXMmiYmJUre+J1u2bNy+fVt5nTZtWqpXr65MKA2wevVqHjx4gJ+fH7GxsYSEhHDr1i0cHBw0+sGJ38t/OoED8Pb2pnXr1nTo0IHDhw/j6+tLv379MDc3Z/LkyTx8+BBPT09UKhWVKlVi7ty5SmdlqWDe+Zbka86cOQCMGjWKhIQEpclfkosvu3fvHv3792fChAksX74ceDedRceOHdHX10dXV5c3b96gUql4+PAhM2bM0HLEKVfSNatSqTAxMeHhw4c8ffoUb29vwsPDOXz4sPJ+0rX54XXu6upK//796dOnjzyi+oz4+HgyZcpEYGAgxYsXZ8uWLUyYMIHHjx8zevRoChUqxL59+9i3bx+A1K3vWbhwIS9fvqR79+7KNgMDA7JkyaLROqlSqZgzZw6VKlVi+PDhJCQkMHPmTGXQkrRm/p7+c49QP1zCZeTIkYSGhvL333/ToEEDgoKCWLduHW3btmXWrFlMnTqVe/fuoaury+zZsyldujTu7u78888/Wv4kKYOenh7Lli0jLCyMW7duMXv2bKWi+Fyl0b59e8aNG0erVq3YsWPHrww51ciZMyc2NjZkyJCByMhIZTSvlZUVgwcPplWrVjg4OHD48GHSpUun3FioVCplMllp2Uzu/TKxt7fn0qVLXLt2DYDNmzeTIUMGevTowZEjR5TrN2l1iySurq4MGzaMHj16sHHjxl//IVKg98vVxMSE2NhYYmNjAWjYsCF2dnZcu3aNAwcO8Pz5c9KnT8+6desYNmyYDFj4hPTp0xMdHU1cXBwWFhY8ffoUgHHjxlGhQgXc3NyUaxfAz8+PwoULU6FCBdq1ayfX5m/uP5fAJUnqPO/k5MTmzZspUKAAc+bMITg4mFmzZuHn50enTp1Yvnw5w4YN4+nTp+jp6REaGkq2bNmoXLkyMTEx2v4YKYK9vT3p0qVj4MCBXLhwgf379zNjxgxiY2M/m8RVrlxZuesWmlq2bKl0UraysiJdunTs27ePOXPmsGnTJqysrAgKCqJEiRI0b96cc+fOAXy2xUhoGjp0KA0bNmTZsmUsXLhQ6Vi/efNmLCws6Nu3LydPnmT69Ok8fPhQWWUlKXnz9PRkw4YN2vwIKcb711qXLl2oVasWOjo6REREKK1Henp6xMfHo6enR9q0aQkJCSF9+vQ0bNhQWoc+4v26093dHRcXF9zd3bl48SJly5bFx8eH169fM2TIEMLDwzE0NGT27NksXLiQOnXqUKxYMZo0acLLly+1/EnEz/KfSeDer2BGjBhB586dyZUrF/Hx8bx584ahQ4eSNWtWunfvTkxMDH379qVkyZIYGxtjb2+vHKurq0vGjBnlkclHmJub4+3tTcmSJXn+/DkdOnTgzZs3X2y+l0RDU/PmzQkICMDHx4d//vmHhIQEihYtyoQJE4iKimLEiBH8/fffWFtbM2HCBEqXLk2zZs2UJE58mbu7O3369KFFixacP3+euLg4jX5X69atI2fOnLx+/Zq4uDiqVatGfHw8jRo1Ytq0aXTu3FlaNz7C19eXli1bEhwczIsXLxg0aBCXLl2ibdu2vHnzhjRp0tC1a1fKly9PunTpqFevngxY+AoZMmRg7969XL9+nZ49e3L9+nXs7e1xc3OjaNGiHD9+nKxZs5KYmEilSpXo1asXdevWpXbt2toOXfxE/5kELomdnR2dO3dm1apVHDp0CHiXQCxcuJC4uDjc3d1JSEhQ1jdNelQqScbnJVXAhoaG1KlThx49evD27VuaNm1KTEyMlN9XypIlCwsWLGD+/PksWrRI4708efKwatUq7t27R8uWLXn27BmZMmVi5syZvHnzhhYtWmgp6tTFwMCAqVOncunSJQICAjT6w71/jbZs2RK1Ws3KlSuVxC5nzpxkzZqVvXv3aiv8FCNDhgw8fvxYeV27dm2GDBlCz549OX78OHXq1FEGJVy7do1GjRrx5s0batWqRZEiRQgMDJTBYB/xqboyQ4YM7N69m4iICLp27cqNGzfIkSMH1atXJ1++fDx+/JjAwEDi4uIICAjAzMyMrl27ypOi39h/KoFzcHBg6NChvHz5kubNm/Po0SPli9K0aVOmT5/O/v37sba2BqBKlSpSsXxCpUqVsLCwQE9Pjw0bNih9XeBdMlelShX69+/PuXPn6Nevn5TjVypWrBiLFy+mdevWyghd+F+C/Oeff7Jlyxa6d+/OsmXLgHctn8+fP5cE+RM+/EHU09Nj69atHDx4EF9fX419DQ0NsbOz48KFCxrbdXV1SUxMlDL+f5MmTUJHR4eAgABu3boFvJunMF++fAQEBFCzZk2mTZvG2LFjOX/+vHLD7ObmRnR0tHIeaXnT9P612rBhQ3LmzElcXBwnT57kyJEjZMiQgZ07d3L37l08PT25evWqxvFWVlZ4eXnRunVr6tev/9VTDInU6T+1FuqbN28IDw8nR44cmJqaolarlfUhV65cSYcOHbhx4wZbt25VkjdZUDm5wYMHM2nSJHr16sX06dOZMWOGsqh3UoW8f/9+Vq1aRZ48eShZsiQga0R+jUyZMmFkZERUVBSAcv0lTa1w4sQJTp48Sa5cuZRjnj17prQgieSSfhCT1uE1MDDgzp075M6dG3Nzc41yy5o1K15eXuTJk0fjHAkJCZK8vef8+fPKJOd//PEHAJs2bWLZsmUYGxvj7e3NzJkzmTt3Ljdu3CA8PJxq1arh7++vcR5J3jQlXWNDhw7Fz8+PcuXKUaxYMTZu3EijRo14/PgxVatWxcbGhoCAAIoUKaIca2lpiZubG8WLF6dx48aSvP0H/CeyEycnJ+zt7dm6dSvBwcFcuXKFGTNm8Mcff2gkaevXr6dPnz74+fkpTftSwWjq0aMHrVu3plOnTjRs2JAyZcpQo0YN2rRpA/yvQo6Pj2fx4sXo6urSqlUrQKYK+RpXr17FxMQEBwcH4F15JiUYSa2YKpXqoxMfS/l+WuPGjTlw4AD58uUjOjqaKVOmUKlSJQYPHoytrS26urqYm5szYsQIzM3Nk7VsCE1z5sxh9OjRNGnShPbt25MzZ04A7t69S6ZMmbCxsWHXrl3Au+vywoUL1KpVix49emgz7FTB3t6eZs2a0bFjR1q1asW2bdsAMDY2BuDp06fUqFGDUqVK4eLiohz35MkTFi9ejIuLS7LVWcTv6befB87IyEjpQLthwwZ27tyJvr4+nTp1YvLkyfTo0YPw8PCP9sOQx36a8ubNS61atRgwYACnTp1CV1eXmzdvsnXrVnLnzq2xr46ODtHR0QwYMIDg4GDy5MnDlStXtBR56nH37l3WrFmDh4cH4eHhrFmzRiMxMzc3x8DAgDJlymBsbMyOHTu4dOkSb9++1WLUKd+TJ084f/48M2fOxMPDgxMnTtCmTRvmzZtHsWLFMDQ05PXr1xgaGlKzZs2P9okTmo/4QkND0dfXp2/fvsC7pC48PFyZe6xPnz6EhITQs2dPdHR0OHPmjCzn9hX++OMPdu3axYkTJ2jYsCEBAQH06tWL0NBQTE1NyZQpE1evXiVPnjy8evVK49gHDx5oKWqhDb9dC9z7j0P09PSIiYmhZ8+eVKhQQbn727JlC7NmzSImJoagoCDs7OwkWfsKT548ITo6muvXrwP/S3AjIyOVO/D3H/klvXf79m0MDAy0EHHK9rFHnjExMSxatIi7d+8ybNgw5Q47TZo0WFtbExwcTIYMGciYMSOWlpZkyJBBkrcPfKxcDxw4wMSJE4mIiGDOnDnky5ePPXv2ULNmTebNm8f69etZuHAhNWrUID4+Hl1dXUnePvB+8la2bFkAFi1axJgxY3BwcKBDhw7kzJmTN2/e4OPjg52dHePGjUNHR4emTZsqSbEkbx+XVHe+P0fh1KlTGTp0qDKgqVatWrRp0wYzMzNevnxJYmKidPP5D/ttBzG4u7ujq6vLli1buHnzJu7u7jRv3pwhQ4Zw8OBB4N2oqf79+3PixAllQWDxeSYmJspdX9Kd9MCBA8mVKxcdOnQA3jX158iRg4sXLwLv1kIMCwuTu8P/5+vry8qVK7l06dInW3mqVq1Kt27dqFq1KleuXEFPT4/IyEj09fVlaoCv5OTkxKFDhzSm/ClTpgxeXl5kz56d9u3bc/ny5WT/D6SF6PMGDhyIg4MDM2bMUFZWad26NQMHDmTt2rVMmzaNe/fuYWRkhK2tLTdu3ACQ0aaf0bRpU4yNjVm4cCH29vb4+vqSKVMm/Pz8mDVrFvCu7p09ezZXrlxhyJAhWo5YpAS/5SPUjBkz4unpibGxMU2bNsXPz49du3ZRsWJFKlasyMmTJ4mJiWHbtm08f/6cY8eOaTvkVOPDJnt4198tiZmZGTt27GDp0qVKAiezrP9P4cKFqVy5MuXKlcPLy4urV69+NInbs2cPly9fJm/evFSpUoU3b94QFhamzD0mP4aa5s2bR3h4OMOHDwfeTbnSo0cPXF1dcXd3V24ejhw5wsyZM5kyZQrTpk2je/fuXLp0SeNckrx9Wp8+fXB1daVt27Ya63MuWbIEtVrNwIEDSUxMZNGiRVy9elVJ3lQqlVyvn6Crq4uDgwPp06dn4cKFbNiwgUqVKtGmTRvevHlDwYIF0dHRwdfXFysrK6W/sRC/ZQuckZERHh4elC5dmmPHjtG1a1eCgoIoVqwY5cuXx8nJKdmC1dLf5fsNHjyYvHnz0q1bNzZv3szDhw9p0qSJtsNKsapVq0bnzp0xMzOjR48eXLly5ZuuP2kh0qSrq0uXLl0YPHgw48aNY9KkScC7gQuurq6oVCq6du2q9M3S1dVl7dq15MqVi/379+Ph4aHN8FMNS0tLZX7CpLV4QfNmonXr1kyaNInBgwcrLUdC0/vfdQMDA2JjYzEzM+Po0aMsXbqUoUOHAjB16lQKFSpEvnz5OHXqFG/evKFZs2Yy8bFQ/FYPz5s1a0bRokWJiYlh2bJl5MqVi/v371O7dm2yZs3K27dvsba2JiQkBFNTU41jJXn7fq9fvyZ9+vRs2LCBBw8eKMmbTGuhKWmqld27d7NkyRKioqIICAjgjz/++KZpQKTi1pSQkMD06dMZMGAAffv2VZa8WrduHfPmzUNHR4epU6diYWEBvHvEf+fOHby8vOjcubM2Q09VTExMKFq0aLJW+ISEBNKkSQO8a4lzcXFRHq2K5N5fcszLy4sCBQrw/PlzhgwZQsWKFalTpw4A3bp1w9XVlcaNG9O1a1eaNGmi9M+UOkDAb5TAZcmSBUdHR7Zs2YK7uzuvXr2ic+fOeHt7Y2lpyYgRI1i4cCFhYWG8efPmo48CxffR19enXLlyXLp0CScnJ0BaND8m6VGzl5cXjo6OWFtbU6ZMGYKDg8mdO7fM5fYdkjpwJyQkcObMGebNm0f//v3p0qULABs2bGD27Nno6uqyc+dOfHx8CA0NJXPmzOzYsUPK/Bu8fPmSsLAw8ubNi6GhIfC/m7SqVasycOBAALZu3Sqd67/A0tISV1dXunfvzowZM3B0dOTQoUPcunWLsmXLYmJiAkB4eDiHDx/mxo0byrUqj6JFkt/qEaqRkRGtWrWie/fuXLx4kf3796Orq4ulpSVBQUHK5KifWjpHfJ+8efPSvXt3ZZSvlOunubu7M2jQIFxdXbl16xbVqlXD0dERPT09evTowbVr16T8vsOQIUOoWbMm586do1SpUvzxxx+MHTuWiRMnAvDnn3/i7OxM7ty5lQXW4+Pjpay/0cSJE6lTpw59+vRhx44dxMfHY2RkxOzZs4mLi6Ndu3baDjFV0NPTo127dtSoUYMdO3bQq1cv5s2bR/bs2alXrx4tW7bk2LFjcn2Kz/qtErgkJUuWpH79+jRq1AgzMzMePnxIjx49OHnypLKPfDF+DinXT9PT02PatGk8ffqU/v37K9vr169P//79efHihTIvofh6tWvXZtasWTRt2pRjx46RKVMmmjVrhq+vL2PHjiUgIEDZ19TUVLmRk4EgX+/97/WCBQsoVKgQ58+f59GjRxQsWBBTU1OqVaumMaBJJNeqVSvu3bvH3r17MTU1Zf369SxfvpwVK1bg6emJqakpzs7O3Llzh9q1axMZGantkEUK9lu2cR8/fpyAgADat2/PxYsXyZMnD+7u7hr7SJLxc0i5flp8fDwxMTHkypVL4/HSpk2b2Lt3L2XLlmXZsmVky5ZNi1GmPhkyZODWrVvKaPIHDx4wb948AgMDGTBggDK9DaAkbyATdX+LpAl4AVxdXZkxYwZPnz7FxsaGI0eOULVqVaV/lvg4GxsbqlevzsqVK/Hx8cHQ0JAOHTrQqlUrChUqhJ+fH3PnzuXw4cM8evSIJ0+eaDtkkcKlqha473n0qaenR7NmzVi+fLlU2OKX+dQ12q5dOzw8PBg4cCAHDhwgNjYWgDZt2mBvb6/cfEgn5a9XtWpVZf6sM2fOKNvLli3LunXr0NHRwdPTk6VLl2oxypStSJEiysj82NjYT16/H45+fH8/adH8Mj09PRo0aMDAgQO5efMmx44dIyoqCltbW4KCgnj27Bkg3XzE10k1CZyDgwNVq1YlKCiIBw8e8ObNmy8e82FlIxWM+BXer3Tr1KmDhYUFBgYGrF27lhcvXrBkyRJy5MjBuHHjOHLkCK9fv2batGmcPXtWWexbpglI7lM/Zkkjyx89esTkyZOV+Qft7Ozw9PRk8+bNbNu2Tb77n1C9enWWLVvGggUL0NHRISgoiFu3bmk7rN9a/vz5adiwIY6OjmTJkoXHjx/Tu3dvjTkzJXkTX5IqEjhTU1P27NmDiYkJDx484OTJkxw8eJAVK1Yo+8gPnkhphg4dSrNmzThz5gx58+blxYsXjBw5kt27d7N48WKyZctGpkyZePz4Mbq6ulSoUEGSjE94/8esRYsWZM2aFQsLC1atWsXJkyepVasW3t7evHjxgtDQUO7evUufPn2Ij49XJj6VG7iPK1++PEuWLCEoKIgMGTLQuHFjli9fzsmTJ9mwYYOyn9SxP5aRkRGZM2dm+PDh1KlTh02bNuHq6qrtsEQqkipWYnj9+jVr164lPDycc+fOUalSJcaNG0f16tW5dOkSwcHBUrGIFKVFixY0a9aMVq1ace7cOZo1a8a0adOU6RecnZ0pWbIkuXPnJiEhgVWrVpGQkCA/kp+QlLwNHz6cVq1a8c8//1CwYEFq1KjBpk2bGD16NLGxsbRo0YIZM2Zw/fp1oqKiaNCggXIOSd4+7vDhw8q8hJMmTeLkyZNkypSJyZMnU79+fQ4ePMjixYvluvyCRo0asX//fuUx6JfExMRw/fp1nJ2dady4sUayLMTXSBUtcPBupNmMGTOoW7culy9fxtjYGC8vL3r16sWZM2dYs2YNO3fuTLbCghDa0L9/f6ytrenVqxeOjo5MnDgRPz8/5s2bh4mJCYaGhsk6KUvy9nnVq1cnMDAQZ2dnzp49C0Dv3r2pUaMGO3fuVKYMyZIlC7q6uty+fRu1Wi0tb5/wfquml5cXzZo1o3r16sTGxqKnp8eZM2d4/vw5MTExGBsbs3TpUlasWKGsaCH+p2XLlgwYMICFCxcya9YsXr58+VXHSTcf8W+k2FGoSS0VSSOftm3bxqpVq5R5hqKjo2nYsCGbN2/m0KFDVKtWjf3799OiRQutxSxE0vWaOXNmHjx4QOHChQkMDGTEiBHMmzcPlUpFy5YtadSokbIyQxJJ3jR9OBGsqakpsbGxGovTBwQEcOTIEZo3b46xsTEAERER3Lp1SyY+/YTy5csDKMktQFBQENHR0bi4uACwc+dOLl++TLNmzWjbti1nz56lePHiypqyQlNoaChr1qyhfv36eHh4kD59+q86Luk7nzQhslyr4lukyASuUqVKTJkyBRsbG40Zvc+cOaPMObRr1y6eP39O165d8fX1xdPTk86dO7Ny5UotRy/+Sz6cxT+pQt6yZQuenp7s2rULb29v5s+fD0CaNGmoU6cO2bJlkzmzviCpLDt37kyJEiXQ19dHV1dXubnT09NDrVYTGBhIlixZlMTkfdIJXJOZmRlz585l06ZNwLuEIelG4u+//6ZixYocO3aMFy9e0LlzZ+7du8e9e/fw8PCgXbt2snLFRxgYGAAwbNgwpTGhY8eOyZZr/JxcuXL9rPDEbyxFJnAFChQgV65c9O/fn0yZMikV+aJFizA2NubGjRtERUXRpk0bZUmse/fuKf2IZC4i8askJQg1atSgZcuW5MuXD2NjY7Zs2cKiRYt49OgRcXFxpE2bljx58jBv3jwsLS0ZOXKkliNPud5PEJydnRkxYgQvX77k77//BmDcuHGoVColAba0tCQ8PPyr+x79lz1//hwXFxcyZszImjVrgP8t8bZ+/XrKly9PbGwsjo6OPHr0CEjeEipJsaakqYBatWpFfHw8dnZ2dOnShU6dOpEuXbovHt+hQwcOHTpElixZfnao4jeTYvvAdezYkcaNG3P79m2GDx+uVCYtW7akW7dudOvWTekHI4Q2DR8+nObNm6NWq3n16hVr1qwhODgYMzMzvLy8aNOmDZGRkTx9+pTnz5/j5OREfHy89Hn7gqpVq5I5c2ZiY2OVEefFixdn8eLFXLlyhblz5/Ly5Us6d+6MpaUldevWlfL8Crq6uuTLl4+FCxdy4cIF3NzclHLr1q0b1atXp0ePHhqPqsXn+fj40LlzZ/r27cubN29o0aIFefLkYeXKlcycOVNjAun3ubq6MmjQIHx8fFi3bt0vjlqkdimuBS7pbm/27NmEhoZSoUIFBg8eTKZMmQD4559/sLS0pFKlStoMU/yHvd9CVLJkSYoWLYqzszNlypRh9erVVK9enX79+vHixQv69OlDrVq16NOnD97e3jg4OCgz1kuy8Wl2dnasWLGCwMBAjVaMU6dO0ahRI9KkScOQIUMYN24c+vr61K9fXxZQ/4SkhdHh3WPnhIQELly4wJ07d6hXrx4rVqxQrulLly5hZ2dH/vz5tRVuqmNubk69evUYNWoUq1atUqYD2bdvH25ubhqPU9+/Pl1dXRk2bBi9e/eW5E18lxRR2xUsWBBLS0tAs3m+Ro0aGBoaki9fPnx9fbG1teXOnTsEBwfj5eVFnjx5tBWy+A8qVKgQ8L9r1MHBgQ4dOnD9+nVOnDhBVFQUY8eOZdOmTZQpU4a+ffuSMWNGzp8/z9atWzl58qR0rP9Kt27dwtXVlQcPHlCxYkVlu46ODtevX6d+/fo0btyYZs2a0axZM0mKP6FChQrMnDmTvHnzAv97XDp37lzMzc1xc3Pjjz/+UB6n7tq1i4iICNq2bau1mFOb6OhoEhISlEQ5qQtPv379uHv3Ls7OzvTp0wdTU1Pl+nRzc1P6bsv0IeJ7aTWBU6lU2NrasmfPHnx8fLC2tlZ+HOfPn0+uXLmoUaOGMnP9wIEDyZAhAwcPHmTXrl1cvXpVm+GL/5ARI0bg5uYG/K8Frk6dOtStW5ciRYpo9LucNGkSmzZt4s8//8TX1zdZPxjpQ6TpY53i4+Li2LRpEwMHDqRGjRrKFCGJiYlKonbv3j3u3LkjSfFnZM+eHUtLS/r160fWrFmBd3Vr7ty5adWqFX///Tfu7u5ky5aNtWvXAtC9e3fat2+vxahTro9dq7GxsTx69Ig6deooLZxJLW1XrlwhPj4eIyMj5TFqlSpV8Pf3x9vbW5I38a+kiD5wTk5OBAcHM23aNMaNG0dISAi5c+embdu2hIeHA//rE/f06VM6deqkdByVfkTiVyhVqhSnTp0iPj6erFmzcufOHVQqFUOGDMHe3p6lS5cm6+syePBgLCws6N27tyRtX6FLly4ULFgQa2trFi1axIkTJ7h37x729vZMmzaNZcuW0adPH22HmSpYW1sr/YabNWtGmzZtePz4MZaWlpiZmeHm5sbt27eV/f/88082bNjAnDlz8PX1BaRu/dD78+YVL14clUqFrq4ux44dw9bWlm3btnH8+HF69OhBTEwMcXFxzJo1izVr1rB582blRiMpqT5x4oSWP5FI7bSWwJUoUYKXL19y/fp11Go1Dg4OzJo1i/v37/P8+XNatmzJvXv3NCY29PT0JHv27PTp00d+EIVWODo60qVLF0aNGsXevXtRqVSMHTuWEiVK8PfffzN79mxlZPT7ZF3D5N4vk379+uHh4cHKlSvJmTMn2bNn59ixYwQGBnLlyhXs7e2ZPHkyO3bsoFOnTlqOPGVzcHCge/fuBAYGsnHjRuDdyiBubm7ky5ePdu3asWfPnmTXZN68ebl69aokbV8waNAgGjVqRGxsLDY2Nqxfvx5/f3+yZMnC/Pnzef78OY8fP8bU1BQTExPKlSun9M+UshU/klYeodrb27N161YGDhxIjhw5AFi7di0uLi7Y2Nhw8uRJpSXj/eboyZMnK60ZMheR0IY3b97w4sULunTpQuXKlVGr1fTv359Tp07RoEED2rdv/9H5nyR5Sy6pTDJmzEiOHDlo06YNffv2pWnTpkycOBFbW1s8PDwwNTVl06ZN+Pj4YGlpKd/9z7CysqJv377kzp2bpk2b0qhRIwCWLVvG7NmzOXv2LC4uLuTNmzfZNXn58mUZCPIFXbp0wcXFha5du1KpUiVCQkJo27YtGTNm5NixY5QpU4a1a9dy+vRp9uzZQ/ny5SV5Ez+NVr6p+vr6ADRo0IAxY8aQPXt2ADZv3kyHDh1o06aNUlnDu34vH1ba8oMofraPJQpbtmwhJCQEeNcinJTE9evXjxMnTtCuXTvq1av3q0NNtVq0aMHp06f5888/iY6OVrYnzWxfr149rKyslPVimzRpIjdwnxEZGck///yDWq0mMTGRFi1a0LBhQwBWrVrFkiVLsLCwoH///p8cBCaJxqcVLlwYf39/Tpw4gb29PV26dKFv376cPn2aNGnS8OrVK/z9/Rk2bBh+fn7KvKRSpuJn0EoCd/DgQZYsWcKQIUOws7NjypQpZMuWDXg3mWTHjh3p0qUL3bt3x8rKCpCETfx6SddcnTp1aNy4MY6OjsC7ZYZCQkJISEigR48eVKpUCbVazYABA5g1a5asBvINNm/ezM6dO/njjz+UOiApOVuwYAFqtZrq1asDmnWA1AfJJa2oEBgYyN69ewkLC0NXV5d27drRoEED4F1L3NKlS0mfPj3jx49XBjaILzMyMqJkyZI8fPiQUqVKERwcrKxvrKenR79+/T46vZUMrhE/i1YSuAcPHpCYmEilSpWoV68emTNnZvLkyUoFvm7dOjp06ED37t1xcHDQRojiP2rMmDEaqySMHj2aqVOn4uvrq/R7K1iwIHv27CEkJIT4+Hi6d+9OzZo1UavVTJs2TR5DfYOXL1/SpUsX9u/fj5+fH0WLFlWSMysrK6Kjo3n69KmWo0zZkp5UJE0R8vr1axITE3nx4gU+Pj7Ex8cnS+LWr19PWFgYERERWos7JftYC29MTAwrV66kR48erF27lgEDBihL5JmYmFC4cGEKFiz4iyMV/2W/ZBDDn3/+SVRUFHfv3uX169fAu4Wp161bx4gRI7h69Spbt27lypUr9OzZUxkdValSJQ4ePCh3MOKXSJcuHX369KFGjRqsWbOG5cuXM3v2bHr16sXjx4/R09Nj9uzZZMiQAQcHByIiIqhZsyZ9+/bl6NGjDB48WNsfIdUyNTVl8eLF2NnZ8ddff3H79m3q1atHtmzZqFq1qtQBn+Dg4IC3tze7du1i+vTpREdH8+rVK6pWrcrMmTOpV68eRkZG+Pr6oqOjw/z585V1UJPIABtN75dH3rx5sbS0JCIiggcPHlCoUCECAgJ4+fIlnp6ehIeHkyFDBiZPnkz69Olp2LChPC4Vv8xPT+AaN27M7NmzOXPmDM+ePWP06NHcuXOHyMhIJk6cyNu3bxk4cCDZs2dnw4YNXLlyhb59+3Ljxg3lHO+PRBXiZ8qYMSNt27alUaNG3L59m8TERDp06EBcXJyyz549e4iMjKRp06bAuxuUpEl6xfczNTVl1qxZ1KhRg6VLl3L9+nWCg4OVfkRSB2jKlCkTy5cv548//kCtVrNz505iY2OZMWMGFy9eZPjw4Vy+fJm5c+dSpkwZPD09yZQpE4MHD+bQoUPaDj/F8/X1pU6dOlhYWHDt2jUeP35M9+7dsbe3p127dmTJkoWHDx8qCV/dunVliTzxS/30BK5q1aqsWLGC06dPc/PmTQoXLsy5c+fYvn074eHhLFu2jKZNm3Ly5EmyZs3KsWPHmDt3LgMHDvyZYQnxSZkyZaJt27Y0b96c6OhoqlSpAoChoSFv376lQYMGjBgxAicnJ2WeQpCWjI/51h+zdOnSMWvWLLJnz46rqyuXL1+WH8TPaNKkibI828WLF1GpVHTs2JHly5dTs2ZNYmJiqFOnDrGxsZQtW5Z69eoxbNgwuU6/oHPnzvTs2ZN27dpx6NAh/P39ad26NU2aNOHIkSMUKlSIggULYmNjQ3h4OOvXr1cmmZYbDfGr6P3Mk6tUKvbs2UPz5s1Zvnw5W7ZsYe3atZiZmTFixAiOHDmCqakpVapU4ezZs9y5c4ciRYoQGRn5M8MSQsOHideDBw/466+/AOjRowdDhw5l+PDhvH37FnjXFwaSd06WH0VNKpVKSbwyZszIw4cPv3jMy5cv6dixI3/99RcLFiygQ4cOXLhw4WeHmuokXbOrV69GV1cXJycnihYtSq9evdi6dSsVK1ZEV1eXnDlzYm1tTUREBIcPH+bw4cMax4vkDA0NKVu2LOPHj+fQoUPUrFmT5s2b079/f44cOYK+vj7Xrl3j/PnzGsfp6OhI8iZ+qV82kW/9+vVZsGABISEhjBgxgrRp01K/fn1q1KhBQEAAFy5c0KhU5K5b/ArvX3P58+fn7du3PH78mKioKGxsbHB2dqZly5Zs3bqVKVOmkC5dOvz8/EiTJg2NGjWSH8FPqFKlCuXKlWPs2LGMHz8ea2tr3N3dlRVUPiXp/0fatGlZt24dBgYG1KhRQ+MRtkiuSZMmuLq68vLlS4YOHcqNGzdIly4dZmZm3L59WxK2bxQaGsr06dPR19dnzpw5DB06lPnz56Onp0fLli159OgR27Zt03aY4j/ul67EULduXRYtWsT8+fMZPnz4R2esF0IbfH19cXZ25uXLl0RFRdG2bVvu3r2rJHFeXl7ExMSwefNmjI2N8fDwID4+Xn4YP8LAwIBhw4ZRunRpoqOjKViwIHXr1v2mtYtLliypjJB88ODBzwr1t+Lo6IirqytRUVGMHz+ec+fOAdLa9jkfKxsdHR0WLFhA9uzZsbW1xc/PjwULFgBgY2NDcHAwa9euZdGiRdoIWQjFL53rYMuWLbRt2xY3NzcGDx6szPEmhDaVL1+ehg0b0qlTJ0aOHElkZCS7du0ib9683L9/n4ULFzJp0iTi4uK4ePEiHTp0ID4+Hl1dXflh/IjY2FiGDBlCbGws5cqVY9WqVUry9jUT8LZr144FCxaQIcP/tXefcVHc69/HP3QBCQKKiooFsZf4t3esGBUxYkUEkaDGBvYCigRUUCwoKhEVscRCLBwrUQm2WGJLYizEWECwIwpSpN0PuJlIbCnGBbneT06yO7uvK3OG2e/8ahkJb3/Dzp07CQsLQ19fn4kTJ1K/fn1Auvbf5OXw1qBBA6pWrYqZmRk5OTlMmzYNHR0d4uLi2Lx5M7q6uhgZGbF48WJKlCihDLEQQpXeSwtc27ZtSU5O5uLFi3988Vue+qytrVm3bh07d+5k+vTpPH369N+WIMRf9udrs3nz5rRs2ZIlS5YAYGZmRkBAAI0bN6ZXr15cu3aNSpUq0bZtW7Zs2SJd+++goaFBqVKlmDx5MgYGBlStWpXo6Gjmz5+vvP/yWKGX//9wcnJi9uzZjBs3jt27d6uk/sLm77ag5S8tcujQIXx8fP7Dyj4OXl5e9O3bFzU1NWJiYli5ciUHDx6kQ4cOrF69mrt375KdnU1KSgq6urp07dpVZpuKQuFfB7jWrVszZcoUzMzM+Omnn4iIiODAgQNkZma+dUaOra0tw4cPp2fPnvKEKFRizJgxWFpa0qBBAy5duoS7u7tyvZqZmbFgwQIaNWpEv379Cgyklxv3q94UMgwMDJgyZQotWrTg4MGDSogDqFSpEvHx8cq5zA9vY8eOVTZhF3+oV68eJUqU4OLFi8qivW/Srl07jh8/LtfpOzRr1ozg4GBGjRpF5cqVadOmDW3btmXy5MkcPHgQExMTBg4ciLq6Onfv3mXHjh0y21QUGu+lBU5bWxtTU1N8fHwwNjYmPT0dZ2dnUlNTX/tj9+ebvYzREB/Cy9fZ2LFjcXd35+DBg5ibm1O7dm2GDBnC8ePHlePLly9PaGgoT548YdCgQaoqu9B7+bz269ePGjVqoK6uzqFDhzh58iSGhoZMmDCBZs2aceLECQIDAwkLCyMuLg43NzcAXFxcmDFjBu7u7tLyBkybNo0LFy4QGRkJgLe3N71798bExITz588THBxMZGTkKyHiz/dSedh4s4EDB1K3bl2ePHnCokWLAKhVqxYjRoygY8eOzJgxg717977yOTmnorB4LwEu/6ahq6tL+/btmTBhAkZGRnTr1o3Hjx/LBS8KlSpVqjB69Gi2b9/OqVOn0NPTY9myZbRp0wYnJydlqQXI26YoMTFRHjD+Am9vb/r168eVK1coUaIEzZo1Y+7cuSxevJhSpUoxZswYevbsiZ6eHg8fPqRbt25kZmbSvHlz1qxZg4eHBxEREar+z1A5Q0NDoqOjuX37NosWLaJEiRLMmjULT09PkpKSmDVrFrq6uoSGhrJ9+3ZpCfoHKlSowKJFi2jevDmhoaF4e3sr79WsWZMRI0ZgZWXFV199xa5du1RXqBBv8Z/MQq1Zs6Zy0+7YsaOybpYQqpa/nE1CQgIjR45UVqTX1NTk66+/pnXr1jg5OXH69OkCn5NW4rfr0KEDK1asYODAgfz0008ADB06FH9/fzw8PFi9ejUGBgaYm5tjbm5OZGSk8lBXtWpV9PT0ZL23l5QrV46wsDAePnzI5cuXef78OYGBgUBet/SKFSsoU6YMa9asYceOHRLi/oFWrVoxatQomjZtypAhQzhz5ozyXs2aNZk6dSoaGho4OTmpsEoh3uwfzUJt0aIFTZo0eeOMsmvXrjFlyhSSk5Px9fVFQ0PjXxUpxPuyb98+wsLCMDMzo27duujo6AB5G4GPGDGCY8eOsWfPHurUqVPgcxLe/jBx4kRq1KhR4DUjIyPu37/PtWvXlPvCunXr8PHxwdPTk2rVqpGcnMyvv/7K/v37lXFEADdv3pTw9hI1NTXu3buHs7Mz5cuXZ/z48VhaWirvJycnM2rUKB4+fMjQoUMZPHgw6uofdEGBj8IPP/xAYGAgp06dws/Pj6ZNmyrvXbt2jdmzZzN06FDVFSjEO/ztv/rPP/+c3bt3s2DBAho0aPDG465cucK3335LtWrVqFChwr8qUoh/4k0PGJMmTWL79u14enrSuXNntLW1gbwQ9+WXX7Jo0SKuXr36IUstMsqUKcO0adP46quvqFq1qvJ6dnY2NWvWxNjYmNzcXDQ18zZ5+e6770hOTsbU1PSV75JWo4Lyr9fc3FxMTExISEjA3t6eM2fO0LBhQzp16qQcmx/icnNzadiwoQxR+Yd+/PFHgoODiY2Nxd/fnyZNmijvxcbGkpub+5eWvhFCFf5WgKtVqxZjxowhICAATU1Nli1bxqeffvraY7Ozs9m8eTNly5bF2dn5fdQqxF/2cpenjY0N48ePZ9iwYcq+piNHjuTgwYMEBQXRpUuXAiFu3rx5BVqIRB41NTUePnxI48aN+fTTT/H391daho4cOcLp06fx9/enYsWKyizJ1NRUZTKTeLOXr1d3d3eCgoKoXr069+/fZ9iwYaSnpzN27FisrKyUzyQnJ9OvXz8mTZqkoqo/DidPnuTrr7/m5s2bhIaGUqtWrQLvS+u7KKz+1l21ZMmSnDp1io0bN9KuXTs0NDQIDAx8Y4hLSUnB29ubypUrY2Bg8D7qFeIvyb/pzp49m4CAAFq3bs2XX36Jv78/Xl5eALi6uvLdd9+xZMkSbGxslFajfNJCVFB+S0RsbCy2trY0b96ciRMnUr16dZKSkggLC8PAwICVK1diZWWFlZUVCxYs4OnTpwUmhohX5V+vs2bNwtXVld27dyutavfv32fIkCGULFkSd3d35SEEIC0tTVqJXkNLS0v551KlShV473Xn6uTJk2zYsIEtW7YQExPzX5cnxHvxtyYxlChRAlNTU2JjY4G8TX+joqLIysrCzc1NWchXV1eXtLQ0AOrUqcPAgQNZsGABycnJ7/+/QIg36NKlC0uWLMHZ2ZkzZ85Qrlw5+vTpg6urK5s3b1bWJNu8eTOampr069dPxRUXDbNmzUJTU5Pu3btTuXJloqKimDBhAvHx8VhbW2Nvb0/Xrl2JiYnh8ePH9O/fXxY+/QuaNm1KcHAw7u7uHDt2THk9f82x/IkNenp6jBs3jgsXLqiw2sKpT58+7Nq1S7nOxo8fT7du3UhJSSEyMpKwsDAyMjLeeS3KtSqKgn88C1VLS4vMzEy0tLSIjo4mKyuLsWPHcv/+fWbPns3333/Ptm3bAKhYsaKyr6EQH4qrqyv29vZ06tRJuRmbmJgwYsQIWrVqhaurK3fv3gVklulfNWLECCZNmoS9vT0vXrzA2NiYkJAQLl68iLu7u/J3bmFhQXJyMg8fPiQ3N1cWPn2NP19zn332GT4+PnTo0OGVh938+23FihWZNGkSEyZMkIDxJ/3792fq1KmEh4fj5+fHwIED8fHxYcGCBbRr1w4TExOuXr3K9OnTSU9Pl5Amirx/tYxI/k1ZS0uLqKgoZfCyhoYGrVq1khu2+GBe/jHM/2cbGxs8PT1xdHTk2rVryrEtWrQgIiKCbt26FWjFkBD3bsuXLyc7O5tx48Ypr9WsWZN9+/Zx4sQJ5syZU+Bcg5zXdxk6dCgxMTHo6uqyaNEi7O3tlVm5+efO3t6eixcvcvnyZeVzEkAKMjQ0xM3NjdatW3P06FHU1dU5f/48e/fuRUNDA1dXVz7//HOuXr3K1KlTJcSJIu9fjSzOzs5GXV2dzMxMBgwYQO3atUlKSqJ169bKe0L8114OCLa2trRu3RpdXV2uX7+OlpYWAwYMoGzZssrxDx484OrVq69sRyQh492MjIwKjGfV1tbm2rVrLFu2jM8++wx/f3/MzMwKfEbOa0Evj8FydXVl2rRpJCYmkpCQgJqaGoMGDaJ8+fIASutl//79+fzzzwt8jwSPP2hqavL06VMCAwM5fvw4bdq0oV+/fiQmJgJ5v1WhoaHs3LmTmjVrMm/ePHR1deUciiLtXyesnJwcTExMCAsLIyYmBhsbG7KystDQ0JA/DvFB5AcELy8v5s6dqywMe+XKFfz8/Bg2bBiTJk2iV69e1K9fH39/f9LT07l06ZKKKy96Nm/eTKdOnejduzcAL168AODJkyds376d9PR0pVtavF7+9VqvXj3KlSvHjBkzuHr1KleuXMHHx4chQ4Ywbdo07O3t6dKlC+Hh4ZQqVQo/Pz8VV144aWpqKg9j5cqVY+7cuRw/fpwSJUowYMAAJTBnZGSwdu1aduzYQdu2bRkxYoQqyxbiX9N89yHvVqpUKWJiYnBzcyM7O1vGu4gPzsnJiQEDBijdT5mZmQBs27aNzMxMhg4dSp8+fUhISODJkyf06NFDmb0nLUR/3cmTJ/nmm2/w8PBAS0uLHTt28Mknn9CtWzd2797NN998A0i36bs0adKE/fv3k5WVhbu7u/J6eHg4aWlpODo64u3tTWxsLPfv36dz585Kr4Y8GP/BxsaGtm3bMmXKFHx9fenUqRNt27Zl6dKlqKmpYWVlxbRp05g3bx6Q98ARFhbG/fv3Zc9dUeS99620JLwJVQgMDCQzM7PAmlgvX4sGBgYYGxtTokQJYmJiZGD9v1CjRg0GDRrEyJEjuXPnDpqamiQnJ9OxY8dXuqXFmw0dOpQFCxawfv165syZo3T3Aejp6VGyZEkgr8sf5N76svwHBGtrazZu3MiFCxewtLSkR48eyjhBAwMDxo8fT5s2bfj+++/x8/N75aFCArEoyt5LC9zL5AYjPjQtLS3q16//ylpj2dnZaGtrU6tWLa5fv87t27eV99TU1ORafUnnzp05d+4cT548eeexMTExzJkzh/DwcBo1akRGRgY7d+6U1vc3eNM5WbduHbq6unz11VfcunWLdevWKbNP8xdAzifX6x/Wr1/P4sWLuXDhApGRkRw7doy2bduyffv2ApM8kpOTWbx4MQDt2rXjk08+Yfr06QW+S8KbKMree4AT4kPLzMzk4MGD9O/fn02bNhXYV7NSpUq4uLiwYsWKArMjpXvvD/b29vj7++Pt7U14eDhPnz5952eys7O5fPnyK7MiJWS8Kv+c2NvbU7t2bdTU1Pjpp58IDw9n5cqVaGlpMWvWLHJzcwkLC3vteplyvf4hMTGxwPjVAwcOcOjQITw8PEhKSmLWrFm8ePECDQ0NJcTp6emhp6enwqqFeP/eexeqEKrQrFkzpk2bxosXL/D19eXSpUuULl2awMBADA0NsbGxkR/Bt/D19aVbt258/fXXfPvtt3+pJe7PZNxbQb169UJPT48tW7Ywe/ZsBg8ezIEDB6hTpw46OjrcuHEDR0dHAMaMGYOnpydLliwhMDBQWQhd/OHP3Z0jR47k8uXLHD16FIDu3bsTEhLChg0b8PT0VLrzmzRpwtmzZ1VSsxD/JQlw4qPRs2dPBg4cSLt27YiNjUVNTY309HSsra3JysqSgPEa2traykxSPz8/2rVrR2hoKFu3buXZs2d/6Ttq1arF1atX/8syi5z88W22trbK7Mfhw4dz+vRpNDU16dWrF+PGjePq1auMHDkSgEmTJtGhQwd69Oih4uoLp/y/3/z/PXLkCGXKlMHV1ZVTp06RnZ3NZ599RkhICNu2bWP9+vVMnjwZQ0NDevbsqeryhXjvJMCJQu91wetNYax8+fI0bNiQypUr8+DBAyIiIpSN6aV7783s7e0pU6YMkydP5vnz5wQEBPylEDd06FCmT5+OtbU1t27d+jDFFnL29vYEBAQwYsQIdu/eTe/evfH19aVNmzYkJSUBedsNDhw4kCFDhjBy5EjZf/NvsLKyIjo6GoCdO3diYWHBqFGjOHnyJNnZ2VhZWREWFkZcXBwZGRnKA5wQHxtZaVcUalpaWkpQs7S0pGrVqmhqar5xA++7d+9y4MABvv76a3bu3ElOTo6MzXqHyZMn4+Pjw+3bt3Fzc+PYsWNMmjSJAQMG8Mknn7zxc05OTsyaNYtJkyZJePv/+vXrR2BgIAsXLlSWqbhz5w6pqanUr19fOS4tLY1Dhw5Rq1Ytqlevrqpyi5yqVasSHh6Os7MzAJ9//jk3btxgxYoVtGzZEk1NTaKjo2nevDljxoyhc+fOyrqkQnxsJMCJQmnOnDkYGRkp67nNnDmTXbt2sWPHDiIjIylXrtxf7g6VmWZvVqpUKXr27Mm8efPYtWsX27dv54svvmDPnj3MmDGD/v37Y2hoCBTcQcDJyYnZs2fj5uYm62n9f05OTixfvpyzZ88yevRoWrZsCUB8fDxpaWkMHTq0QFh78eIF165dIyUlRVUlF3p/3s3nwYMHBAUF0bZtW2rWrAlA7969uXHjBkFBQbRo0QItLS3u3bvHxYsXlQc9eYATHyMJcKLQMTMzo1evXkRERGBgYKBsi+Pu7o6XlxfJyckcPHhQuYGLfy6/ayn/B05HRweAiRMncvnyZVxdXXF2dsbAwEAJzM7OzsycOZNx48ZJePv/nJ2d8ff3Z+jQodjY2LB//362bdtGq1atuHv3Lm5ubrRq1Qpvb2++/PJLOnToQFBQEDk5ORw/flzV5Rda+Q9f1tbWqKmp8fz5c/bu3UutWrVo1aqVclzv3r35/fffCQ8Pp3bt2gW+Q8a9io+VBDhR6CQkJGBnZ0dmZiZ79uyhUqVKLFu2jIMHD/K///0PFxcXLl++zPbt2yXE/Q2v63JOSUkhISEBe3t7IG+7IU3NvNWF4uLi0NXVpXbt2srSFu3bt2fu3LmMHz9ewtv/p6enh52dHa6uruzbt4+srCxmzpzJ//73P7Zu3Urr1q25ePEi/fv3JzMzExcXF7y8vMjOzsba2lrp5hevZ2Njw8aNG/n222/p2rUrFy9eJCgoCF9fX6pVq6YcZ2dnx7p162SLPFFsyCQGUai8PDnB0tKS5cuX06hRIxYvXszcuXOV44yNjVmxYgW1atVi8ODBBdZ+E696+bw2bNgQNTU1dHR0OH36NBYWFmzfvp0rV64waNAgZbmGkJAQVq9ezZkzZwp8VlNTk3PnzqnyP6fQeNtK/sbGxvj4+NCrVy8GDhzIiRMn0NPTQ0tLC319fRISEgDZYeHP/jxBqWLFiuzfvx9tbW12797NJ598wvr16+nduzfGxsa4ubm9snae7LAgigMJcKLQqFSpEnFxcUBel0hkZCTm5ub4+/tjZmbGZ599xuPHj5XjjYyM2LZtG/fu3WPIkCGqKrtI8fDwoHv37mhqaqKrq8vhw4eZPXs2//d//0dAQAA5OTnExMRQvnx59PX1admypdJCJD+Ib5Z//W3YsKFAIMsPcTY2NvTv3/+V3UJkaZs3q1ChAk+fPiUlJYWePXvSv39/oqOj0dPTY9KkSVy5cgVDQ0O8vb2JjIxUdblCfHDSbi8KhZYtWxIcHEzXrl3x9fUlJCQEIyMjrl27xpQpU0hOTiYiIqLArMgnT57Qp08fZTFU8XZjxozByckJNzc32rRpw6ZNm3BwcKBSpUp8//33dOnShd27d3Pz5k2OHj1Kq1atJLz9RZ9//jl9+vQBCm4nmJiYqEzA2b17N3Xq1CnwOQlvr9erVy++++473NzcqFq1Kt9//z2PHz8mJyeHoKAgnJycSElJwdLSks6dO6u6XCFUQlrghEoZGxuTmJiIubk58+fPp1atWhgYGNCjR48Ci8PWqFGD4OBgtLS06NGjxyvrk0lLRkGampqvLF4cHBzM0aNH+eabb+jZsyeBgYF89dVXhIWFUaJECdLT01/5Hunee7v8cFunTh02bNjA7NmzXzs2sHTp0jg5ObFkyRI5n3/RhAkT+PTTT6lbty7u7u5Ur16dESNG0K9fP+Li4qhQoQKffvop+/fvlwcMUSxJC5xQmYCAAEaOHIm6ujqxsbGcOXOG0qVLc+PGjQKDkyFvA/WRI0eSkZHBmTNn0NfXL/C+hLc/+Pn5cfr0aXR0dJRlFEqUKEGTJk1ITU2ldevWLF++HB8fH8LCwtDU1MTd3f21LRkSNt4uPzjcv3+fmJgYWrRoAbw6YeTRo0csXLiQ7OxsWZPsHfIndCxatAhvb2927drFxo0bKVu2LHp6esyZMwd9fX3i4+PZu3evslC3EMWNBDihMsePH2f+/Pnk5OSgra3NgQMHGDRoEPfv32f48OFKl1S+mJgYxowZw6FDh2SvyLfYunUrGRkZREREKCEuPT2d7du3M3jwYDZv3oyHhwfr1q0D8taC+/TTTzE3N1dt4UWIo6MjM2bMwMDAAE1NTR4/fszWrVsZOnQoDRo0eOsDhYTit3u5Ne3333/Hx8cHFxcXLCwsSE9P57PPPiuwhAjIORXFk3ShCpWzt7ena9euzJgxg4SEBCwsLPD19UVXV5fQ0FAiIiIAcHV1Zf369WRkZAAy0+xt6tWrx+rVq3n69Cm9evUiIyOD7t274+vry82bN5k0aRI3b97E1NSUwMBAZb9IOZ+vV7NmTUxMTFBTU+Pq1auMGjWKwYMHc/36da5evcrChQtJTk4mICCA2NhY/P39ycnJkZbh98jMzIwmTZrQq1cvhg8fLteqKPYkwIkP7s/j1YYPH07fvn2JiYlh3rx5xMfHU61aNXx9fSlZsiQ//vgjtWrVomnTptSqVUtu3H9RfohLTk6mR48evHjxAgcHB0aPHk1OTg5paWlK91P+fpESil81aNAgpkyZgo6ODmXKlCE4OJglS5aQkZHB0KFD6dixI3Xq1GHLli20bt2atLQ0Bg4cSGpqqqpLL7SaNWvGgwcPePDgwT8+T3KtiuJOApxQGTs7O65evcqvv/6Ki4sLffr04fbt28yZM4f4+HiqVKnCmDFjsLCwIDU1FScnp1cG5os8rzsnampq1KtXj5CQEFJSUvjss8/IzMykefPmVK5cmSpVqvDbb78RERGhBDnpiipoyJAhLFiwgFGjRhEXF0eNGjUICAhg8eLFzJ8/v8BxNWvWpH///hgZGTF//nwWLFigwsoLr6ZNm7Jv3z62bNlChQoV8PLyIi4ujqSkJFWXJkSRIgFOqISuri4//PADp0+fZuTIkUBeF2nv3r0LhDh9fX1yc3OVp3QJGa96ObxVr16drKws0tLSuH//PmpqatStW5fVq1fz/PlzunfvrnRBv0xaM15la2vL6tWrcXJyYt++fcrr69ato2LFitja2vL8+XPldXV1dSwtLZkxYwYlSpRg4MCB8qDxGg0bNmTfvn1Mnz6dcuXK0bt3by5fvszx48eVcZkg16QQ7yKTGMQH8fKsPDU1NdLS0nB1daVr167KNk4hISHs2LEDc3Nzpk+fTsWKFXn+/HmBLhYJb6/KDwmTJ09mw4YNbNu2jaioKKysrMjNzeXSpUu4uLigp6dHREQEurq6r3yH/FC+qmTJkgCYmpoq24sBpKWl8eTJE2Uf2Xy5ublcu3aNr776inbt2tGhQ4cPWm9R8dNPPxEUFET16tWZP38+06dPZ//+/Xh5efHtt98yffp0SpQoIdekEO8gAU58EPkhw8nJic8++wxTU1POnj1LWFgY3bt3p1atWgCsWbOG7du306RJEwYMGKDKkouUKVOmKJvM9+7dmwsXLhAaGkr//v0B+PXXX/niiy+oXr16gS3JxJtt2rSJKVOmKF2oAN27d6dPnz6sXLnylZbM3Nxc1NXV+f333zl//jyGhoaqKLtI+O2332jbti2lS5fmyJEjbN++nSdPnmBgYECXLl348ccfCQwMfGU5ISHEHzTffYgQ74elpSXz5s3jwYMHnDt3jmXLlrFp0yZWrVpFkyZNlIV7Q0NDefjwYYFuK/FmDRo0oHXr1owaNYro6Gi6detGixYtuHDhAkuXLiU3N5fw8HB+/fVXunTpwu3bt1VdcpERGhqKuro68+bNo27dunTs2JGJEycSFRX12nGHOTk5DB48mGbNmjF69GgVVV24WFlZcePGDWJjY5XXvv32W5ydnRk7dixeXl4cOXKE2NhYhg8fzsOHD5k1axZGRkbcunVLdYULUcjJGDjxwRgYGODl5UW9evWIiIjAw8MDNzc3unTpgpWVFR07dlQ2+M4n42Be9efgYGFhQadOnVi1ahVt2rTh66+/ZvHixaxevZqdO3dSv359vvrqK9avX698Rs7r3+Pk5ERAQACRkZE4ODi89Vh9fX0qVqzItWvXPlB1hZe2tjbHjh0jNzeXvn37cufOHeX6tbGxwcHBgTp16nDr1i1cXFx48ODBK98hk5aEeD3pQhX/ua5du2JpaUlycjJLly6lSpUqxMXF0atXL+zs7MjKysLExAR/f3/09PQKfFZCRkEv/5g1bdoUyFvsdOvWrQAMHjyYffv2sXbtWgASEhJ49OgR/fr1K/A9cl7/nrCwMCZOnIi1tfVbW9Y0NDR4/vy5hLf/78WLF9ja2pKamsqGDRuoVKmScv2eO3eOKlWqkJqaio2NjRLe/ryLhYQ3IV5PApz4T9WuXZuxY8eya9cubG1tiY2NZcKECbi4uPDw4UMmTpzIkSNHePjwIYaGhrJ21jvk/5jNmDGD5cuXM3ToUACePn2Knp4etWrV4v79+8qyIPr6+owePRobGxsVVl141a9fHzMzswKv/TlA5Fu/fj1Tp07F09OTadOmvfYYmWTzqnv37mFra0tOTg7Lli2jcuXKQN7DxYIFC8jOzqZ27drK8RLYhPhrZAyc+E9duXIFNzc37OzsWLp0KW3btiUmJoZz587RuXNnQkNDCQ8PZ8+ePa9d3kK8atKkSTg5OeHo6FhgXFFqairHjx9n3LhxlCpViubNm6OlpcWFCxcA6Yr6s65du+Lj48PTp0+5dOkSoaGhXL58mezs7Dd2Ma9duxZ9fX2sra1VUHHRYGhoyNOnTwHQ1NQkKyuL5ORk7t27R9euXVm7di3Ozs7ExsZy+fJlMjIyaNmyJVeuXFFx5UIULTIGTnwwnTt3pm/fvlhYWFCtWjXu3LnD4MGDuXPnjnKMjM16OxMTE8LCwggLCyM8PFx5PX99PD09PaZMmULt2rV5+PAh7u7ussPCW5iamlK+fHkWLVpEcnIy169fx9PTk/T0dDln/0CzZs1YvHgxbm5unD17Vnk9NDSUqlWr4u7uzuLFi1FTU8PBwYE7d+6watUqjIyMXunmF0K8nQQ48UFVqFCBhg0bMnnyZOrVq8eqVavw8PBQdVlFRtWqVTly5Aiurq5ERkYWeE9bW5sXL14AeQPp8xeZlcWP361kyZLY29tjZ2dHeno6AwcOJC0tTULc32RlZcWXX36JsbExY8eO5erVq6xbtw4LCwvs7e2Ji4vD1NRUGbM5dOhQkpKSePbsmbQOC/E3SYAT/1p+19zf+bErWbIkLi4uBAUFSbh4g5e7PPPPrZGREdu2bWPPnj0EBweTkZGhHNejRw/q1auHv7+/iisv3AYMGEBqaiq7d+8G/jjPmpqaWFlZMXXqVJ48eYKDg4MSiMXbVahQgfj4eADatm2Lq6sr5cuXJz09HT09PaXLNF+ZMmWIjo7m8OHDjBs3DpAufiH+LpnEIP6V7t27M3HiREqXLv2Xw5u6ujopKSkEBgaSnZ2NhobGf1xl0fPyj9mIESP44osvMDAw4MmTJ8qivO3bt0dDQ4Pc3FxKlCjBoEGDqFmzpoorL9wcHR0JCgoiLS1NeS3/4SMrK4uoqCiWLFlCyZIlGTFihAorLTo+//xzDh8+zJAhQwA4duwYa9as4e7du3z66acsXLiQ2NjYApNDHj58SOvWrXF3d1dek/AmxN8jLXDiHytXrhzR0dGkpKSgpqbGli1bOH/+PIcPH1aOkS6of8fLy4v+/fsTGBhIREQE9+/fB2Djxo3UqVOHn376iQcPHlC/fn0MDAzo0KHDK1s8iTxOTk74+fnx5ZdfsmvXrjcep6Ojw8yZM6lTpw6DBg2SyTVvYWBgwNq1a2nVqhU///wzO3bsICQkBIB27drxxRdfUKFCBSZPnsz58+df28om9wgh/hlpgRP/WGpqKidOnMDX15cxY8ZgaGjIqlWrWLBgAZ9//jkg6439Gw4ODgwaNIh+/fqxatUq7t+/r+xj6uDgQGBgIM+fP6dChQqcPn0aKysrsrKypEXzNTp37kxAQAAuLi7s2rWL6tWrM3nyZFavXs3MmTNp3LixcmxGRgbz58/H0tISZ2dnFVZd+CUnJ3Pq1CnS0tI4e/Ysffr0YdiwYQAcPXqUNWvWEB8fz4IFC2jUqNFrW9nkHiHEPyMBTvxjz549IzIyEj8/P27fvo2npyetW7fGwMCAoKAg9uzZQ8+ePTE3N1d1qUVS5cqV2bt3L5cvX6Z69eo4OTnx3XffsWvXLhwcHAgLC2PMmDEMGTIEb29vpTtaxhQWpKGhQe3atYmLi6N27dpUr16d9evX07x5c7S1tenTpw+zZ8+md+/eyvHPnj0jMDCQqlWrqrb4QkxTM28VqhUrVvDLL7+Qm5vLr7/+ypAhQ5T1CY8cOcLq1auJi4sjLCyMGjVqqLBiIT4uEuDE35J/01ZXz7t0tm/fTnR0ND179gTyFu1s2LAhBw8eJD4+nnHjxvHDDz/QqVMnldVcVGlra9OvXz/c3d1ZtWoVnTt35sCBAzx+/JihQ4dSqlQpoODYIQlvr8rOziYsLIzg4GD69u3L0aNHiYyMZOjQoTg6OtKlSxeysrKUMVz55/DKlStoa2ujo6OjyvILnfyFj/O76nNycrh48SKZmZksWrSI8+fP4+TkpIS4o0ePsmnTJsLDw7l+/bqqyhbioyNj4MRfZmVlRatWrVi5ciVPnjxRXp8xYwYtW7bExsaGqKgo0tLSGDBgACkpKTRu3JjGjRuzZs0aCRf/QGBgIJaWlvzvf//j+++/59q1a7Ro0QJfX18cHBy4d++eqkssMj755BMcHByoVKkSy5cvL7AvZ+vWrdm1axdt2rQpsA1W1apVuXnzpgqrLlx69+7NwoUL2b17N6GhocTGxvLkyRMaNmzIrl27GDBgALdv32by5Mk0atSI9evXExYWVuA7ZMybEO+HBDjxl/n6+tKxY0e2b9/OmjVrSEpKAvK6nL7//ntq167NqVOncHJyIjEx8ZXPS/feX/fyYO+X13TT1NRk06ZNZGZmvnNTdfEqAwMDzMzMXtmr1MbGRtkxJH8XAVGQkZERy5cvp23btrx48YK9e/dSq1YtFixYwKlTp3B1dcXQ0BAvLy9q1qzJF198gbW1NdOmTWPfvn2qLl+Ij44EOPG3eHl50a5dOw4cOMCqVat4+vQpmpqaTJw4EVtbW/r06SOtQv8BPT09+vXrR/fu3SlXrhydOnUiKytL1s56D7S1tVm7di1paWm4urqqupxCrW3bttjZ2VG/fn02b95MTk4OI0aM4NKlS9SqVYvc3Fx69uxJUlIStWrVonPnzqxYsUJa3IT4D8gYOPGX5M9s9Pb25tChQ/Tt2xdXV1eMjIzIyspi165dmJub0759exVXWjRoaWkp/6yvr1/gvddtpq6jo4OpqSkPHz6kY8eOymxTCW//nL6+Pt27dycsLIzKlSszcuRI4M2b2Yu8Nd6+/fZbbty4gYODAwcPHsTW1paIiAgAypcvj7GxMQBXr14lKCiInJwcZcysEOL9kRY48UbVqlXjxo0bQMFxK2vXrqVjx47cvHmT/fv3s2bNGh4/fszs2bNp164dDg4OJCQkqLL0Qqt9+/YcO3ZMOZejR4+mXbt2JCcnEx4ezuHDh9/YsqalpUVmZiYg44jehzJlyrBgwQI0NDRwdnZWQrF0879by5Yt+fLLL6lUqRKTJk3i3Llz6OvrY2BgwL1796RlWIgPQB6LxGtZWFhw+vRpRo8ejYaGhhIWwsLCqFatGq1atSIqKgpra2uGDRuGvr4+Z8+e5e7duxLe3mD06NH4+/szaNAgAL744gsmTpzI+fPnqVatGuPHj2fcuHFoaWmRm5v7SktQfngDWTvrfXj48CHjx4/H0dFRwtvfdPLkSVauXMnt27cJCAigZcuWPH/+XMKbEB+QtMCJN3Jzc2PKlCl4eHiwbt06QkNDqV69OkOGDOHWrVtA3pi4tm3bcuzYMby9vZXPyk38VaampsydO5dy5coRHh5OvXr12Lt3L9HR0WhqauLj40OjRo04ePAgS5cuJTMzU87jByLn+Z9p2bIlrq6uVK5cmVmzZnHixAlVlyREsSEBThRQt25dfvvtN2UT71GjRjF79mxu3LhBWloaQ4YM4c6dOwVaKxYuXIiOjg5jxoxRZemFWn6Xp4mJCQEBAZQuXRpTU1OGDRvGr7/+CuSNyfLw8FBCXFBQkGymLj64vxtmW7RowbRp07hz547cA4T4gKQLVSjs7OyIjo5m7ty5BVZZnzp1KhYWFkRGRnLnzh0gb7HT/IHJEydOlBv3W6ipqSldno8fP2bKlCnEx8dTvnx5rK2tleOeP3+Or68v586dY9CgQdjZ2amqZFGM5Ye3Vq1a0aVLF8zMzJS/9ddN8Dh16hQeHh6MHTv2g9YpRHGnqeoCROGRP3vM0dERfX19Ro8eTU5ODqGhoWhra+Pj40NiYiKrVq0C8sZhSdfT2718fvr27Ut8fDwnT55k+vTpqKur06VLFx48eMDGjRuBvP1l582bR1xcHFu3blVl6aIY8fDw4NGjR3z99dcA+Pj48Pnnn2NgYEBMTAzbt29n7dq1vHjx4rV/8/mtyHI/EOLDkQAnFKdOnSIqKoro6GhGjRpFSEgIrq6u5OTk8PXXX6Ouro6Pjw+5ubmEhIQAyM36HfLPj5eXF3379mXNmjVcuXKFJ0+eMH36dObPn69MasgPcc+fP1d+SGW2qfivGRgY0LhxY7S0tEhJSeHmzZu0aNGCoUOHkpiYyJgxY+jduzf6+vosW7bsjSEO5H4gxIckY+BEAevXryc7O5ulS5eyefNmjh49ysiRI5UQMXLkSHx8fHBxceF///ufiqstGoYNG8bUqVPp168fV69eLfADaGJigr+/P6ampuzZs0dp3RTiQzI2Nsbf3x9DQ0Nu3rxJWloas2fPBvIWkfbw8KBJkyZ89913SogTQqiWjIErxho0aIC+vj7a2trKa3PmzMHY2Jjc3FxcXFzo3LkzK1euVMbABAcHM3z4cPbu3auqsouchg0bsnnzZn7++WdlA/B8+WPisrKyqFGjhooqFMWZmpoaiYmJTJ8+nZSUFAYMGECdOnWU91NTU/H19eXHH3+kU6dOTJ8+XRkjK4RQHQlwxZStrS2HDx9m/fr1+Pn5YWFhAUBsbCyZmZl06tSJEydO4OTkRKdOnVixYoUS4nbu3El2drayO4N4vUaNGgF5M3tNTEyAP9Zvy83NRVtbmxo1apCYmIizszOTJ09WWa2i+MmfkJCbm4uZmRmPHj1iwoQJ7N+/n8qVK+Ps7Kwck5aWxpw5c/j9998xNDR85UFECPHhSYArpvT09IC8Daq1tLTYu3cv3t7eNG3alPnz5+Pg4ICFhQXHjh3D0dEROzs7Jk2aVOA7ZNHTN/P09GTevHmYmZkRFRWFhYUFn376aYFjqlSpgqenJzVr1uTp06evXbxXiP/Cy2PYJk6cSFBQEI0aNSIpKQkPDw9++eUX7OzscHBwUD6TlpbGpEmTmDhxoqrKFkK8RAJcMbV582bGjRtH/fr1OXnyJOPHjyclJYU1a9YwefJkypYtS+PGjQH44Ycf6NixIwEBASquumho0KABjRs3xtPTk4SEBL7//ntKly6Nk5MTzZs3B6BcuXLMmjWLUqVK8dtvvymflUHg4kPIv848PT1xcXFhw4YNPHjwAIDExESmTp3K/fv3GTBgAIMHD1Y+l5GRIQ8aQhQSMomhmHN1dWXOnDl4eHgQEhKCmZkZzs7ONG7cGA8PD65cuVLgeNlu6O1cXFxo164dWlpauLi4kJaWBoC1tTUTJkxQulJTUlLIzc2lS5cub9z7VIj/Ut26dVm7di0zZszg8OHDyuv5f+PGxsb4+fnRoEEDvLy8iIyMVGG1Qog/k5GoxVxISAi5ubnMmzcPfX19lixZwrx589DU1HztTDMJb2+XnZ2NlZUVz58/p3r16vzyyy8AREZGcuPGDcqXL0+jRo24desWu3fvJicnR0Kx+CD+/JBQsmRJSpYsyc8//1zguOzsbLS1tUlMTMTDwwMXFxcOHjz4ocsVQryDtMAJIK/lyM/Pj6+++oply5apupwi4U2tZn369GHu3Lns2bOH5cuXc/PmzTd+h6zzJj60sWPHEhcXx2+//caOHTv48ssviYqKAv64Hm1sbEhMTCywt6lcq0IULtIC9xGrX78+jx8/JiEhQXntTaFjzZo15ObmMmfOHPT09PD39/+QpRZJ+eexXr166Orq8uzZM65du8aOHTvQ09Nj2rRppKWlsWbNGm7duvXa75AfRPFfe/lvftCgQYwYMQIHBweSkpK4ceMG/fv359GjR/z888/k5OSgrq6Os7MzV65cKRDg5FoVonCRFriPVNeuXfHx8eHp06dcunSJ0NBQLl++rOxh+qab8dixY7G2tqZnz54fuOKio2HDhvz0008AzJo1ix49emBqakp8fDzx8fEMGDAAyNuSbNKkSezcuZP169fz+++/q7JsUcw1adKE3r17c+3aNTZs2ADk3Sdmz57N7du3OXXqFPfv32fgwIEYGxvToUMH6doXohCTAPcRMzU1pXz58ixatIjk5GSuX7+Op6cn6enp0h3yDzk5OTFlyhS6dOlCz549mTx5Mk5OTjx9+pTq1aszdepU0tLS6NSpEwD29vYsXrwYT09PZfsxIT60unXr8t1336Guro6vry/Lly9X3mvTpg19+vTB2tqaW7ducf/+fYYPH05WVpbcJ4QoxCTAFQMlS5bE3t4eOzs70tPTGThwIGlpaXJz/pscHR0JCAjA2dmZvXv3smLFCuLj45kzZw6Q11XVsGFDgoODOXr0KFOmTAGgc+fOREVFybkWKtWnTx/mzZvH+fPn8fLyIiYmpsD7+vr6QN5evCAzzoUo7GQduI/MJ598gqmpaYHXUlJSWLt2LQsWLEBPT4+wsDC0tbUlUPwNtra2LFy4EEdHR2UbsXLlyhXYcig3N5eLFy+yf/9+atSoQYkSJQA4dOiQMrZIiP/ayzukvHzN7dixg9mzZ9OgQQMcHR2pUqVKgeOeP3+uhDeQGedCFHbyi/IR6dOnD2FhYURFRbFhwwYaNmwI5LUMZWVlERUVxZIlSyhZsiQjRoxQcbVFh5OTE6tXr37l9QMHDlC6dGk6dOhQ4PVbt26hr6+PlpZWgdclMIsPIT94DRs2jKCgIIKDg5kwYQKQt4D3vHnzsLW1xcXFRQlxcm0KUfRIgPtIDBo0iEWLFnHo0CE8PDxo0KABQ4cOBf6YLZmTk8OhQ4c4f/48HTp0QEdHR4UVFw1Dhw5Vthbz8/MjNDSUvn37AnkBLjs7GxcXF3r27ImamhpGRkbY2Nhw8+ZNkpOTVVy9KE4GDhyIu7s7AF5eXkydOpXHjx9TsmRJ+vfvz6FDh1BTU2Pjxo34+flhY2PD+PHjKV++vGoLF0L8IzIG7iPQpk0bVq5cycyZM9m1axcAzs7OmJubs2bNGh4/fqzsCAB53awnTpxg+fLlBAcHq6jqwq9du3Zs2LCBUaNGKd2mM2fOZNSoUbi5ubFt2zZq1Kih7HlqaGjIvXv30NDQoFOnTrLht/hgnJycWLBgAYMGDSIuLo4tW7bg5ubGsWPHgLwZqIsXL+bJkyf06tULyGuh69ChA46OjrILiBBFkAS4Ik5dXZ0BAwZgYmLC2rVrSU1NBWDXrl2YmZlhYmLCzz//zPHjx1m4cKHyuS+++AJLS0umTp2qqtILvVKlSmFubs7PP/9cYED3zJkzGT16NG5ubmzdupUyZcpQsWJFmjdvzt27d2WHBfFBDRo0iMWLFzNs2DD27duHlZUVq1evpm3btty9exfIu0+0a9eOefPmMX36dKKjowt8h2zlJkTRIwv5FnE5OTns2bOHUqVKKeFt/fr1VK1aFXd3d5KSkhgwYACfffYZu3fvVmaeXblyhbp166Kjo0NGRoYq/xMKraSkJBo3boyBgUGB7lAfHx8AAgMDycnJITw8nIcPH3LhwgXlGHV1dQlv4j/Xv39/li5dypo1a9i3bx8Av/32G0+fPsXKyorNmzcDefeJX3/9lVKlSmFmZvbK90h4E6LokQD3EUhOTlYChpaWFnv37sXDw4O4uDgAnj17houLC1WrVlUC3IkTJ0hISJDw9haNGzfGz88PV1dXLl68WGDZlfwQt3jxYnR0dNi4cWOBz8qgcPFfc3Jywt/fn++++w57e3vOnz/Ptm3bePbsGZcvX8bW1paEhASOHDkCQHp6OgkJCcqDnhCiaJMu1GKgbt26LFiwgIkTJ3LlyhVVl1NkaGhocOTIES5cuMDYsWNfe8z8+fOpVauWMq5IiA9h8ODBLFmyBCcnJ/bt26eMzXR3d2fr1q1Ur16doKAgXrx4wU8//cTFixdxcHBQdliQBwwhij4JcB85bW1t1q5di6amJoMGDZKukjf48xggTU1NsrKysLa2xsPDA3d3d86fP6/CCoXIU6JECZYsWUJERAT79+9XXvf09GTMmDG4u7uzZcsWqlatipOTE507dyYlJYX79+/j4uIiOywI8ZGQAPeR0tPTo127dgwZMgRzc3M6dOhAVlaWDFZ+h2bNmnHmzBnl36tXr05YWBhhYWGsWrVKhZUJ8e7JBn8OcZDXkqyvr8+zZ8+Uf5fxmUIUfbIO3EdKT0+P3r17k5aWhpWVFVlZWWhoaEh4+5MSJUpgYGAAQNOmTYmIiCAiIoLhw4djaGjI9evXWbNmDW5ublSrVk3F1YriLv/v197eHi8vLyAv1OXz9fUlKCiIRYsWKesVZmdnK+Et/9+FEEWfTGL4SD169IipU6fy9OlTQGZFvo6NjQ39+/enRo0aHDhwgO+++44mTZowadIkevXqhbu7OwEBASQnJ/PDDz/QrFkzbty4Id1PQuWaNWtG/fr1gVdnkPr6+pKTk8PKlSt59OjRK0uGCCE+DtKFWgxIt+mrnJyc8Pb25ttvv0VbW5s+ffpw8uRJBgwYgLq6Ovr6+owcOZJGjRphaWmJubk5J0+elMkKQqXy/5ZNTEw4cuQIgYGBhISEvPZYJycnNm7cKA9uQnykJMCJYmfw4MHMnz+fYcOGERkZCUD79u359ttv+eKLL4iIiFCONTMzo3LlyowePZpGjRrh6+urrK0lhKqUKFECHx8fDAwMGDly5FuPlTFvQnycZAycKFZKly7NkiVL+PHHH5WuJTU1NS5cuEBcXBx6enrKawAJCQmcPHmSMWPG8OOPP9K8eXNVlS6KseHDhxMYGEjNmjXR0tIiPT2dPXv20Lt3b6ysrN76WQlvQnycJMCJYuXRo0c4OjrSrFkzvL29KVu2LLm5ubRv354KFSpw8eJFoOC4InV1dZKSkggPD6d9+/aYmpqqqHpRXDRo0IAePXrQo0cPKlasSFpaGi1atGDp0qVs3LiRevXqcfr0aVauXIm9vb0yEUcIUXzIJAZR7Ozfvx8XFxfWr1/P06dPuXXrFn5+fowbN+61Cx3nT1ho1qwZycnJpKWlfeiSRTFib2+Ph4cHL168oGLFihw4cAAvLy82bdpE9+7d6devH5s2beLChQuUKlUKHR0dZbs3Ge8qRPEhY+BEsdW9e3fCwsIAmDVrFitXrnzjsRoaGqxbt46FCxcqrXRCvG+DBw9m4cKFuLq6cunSJczNzfnmm2/Ytm0b48ePV47r2rUrdevWZcSIEZiYmLBp0ybc3d1VV7gQ4oOTACeKNSsrK8LDwwkODiYwMJBHjx6puiRRTNna2rJ69WrGjh3Lli1blNa0efPm0bFjR6ytrUlKSirwGTMzM1xdXfm///s/XF1defDggWqKF0J8cDIGThRr0dHRODo6MmLECNzd3SlbtqyqSxLFVHJyMgCWlpaUK1dO6QrV1NQkKSnplckIampqJCQkEBISQsOGDd85mUEI8XGRACc+Si+vTv+21yBvTFx+iOvdu/d/XJkQr1JTUyMqKgoHBwfGjRundJd269YNBwcHFi1apAS8fLm5uUqI+/HHHzE2NlZF6UIIFZEuVPHR0dLSIjMzE8hrzcjKyiIuLu6de8E2b96cs2fPyrILQqWsra3ZuHEjR44coWHDhnh7e7Nx48Y37gCS3/XasmVLrl+/roKKhRCqIAFOfDTmzJlDQEAAT548AWDmzJkMHDiQFy9ekJiYyODBg7l37947v0cWPhWq1rlzZzZv3szp06cZPHiwsiXe6+jq6lKuXDlu3rz5ASsUQqiadKGKj4KZmRm9evUiIiICAwMD2rRpQ79+/XB3d8fLy4vk5GQOHjxIzZo13/ldEt6Eqh06dIhBgwbRvHlzpk2bRunSpV97nLq6OmlpaRLehCiGpAVOfDRq1KjBypUr0dTUJDg4mJIlSyr7RJqYmLBixQrq1q2LnZ0d165dU3G1ojhq27YtycnJBZaieVu3vrW1NevWrWPnzp1Mnz79rS1xQojiRVrgRJGXPzkhJiaGkSNHkpGRwdKlSylTpoxyzOPHj/nyyy+5dOkSW7dupW7duqoqVxRTrVu3ZtKkSYSEhLB69WpsbGzQ0tIiNzcXDQ2N134mMjKSkSNHUrlyZZ49e/aBKxZCFGbSAieKtEqVKhEXFwdA7969iYyMxNzcHH9/f8zMzPjss894/PixcryRkRHbtm3j3r17DBkyRFVli2JKR0eHMmXK4OPjg7GxMenp6Tg7O5OamvraSQp/bp2TnRaEEPkkwIkiq2XLlnh6ehIYGEi7du0YMWIEDRs2JCEhQelO1dHRoXv37gVaLwwMDEhJSZEfQvHBlChRgvT0dOXfdXV1ad++PRMmTMDIyIhu3brx+PHjN840FUKIP5MAJ4ocY2NjEhMTMTc3Z/78+dSqVQsDAwN69OjB1atXleNq1KhBcHAwWlpa9OjR45UuKGnNEB+Cra0tVatW5ZtvvuHBgwcFrruaNWuyePFiSpUqRceOHQuEPCGEeBsZAyeKlICAAEaOHIm6ujqxsbGcOXOG0qVLc+PGDapVq1bg2JfHxJ05cwZ9ff0C70t4E/+1wYMHExQUxIsXL5S1CV++7q5du8aUKVNITk7G19f3jWPhhBDizyTAiSLl+PHjzJ8/n5ycHLS1tTlw4ACDBg3i/v37DB8+nD59+hQ4PiYmhjFjxnDo0CHS0tJUVLUojho3bsy0adMYN24cK1asIDU1FRMTE4yMjAocd+XKFb799luqVatGhQoVVFStEKKokQAnipRdu3aRlZWFvb09q1atIikpiWPHjuHl5UVaWhqOjo7Y2toqx7u6unLz5k3GjBlDTk4O6upyyYsPo3Tp0ly6dImdO3dSt25dQkND2bt3L1u3biUgIEA5Ljs7m82bN1O2bFmcnZ1VWLEQoiiRXzNRJPx5H9OSJUtiZmbGjBkzqFChAr///jseHh6kpqbi4uLCzJkz2bRpE5MnT1a6rgAZIC4+mDp16lCmTBl0dXUJDg7m5s2bzJs3j3379tG0aVM2btyoHJuSkoK3tzeVK1fGwMBAhVULIYoKCXCiSMgfN2RnZ0fdunVZtWoVW7dupWrVqnh4eFChQgVu3LjBjBkziImJ4f/+7/+AvB/RnJycN25kL8R/5ejRo2RmZjJ69Ghu3brF/PnziYiIYNmyZQQEBFChQgVat26tHH/nzh3u3LmjwoqFEEWJzEIVRYauri4//PADp0+fZuTIkUBeF2nv3r25ffs2c+bMIT4+Hn19fXJzc0lNTQVkb1OhGqampoSFhVGzZk1iYmLo1q2b8l7p0qU5duwYs2fPZuvWrcrrFSpUID4+XhXlCiGKGGmBE4XWy61mampqpKWl4erqSteuXbG3twcgJCSEHTt2YG5uzvTp06lYsSLPnz9XwhvI3qZCNR48eIC7uzsvXrygcePGDBw4UHkvNTWV69ev8+TJE+CPa13CmxDir5IWOFHoOTk58fDhQ86ePcuDBw/w8vLC0tISX19fZd03Z2dnRowYQXh4OAsXLlRxxUL8oWbNmmzatInU1FROnz7NqVOnGDx4MIaGhnTp0kXGZQoh/hEJcKJQs7S05MiRIzx48IBz586xbNkyUlJSWLVqFWvXri0wELxnz57s27dPfhBFoVOlShWcnJzo2LEjSUlJJCYm4urqSlZWluy+IIT4RyTAiULNwMAALy8v6tWrR0REBB4eHri5udGlSxesrKzo2LEjCQkJBT4jP4iisNLU1ERbW1vGZwoh/jUZAycKpa5du2JpaUlycjJLly6lSpUqxMXF0atXL+zs7MjKysLExAR/f3/09PQKfFbCmyissrKyZHymEOK9kAAnCp3atWszduxYdu3aha2tLbGxsUyYMAEXFxcePnzIxIkTOXLkCA8fPsTQ0LDAD6IQQghRHEgXqiiUqlWrhp2dHWPGjCE8PJyYmBhMTU2Jj48nNDQUyFtWJCMjQ1rchBBCFDsS4ESh1rlzZ/r27YuFhQXVqlXjzp07DB48uMCCpzLmTQghRHEjAU4UehUqVKBhw4ZMnjyZevXqsWrVKjw8PFRdlhBCCKEyEuCESqipqZGbm/u3Ws9KliyJi4sLQUFBMvhbCCFEsSYBTnxw3bt3p06dOqxbt45Hjx79pc/8OejJ8gtCCCGKMwlw4oMqV64c0dHRpKSkoKamxpYtWzh//jyHDx9WjpExbUIIIcTbaaq6AFG8pKamcuLECXbv3s39+/fp0aMHq1atYseOHfzwww/s3LlTwpsQQgjxDrIOnPignj17RmRkJH5+fty+fRtPT09at26NgYEBQUFB7Nmzh549e2Jubq7qUoUQQohCSwKc+M9pauY19Kqr511u27dvJzo6mp49ewJw7949GjZsyMGDB4mPj2fcuHH88MMPdOrUSWU1CyGEEIWZdKGK/5SVlRWtWrVi5cqVPHnyBMjbPig2NhYbGxtWrVpFVFQUiYmJjBkzhpSUFBo3bkzjxo2Jjo5WbfFCCCFEISUtcOI/1blzZ3r27MmwYcMoVaqU8rq/vz+GhoY8fPiQ58+fM2TIEFJSUgA4d+4cq1atIjs7Gw0NDRVVLoQQQhReEuDEf8rT05PIyEi6d++Oq6srhoaGQN46cHv37uW3337D1dWVxMTE135elgoRQgghXiUBTvxn8lvPvL29OXToEH379sXV1RUjIyOysrLYtWsX5ubmtG/fXsWVCiGEEEWLBDjxXlWrVk3559zcP5YYtLS0pGzZsvTo0QNXV1dMTEy4du0aq1evZsSIEZiZmamiXCGEEKJIkgAn3hsLCwtOnz7N6NGj0dDQUNZzCwsLo1q1arRq1YqoqCisra0ZNmwY+vr6nD17lrt375KQkKDi6oUQQoiiQ2ahivfm999/x9fXlxkzZvD8+XPWrVtHaGgo1apVY8iQISQkJODj44O6ujrW1tbo6enh7e3Nnj17gD/2RxVCCCHE28lWWuJfq1u3Lr/99hsvXrwAYNSoUcyePZsbN26QlpbGkCFDuHPnToH9SxcuXIiOjg5jxoxRZelCCCFEkSRdqOJfsbOzIzo6mrlz5yoL9q5YsYKpU6diYWFBZGQkd+7cAfJmlOYv5jtx4kQJb0IIIcQ/JF2o4l8xNjYGwNHREX19fUaPHk1OTg6hoaFoa2vj4+NDYmIiq1atAiAnJ0e6SoUQQoh/SQKc+FdOnTpFVFQU0dHRjBo1ipCQEFxdXcnJyeHrr79GXV0dHx8fcnNzCQkJAZDwJoQQQvxL0oUq/pVffvmFjIwMmjVrhqOjI61btyY4OFjpKl25ciVeXl7MnTuXXr16qbhaIYQQ4uMgAU78LQ0aNEBfXx9tbW3ltTlz5mBsbExubi4uLi507tyZlStXKiEuODiY4cOHs3fvXlWVLYQQQnxUJMCJv8zW1pbDhw+zfv16/Pz8sLCwACA2NpbMzEw6derEiRMncHJyolOnTqxYsUIJcTt37pS9TYUQQoj3RAKc+Mv09PQAMDIyQktLi7179+Lt7U3Tpk2ZP38+Dg4OWFhYcOzYMRwdHbGzs2PSpEkFvkP2NhVCCCH+PZnEIP6yzZs3A7B06VJWr17Nvn37qFevHmvWrOHixYuULVuWxo0b8/vvv/PDDz/QsWNHfv31VxVXLYQQQnx8pAVO/C2bN29mxowZLFmyhIoVK7JgwQLat2/PxYsXOXXqFL/88oty7C+//EJOTo50mwohhBDvmbTAib8tJCSE3Nxc5s2bh76+PkuWLGHevHloamoquzG8TLpNhRBCiPdLApz4R1avXk1ubi5+fn5kZ2ezbNmy14Y3IYQQQrx/EuBEAfXr1+fx48ckJCQor71p54Q1a9aQm5vLnDlz0NPTw9/f/0OWKoQQQhRbspm9UHTt2hUfHx+ePn3KpUuXCA0N5fLly8oepjk5Oa/93NixY7G2tqZnz54fuGIhhBCieJIAJwowNTWlfPnyLFq0iOTkZK5fv46npyfp6elvDXFCCCGE+HAkwIkC8rtLS5Ysib29PXZ2dqSnpzNw4EDS0tIkxAkhhBCFgAQ4QZMmTUhPT+fSpUsAaGhokJ2djaamJlZWVkydOpUnT57g4OAgExWEEEKIQkDWgSvmWrZsyf79+xk7diyffvopkLfsh5qaGllZWURFRbFkyRJKlizJiBEjVFusEEIIIQAJcMWeqakpmZmZlC1bluHDh9OgQQMAcnNzUVNTIycnh0OHDnH+/Hk6dOiAjo6OiisWQgghhAS4Yu7cuXPs3LmT0NBQatasyahRozA3NwfyxsMBZGRkMH/+fCwtLXF2dlZluUIIIYRAAlyxp6GhQfPmzYmKimLp0qVUqVKF6dOnc+PGDby9vQHQ1NTk2bNnBAYGUrVqVRVXLIQQQghZyLcYU1NT4/bt21y9ehVzc3MiIiLQ0NBg8eLFJCcnc/jwYQCysrIAuHLlCnXr1kVHR4eMjAxVli6EEEIUaxLgirH83RXU1NRo0KABv/76K2PHjiU+Pp709HT69OlDcnIy586dA+DEiRMkJCRIeBNCCCFUTLpQBWfPnqVatWpERkaSnJxMmzZtWLx4MW3btsXKyqrAsTdv3lRNkUIIIYRQSAuc4NKlS3zzzTecOHECV1dXcnJy2L17N0lJSZw4cULV5QkhhBDiT2QhX4GOjg6dOnXizJkzPHr06JX3ZfcFIYQQonCRACeEEEIIUcTIGDghhBBCiCJGApwQQgghRBEjAU4IIYQQooiRACeEEEIIUcRIgBNCCCGEKGIkwAkhhBBCFDES4IQQQgghihgJcEIIIYQQRYwEOCGEEEKIIkYCnBBCCCFEESMBTgghhBCiiJEAJ4QQQghRxEiAE0IIIYQoYv4fivLASo9yYzkAAAAASUVORK5CYII=",
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"\n",
"import matplotlib.pyplot as plt\n",
"\n",
"\n",
"embeddings_cuda = embeddings.to(torch.device(\"cuda\"))\n",
"\n",
"functions = {\n",
" \"1) MHA wrapper class\": mha_ch03_wrapper,\n",
" \"2) MHA Ch03\": mha_ch03,\n",
" \"3) MHA with combined QKV weights\": mha_combined_qkv,\n",
" \"4) MHA with PyTorch scaled_dot_product_attention\": mha_pytorch_scaled,\n",
" \"5) PyTorch MHA class defaults\": mha_pytorch_class_default,\n",
" \"6) PyTorch MHA with need_weights=False\": mha_pytorch_class_noweights\n",
"}\n",
"execution_times = [time_pytorch_function(fn, embeddings_cuda) for name,fn in functions.items()]\n",
"\n",
"\n",
"# Plotting\n",
"\n",
"# Customize further for dark mode aesthetics\n",
"plt.rcParams['figure.facecolor'] = '#121212' # Dark figure background\n",
"plt.rcParams['axes.facecolor'] = '#121212' # Dark axes background\n",
"plt.rcParams['axes.edgecolor'] = 'white' # White axes border\n",
"plt.rcParams['axes.labelcolor'] = 'white' # White labels\n",
"plt.rcParams['text.color'] = 'white' # White text\n",
"plt.rcParams['xtick.color'] = 'white' # White x ticks\n",
"plt.rcParams['ytick.color'] = 'white' # White y ticks\n",
"plt.rcParams['grid.color'] = '#444444' # Lighter grid lines for contrast\n",
"plt.rcParams['lines.linewidth'] = 2 # Thicker plot lines for visibility\n",
"plt.rcParams['lines.markersize'] = 8 # Larger markers for visibility\n",
"\n",
"fig, ax = plt.subplots()\n",
"bars = plt.bar(functions.keys(), execution_times)\n",
"\n",
"plt.ylabel('Execution time (ms)')\n",
"plt.xticks(rotation=45, ha=\"right\")\n",
"\n",
"# Calculate new ylim with a margin\n",
"max_execution_time = max(execution_times)\n",
"upper_ylim = max_execution_time + 0.2 * max_execution_time # Adding a 20% margin\n",
"\n",
"plt.ylim(0, upper_ylim) # Setting new ylim\n",
"\n",
"# Annotate bars with execution times\n",
"for bar in bars:\n",
" yval = bar.get_height()\n",
" plt.text(bar.get_x() + bar.get_width()/2, yval + (0.05 * upper_ylim), round(yval, 2), ha='center', va='bottom')\n",
"\n",
"\n",
"plt.tight_layout()\n",
"plt.savefig(\"1.pdf\")\n",
"plt.show()\n"
]
}
],
"metadata": {
"accelerator": "GPU",
"colab": {
"gpuType": "A100",
"machine_shape": "hm",
"provenance": []
},
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.4"
}
},
"nbformat": 4,
"nbformat_minor": 5
}