From 3f51f48ee55690c7f64e591a729d9cb034264ca4 Mon Sep 17 00:00:00 2001
From: SAM <8dmasters@gmail.com>
Date: Fri, 31 May 2024 10:00:10 +0530
Subject: [PATCH 01/76] Create Transformers.md
Added a empty .md file to be served as a guide for Transformers
---
contrib/machine-learning/Transformers.md | 0
1 file changed, 0 insertions(+), 0 deletions(-)
create mode 100644 contrib/machine-learning/Transformers.md
diff --git a/contrib/machine-learning/Transformers.md b/contrib/machine-learning/Transformers.md
new file mode 100644
index 0000000..e69de29
From b832f8f0c90673462ecc990f45595761565b027d Mon Sep 17 00:00:00 2001
From: SAM <60264918+SAM-DEV007@users.noreply.github.com>
Date: Fri, 31 May 2024 13:40:20 +0530
Subject: [PATCH 02/76] Update Transformers.md
Added topics
---
contrib/machine-learning/Transformers.md | 21 +++++++++++++++++++++
1 file changed, 21 insertions(+)
diff --git a/contrib/machine-learning/Transformers.md b/contrib/machine-learning/Transformers.md
index e69de29..7bcc102 100644
--- a/contrib/machine-learning/Transformers.md
+++ b/contrib/machine-learning/Transformers.md
@@ -0,0 +1,21 @@
+# Transformers
+## Introduction
+A transformer is a deep learning architecture developed by Google and based on the multi-head attention mechanism. It is based on the softmax-based attention
+mechanism. Before transformers, predecessors of attention mechanism were added to gated recurrent neural networks, such as LSTMs and gated recurrent units (GRUs), which
+processed datasets sequentially. Dependency on previous token computations prevented them from being able to parallelize the attention mechanism.
+
+## Key Concepts
+
+## Architecture
+
+## Implementation
+### Theory
+Text is converted to numerical representations called tokens, and each token is converted into a vector via looking up from a word embedding table.
+At each layer, each token is then contextualized within the scope of the context window with other tokens via a parallel multi-head attention mechanism
+allowing the signal for key tokens to be amplified and less important tokens to be diminished.
+
+### HuggingFace
+
+### Tensorflow and Keras
+
+### PyTorch
From 5b3c8d7ad108d4a65b4a946d719d242fe9eae308 Mon Sep 17 00:00:00 2001
From: SAM <60264918+SAM-DEV007@users.noreply.github.com>
Date: Fri, 31 May 2024 15:14:09 +0530
Subject: [PATCH 03/76] Update Transformers.md
Added information about Model Architecture
---
contrib/machine-learning/Transformers.md | 59 +++++++++++++++++++++++-
1 file changed, 57 insertions(+), 2 deletions(-)
diff --git a/contrib/machine-learning/Transformers.md b/contrib/machine-learning/Transformers.md
index 7bcc102..49a1b97 100644
--- a/contrib/machine-learning/Transformers.md
+++ b/contrib/machine-learning/Transformers.md
@@ -4,9 +4,53 @@ A transformer is a deep learning architecture developed by Google and based on t
mechanism. Before transformers, predecessors of attention mechanism were added to gated recurrent neural networks, such as LSTMs and gated recurrent units (GRUs), which
processed datasets sequentially. Dependency on previous token computations prevented them from being able to parallelize the attention mechanism.
-## Key Concepts
+## Model Architecture
+

-## Architecture
+### Encoder
+The encoder is composed of a stack of identical layers. Each layer has two sub-layers. The first is a multi-head self-attention mechanism, and the second is a simple, positionwise fully connected feed-forward network. Each encoder consists of two major components: a self-attention mechanism and a feed-forward neural network. The self-attention mechanism accepts input encodings from the previous encoder and weights their relevance to each other to generate output encodings. The feed-forward neural network further processes each output encoding individually. These output encodings are then passed to the next encoder as its input, as well as to the decoders.
+
+### Decoder
+The decoder is also composed of a stack of identical layers. In addition to the two sub-layers in each encoder layer, the decoder inserts a third sub-layer, which performs multi-head attention over the output of the encoder stack. The decoder functions in a similar fashion to the encoder, but an additional attention mechanism is inserted which instead draws relevant information from the encodings generated by the encoders. This mechanism can also be called the encoder-decoder attention.
+
+### Attention
+#### Scaled Dot-Product Attention
+The input consists of queries and keys of dimension dk, and values of dimension dv. We compute the dot products of the query with all keys, divide each by √dk, and apply a softmax function to obtain the weights on the values.
+
+> Attention(Q, K, V) = softmax(QKT / √dk) * V
+
+#### Multi-Head Attention
+Instead of performing a single attention function with dmodel-dimensional keys, values and queries, it is beneficial to linearly project the queries, keys and values h times with different, learned linear projections to dk, dk and dv dimensions, respectively.
+
+Multi-head attention allows the model to jointly attend to information from different representation
+subspaces at different positions. With a single attention head, averaging inhibits this.
+
+> MultiHead(Q, K, V) = Concat(head1, ..., headh) * WO
+
+where,
+
+> headi = Attention(Q * WiQ, K * WiK, V * WiV)
+
+where the projections are parameter matrices.
+
+#### Masked Attention
+It may be necessary to cut out attention links between some word-pairs. For example, the decoder for token position
+𝑡 should not have access to token position 𝑡+1.
+
+> MaskedAttention(Q, K, V) = softmax(M + (QKT / √dk)) * V
+
+### Feed-Forward Network
+Each of the layers in the encoder and decoder contains a fully connected feed-forward network, which is applied to each position separately and identically. This
+consists of two linear transformations with a ReLU activation in between.
+> FFN(x) = (max(0, (x * W1) + b1) * W2) + b2
+
+### Positional Encoding
+A positional encoding is a fixed-size vector representation that encapsulates the relative positions of tokens within a target sequence: it provides the transformer model with information about where the words are in the input sequence.
+
+The sine and cosine functions of different frequencies:
+> PE(pos,2i) = sin(pos/100002i/dmodel)
+
+> PE(pos,2i+1) = cos(pos/100002i/dmodel)
## Implementation
### Theory
@@ -19,3 +63,14 @@ allowing the signal for key tokens to be amplified and less important tokens to
### Tensorflow and Keras
### PyTorch
+
+## Application
+The transformer has had great success in natural language processing (NLP). Many large language models such as GPT-2, GPT-3, GPT-4, Claude, BERT, XLNet, RoBERTa and ChatGPT demonstrate the ability of transformers to perform a wide variety of such NLP-related tasks, and have the potential to find real-world applications.
+
+These may include:
+- Machine translation
+- Document summarization
+- Text generation
+- Biological sequence analysis
+- Computer code generation
+- Video analysis
From 35357d2fe3f6650faca3d0f19327fe4fa7e8629d Mon Sep 17 00:00:00 2001
From: SAM <60264918+SAM-DEV007@users.noreply.github.com>
Date: Fri, 31 May 2024 17:04:07 +0530
Subject: [PATCH 04/76] Update Transformers.md
Added Tensorflow implementation
---
contrib/machine-learning/Transformers.md | 136 ++++++++++++++++++++++-
1 file changed, 134 insertions(+), 2 deletions(-)
diff --git a/contrib/machine-learning/Transformers.md b/contrib/machine-learning/Transformers.md
index 49a1b97..a5a56ec 100644
--- a/contrib/machine-learning/Transformers.md
+++ b/contrib/machine-learning/Transformers.md
@@ -58,12 +58,144 @@ Text is converted to numerical representations called tokens, and each token is
At each layer, each token is then contextualized within the scope of the context window with other tokens via a parallel multi-head attention mechanism
allowing the signal for key tokens to be amplified and less important tokens to be diminished.
-### HuggingFace
+### Tensorflow
+Tensorflow provides the transformer encoder and decoder block that can be implemented by the specification of the user. Although, the transformer is not provided as a standalone to be imported and executed, the user has to create the model first. They also have a tutorial on how to implement the transformer from scratch for machine translation and can be found [here](https://www.tensorflow.org/text/tutorials/transformer).
-### Tensorflow and Keras
+More information on [encoder](https://www.tensorflow.org/api_docs/python/tfm/nlp/layers/TransformerEncoderBlock) and [decoder](https://www.tensorflow.org/api_docs/python/tfm/nlp/layers/TransformerDecoderBlock) block mentioned in the code.
+
+Imports:
+```
+import tensorflow as tf
+import tensorflow_models as tfm
+```
+
+Adding word embeddings and positional encoding:
+```
+class PositionalEmbedding(tf.keras.layers.Layer):
+ def __init__(self, vocab_size, d_model):
+ super().__init__()
+ self.d_model = d_model
+ self.embedding = tf.keras.layers.Embedding(vocab_size, d_model, mask_zero=True)
+ self.pos_encoding = tfm.nlp.layers.RelativePositionEmbedding(hidden_size=d_model)
+
+ def compute_mask(self, *args, **kwargs):
+ return self.embedding.compute_mask(*args, **kwargs)
+
+ def call(self, x):
+ length = tf.shape(x)[1]
+ x = self.embedding(x)
+ x = x + self.pos_encoding[tf.newaxis, :length, :]
+ return x
+```
+
+Creating the encoder for the transformer:
+```
+class Encoder(tf.keras.layers.Layer):
+ def __init__(self, num_layers, d_model, num_heads,
+ dff, vocab_size, dropout_rate=0.1):
+ super().__init__()
+
+ self.d_model = d_model
+ self.num_layers = num_layers
+
+ self.pos_embedding = PositionalEmbedding(
+ vocab_size=vocab_size, d_model=d_model)
+
+ self.enc_layers = [
+ tfm.nlp.layers.TransformerEncoderBlock(output_last_dim=d_model,
+ num_attention_heads=num_heads,
+ inner_dim=dff,
+ inner_activation="relu",
+ inner_dropout=dropout_rate)
+ for _ in range(num_layers)]
+ self.dropout = tf.keras.layers.Dropout(dropout_rate)
+
+ def call(self, x):
+ x = self.pos_embedding(x, length=2048)
+ x = self.dropout(x)
+
+ for i in range(self.num_layers):
+ x = self.enc_layers[i](x)
+
+ return x
+```
+
+Creating the decoder for the transformer:
+```
+class Decoder(tf.keras.layers.Layer):
+ def __init__(self, num_layers, d_model, num_heads, dff, vocab_size,
+ dropout_rate=0.1):
+ super(Decoder, self).__init__()
+
+ self.d_model = d_model
+ self.num_layers = num_layers
+
+ self.pos_embedding = PositionalEmbedding(vocab_size=vocab_size,
+ d_model=d_model)
+ self.dropout = tf.keras.layers.Dropout(dropout_rate)
+ self.dec_layers = [
+ tfm.nlp.layers.TransformerDecoderBlock(num_attention_heads=num_heads,
+ intermediate_size=dff,
+ intermediate_activation="relu",
+ dropout_rate=dropout_rate)
+ for _ in range(num_layers)]
+
+ def call(self, x, context):
+ x = self.pos_embedding(x)
+ x = self.dropout(x)
+
+ for i in range(self.num_layers):
+ x = self.dec_layers[i](x, context)
+
+ return x
+```
+
+Combining the encoder and decoder to create the transformer:
+```
+class Transformer(tf.keras.Model):
+ def __init__(self, num_layers, d_model, num_heads, dff,
+ input_vocab_size, target_vocab_size, dropout_rate=0.1):
+ super().__init__()
+ self.encoder = Encoder(num_layers=num_layers, d_model=d_model,
+ num_heads=num_heads, dff=dff,
+ vocab_size=input_vocab_size,
+ dropout_rate=dropout_rate)
+
+ self.decoder = Decoder(num_layers=num_layers, d_model=d_model,
+ num_heads=num_heads, dff=dff,
+ vocab_size=target_vocab_size,
+ dropout_rate=dropout_rate)
+
+ self.final_layer = tf.keras.layers.Dense(target_vocab_size)
+
+ def call(self, inputs):
+ context, x = inputs
+
+ context = self.encoder(context)
+ x = self.decoder(x, context)
+ logits = self.final_layer(x)
+
+ return logits
+```
+
+Model initialization that be used for training and inference:
+```
+transformer = Transformer(
+ num_layers=num_layers,
+ d_model=d_model,
+ num_heads=num_heads,
+ dff=dff,
+ input_vocab_size=tokenizers.pt.get_vocab_size().numpy(),
+ target_vocab_size=tokenizers.en.get_vocab_size().numpy(),
+ dropout_rate=dropout_rate
+)
+```
### PyTorch
+
+### HuggingFace
+
## Application
The transformer has had great success in natural language processing (NLP). Many large language models such as GPT-2, GPT-3, GPT-4, Claude, BERT, XLNet, RoBERTa and ChatGPT demonstrate the ability of transformers to perform a wide variety of such NLP-related tasks, and have the potential to find real-world applications.
From f56131349d1ad0e6e27209ea3d9217c13e695e8e Mon Sep 17 00:00:00 2001
From: SAM <60264918+SAM-DEV007@users.noreply.github.com>
Date: Fri, 31 May 2024 17:13:19 +0530
Subject: [PATCH 05/76] Update Transformers.md
Added python tag to code blocks
---
contrib/machine-learning/Transformers.md | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/contrib/machine-learning/Transformers.md b/contrib/machine-learning/Transformers.md
index a5a56ec..ffdea05 100644
--- a/contrib/machine-learning/Transformers.md
+++ b/contrib/machine-learning/Transformers.md
@@ -64,13 +64,13 @@ Tensorflow provides the transformer encoder and decoder block that can be implem
More information on [encoder](https://www.tensorflow.org/api_docs/python/tfm/nlp/layers/TransformerEncoderBlock) and [decoder](https://www.tensorflow.org/api_docs/python/tfm/nlp/layers/TransformerDecoderBlock) block mentioned in the code.
Imports:
-```
+```python
import tensorflow as tf
import tensorflow_models as tfm
```
Adding word embeddings and positional encoding:
-```
+```python
class PositionalEmbedding(tf.keras.layers.Layer):
def __init__(self, vocab_size, d_model):
super().__init__()
@@ -89,7 +89,7 @@ class PositionalEmbedding(tf.keras.layers.Layer):
```
Creating the encoder for the transformer:
-```
+```python
class Encoder(tf.keras.layers.Layer):
def __init__(self, num_layers, d_model, num_heads,
dff, vocab_size, dropout_rate=0.1):
@@ -121,7 +121,7 @@ class Encoder(tf.keras.layers.Layer):
```
Creating the decoder for the transformer:
-```
+```python
class Decoder(tf.keras.layers.Layer):
def __init__(self, num_layers, d_model, num_heads, dff, vocab_size,
dropout_rate=0.1):
@@ -151,7 +151,7 @@ class Decoder(tf.keras.layers.Layer):
```
Combining the encoder and decoder to create the transformer:
-```
+```python
class Transformer(tf.keras.Model):
def __init__(self, num_layers, d_model, num_heads, dff,
input_vocab_size, target_vocab_size, dropout_rate=0.1):
@@ -179,7 +179,7 @@ class Transformer(tf.keras.Model):
```
Model initialization that be used for training and inference:
-```
+```python
transformer = Transformer(
num_layers=num_layers,
d_model=d_model,
From ca9c6a9a1f23d1e6b5901954c8a5c5c772bd80a7 Mon Sep 17 00:00:00 2001
From: SAM <60264918+SAM-DEV007@users.noreply.github.com>
Date: Fri, 31 May 2024 17:44:28 +0530
Subject: [PATCH 06/76] Update Transformers.md
Added PyTorch implementation
---
contrib/machine-learning/Transformers.md | 24 +++++++++++++++++++++++-
1 file changed, 23 insertions(+), 1 deletion(-)
diff --git a/contrib/machine-learning/Transformers.md b/contrib/machine-learning/Transformers.md
index ffdea05..cb90cf1 100644
--- a/contrib/machine-learning/Transformers.md
+++ b/contrib/machine-learning/Transformers.md
@@ -56,7 +56,9 @@ The sine and cosine functions of different frequencies:
### Theory
Text is converted to numerical representations called tokens, and each token is converted into a vector via looking up from a word embedding table.
At each layer, each token is then contextualized within the scope of the context window with other tokens via a parallel multi-head attention mechanism
-allowing the signal for key tokens to be amplified and less important tokens to be diminished.
+allowing the signal for key tokens to be amplified and less important tokens to be diminished.
+
+The transformer uses an encoder-decoder architecture. The encoder extracts features from an input sentence, and the decoder uses the features to produce an output sentence. Some architectures use full encoders and decoders, autoregressive encoders and decoders, or combination of both. This depends on the usage and context of the input.
### Tensorflow
Tensorflow provides the transformer encoder and decoder block that can be implemented by the specification of the user. Although, the transformer is not provided as a standalone to be imported and executed, the user has to create the model first. They also have a tutorial on how to implement the transformer from scratch for machine translation and can be found [here](https://www.tensorflow.org/text/tutorials/transformer).
@@ -192,10 +194,30 @@ transformer = Transformer(
```
### PyTorch
+Unlike Tensorflow, PyTorch provides the full implementation of the transformer model that can be executed on the go. More information can be found [here](https://pytorch.org/docs/stable/_modules/torch/nn/modules/transformer.html#Transformer). A full implementation of the model can be found [here](https://github.com/pytorch/examples/tree/master/word_language_model).
+Imports:
+```python
+import torch
+import torch.nn as nn
+```
+
+Initializing the model:
+```python
+transformer = nn.Transformer(nhead=16, num_encoder_layers=8)
+```
+
+Sample Implementation:
+```python
+src = torch.rand((10, 32, 512))
+tgt = torch.rand((20, 32, 512))
+
+output = transformer(src, tgt)
+```
### HuggingFace
+
## Application
The transformer has had great success in natural language processing (NLP). Many large language models such as GPT-2, GPT-3, GPT-4, Claude, BERT, XLNet, RoBERTa and ChatGPT demonstrate the ability of transformers to perform a wide variety of such NLP-related tasks, and have the potential to find real-world applications.
From d81e875a3235d7c9f50f9a42bb359bf80d3146a1 Mon Sep 17 00:00:00 2001
From: SAM <60264918+SAM-DEV007@users.noreply.github.com>
Date: Fri, 31 May 2024 18:32:47 +0530
Subject: [PATCH 07/76] Update Transformers.md
Added outputs for PyTorch and HuggingFace implementation
---
contrib/machine-learning/Transformers.md | 114 ++++++++++++++++++++++-
1 file changed, 112 insertions(+), 2 deletions(-)
diff --git a/contrib/machine-learning/Transformers.md b/contrib/machine-learning/Transformers.md
index cb90cf1..e9e50e8 100644
--- a/contrib/machine-learning/Transformers.md
+++ b/contrib/machine-learning/Transformers.md
@@ -61,6 +61,8 @@ allowing the signal for key tokens to be amplified and less important tokens to
The transformer uses an encoder-decoder architecture. The encoder extracts features from an input sentence, and the decoder uses the features to produce an output sentence. Some architectures use full encoders and decoders, autoregressive encoders and decoders, or combination of both. This depends on the usage and context of the input.
### Tensorflow
+TensorFlow is a free and open-source software library for machine learning and artificial intelligence. It can be used across a range of tasks but has a particular focus on training and inference of deep neural networks. It was developed by the Google Brain team for Google's internal use in research and production.
+
Tensorflow provides the transformer encoder and decoder block that can be implemented by the specification of the user. Although, the transformer is not provided as a standalone to be imported and executed, the user has to create the model first. They also have a tutorial on how to implement the transformer from scratch for machine translation and can be found [here](https://www.tensorflow.org/text/tutorials/transformer).
More information on [encoder](https://www.tensorflow.org/api_docs/python/tfm/nlp/layers/TransformerEncoderBlock) and [decoder](https://www.tensorflow.org/api_docs/python/tfm/nlp/layers/TransformerDecoderBlock) block mentioned in the code.
@@ -193,7 +195,22 @@ transformer = Transformer(
)
```
+Sample:
+```python
+src = tf.random.uniform((64, 40))
+tgt = tf.random.uniform((64, 50))
+
+output = transformer((src, tgt))
+```
+
+O/P:
+```
+
+```
+
### PyTorch
+PyTorch is a machine learning library based on the Torch library, used for applications such as computer vision and natural language processing, originally developed by Meta AI and now part of the Linux Foundation umbrella.
+
Unlike Tensorflow, PyTorch provides the full implementation of the transformer model that can be executed on the go. More information can be found [here](https://pytorch.org/docs/stable/_modules/torch/nn/modules/transformer.html#Transformer). A full implementation of the model can be found [here](https://github.com/pytorch/examples/tree/master/word_language_model).
Imports:
@@ -207,7 +224,7 @@ Initializing the model:
transformer = nn.Transformer(nhead=16, num_encoder_layers=8)
```
-Sample Implementation:
+Sample:
```python
src = torch.rand((10, 32, 512))
tgt = torch.rand((20, 32, 512))
@@ -215,8 +232,101 @@ tgt = torch.rand((20, 32, 512))
output = transformer(src, tgt)
```
-### HuggingFace
+O/P:
+```
+tensor([[[ 0.2938, -0.4824, -0.7816, ..., 0.0742, 0.5162, 0.3632],
+ [-0.0786, -0.5241, 0.6384, ..., 0.3462, -0.0618, 0.9943],
+ [ 0.7827, 0.1067, -0.1637, ..., -1.7730, -0.3322, -0.0029],
+ ...,
+ [-0.3202, 0.2341, -0.0896, ..., -0.9714, -0.1251, -0.0711],
+ [-0.1663, -0.5047, -0.0404, ..., -0.9339, 0.3963, 0.1018],
+ [ 1.2834, -0.4400, 0.0486, ..., -0.6876, -0.4752, 0.0180]],
+ [[ 0.9869, -0.7384, -1.0704, ..., -0.9417, 1.3279, -0.1665],
+ [ 0.3445, -0.2454, -0.3644, ..., -0.4856, -1.1004, -0.6819],
+ [ 0.7568, -0.3151, -0.5034, ..., -1.2081, -0.7119, 0.3775],
+ ...,
+ [-0.0451, -0.7596, 0.0168, ..., -0.8267, -0.3272, 1.0457],
+ [ 0.3150, -0.6588, -0.1840, ..., 0.1822, -0.0653, 0.9053],
+ [ 0.8692, -0.3519, 0.3128, ..., -1.8446, -0.2325, -0.8662]],
+
+ [[ 0.9719, -0.3113, 0.4637, ..., -0.4422, 1.2348, 0.8274],
+ [ 0.3876, -0.9529, -0.7810, ..., -0.5843, -1.1439, -0.3366],
+ [-0.5774, 0.3789, -0.2819, ..., -1.4057, 0.4352, 0.1474],
+ ...,
+ [ 0.6899, -0.1146, -0.3297, ..., -1.7059, -0.1750, 0.4203],
+ [ 0.3689, -0.5174, -0.1253, ..., 0.1417, 0.4159, 0.7560],
+ [ 0.5024, -0.7996, 0.1592, ..., -0.8344, -1.1125, 0.4736]],
+
+ ...,
+
+ [[ 0.0704, -0.3971, -0.2768, ..., -1.9929, 0.8608, 1.2264],
+ [ 0.4013, -0.0962, -0.0965, ..., -0.4452, -0.8682, -0.4593],
+ [ 0.1656, 0.5224, -0.1723, ..., -1.5785, 0.3219, 1.1507],
+ ...,
+ [-0.9443, 0.4653, 0.2936, ..., -0.9840, -0.0142, -0.1595],
+ [-0.6544, -0.3294, -0.0803, ..., 0.1623, -0.5061, 0.9824],
+ [-0.0978, -1.0023, -0.6915, ..., -0.2296, -0.0594, -0.4715]],
+
+ [[ 0.6531, -0.9285, -0.0331, ..., -1.1481, 0.7768, -0.7321],
+ [ 0.3325, -0.6683, -0.6083, ..., -0.4501, 0.2289, 0.3573],
+ [-0.6750, 0.4600, -0.8512, ..., -2.0097, -0.5159, 0.2773],
+ ...,
+ [-1.4356, -1.0135, 0.0081, ..., -1.2985, -0.3715, -0.2678],
+ [ 0.0546, -0.2111, -0.0965, ..., -0.3822, -0.4612, 1.6217],
+ [ 0.7700, -0.5309, -0.1754, ..., -2.2807, -0.0320, -1.5551]],
+
+ [[ 0.2399, -0.9659, 0.1086, ..., -1.1756, 0.4063, 0.0615],
+ [-0.2202, -0.7972, -0.5024, ..., -0.9126, -1.5248, 0.2418],
+ [ 0.5215, 0.4540, 0.0036, ..., -0.2135, 0.2145, 0.6638],
+ ...,
+ [-0.2190, -0.4967, 0.7149, ..., -0.3324, 0.3502, 1.0624],
+ [-0.0108, -0.9205, -0.1315, ..., -1.0153, 0.2989, 1.1415],
+ [ 1.1284, -0.6560, 0.6755, ..., -1.2157, 0.8580, -0.5022]]],
+ grad_fn=)
+```
+```
+>> output.shape
+torch.Size([20, 32, 512])
+```
+
+### HuggingFace
+Hugging Face, Inc. is a French-American company incorporated under the Delaware General Corporation Law and based in New York City that develops computation tools for building applications using machine learning.
+
+It has a wide-range of models that can implemented in Tensorflow, PyTorch and other development backends as well. The models are already trained on a dataset and can be pretrained on custom dataset for customized use, according to the user. The information for training the model and loading the pretrained model can be found [here](https://huggingface.co/docs/transformers/en/training).
+
+In HuggingFace, `pipeline` is used to run inference from the trained model available in the Hub. This is very beginner friendly. The model is downloaded to the local system on running the script before running the inference. It has to be made sure that the model downloaded does not exceed your available data plan.
+
+Imports:
+```python
+from transformers import pipeline
+```
+
+Initialization:
+
+The model used here is BART (large) which was trained on MultiNLI dataset, which consist of sentence paired with its textual entailment.
+```python
+classifier = pipeline(model="facebook/bart-large-mnli")
+```
+
+Sample:
+
+The first argument is the sentence which needs to be analyzed. The second argument, `candidate_labels`, is the list of labels which most likely the first argument sentence belongs to. The output dictionary will have a key as `score`, where the highest index is the textual entailment of the sentence with the index of the label in the list.
+
+```python
+output = classifier(
+ "I need to leave but later",
+ candidate_labels=["urgent", "not urgent", "sleep"],
+)
+```
+
+O/P:
+
+```
+{'sequence': 'I need to leave but later',
+ 'labels': ['not urgent', 'urgent', 'sleep'],
+ 'scores': [0.8889380097389221, 0.10631518065929413, 0.00474683940410614]}
+```
## Application
The transformer has had great success in natural language processing (NLP). Many large language models such as GPT-2, GPT-3, GPT-4, Claude, BERT, XLNet, RoBERTa and ChatGPT demonstrate the ability of transformers to perform a wide variety of such NLP-related tasks, and have the potential to find real-world applications.
From 251c4d86894b26e113b1b4426aeaf412c0b4c977 Mon Sep 17 00:00:00 2001
From: SAM <60264918+SAM-DEV007@users.noreply.github.com>
Date: Fri, 31 May 2024 18:43:19 +0530
Subject: [PATCH 08/76] Update Transformers.md
Added Tensorflow output and fixed wrong mentions
---
contrib/machine-learning/Transformers.md | 93 +++++++++++++++++++++++-
1 file changed, 91 insertions(+), 2 deletions(-)
diff --git a/contrib/machine-learning/Transformers.md b/contrib/machine-learning/Transformers.md
index e9e50e8..d30bd63 100644
--- a/contrib/machine-learning/Transformers.md
+++ b/contrib/machine-learning/Transformers.md
@@ -189,8 +189,8 @@ transformer = Transformer(
d_model=d_model,
num_heads=num_heads,
dff=dff,
- input_vocab_size=tokenizers.pt.get_vocab_size().numpy(),
- target_vocab_size=tokenizers.en.get_vocab_size().numpy(),
+ input_vocab_size=64,
+ target_vocab_size=64,
dropout_rate=dropout_rate
)
```
@@ -205,7 +205,96 @@ output = transformer((src, tgt))
O/P:
```
+
+```
+```
+>>> output.shape
+TensorShape([64, 50, 64])
```
### PyTorch
From 44303ff8e04e496e0f545d92a60e9758b971e314 Mon Sep 17 00:00:00 2001
From: SAM <60264918+SAM-DEV007@users.noreply.github.com>
Date: Fri, 31 May 2024 18:45:07 +0530
Subject: [PATCH 09/76] Update Transformers.md
Extended Introduction
---
contrib/machine-learning/Transformers.md | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/contrib/machine-learning/Transformers.md b/contrib/machine-learning/Transformers.md
index d30bd63..97346bd 100644
--- a/contrib/machine-learning/Transformers.md
+++ b/contrib/machine-learning/Transformers.md
@@ -1,8 +1,9 @@
# Transformers
## Introduction
A transformer is a deep learning architecture developed by Google and based on the multi-head attention mechanism. It is based on the softmax-based attention
-mechanism. Before transformers, predecessors of attention mechanism were added to gated recurrent neural networks, such as LSTMs and gated recurrent units (GRUs), which
-processed datasets sequentially. Dependency on previous token computations prevented them from being able to parallelize the attention mechanism.
+mechanism. Before transformers, predecessors of attention mechanism were added to gated recurrent neural networks, such as LSTMs and gated recurrent units (GRUs), which processed datasets sequentially. Dependency on previous token computations prevented them from being able to parallelize the attention mechanism.
+
+Transformers are a revolutionary approach to natural language processing (NLP). Unlike older models, they excel at understanding long-range connections between words. This "attention" mechanism lets them grasp the context of a sentence, making them powerful for tasks like machine translation, text summarization, and question answering. Introduced in 2017, transformers are now the backbone of many large language models, including tools you might use every day. Their ability to handle complex relationships in language is fueling advancements in AI across various fields.
## Model Architecture

From f75b2bab640e8b52e94d383ad2805ef487d2984a Mon Sep 17 00:00:00 2001
From: Arihant Yadav <147732947+arihunter-18@users.noreply.github.com>
Date: Fri, 31 May 2024 22:22:54 +0530
Subject: [PATCH 10/76] Add files via upload
---
contrib/advanced-python/closures.md | 101 ++++++++++++++++++++++++++++
1 file changed, 101 insertions(+)
create mode 100644 contrib/advanced-python/closures.md
diff --git a/contrib/advanced-python/closures.md b/contrib/advanced-python/closures.md
new file mode 100644
index 0000000..87ad058
--- /dev/null
+++ b/contrib/advanced-python/closures.md
@@ -0,0 +1,101 @@
+# Closures
+In order to have complete understanding of this topic in python, one needs to be crystal clear with the concept of functions and the different types of them which are namely First Class Functions and Nested Functions.
+
+### First Class Functions
+These are the normal functions used by the programmer in routine as they can be assigned to variables, passed as arguments and returned from other functions.
+### Nested Functions
+These are the functions defined within other functions and involve thorough usage of **Closures**. It is also referred as **Inner Functions** by some books. There are times when it is required to prevent a function or the data it has access to from being accessed from other parts of the code, and this is where Nested Functions come into play. Basically, its usage allows the encapsulation of that particular data/function within another function. This enables it to be virtually hidden from the global scope.
+
+## Defining Closures
+In nested functions, if the outer function basically ends up returning the inner function, in this case the concept of closures comes into play.
+
+A closure is a function object that remembers values in enclosing scopes even if they are not present in memory. There are certain neccesary condtions required to create a closure in python :
+1. The inner function must be defined inside the outer function.
+2. The inner function must refer to a value defined in the outer function.
+3. The inner function must return a value.
+
+## Advantages of Closures
+* Closures make it possible to pass data to inner functions without first passing them to outer functions
+* Closures can be used to create private variables and functions
+* They also make it possible to invoke the inner function from outside of the encapsulating outer function.
+* It improves code readability and maintainability
+
+## Examples implementing Closures
+### Example 1 : Basic Implementation
+```python
+def make_multiplier_of(n):
+ def multiplier(x):
+ return x * n
+ return multiplier
+
+times3 = make_multiplier_of(3)
+times5 = make_multiplier_of(5)
+
+print(times3(9))
+print(times5(3))
+```
+#### Output:
+```
+27
+15
+```
+The **multiplier function** is defined inside the **make_multiplier_of function**. It has access to the n variable from the outer scope, even after the make_multiplier_of function has returned. This is an example of a closure.
+
+### Example 2 : Implementation with Decorators
+```python
+def decorator_function(original_function):
+ def wrapper_function(*args, **kwargs):
+ print(f"Wrapper executed before {original_function.__name__}")
+ return original_function(*args, **kwargs)
+ return wrapper_function
+
+@decorator_function
+def display():
+ print("Display function executed")
+
+display()
+```
+#### Output:
+```
+ Wrapper executed before display
+ Display function executed
+```
+The code in the example defines a decorator function: ***decorator_function*** that takes a function as an argument and returns a new function **wrapper_function**. The **wrapper_function** function prints a message to the console before calling the original function which appends the name of the called function as specified in the code.
+
+The **@decorator_function** syntax is used to apply the decorator_function decorator to the display function. This means that the display function is replaced with the result of calling **decorator_function(display)**.
+
+When the **display()** function is called, the wrapper_function function is executed instead. The wrapper_function function prints a message to the console and then calls the original display function.
+### Example 3 : Implementation with for loop
+```python
+def create_closures():
+ closures = []
+ for i in range(5):
+ def closure(i=i): # Capture current value of i by default argument
+ return i
+ closures.append(closure)
+ return closures
+
+my_closures = create_closures()
+for closure in my_closures:
+ print(closure())
+
+```
+#### Output:
+```
+0
+1
+2
+3
+4
+```
+The code in the example defines a function **create_closures** that creates a list of closure functions. Each closure function returns the current value of the loop variable i.
+
+The closure function is defined inside the **create_closures function**. It has access to the i variable from the **outer scope**, even after the create_closures function has returned. This is an example of a closure.
+
+The **i**=*i* argument in the closure function is used to capture the current value of *i* by default argument. This is necessary because the ****i** variable in the outer scope is a loop variable, and its value changes in each iteration of the loop. By capturing the current value of *i* in the default argument, we ensure that each closure function returns the correct value of **i**. This is responsible for the generation of output 0,1,2,3,4.
+
+
+For more examples related to closures, [click here](https://dev.to/bshadmehr/understanding-closures-in-python-a-comprehensive-tutorial-11ld).
+
+## Summary
+Closures in Python provide a powerful mechanism for encapsulating state and behavior, enabling more flexible and modular code. Understanding and effectively using closures enables the creation of function factories, allows functions to have state, and facilitates functional programming techniques.
\ No newline at end of file
From 50f1e2c3deb4d244b052fcc4c9e4b7d3b6836551 Mon Sep 17 00:00:00 2001
From: Arihant Yadav <147732947+arihunter-18@users.noreply.github.com>
Date: Fri, 31 May 2024 22:24:16 +0530
Subject: [PATCH 11/76] Updated index.md
---
contrib/advanced-python/index.md | 1 +
1 file changed, 1 insertion(+)
diff --git a/contrib/advanced-python/index.md b/contrib/advanced-python/index.md
index b95e4b9..d9a9056 100644
--- a/contrib/advanced-python/index.md
+++ b/contrib/advanced-python/index.md
@@ -7,3 +7,4 @@
- [Regular Expressions in Python](regular_expressions.md)
- [JSON module](json-module.md)
- [Map Function](map-function.md)
+- [Closures](closures.md)
From f65c949564f003591c746088bf8c25be44a65853 Mon Sep 17 00:00:00 2001
From: Ritesh
Date: Sun, 2 Jun 2024 09:17:22 +0530
Subject: [PATCH 12/76] Create hierarchical-clustering.md
---
.../hierarchical-clustering.md | 119 ++++++++++++++++++
1 file changed, 119 insertions(+)
create mode 100644 contrib/machine-learning/hierarchical-clustering.md
diff --git a/contrib/machine-learning/hierarchical-clustering.md b/contrib/machine-learning/hierarchical-clustering.md
new file mode 100644
index 0000000..3f9e606
--- /dev/null
+++ b/contrib/machine-learning/hierarchical-clustering.md
@@ -0,0 +1,119 @@
+# Hierarchical Clustering
+
+Hierarchical Clustering is a method of cluster analysis that seeks to build a hierarchy of clusters. This README provides an overview of the hierarchical clustering algorithm, including its fundamental concepts, types, steps, and how to implement it using Python.
+
+## Table of Contents
+
+1. [Introduction](#introduction)
+2. [Concepts](#concepts)
+3. [Types of Hierarchical Clustering](#types-of-hierarchical-clustering)
+4. [Steps in Hierarchical Clustering](#steps-in-hierarchical-clustering)
+5. [Linkage Criteria](#linkage-criteria)
+6. [Implementation](#implementation)
+ - [Using Scikit-learn](#using-scikit-learn)
+ - [Code Example](#code-example)
+7. [Evaluation Metrics](#evaluation-metrics)
+8. [Conclusion](#conclusion)
+9. [References](#references)
+
+## Introduction
+
+Hierarchical Clustering is an unsupervised learning method used to group similar objects into clusters. Unlike other clustering techniques, hierarchical clustering does not require the number of clusters to be specified beforehand. It produces a tree-like structure called a dendrogram, which displays the arrangement of the clusters and their sub-clusters.
+
+## Concepts
+
+### Dendrogram
+
+A dendrogram is a tree-like diagram that records the sequences of merges or splits. It is a useful tool for visualizing the process of hierarchical clustering.
+
+### Distance Measure
+
+Distance measures are used to quantify the similarity or dissimilarity between data points. Common distance measures include Euclidean distance, Manhattan distance, and cosine similarity.
+
+### Linkage Criteria
+
+Linkage criteria determine how the distance between clusters is calculated. Different linkage criteria include single linkage, complete linkage, average linkage, and Ward's linkage.
+
+## Types of Hierarchical Clustering
+
+1. **Agglomerative Clustering (Bottom-Up Approach)**:
+ - Starts with each data point as a separate cluster.
+ - Repeatedly merges the closest pairs of clusters until only one cluster remains or a stopping criterion is met.
+
+2. **Divisive Clustering (Top-Down Approach)**:
+ - Starts with all data points in a single cluster.
+ - Repeatedly splits clusters into smaller clusters until each data point is its own cluster or a stopping criterion is met.
+
+## Steps in Hierarchical Clustering
+
+1. **Calculate Distance Matrix**: Compute the distance between each pair of data points.
+2. **Create Clusters**: Treat each data point as a single cluster.
+3. **Merge Closest Clusters**: Find the two clusters that are closest to each other and merge them into a single cluster.
+4. **Update Distance Matrix**: Update the distance matrix to reflect the distance between the new cluster and the remaining clusters.
+5. **Repeat**: Repeat steps 3 and 4 until all data points are merged into a single cluster or the desired number of clusters is achieved.
+
+## Linkage Criteria
+
+1. **Single Linkage (Minimum Linkage)**: The distance between two clusters is defined as the minimum distance between any single data point in the first cluster and any single data point in the second cluster.
+2. **Complete Linkage (Maximum Linkage)**: The distance between two clusters is defined as the maximum distance between any single data point in the first cluster and any single data point in the second cluster.
+3. **Average Linkage**: The distance between two clusters is defined as the average distance between all pairs of data points, one from each cluster.
+4. **Ward's Linkage**: The distance between two clusters is defined as the increase in the sum of squared deviations from the mean when the two clusters are merged.
+
+## Implementation
+
+### Using Scikit-learn
+
+Scikit-learn is a popular machine learning library in Python that provides tools for hierarchical clustering.
+
+### Code Example
+
+```python
+import numpy as np
+import pandas as pd
+import matplotlib.pyplot as plt
+from scipy.cluster.hierarchy import dendrogram, linkage
+from sklearn.cluster import AgglomerativeClustering
+from sklearn.preprocessing import StandardScaler
+
+# Load dataset
+data = pd.read_csv('path/to/your/dataset.csv')
+
+# Preprocess the data
+scaler = StandardScaler()
+data_scaled = scaler.fit_transform(data)
+
+# Perform hierarchical clustering
+Z = linkage(data_scaled, method='ward')
+
+# Plot the dendrogram
+plt.figure(figsize=(10, 7))
+dendrogram(Z)
+plt.title('Dendrogram')
+plt.xlabel('Data Points')
+plt.ylabel('Distance')
+plt.show()
+
+# Perform Agglomerative Clustering
+agg_clustering = AgglomerativeClustering(n_clusters=3, affinity='euclidean', linkage='ward')
+labels = agg_clustering.fit_predict(data_scaled)
+
+# Add cluster labels to the original data
+data['Cluster'] = labels
+print(data.head())
+```
+
+## Evaluation Metrics
+
+- **Silhouette Score**: Measures how similar a data point is to its own cluster compared to other clusters.
+- **Cophenetic Correlation Coefficient**: Measures how faithfully a dendrogram preserves the pairwise distances between the original data points.
+- **Dunn Index**: Ratio of the minimum inter-cluster distance to the maximum intra-cluster distance.
+
+## Conclusion
+
+Hierarchical clustering is a versatile and intuitive method for clustering data. It is particularly useful when the number of clusters is not known beforehand. By understanding the different linkage criteria and evaluation metrics, one can effectively apply hierarchical clustering to various types of data.
+
+## References
+
+- [Scikit-learn Documentation](https://scikit-learn.org/stable/modules/clustering.html#hierarchical-clustering)
+- [Wikipedia: Hierarchical Clustering](https://en.wikipedia.org/wiki/Hierarchical_clustering)
+- [Towards Data Science: Hierarchical Clustering Explained](https://towardsdatascience.com/hierarchical-clustering-explained-925d9e1600c1)
From 5b6cd66f79e2cc0341f94bd0ec3ee935ccf8a20c Mon Sep 17 00:00:00 2001
From: Ritesh
Date: Sun, 2 Jun 2024 09:21:45 +0530
Subject: [PATCH 13/76] Update index.md
---
contrib/machine-learning/index.md | 1 +
1 file changed, 1 insertion(+)
diff --git a/contrib/machine-learning/index.md b/contrib/machine-learning/index.md
index b6945cd..20305dc 100644
--- a/contrib/machine-learning/index.md
+++ b/contrib/machine-learning/index.md
@@ -15,4 +15,5 @@
- [Logistic Regression](logistic-regression.md)
- [Types_of_Cost_Functions](cost-functions.md)
- [Clustering](clustering.md)
+- [Hierarchical Clustering](hierarchical-clustering.md)
- [Grid Search](grid-search.md)
From 8ef6a2febc2ec303ffdc0702d838e2caa1462b17 Mon Sep 17 00:00:00 2001
From: Soubeer Koley
Date: Sun, 2 Jun 2024 09:27:46 +0530
Subject: [PATCH 14/76] added ensemble learning
---
contrib/machine-learning/ensemble_learning.md | 140 ++++++++++++++++++
contrib/machine-learning/index.md | 3 +-
2 files changed, 142 insertions(+), 1 deletion(-)
create mode 100644 contrib/machine-learning/ensemble_learning.md
diff --git a/contrib/machine-learning/ensemble_learning.md b/contrib/machine-learning/ensemble_learning.md
new file mode 100644
index 0000000..940bd09
--- /dev/null
+++ b/contrib/machine-learning/ensemble_learning.md
@@ -0,0 +1,140 @@
+# Ensemble Learning
+
+Ensemble Learning is a powerful machine learning paradigm that combines multiple models to achieve better performance than any individual model. The idea is to leverage the strengths of different models to improve overall accuracy, robustness, and generalization.
+
+
+
+## Introduction
+
+Ensemble Learning is a technique that combines the predictions from multiple machine learning models to make more accurate and robust predictions than a single model. It leverages the diversity of different models to reduce errors and improve performance.
+
+## Types of Ensemble Learning
+
+### Bagging
+
+Bagging, or Bootstrap Aggregating, involves training multiple versions of the same model on different subsets of the training data and averaging their predictions. The most common example of bagging is the `RandomForest` algorithm.
+
+### Boosting
+
+Boosting focuses on training models sequentially, where each new model corrects the errors made by the previous ones. This way, the ensemble learns from its mistakes, leading to improved performance. `AdaBoost` and `Gradient Boosting` are popular examples of boosting algorithms.
+
+### Stacking
+
+Stacking involves training multiple models (the base learners) and a meta-model that combines their predictions. The base learners are trained on the original dataset, while the meta-model is trained on the outputs of the base learners. This approach allows leveraging the strengths of different models.
+
+## Advantages and Disadvantages
+
+### Advantages
+
+- **Improved Accuracy**: Combines the strengths of multiple models.
+- **Robustness**: Reduces the risk of overfitting and model bias.
+- **Versatility**: Can be applied to various machine learning tasks, including classification and regression.
+
+### Disadvantages
+
+- **Complexity**: More complex than individual models, making interpretation harder.
+- **Computational Cost**: Requires more computational resources and training time.
+- **Implementation**: Can be challenging to implement and tune effectively.
+
+## Key Concepts
+
+- **Diversity**: The models in the ensemble should be diverse to benefit from their different strengths.
+- **Voting/Averaging**: For classification, majority voting is used to combine predictions. For regression, averaging is used.
+- **Weighting**: In some ensembles, models are weighted based on their accuracy or other metrics.
+
+## Code Examples
+
+### Bagging with Random Forest
+
+Below is an example of using Random Forest for classification on the Iris dataset.
+
+```python
+import numpy as np
+import pandas as pd
+from sklearn.datasets import load_iris
+from sklearn.ensemble import RandomForestClassifier
+from sklearn.model_selection import train_test_split
+from sklearn.metrics import accuracy_score, classification_report
+
+# Load dataset
+iris = load_iris()
+X, y = iris.data, iris.target
+
+# Split dataset
+X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)
+
+# Initialize Random Forest model
+clf = RandomForestClassifier(n_estimators=100, random_state=42)
+
+# Train the model
+clf.fit(X_train, y_train)
+
+# Make predictions
+y_pred = clf.predict(X_test)
+
+# Evaluate the model
+accuracy = accuracy_score(y_test, y_pred)
+print(f"Accuracy: {accuracy * 100:.2f}%")
+print("Classification Report:\n", classification_report(y_test, y_pred))
+```
+
+### Boosting with AdaBoost
+Below is an example of using AdaBoost for classification on the Iris dataset.
+
+```
+from sklearn.ensemble import AdaBoostClassifier
+from sklearn.tree import DecisionTreeClassifier
+
+# Initialize base model
+base_model = DecisionTreeClassifier(max_depth=1)
+
+# Initialize AdaBoost model
+ada_clf = AdaBoostClassifier(base_estimator=base_model, n_estimators=50, random_state=42)
+
+# Train the model
+ada_clf.fit(X_train, y_train)
+
+# Make predictions
+y_pred = ada_clf.predict(X_test)
+
+# Evaluate the model
+accuracy = accuracy_score(y_test, y_pred)
+print(f"Accuracy: {accuracy * 100:.2f}%")
+print("Classification Report:\n", classification_report(y_test, y_pred))
+```
+
+### Stacking with Multiple Models
+Below is an example of using stacking with multiple models for classification on the Iris dataset.
+
+```
+from sklearn.linear_model import LogisticRegression
+from sklearn.neighbors import KNeighborsClassifier
+from sklearn.svm import SVC
+from sklearn.ensemble import StackingClassifier
+
+# Define base models
+base_models = [
+ ('knn', KNeighborsClassifier(n_neighbors=5)),
+ ('svc', SVC(kernel='linear', probability=True))
+]
+
+# Define meta-model
+meta_model = LogisticRegression()
+
+# Initialize Stacking model
+stacking_clf = StackingClassifier(estimators=base_models, final_estimator=meta_model, cv=5)
+
+# Train the model
+stacking_clf.fit(X_train, y_train)
+
+# Make predictions
+y_pred = stacking_clf.predict(X_test)
+
+# Evaluate the model
+accuracy = accuracy_score(y_test, y_pred)
+print(f"Accuracy: {accuracy * 100:.2f}%")
+print("Classification Report:\n", classification_report(y_test, y_pred))
+```
+
+## Conclusion
+Ensemble Learning is a powerful technique that combines multiple models to improve overall performance. By leveraging the strengths of different models, it provides better accuracy, robustness, and generalization. However, it comes with increased complexity and computational cost. Understanding and implementing ensemble methods can significantly enhance machine learning solutions.
\ No newline at end of file
diff --git a/contrib/machine-learning/index.md b/contrib/machine-learning/index.md
index 073bca9..a46cd7a 100644
--- a/contrib/machine-learning/index.md
+++ b/contrib/machine-learning/index.md
@@ -9,4 +9,5 @@
- [TensorFlow.md](tensorFlow.md)
- [PyTorch.md](pytorch.md)
- [Types of optimizers](Types_of_optimizers.md)
-- [Random Forest](Random_Forest.md)
\ No newline at end of file
+- [Random Forest](Random_Forest.md)
+- [Ensemble Learning](ensemble_learning.md)
\ No newline at end of file
From 402289d3ceae6add26458f41d33364eff322f8e2 Mon Sep 17 00:00:00 2001
From: SAM <60264918+SAM-DEV007@users.noreply.github.com>
Date: Sun, 2 Jun 2024 10:23:09 +0530
Subject: [PATCH 15/76] Rename Transformers.md to transformers.md
---
contrib/machine-learning/{Transformers.md => transformers.md} | 0
1 file changed, 0 insertions(+), 0 deletions(-)
rename contrib/machine-learning/{Transformers.md => transformers.md} (100%)
diff --git a/contrib/machine-learning/Transformers.md b/contrib/machine-learning/transformers.md
similarity index 100%
rename from contrib/machine-learning/Transformers.md
rename to contrib/machine-learning/transformers.md
From e13fb234f2829a1b918aac3c044e4feef5066d34 Mon Sep 17 00:00:00 2001
From: SAM <60264918+SAM-DEV007@users.noreply.github.com>
Date: Sun, 2 Jun 2024 10:23:56 +0530
Subject: [PATCH 16/76] Add transformers.md
---
contrib/machine-learning/index.md | 1 +
1 file changed, 1 insertion(+)
diff --git a/contrib/machine-learning/index.md b/contrib/machine-learning/index.md
index b6945cd..13d9440 100644
--- a/contrib/machine-learning/index.md
+++ b/contrib/machine-learning/index.md
@@ -16,3 +16,4 @@
- [Types_of_Cost_Functions](cost-functions.md)
- [Clustering](clustering.md)
- [Grid Search](grid-search.md)
+- [Transformers](transformers.md)
From 229458043245de432c96f05457dd79330d568a92 Mon Sep 17 00:00:00 2001
From: SAM <60264918+SAM-DEV007@users.noreply.github.com>
Date: Sun, 2 Jun 2024 10:39:37 +0530
Subject: [PATCH 17/76] Update transformers.md
Added source for the image
---
contrib/machine-learning/transformers.md | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/contrib/machine-learning/transformers.md b/contrib/machine-learning/transformers.md
index 97346bd..9fbfa6d 100644
--- a/contrib/machine-learning/transformers.md
+++ b/contrib/machine-learning/transformers.md
@@ -6,7 +6,10 @@ mechanism. Before transformers, predecessors of attention mechanism were added t
Transformers are a revolutionary approach to natural language processing (NLP). Unlike older models, they excel at understanding long-range connections between words. This "attention" mechanism lets them grasp the context of a sentence, making them powerful for tasks like machine translation, text summarization, and question answering. Introduced in 2017, transformers are now the backbone of many large language models, including tools you might use every day. Their ability to handle complex relationships in language is fueling advancements in AI across various fields.
## Model Architecture
-
+
+ 
+Source: Attention Is All You Need - Figure 1
+
### Encoder
The encoder is composed of a stack of identical layers. Each layer has two sub-layers. The first is a multi-head self-attention mechanism, and the second is a simple, positionwise fully connected feed-forward network. Each encoder consists of two major components: a self-attention mechanism and a feed-forward neural network. The self-attention mechanism accepts input encodings from the previous encoder and weights their relevance to each other to generate output encodings. The feed-forward neural network further processes each output encoding individually. These output encodings are then passed to the next encoder as its input, as well as to the decoders.
From 07a1e737606d10d7e96e2291cec21003309bfbf0 Mon Sep 17 00:00:00 2001
From: SAM <60264918+SAM-DEV007@users.noreply.github.com>
Date: Sun, 2 Jun 2024 10:46:21 +0530
Subject: [PATCH 18/76] Update transformers.md
Added bibliography
---
contrib/machine-learning/transformers.md | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/contrib/machine-learning/transformers.md b/contrib/machine-learning/transformers.md
index 9fbfa6d..0799b7f 100644
--- a/contrib/machine-learning/transformers.md
+++ b/contrib/machine-learning/transformers.md
@@ -430,4 +430,11 @@ These may include:
- Text generation
- Biological sequence analysis
- Computer code generation
-- Video analysis
+
+## Bibliography
+- [Attention Is All You Need](https://arxiv.org/pdf/1706.03762)
+- [Tensorflow Tutorial](https://www.tensorflow.org/text/tutorials/transformer)
+- [Tensorflow Models Docs](https://www.tensorflow.org/api_docs/python/tfm/nlp/layers)
+- [Wikipedia](https://en.wikipedia.org/wiki/Transformer_(deep_learning_architecture))
+- [HuggingFace](https://huggingface.co/docs/transformers/en/index)
+- [PyTorch](https://pytorch.org/docs/stable/generated/torch.nn.Transformer.html)
From 643cadfa330d4fa5443ba1f4044ecc5942e2b7f3 Mon Sep 17 00:00:00 2001
From: manishh12
Date: Sun, 2 Jun 2024 11:54:06 +0530
Subject: [PATCH 19/76] Added Generators under Advance python
---
contrib/advanced-python/Generators.md | 87 +++++++++++++++++++++++++++
contrib/advanced-python/index.md | 1 +
2 files changed, 88 insertions(+)
create mode 100644 contrib/advanced-python/Generators.md
diff --git a/contrib/advanced-python/Generators.md b/contrib/advanced-python/Generators.md
new file mode 100644
index 0000000..ff5226e
--- /dev/null
+++ b/contrib/advanced-python/Generators.md
@@ -0,0 +1,87 @@
+# Generators Under Advanced Python
+
+## Introduction
+
+Generators in Python are a sophisticated feature that enables the creation of iterators without the need to construct a full list in memory. They allow you to generate values on-the-fly, which is particularly beneficial for working with large datasets or infinite sequences. In this README, we'll explore generators in depth, covering their types, mathematical formulation, advantages, disadvantages, and implementation examples.
+
+## Function Generators
+
+Function generators are created using the `yield` keyword within a function. When invoked, a function generator returns a generator iterator, allowing you to iterate over the values generated by the function.
+
+### Mathematical Formulation
+
+Function generators can be represented mathematically using set-builder notation. The general form is:
+
+```
+{expression | variable in iterable, condition}
+```
+
+Where:
+- `expression` is the expression to generate values.
+- `variable` is the variable used in the expression.
+- `iterable` is the sequence of values to iterate over.
+- `condition` is an optional condition that filters the values.
+
+### Advantages of Function Generators
+
+1. **Memory Efficiency**: Function generators produce values lazily, meaning they generate values only when needed, saving memory compared to constructing an entire sequence upfront.
+
+2. **Lazy Evaluation**: Values are generated on-the-fly as they are consumed, leading to improved performance and reduced overhead, especially when dealing with large datasets.
+
+3. **Infinite Sequences**: Function generators can represent infinite sequences, such as the Fibonacci sequence, allowing you to work with data streams of arbitrary length without consuming excessive memory.
+
+### Disadvantages of Function Generators
+
+1. **Single Iteration**: Once a function generator is exhausted, it cannot be reused. If you need to iterate over the sequence again, you'll have to create a new generator.
+
+2. **Limited Random Access**: Function generators do not support random access like lists. They only allow sequential access, which might be a limitation depending on the use case.
+
+### Implementation Example
+
+```python
+def fibonacci():
+ a, b = 0, 1
+ while True:
+ yield a
+ a, b = b, a + b
+
+# Usage
+fib_gen = fibonacci()
+for _ in range(10):
+ print(next(fib_gen))
+```
+
+## Generator Expressions
+
+Generator expressions are similar to list comprehensions but return a generator object instead of a list. They offer a concise way to create generators without the need for a separate function.
+
+### Mathematical Formulation
+
+Generator expressions can also be represented mathematically using set-builder notation. The general form is the same as for function generators.
+
+### Advantages of Generator Expressions
+
+1. **Memory Efficiency**: Generator expressions produce values lazily, similar to function generators, resulting in memory savings.
+
+2. **Lazy Evaluation**: Values are generated on-the-fly as they are consumed, providing improved performance and reduced overhead.
+
+### Disadvantages of Generator Expressions
+
+1. **Single Iteration**: Like function generators, once a generator expression is exhausted, it cannot be reused.
+
+2. **Limited Random Access**: Generator expressions, similar to function generators, do not support random access.
+
+### Implementation Example
+
+```python
+# Generate squares of numbers from 0 to 9
+square_gen = (x**2 for x in range(10))
+
+# Usage
+for num in square_gen:
+ print(num)
+```
+
+## Conclusion
+
+Generators offer a powerful mechanism for creating iterators efficiently in Python. By understanding the differences between function generators and generator expressions, along with their mathematical formulation, advantages, and disadvantages, you can leverage them effectively in various scenarios. Whether you're dealing with large datasets or need to work with infinite sequences, generators provide a memory-efficient solution with lazy evaluation capabilities, contributing to more elegant and scalable code.
\ No newline at end of file
diff --git a/contrib/advanced-python/index.md b/contrib/advanced-python/index.md
index fa2fd7b..df4b545 100644
--- a/contrib/advanced-python/index.md
+++ b/contrib/advanced-python/index.md
@@ -9,3 +9,4 @@
- [Map Function](map-function.md)
- [Protocols](protocols.md)
- [Exception Handling in Python](exception-handling.md)
+- [Generators](Generators.md)
From 3e35783d59919de603e6d11e81df54fdf47662b2 Mon Sep 17 00:00:00 2001
From: SAM <8dmasters@gmail.com>
Date: Sun, 2 Jun 2024 16:19:11 +0530
Subject: [PATCH 20/76] Create transformer-architecture.png
---
.../assets/transformer-architecture.png | Bin 0 -> 57018 bytes
1 file changed, 0 insertions(+), 0 deletions(-)
create mode 100644 contrib/machine-learning/assets/transformer-architecture.png
diff --git a/contrib/machine-learning/assets/transformer-architecture.png b/contrib/machine-learning/assets/transformer-architecture.png
new file mode 100644
index 0000000000000000000000000000000000000000..2854ab0a8d3e6f0ba0c53735afb57e81eb825a3f
GIT binary patch
literal 57018
zcmdqIRZv|)*DV?%xVsbF-95qG-7UDgLy(QT2A5#L-66OJx1hn@-S66bztpMobgS;u
zO;rk5d+jB&yXPEpjL~5Va^eWExUlcuy+e?c5K(&f4&vzDyZ3_7kie08^8;_-2ZWQ7
zxX`=s${TdPs6MPR7eE!rEr69}OZ{1Y}tiUgkxW5DbfIqW*2gd=N
z(NhdTxA1lXWhc^tL!`_Cq5f
z4fKQ{IULT>pEp@dQ$!N-;}*$hC&xUo`uh4ZadM_e!vuDA3aXW95T{^^u^V=oVpB~1
zZLq=PaXkwC{sBrzSy_2zQo%hMpWA+;7bcQOP!HO{8wLjEtAT+*2AwNN5z8GBe7|5@
z=4Pke{*e^M-12hzd9iP=FV99AE$Wqeds77w*gW&Ivq{5O^Hc{_XAj
zJkERm)XIhbUY;L~xE^ngm-0S*!E-?#gk{vKD|Yvtuh6YF9*CyeZFAfXM|S7u=eJ&N
zLYc4B7cY{_Wb(K$F&R#j6bLs|foiD20ZFP1I&!eFMXHvCJzO7d9{sH~%I>{Y=M4yA
zXJbQG{iV|NOSLrU9iv9omr(k8*;FQo=|Wkwp2JLDxBe34qKQ^FyN@3~hVJ^^Uu?(h
z{)~Q~nb4P}sH*XHY4g>zpli1YCho9<&Z$N_{h!h)atsg_Q$M?a`#7w3e`*F9&JbZ*
zgA7qI5D*Z&pY9YH`XVbU8ClH6=u$T1`ob}`+p`lH^$CHON|1)ne*sV7gyNs_*@UO7
z?Ge~n(P7dl47oN-4UW`2CIzpEj%<%fRaH#Py99WCJd{1fxZt@@Mze*`Vf~k-x^HV!
zO8OTjgHGtqpY({D@QVm_prmgy-;JXtG(~};xXYq8_$ooP44G;I`Otz
zug_3WP$5+BkB^S%K5D_VLzb6TXg5|+-5&i#o12?6%09cfiFTE{V2LRa?qd4&p7GEK
zn?f>28v(&*JeB#wcosj}ds)I23JQwBH1<;OM3TYbpOOh@8+~x-)Jh_w^7EzY@}c(C
zMt!{==3#S}yW`9*2P(v?CI!q`;F}t3go{<2b
z-D;a7LeEbb@bDU9iy{0Yb^v~Y6hHhQ2Yg3PZBfwCxAhgXIGM*$n)w@^^(8|OjZ%C4
z>4CBK9hokY7rlsf+#VLSCYMeYDN!oez@U$u<5RXz@Cy#Acix}sUZ>*_@OcRO8VDKY
zBh@Tf!nN=5Z8#CdsupG#QS7@);zY|CFr`PxL5c+6d*73yJ8yLbLZK`rP|AzEn4|VI
z+N}+Xmk^?JeVZv(blj&*q)lh>M6tECb$`06UZ8xq+7G1DtQq|83xu(<>Bvm}rh;sW
zii)bzN;>Zx+9>G7lNt;l|6W98ELFF|A~
zSuFm*WWB&tT^W?6U7|!zT)`$H$6d>)-$7>-p}aKugOwE2D>gwu15J|N*P(hMRKN$&tq+8b1)bx}JBv_;&$Ecd-*4Ed-Ox|0h
zp@S!2SQ!ikWG1kfL)fWf&Gu37BkhQ*+{GTxsmaEy3A1p7K!D2rFk=z{1AkInSu
zNLuJ|bG6-T!y|t7nUs!BW@fG3>ox=!y#>lbxy*;2o*v;N`5!FX1IJ5^eF9&{GdXM)
zV}LyV0u2xUMC?Mbh51QId^=>OBvu+ep{gnv7zrX~R@OkWK}C+5$e}zACMM*8*d$p=
zUaCusYVZ3C^UDEe>%TSMNO#(SOpI;Q;Xazm%;kQTvp~e}>D(LJ;M|ESd<;YtJ(-ml
z#ym**M|=ji6D(!TomPvCi{P1mYW=e8K^&xB;rrOqEmey#>B+TV_<1$XfU7sm9$Wlv5?8*
z5>QxJh^&BmzBv#R!4c5=0Ms}~Nrs9LB~Kuv^IB!!?b-%A5E51|$r4_bKxGiz_5
zI^Vp=@W$)*B#eW{bhAHdFpHlTpMapRjal=+YOV~WyMoVjqc^PE=}A38ki%RflLw`a
zEpjY3C`*1f7g5;2Ajdqv`8-!RB+uh=CvTeBW|56q>Wj^c$3nXor{~okarSpZR7^zK
zVWv?#Vq&k?7k*ewI;wd9Y$OCFB}c@c}Y4*E(RQ
zl{!x=43)f>+dMUk&JHL-DggfJw(N8{#C&>wR_yfkS)WO!l+OySU%29H5op0|fpp}_
z|7Q7--wY_K_HJ$=Kw;c7k4;!!bH&7_VE7-{D7^8%I5IzZqK&2JCj~gvioE7y8FQwW
zk%S3p;A$y$oKv8R{+J%`$RuO=9Y9vt^~NX5&E3O=XGQP{S_X_j(U^gS#rg$QdQguA
zH3mIYzKkw>!jvH#_}qXw5!z#T*hEzB3!I*xCJWub6>V~u3eoo?-zUdvBm`$N>IlYS-EzjoQPchv0K~%HU;@_@FN!^pc^GBInwNwQ75}6
zP{@+RhcU@Wfe{~eJ#mAf9U-Eb(ue=|82P(0HYm*B77ED5UDt>6=j%OC&1%$}ikAQSDSpQ3WEQ7nR#UXsGpXodpd`pNGSg{yF`n(Wk7^rfTq9Y?<$^EJ
ziX*yvc^N;Ih3zqyQAW7D|8ja*NRC(SGHa?*qV(zJ=H{t`C{32LR+Lpy92jG8g>ncq^*bpc(yg1VK{A74r&??LNP5CDGN+>-?R|_TjW+p>%s);ab@*P#nrr!}F=GVvS
zd5uB^sFZg$IY
zW0$%WhS;;7%7OtoD#=v7Wgo
zg*+XY$ZTE+MxCr4dDO|9b{-P(lZ?HIBD6FYbIANWG4ttqdZ}bu@~m_LV^3#-WrV&F
zbn!LC=kvuCG=9+am&vKV31f44J~v;6XeX~5RAbX<;UU-pg8>sG-2vmr__i)mVW&y5Wj`V6F{#9jXIg35}$xx@Sq2ch`g}zY4GNs-ON=-G8!@`O~*(
z?zNQ8_00n{w=36tuA19%=T8`fF_cl*5J^x-$jG=UF!zNMSl~%PIWEr5y^g<(Ips;j
zSMeRCyT45SIazConLi;M{8192$k*uv*oDb*yxQZg
ze70hNn(Ee2wm4Tqfn++Q$Hg{AvcT6yVXcFt5vE4$4tZ$3<_@7E&*M01Ws1AII}yBo
zv((PF>`SZypcjk)7N@_~b;p0@Yz;R)v{ycI9uEvRk0`XcI{;vmY|g_KbdLru2+){}
zdho`^#^zR5`Kl=#M!IZhf3gaQQsc4fE=?yPv-o6BBckWS^Xw5lQE;Q7F@3
zCWJNHKmea>3ANI|<90$P_CW*s$IScx9NJE%D4ktdA@I1^inGF@_|}6)0~9xcEqrmO
z6I!*hSSxIb?H)|vK=!+{^}vOqZ%V;%SfJWkR^Du8YQnby3mS+af)lR=;JyJ+&onx`xgB@MqK1=b
z^8w6vp$MPAvJUP9XF&1f(s|!Tk|(C8(YEoq9doSaE2tE5{Ha*~zbd%@mKyDvyzgtE
zUnLC%RWV>QgdkL?hBbe%eB08RCG~l@+M6xKvYaU<&Z}XT@qVkR$b^K^{ZV*5b8~7H
zhN$ngNaeb8feJd1_oo*Q{B|D!w3*89)nI~Q`sK7EFpe@Nce(qC78t+YlgxA!RMbe9
z!#Ts=P!vbFp58aW@ud+$O7s5uP`ym69fMxmEm1r5?#pRuK_A0t28SN}T5C&-qYB4P
z_WC){CX4LQrq#-&bI{YM{GyqsU?39o{e5Z1;7Cm~HWq^nbciW4Sg5E9YFh2P&g7fVhfrgQ$*uMRvSmfv0EWB`|+C?aF*CYtI=i0tdml9
zU;LJXd3RH_Os
zThl$HtV2Eb5r&nVd8jyrTIh3vaM`{OpQC&B0w)WLRI-(U5+azOCFOECK-a3bBnRNr
zefZ!H+8FSf%OX5+znV~NE5c&X`O*VI*v15f@xEo^J+&(LbEDA=Ze@{=#;A}rPXMbB
z0T4d5!T+_2^Uk(Mu$G&h#J6j|{aswl(`)wxlB;C406&ddStvlA46?&to`OqezUzdJ
z7tl7@pRV$|ZWt*cJjt2?Jxf(%BbfF*-x@*#6JP>wON>CdNDjm4Bm$E@{w=+}^%EGU
zYn@*br@*Yg4D$+Js|3$5q%TTgESt`=l9R(Vcx8rycISBCZu^cv9
z>wmIOnwkg?o}fTR_+P4j8V4vyEf@;$=Q%k!(2Ulq
z-&Wf^z=?Zwbd<@c7rL3_ItSpK3qNsjcpQL}hY~aT#^wHWk&(qrGy(6~&9S!PKrsi|
zOd@Cjw-6Km%OhAkihI8si*&^F)
zmNAe)0FVb1V@*R$WFT+R9VVPSF6=xUGI
z=-APEqB^nTqN^w5E~i`gS27g6q1Qc28oS;1hX1spH%EUJJG}47+}`uLz3{a-pM0dJ
zr;qbZ6l!8QKmrHMUc%pLdzjpIF@}&|Y_3dmJGpq`!Rm5$N2AR=4qNFPDkOiq3@`$a
zAVh+I*Z9H57oZxOPv&vkE;oG#rf#0o?wBK7{ur>zWMItz`U0EF{xcwokOk!Ta@Ya%
zoNXj7yOQ2P5&}@KW3^NL`UVEDR#%CF6;79%;Q*d4MH`jqfloXd|GQWe?s}VvLv3fk
zLl3|mQD*?K>i6Oz?%}+h2*6ToEf!DM8=s3`ZiBLcg@W{dqEYT4OHe!7g^lBJl}qBAriPVy7*ZSs1v4et{eN)^FY~`&uSBT
z^}U=dKa|Pf-~9Yil-lGpfKjd`=bx<~tCRnk?g;PUHZ$bx%+67EDxO%F=lrgydNhde
zh%BTQejj5RmuxZeEhr^HL4@Jt->a*dpk*boZMnHd>Y0U&0H#>G89j5pvK<85r#nM>^WK3XRhX4~Z)vtT@njO$(LYa+Q4R24T9n0UN=7k{X87(y!H@S?Er+qp(
z#Cumt+Q8+!hwAP9w6(YwdlJ83Hk1dT`uYM{b=2&I+OT?aML0b@BKz|V-DV0NXZl9F
zR)eKE<0jW*#h~ry`vk$0RhPb2x9r~fPmmyqM*;WCekiZotl&R=yc_lH?Sh-Xzr%64
zT!3bV?l?!1jJ{`cd7uJPln@}s^m?g$gGbQm_C{k)B7
zFtp*b&@>6FS$nW~fFVw&@A}0Sf{(9lvS8YINJ(vUS1zpy*RIXG_oY%{uIhW>Qnz&%
zLz$KbG{8)X70QGY@1*8zs+ZwMcoZ)6GCftsYRI$xg^qZ}>aG8~u-ugc0f&N%%Mf;H
zR;F$B32ig#j!_o3gmgV-y(igt8y3z)qB0PTvcm9cFG-{2?wzv>yTfS~ZGV(5&Y!eb
zf~>48jXHM}8r5H+ge$GnSx1*Vb@
zVD!2Hk08j)+brF!tP*Mxfyr)Q^Nn(Gd3jY5v#gekWfS1c31m_i5JbbbP;7ehL|d7?
zP7JP&=Y?vR@C=gfxrVg-$Q#9idzk`FRI|CAbWvMxQ<+K*iBc$mWQJqh@7y;-ZI!Qi
zbHkZ=wx7V5T#g;gVRL9OSLREl!^Id)&`z_MnZ-BofPgkxL9m@77wBduXz=|na-EGj
z)+H8v-?nrkvyB*0osBjYtLaV8rI}q`yO{OFnLyv80J-K|rJm@spX8vfPDZ|soP3r<
zo-gKf(T^OR=9*MyW01)#)X2Dz5+1k|_x?zrwEO+p$y4(8$-iz7D3h^xlvdYF^t`LR
zb*GtfZ3v5Ln&jZP-y9ZNvPWoHghUFtLMY85-$kc#wXUZC0f=nw@mBYDUqqw*v@c#;
zyC6(geWM#Fusy(GX+AhJvpb>#JoD0bnY%FWIFb44TeKk9!Q<`+rz3l9o8WL1Q93G
z+h0s7JHDK3K%P6LLEsa#+4yzg=lq>^d;&Bpn9
zcTRU1VKONQ`P{>*&g>0c#3>wq;OCgMNB*1Ngt<6bZQiUi`HP5hIbS+2gFJU1Z9|Ft
za(pXEZ!S5TCrVo7dQ9AQ^YnRMY8M_MW>11@GBHTQf;@prd}sbp3VMJa)e{%Bh-)o|
zh}ZCZgZ%8`Lg?lFCz0;xL2ZCkX7awX*J-kc@VYrdgAuS@>mU#F87I>v=`V&nf4yOb
zPxxi;dJ|;JLlB8Kk3cHs8okgTeN4PlFOAL^s@<3+fX5cXQ7LUS9%};}
zg)GxR)M1SM>kF%-!aT9tNk$$$i^;KCrJ|Ijenbe8zQk@*;P%LE*Mj@ILOD$!GcUCF
zC9zp3!GP{-de-y}qc|*8AoMB)VRu=~C^?Uoyda9?bcM(DWJSFkzbX#CP%wJ@m@4Q)
z=B~C~;X$FylH|8HR{}|PO`NeKQWRucRCKvHs(mBdDi1ic?gw$2boq~8KgQEl1iD13cmPZUU{?*
z=C0LRhd{^c9SB%Q;rB;_6Y;dD2Z!?~#9lXadvonDHB!l!QFt*5O;x!I>KE%hSZyAc
zDBngf4zA+O7r~-GlAR|Vt1M@wLRlR8P6RCHsqxA$QE}AO8C5D4K0G`?+B-Tv&>FrF
zTxHGtQbm)`;_LsVxmfE
z(=?87X!?jm4SoK;wQ;24a(uo|8X}(~7){~+R85S+10?`uNY^Ov6Yn>0`ln9q#IO$F
zOO?;yV$f{)%RJkZoyuyuG4ObsUF|-JIP~xu1c=MwhDSmE@y!6|q+N4U3KvqhV9+I}
z`G#F67y@J`8`3yzU>O-_;Wtah&}me#i{!G!Dl#T_4_-fMrOQI7(y0CVxHQt>XydF=
zrHuLZ;9(I4Xka~n#>q*ix)yC*QMKdF_p5)WAdAVMenTQ?6aC+KPxfQ^`30i?A-J)bvjpiObIEoaF!pA*y`lf4?6ACLzdz
zV!G?oDyn&nzl7*d}9UvjMRDFy{riq~A
zRt5!XIc^+phZV^ahTS?}AF%JuS7tAx3jWW_07F=~8J2Ta
zJP@2+O$PeTIaa;egw7o?@~v^MuD3}y#?#dq|Jm-Yh#Ud
z_kKHMa=X+^J>i00N>Fp(yb*%;%ND4DtK~TEJCgNc^4-^3Zgbblh%ubO0HXwzfI3{<
ziK)r31-AOS-~wwqC&)fpwJz!10r&MQ=>2Pie-!*JtDO<}9+&I2ShOzu1-lm^CHEV{w;PbQI0$4M_Qk&s%
zkv95S^-z!x*S{x^@kl_!cDg(vW=I4M)&mqrT0BS`bIC@!++SnN5RZg%Uey&T
zju^90EI6q*{=UpgGq?Mxr^@5S%rj<(-HflE+Q0nMTki0N7@>?3QSk2!4QjAhbsi;L
zMk~6p#cHm{5mWt-kuLX6hS^4sG28h3F4ugr9xGDaCIAGYz>Fdw`*@!@KzpReLG++m
zo3~UPR>f-@EPuKde(B?R3z`1Mbp7R;Ht&g^5B_nU4o$N|O%4YkAGLtVX$7%vZ%#YZ
zNyD5<==}!=$2CN*mFBRnX;~CZRQPE_AB9#;+WmS?(m*{5{FP=&VAcFr@G>I?5RDyU
z8c>!t9E8t=QNNXzA5QpP$e>_lC>b;y@gwuJ?4KQV>7KWdY}4e)l_JI?ABNxkGa*1j
zjKAhL*r*G=pxvdUQfk4UDxCdt4x6Mw30KC~B=5W6%C4a|Y5a>8B{(-}{0<$;k_p
z5NdhZ^t=+5oe$oLB0TUdI9IWa_KHod0$rElw~Bd6s^0FGTBF+Y=2NK8MWaHM=M;k$
zsrnA@!v=SyQv~<}0^e`P2TT~uYFqX*NrAJTBhaU4AG0Z>95b
zR(VPluKXHhn&DHkmSK+dVt?$7#yLzzGS%VJTbud5e%~OEWA*z=Jho+ib5K&tb)!D4
zRv9HdfCv+j`8lMPwkRgd2c#(ab_))zTb!Vjv6yP3VRH>zrh%icD-$*1ekNma$^s-b
z?6{4;<5rdwbWtty49=CR8tcE#g2Ek3hG*B^{~#`z2zSBT
z0PEd6hS-Rx!BC0mSFBzFoq7u*f~;&a1FH}A#%`@Fxbbtg}Ff*7}`OSl}EoXi26L_WM|GY
zCe;)YqnFqiU;B6vVj1!YU!)T#zV@XVqN0NH$bXys!u(AK&8gBPxrBs;{zr-UA})^i
zgVn4H5Yz4N@8Y1xEhu~^9?gw?6pjEZnXG5FORqrB6Woz=KNy<1@Q?x)Ola2HPez1|
z*0Z=1tb-(tOs~T!F{|$OhlTdG+I8m4V$t~2jlD>&JktM*SWr;|6!XQpx;OlFddNO*
zeFhAGestOm@M)b69T9MJ3nkzT4!|PODj1%ZxpM8QR-=&04nbDK5|T>&PO&ULkF9vi
zcz}Lovz&nj+S`bYzl8e#gIL1A@ayH}#eA~n(^xhyl8ucGK$89r$mB6BVg0S?OyEH3(Ti%sdm8esqRs$(sId^;Krg+%!8B$L$m9HiIo6>$-p+hbtbfP8zphP6HO
z#onH&th}txHzfNbZ-7{;S=Aug$Lf22{iB!T@ubP7>qY;0&oM#iR=77{)_
zoeEVubKeGuI1-XH^)2sl2jT)qxgUJrSU7AKKJ&}aqCZf!T6RE4}>O{=Vi*?c+b8_{4e0*dwc`yJwb8xcS
za$A6Mivf7X07mWvD`mnLU*GPoqtPGJfR5E(e7IPTr&g)=L3c&a?8WFmNZu00#|A7E
z08X-q#8?vxh0&cfcB_}k0|hMe8H3xCm9w)m6~U^#)z<*PYpX9-Ixo(WX57B-``zx!gejArVCb#?$aI
zXb+JTDn%iIjoY8jU4U)nGvHt(n=j;}l52@mBnD^-b*{NS_9ZMh*I|HkSLC&}vFVGF
z5ElNxX}kOdYI3kHTU-t)
zZUDDr22F2TPV*6<*yf4H5E>PG1O7Tdr(`Pm1PBlH-TnPyqpB%@_dPl~8s*SI87^DE
zM+oqx2W`u!v3kV6M*G!Q2@47JI;kdEOcxm*CevdJ8j{FFMr~`&Zg#R+Y%))Dd-3Og^=!omSE}Sl`+z4uB6SL2
zhvT)Cu?dkkLUx{t%_C)u=WC2vT5y)&TQ+1`?I1
zWgqMy50#R-f_=q10sP@oNi;Ch?){NCjy9t}Rsj652--^{RH`k2u#Xc3G{-~qqn#Tk
zFOUFd+Z%yJXg*ztaC5Sf@^hN&zlPn$piAwG(=FIJ-BaeYGa6@Qykqw%9aU;;C_x*Z
zHJiT^akexCtp7T`EE6~%aG?PHg1!Vwx$Y6clK-@mr4U;D*VotVH~WqEaX%=kd-}FGFFH!A
ztaiRedbwpzmU&!LsnNAVVU4W`ZBnSdaQ9kyyFXkZ(5UV)JlnnX=K1*sLfvn%R8!oe
z5Ejk`RPiR~{di<6F`wLY=TfzDk=PVK-tV{4P)Yi@)a(9XEUP)}5^L_W6$gJa2anyF
zZ+GhCh{N95d!Xn6418DBdZt35%-3cyh?OkUv1OWe{V&l+4^2k5?;a6Hzb@8I@(Q{t
zX)n!f+Mk`F0&I$A#i#D>j$Gdrl&ba`f&N``4!Q-fMY=oy$e`$eEi|4kWncp&x;2s_
z-x9WyR2K2`vAPtOOn_|w7Za0$&o05y{9HeG@*OPw!$W-$Re}PrkwVwq8Ql52`0)mn
z+FsapYu^6X)#QWmqH#k~v!9j#Up?U7pK@!74qZ-;tuiR0)2LE~exRcIFb=3QBHl+w
z+>W8o44JK;Q=Ic{CRRgi!YZ$71MV
z(f9>m0yST1C`UeT%ApBp_gnjP{=@lT`5nPxx`f)za44}EmYLgVb|c&ignk@#0z`|e
zRnYNLG)&ja%`vYUouL2WJkeKa=?~k(7+t06dh!3{CBcj@w|22YQ;r3K;Ai#qc85fhz<-POIH4QH|OC80Qop~TyFJs>cyL%zzpL|I;}`BFY#2qMzTISWO&xMcWq)TC@VV%@J6{3jnd&{P5d}wA
zA-OXk4TU7lq5uv=RXTNs^I7l)1LmMAd2+Wr`ds2PGtAOa;zkQ?n^pG2@&Ul{?9<4nf<2LTESYf|iT
zc5|Szf7v6@z{i=I!#aZsrlj1Q)b*Gr!rvcRLz6KmxPY2QjSe)Daxz@5m^(I#zPUPe
zYJgGr!@8H|(t$bL#e~PXIEwAHpewG&)$xyfTf0wz(+}f6Es|WG);J9|*&tQg+nzqr
zID_rtdIErJ`>xws1TZA8z+j^|pH5;YhhQ4a9>DCa`tVrJG63sV#?ZqaBAMu)IRmf<%v8|CpN1&w&A%BA1|2?*V#akS^R*_-Byk$|E_Dq|DUx2%2-w0yDs1_DkD_&&YXQ0>@UGMI
z${Mu7D<9s_>ZYSut)z&JvyE!Q!=r`G_Y(+*9DZjwvoSB+Vag&NXGxGzAKYNMqQ~a7
zBI?2{HcMTg$vq@#z8;VFJ?iH$9e-~T=RqlUZ8rE)b?txGhe&|XV`Zg%bubIh?YI{)
zKQ}$yr!ge8+IX8`&ZydwL-w;kB%LaS2DmJ|=-&w@yhzv`;d;wHQEufz=R4xC>or8B
zm%DR<*4rb@d=kW)8{m4wo4(Hv*XB!o5m;3o(-eT=6_~b3XasGE@u!(!!Jp>cA4A2D
zj4d!gCDCumy$WwKWTHkwO;jWmH@w&y+Av+K`6gXmCY8C+y32-7kU7z@LRL+$-Gau!
z(fDt3lQL{e8++#uF;U59x0BHeLXSr%$;`>@7pN=U@uWkuR@54uff(&z%~~jfg{lT@
zrYiQxqKc5AAmk<^bq-y%4ZRE#-hIQau78WrvU#Nzt#g8Xga!LBbY2XQfUy+_nP4WJ`o1mO>I&{n*z66x$B%$NG3(tFME
zqH5ZG9w7mbd7nSThakI;2@gPdfhZo3Qq*V-THP9gTE_CGV7nL!1M#3Y+M
zOPQNDk;!z$=j>o?W77W$E6}G^KN7z_C=gjWHJ|z!j{Gl^cYKijOBUbX^MhI5=AZxi
zF6ZOx%V<+ji5!!U08c!&YOC)vq*x?A43?Q0u=zu{j4ZV6`slB+iFvEbA;;(_rjWL0
zMSVR6AO+KSlcvsmf^%g-16=vGb9G`V8KLeKJl9|5W|x}_jk1$ZOGXgxw!-ktXNpHJT_4P4NlZYLIS(NS^(EjcFKve5U8&H!E}huE!V3`r{ss(hVdv7t1A-)l@nV`fXDVR8Ar
zZYu}o_Rk=?>y5q$
z-cl97!O*96X+=+HL;K`%jc|@HxXd;A&N4IW#+Ki
z_9pY0ZKX+g0cp&Tdz^EHxXjsxKo*@L1B=_qR5ZUoDdycBpK11qN`1IG3vINM%C>^F9DWd@@@V7q*-P{>C9{R2n_}T%7Oxo;W-0>aH=~)*
zZ6?x-mzjZ*2^2&1
zl5hn|IxWs68jd0;vnn{-^QyB&2M7kf!bJ%BB0XcF{P|(`jR~hRHUGS$HQGwn>fLdHJ_~E%L!&@26C0x
z%C*2XU2P$!%R;m?4nhWj5K8%sf>W_4&>FQjU0=K#;0TaS=ZHA6sqO;s%s=9D6OfQK
zD!CMe{E_MU0;?|{;OZosgR+uR%WY^#9=7G-Su~0$kyYiOr3
z$yB*_M*yEsO_=ZjH4fOH1V@}MXa|UgouRo$ch)9x@jY1l{WD^R`4$l+{1=6M@){a5
zn=e;wYc|79{k!;nwe238AEukba}PJjrM{?hzI6HSaY{+Q52IDD2ooA0PY<_=%O(u2blkb7Oo$*%2l+1M28#DXUd&Ig>{2@G|o}M@byg|!@y#)JX5BIDt
z-a-Xyi5vj_4*>dy;i&V?v$&qUwMwF2^TppUcgIHRCk^4i5=!PMI)jeJmRm!0a}c0N
z57tXU(c}H#^B@^kr}{q9a2P}ipTKmrZxfzB$AQi4h>ppqMK&+C3*=3ptz-pSEQ*^~
z5hECm71*+}vY$?4{eVekON*idkWnPu-GQBzIu&7pyJMLc>k+hXDvDs>)D!GmQ^fm2
z(zZ*lei)zCOinysjNQP+IalY=Ymc_peNr!x!I(@lntf|AmD~3ye6Uof#PO>4*(5CO
z&svzK6XYB=Oup+VW`j!Yvy^+zTnAM)y@aJxQF%ZiakM@^PL;|$_UtzT9
zN=_rmJ=je4$fN4z@mBfyGy&y7$=Dr^ZtDB>>pMH8_IDG+SNlsyfl#l`@0HojVbouq
z?q7!?n1LNJMRFQd2|s=!TXS-9Ds052CPzL;7<77E!t{U`k`o_!%VOv;vl^?borm<$
z>9xMNufM?lA>y#(LF7ouul(wJzJWjysUQ4DTgDy`%WPLXBMU~vM#>d&_{@ocW-3s*
zNL{q-FL_y+_4?7)_kGrfL3YWA7T5+JCXZng>*cDio*ChqtY*XYB~W1DB*w4s{qnM2
z*w}BtVFY#{(QVxsIQ-)SA=t%Oa}OS}A52igeC+$?YHu&3pirz+>aTZuIR9Cv-IFLQ
zi^$iD;B?JLERMvZAUzVS+)=LB=RwVdjy5^?FnMg?27!7zl7CbjpXaAEd0fVYx|MI;
zJ25(x6A5?+mzIWToJtQUfv8*51Uo-FWW!i7WiXYQAdRD(g6^o+j6md}$Magkzf&SvpCUuh?L&(7
z{E*oz3_^N;)emUkJg6&nNfZy)_#6|Kmq;$c&(9}YhROCz778_Rxs4i
zCb+li0@he#
zx`V0fw^kRgm~S#8fF0Dk3e&HF!4+{O3J7on_Bf+^c80l%@Oj=TpqAVk2^x#>JMT>p
z0v7MSBwF=?^p~8YT>;#pD;M7$$!Y5YOdvQ#It4D7^-)LoJbWn}
zQ@M@dXw(_Ld*3^T#>S2$zrw>`3=I$w@XGXkYc0m)2&+GzL7-BaQSi)MUZCW+n@%nP
z?0OAkmPvW^yvneyz((b>yJ|eK7@hEGGj+)gH(}_8p*lVDxgZ|Qh_65L6J^qxAOW}$
z_i-4JfOjsz)>l-7AI3Ge+oNFq7u_(W+;T5%BPTY0VLlZv|G9tjy`J?VpKKyHFkZmE
z`KfzTX*ENwI!m1oqEQ!boW%MyVu<`-smxraNNi>$lXYq6|L~MUGCCiZy1KqTNdQ{}
zBLS08Pp=8?Vx0x45ZVia*0dEPqe`V3hR4ND(#OVZGP%rH@(3Z7%?)R5i
z4Mzlo5x>BYT_~VC5=xPqCNs=rfax8dh=BX!nr|XWx_4OA(-+~!JbzUr?rvZ}rB#nj
zWFzr(K`yStb?W~~3eIhbx~a7@rS~p)F#E|0{%|lrwoKRQH$uz~!KMYKO@
z(S6OAJBp6fjH#V)^#x=7e_1&vi|pXgX$zrUqylvFOuC>;vpzmX%|pQx%!wr`|1wBn
z)Yp4p5D47g`kKo8Ys9Wod35b-<2=ISn88+E(qJ+@FCo9@N8Ob>N?b!g6R(>7k#@Ga
zgy-{kgJGnbFD46QGucFH)9Y;8vKS&k%r4%F1zi+P-$WwC99ESqPjI{%_w
zUgqqlQjLM$GnC_Vz;L9~ej$+vSb%``E5R92lm#{*=x9|JYv>{d7m1gB^Q7YLC~b~m
zz$o*}#N$HESJi4FWNiO`|FXxT;n9FSb+hkKo9{1oyG9P{p@}RkEW!X2^>Ddgq2_d9
zF+d=RH2n1YPG_N|ywad+1LjX+0mvHQ<29b=_o|l4>%|Iq@G09W8m+le;y9Sa5q*ZH
zrh9h&Q1@Rl#rmIyTZ4VnMPC4Kp?t>&iEV7kt?Hsj_j>6vCN#*98b$vG-n-|
z556i`OU)%wg$ll+im&wjm(R4Xo?~nDUM8tczt_mv{{a?iE-GLvD=GQ@ovv$E@5ZH>
zpiJtF>`#uTq&XYQT>JH9Dm@Y+z(__Jwyp<{*MbcBfK%nTjjbpzi76ohXd{!Ar#eT=
z9;lZ)NL@K!%eQNvg7`)ok5h>Q3a<{BKU6NS!G=_pFBNXyv&NWno+1()63b^5|1G${
z8vNe=xSQ|$AsEVG#q%3r`{2Y(JMNw6f{W#jAWUY!qh6gZ{PRnKASat3&zLP5mbLgfNO&u4;Rl0{-O;v6+GEV$
zjnbKFk|SH*9_K$_t6ymuZ0#mp2l<+>5>Dd_=k~amH}GK~_MKN4l#%ftcE_N4$ROz!
zEZ>ns!@$Ff02VL>z?0m}VVILT*;_Q;QLV-NF-&Et!DUNHMWUB_n^+Mb&B}B+VOm|s
z#RlZA6|`uu7|K%d0M_aEpdx{RVe=Q%!!BUAJ=|h_jAb=#?mMaxwaBino(LvaYqOuy
zMf4T9+P7kp-9`ToS8p9vRU7qdtCX}flF}e8-O|#bG}0yA9n#(1UDDm%-6192-OYFN
zyyrdR8)ppvatvVa#bWJy&iT8p$!ekTGvKidbe)k{jANoCzQ%s;sj!;*A=WQ7(dH2x
zaA(R%XS1E|B{fuYzV*QIPn3#~s;?@FCaRvzMDZtGPgwXE*4eka;P0I1F*}(>FuWcL
zo2iPZ=%~0FaEr^>#&luPI!~#}#?6}A_jkW?q<;)S
z&sZLMwWw#W&O^ae|5B&b;p|;LfJjMm3UVXcIXwPEGg*EOD^|SBy|TI*IOKSKiSY>G
zeRiCkb#YNP&M|w!Fo!x?ITlz3~FgLW^
zeeBng(ezgYp$y^z5ei>bN3!?{DiBlV1
z4e^#A%^DTf>?ay?vm}uO6?v7X=UY%Gcb{@uytGCt=f|6mLO>(<^t=srM4AG=L;+lYq^WmvCAo`=|TFYJ|)XD
z*x_=CN4dC)C8|&@?4!$nm^RzzN~C@sR@I_xrTUNQe#P>&5ENkS)|2v~09N3De_nPM1wN0N(ZXlOzijAB4>tE`mc4TOULs^mZkXOt+%PT>(9|x2{o>4+r_*-
z-=k`z(pCxi?ugY4+y*#b67{JaSRpn?&u)sCu>Uha5NE8N7OPYi9Eu$D#UdF(M_KmS
zhWs&&QH+G7shKDjQCYIJ#TYeIvf#*+8~I{ABttybIT7I|`>kI6o+%kCEbPP9AN?2e
zc@0rDcwHEWAmUvInXU?vt@R7VwJHJYA+ahI@lE2Nj
z=WRkFZTId3{30a-tkA;bm@j>^{9*r(1t>l-*up7d*46%|g^$)jXtzr&57b@P2
z^9gwU%yNacMC{zIVa178+8#G3G;>Mq69b6wvjh>4En$|NO(&Odgl)HL{eHi|5S$du
zss|*k;aQi*sRXAo1%ukr0X{MwGDCk@d5(g-ljYrSQ#sw;8HE^8LwOZ!}K$f=u$Y&*}(US9??Y`iWtkuFiW_
z4|c+b??0Yt;^lEEmxD9{)PH4stIZ7^4r>9udGQ7ojc8Zca!+W;ox?7L-Axdpq$rY*
zrR*pB<0aoPe2m0a9f4)2yBUA#kb{
zP=gm!DPXh1G-ul5g&&8qSk1DSafVAl3%CcWF;HNaJR@RNP#cI`X>13_j$POP)5lflAKbF*JJV6AE-V&D^#q9jhB
z#lOz8vk?;-94<(^q`IE=!m88BEk>HTvJYb}ju&vXw5-DZZ@b3r=CSATc;2&NQB3rb
zB3Fk53r)E`OuQ#j_%dqC#Or~G`asC*)|2DQ1%5O(n}tL-M_1cR1Z%|*gE3%&qKhN0
zSadCAB==`Ib(-9_*$I})1<%f7<(d>+u=Pg5?vI)}KdB+r^EmmSxzuL=>;6TlnE3_J
z>!B-Xd20h^FyI%ly4pLF&}@r;4zeZJ&36or3*2IlfA^?4I4qZtyc=14fPj&3)}(zq|sO$tc_|D{sM@T~4$eo+J
zw%0&$%;Qd^5wHCN5z~}hC6nXuPVHJdu_rK~#NdL}(z@B3I?9q-*wy-m)v~;*1Aai&
z-nuz7l^RR5TI%R80bMxZYSoR)?Ss1?e^|=-U!v_!cyxNI_=Ft4<_nqmC4r{AnTaMr
zLDM>JtcrJ9t3Pbi?drozyLNO2@zhm*yWO~)-)(dJO<<|qA2uwl
z$+M;@EOR8o>Gt-oO!dqGBgDpzMc;83R{2A}@bu9a;mSz}A>6J4IGXto@vM^UV2x}F
z5tMT&!?G;?rqS`)nV#XA#jNV1R%_(Hh!+lgS9sIqDq>KIbuD=xFyN%_`~MZcli0~ABl=uJ;J
zt*Hn|fEfIUBG#u4xZ4xfh1u9_#aCw3N3Yx&2zC_?%GM8euQzi*Q3GMb+v5^thAcwW
z8v;n$fzKyg+MKoYsW8D;)Y>(6JgmcMUg9oHh2mhfEt8`~(5)mubTT75Em-JqmkSwrF=And(Kcg=$-k7|({IVNFg-wPt30q@V
z-BUAuN1RSXSJ4SRJi)7siuH9V&M3m-sr|C7>_4UwLXb^P{1bFhu$f=8d6*OPl$SeR
zKmFxO0qhkb2uB5A{kFpqeGI&%jzzD}$sR2nn&(yIik--QZsMcXRl)pe~B
z(3acnbqAR3oQsXm%=YRk$=nqrOw5Q=pQk&RD+ouEG)-?gvPevc-BaPM;B)LPzXN9HNh^|3z+{d|Gfau{=2RTO_YBTV_9bZGg;sgs;^
zQ6%P0*-{7BiOe@@m4+DHhlQg)o%tr4;3RA=2a?Bkf~yBuK83-x*;=jYa>bho;}N4y
zqsO)177Iz$wnHd2|9sdk;|a)!U}(O%3yNf#)%o#{HRaGx4)->BkU`SxV$0->$fY@v
zwZ@D1(0!~g2@dVlm(+3GeKYetY3514q2^v
zd&ic4YqzLn3QiekmHeaMp;qdAA3KB|1yqPQufgf)RlUgWI;96_M%AuH&Lr&FD&PL^
zKiM^rBys;D#tx!m^wpZFV^sBBVKx-}^2znc_bXrEhWBS1j_9s&?q0I~7k1T?jq0m~
z_2I#H(?%Zuh#HE^Ea`=z+0X`WkrV4dNk%wYb&I&P%mPL@NQF#a0w)tSA6x$i!fVRy
zeo0B0jaw;G{)ld0Ku0p9^&ydFFfrOhZQ6Mk8CVN2*5LCbadsNKcTJX`H`D_E2
zaeNjW=8mkVzD@8y#^JTthPCfF6kNqHV6PLdLo2rHL-q7Ue*VKHR7?)(TEkCf&|3YJ
zP^L=u|4uY~va?A(|0wwVx5`i#;NKurG(N%xLP59ZTkz`Y>BJ7D>edpr#U&*Z`7#z2
z>0C-$JJIOWZad{F6`UO1QS|2Z|yyAQq)Cyil`f4CRgNVc24iMPW
z<7FC+{O{iKa6uVc1)jVh&5RjDPZb+Z9=oNtTc`5S&u&w{_jHQzd%7LQS_
zKW+<#_R=yIx*YtVpTA5rU8ol0+%biOw2Ue95l~aZMI)E&j-mPrD{^}_N
zLXeKhNmPFjl3S1H^GbDn9Um4dl$WJd_I?~{*=0A%d9R^eAzjI9Ff1Avq`>GP>#&<6
z5iWj(OQKfF&Hg!XB?Pgl`+}1G8ZZ&%cU~>ZYDd;5tkP?*I$!2ye@5BB@#PllRdBpGK;y!
z9g6E48{TSl79^JrjPtpRy{SzOa$=|^k`JSNadN8*_f@Q{!
z2Fk4la_w($9riXhXg6tqn19utfcZj4o-UvJV*ntN2TI_0J)RMGcv{}Ky(iZOVK4*v
zMlBfexTLtSFm(v#gCz(%wVFO3>5&Sfe{v{2A&v*felt_OU8dm}BDKZAXTc+K^^#Do
zV30uGve04MM8;fWS8l22Ar`@uY1T$WTre0$mPh%$BoDjw;_5DdvxF)wfp
z!C{D(Xny+bF0V7HfY&?53LE(1%GTe20a>5-XDik59Z)uHb4`YNptrKN7WvoeW_h&0
z!YB|m4eUPI+FgG0O%8;BQXrWl$7g;gmu-Ludil@|sJlF&@M1*sz>I==Vgt;hIW|j}
z|44`{wLGHT(?OiI=X`4@l0IDrJw(F<_)I@N
zs2RIU&`pT_wcQy>B?7s>7>pj
z#gxiJ=$lLnPguul>6!)VF0!qTZ?nkJbi-M5p8?1q+p@3{MMnyqR+AsqcE$l(p3MGa
z1qutxP`%YvrBqDyl~D(lKb@CnHxL`N-k+vwc0GIdfQ%PmB2S)_l*I1-$gWyvPV4ii
zWCZEylj!##wklRUP@q*IjR$HPy;YA}qNk@I>vDgg`?i;-N8`!I+LjR&Zrge~m8lxp
zVpo0c77rN0X6{$OrM&r9fLqy}&L;H_bi@S*;W_P%O+#u~Kl0p$dLwV~ADdQrZQBQP
zWdN3rc+C#0UEVOk0zNy~ajD6H-pkv&$6cPkvqt|wv7*gojt<0_OvCTxbH!mL+=?Fg
zUf=?+f&G`!c31XtcS7*6N|^!))GATYQw)SFPW4^?Zi3Yznw_o$mTsvk`pPyLKNAOD
zn6PWKgq+vx^9xGAt2XZWut;?s_>5G%9%8@1#nSmB07lPan|R-YOr}G?@w3X-nl@jF
zMZQZN3VOe^qQ}q~THo5ylg8oVyt+?QQ`5Z6=ap8s;HfBIiDu`m*YS10Q_eb&57fI2
z@gFudv~=q)&im(lhXYQ4o`o6{-KXM#N^%S9b2^Ww`x^BjQs&oJoSm6VTWOhybO7-X
zY)E==ONkkd8}#e*kvbwQ0`TCPfdRE-}{y$;d%N1aJKw5RaXJ>yrPqzX?wy0CV{T)?LINC!P=rvZ?QEQkUw?}Xp
z4c$&x`L%!I;^M;lvyiU=CIdBetqO~%il)3xK3l~kq`AWe
z>+vI2qf{HwF0b`F4R#*Xqi7}5_VNaaoY8kLc+cHW56Gv$3;MonN5B_8*l9BdCyqRG
zr9xNK2jT~axmrElvv9|jQc<0r-H~qF#)cUHR}`3**`J<1x;2i+d<7Q9p2tftB;LXa
z7)$L=@+yqj$hUS}>h6XF$@gGLSK@+_hNSCVAfSH^D2nN!krk!1}f`x*H1eh_+9QH2~J0od*vhS-f6&
zL{&U-y9987Se4$S?g>ZuZ4lYV8)Na$t-T;X-f5GBk!HDtgy`jrb{kBCyIo$
z!6fCcRO-XSq05r0X0mK6Dk=&T$3_?tvRl5{n|gU1VT6SML}S6nYF9r_N_%^Ioxf7@
zlLc}lyZif{djn`zg7j*62(LblxMxx*@IV&^;X`0f%A<^@QIV(dH=W9>p`>8Np#Je8
z8-6ZV8eXo+9^VQP5l3i*Aa8VhM;zs^4yx%NTxO$)^4i$$>RHaboiAsvx&LVd>tm{)
zjn&HVDbq+w-TV^pj7uuTj#d>KY=sRDV{*l`c~}2{Ts6uh8hUW8#dc4AujNLxCn#1n
zD@W&!sEZT|;0-BYN)#YkXIM&H@IIPh9LFjsO+=5C;X^nqlnOm~m|#5s{j|&^Ll*j`
zAqZFNc~nR17dDVB@~nOP^>$CVxU+4#J@B92gLDl=#ko*2phv
z|0bwRYwE~zUEt(z6-;nEjo&{&Hp^ywk`~u7OJF-IQFC^y{4yJ4WyEv6^jEs0GKS4T&z)9y_YYk
zBNB+y&lsu^zc5e~A1BCSF@ef?{)OEu3_0glg!1v5e;%&>V}0S4R#w7VIqhF&ismAo
z+fuBl&N7DXNdq^*6(U``M6q+qmF~*$PG5hB=l8w*{rxXBlmq5_sC;kVv-0z4i+A#$z|vj?;Z2j|MzEC;
zfsl|NpshuUI}`;Z20;{zd{!5&`Y@CeU8PE7^p5ihtB$3678FYfS2t!;?|JQFep{`dD**7w(sfn)T
zI#(|CU8ypp2Yg0MF@8u+=fVOUh!*hI7|(b#RfoTV
z3INU?5buE)@`hPkUq=8SOttY}`BME?Q*DT<<@AE8BR>}Hp8yper*rgX$7ejvHg^nt
zayu(P9_-49+JTe$WwtM_SC;YyN*IpP3e1yT=
zO$(LzmLo|lUpts6g-E0CZ%@y>9douc3g6;D=Wm7JbJ_kriKhopIV<>xA^G2|uhl%G0aiBr#ArqKR2-2$rstvaz_*wfg6+DY2CjYz$zP~n^e&iR$b7xCo2$c_t&w2
z+aV$%a=cxA3raegxG$(ICbONbZVy={qT#Urn=SMpnJ1M78N8@4+IKE4Fh<7YW}Q$9
zpsQ>Suo`Kp-Tl0-`3hY|?M|OIq}BhS7NiZHMarpIt%1{d_27V7hqKOPPQzTxtz4@w
z6kt6nm3qQ)=-lB&nURV&LiapYu|ueS2<2r
zCDd+fMF0?Ph=^ky4t&qknH^5Fro8}NVJdig&hKdC(kTI1!ofiwLQ*;H+ku^F6B880
z84y0i6;36&U9vP9Mv643+
zpBx|m@D|;A^Bcb4rc8QcP)sB@>9fOWgyX)SfA_0g00cjvlHbF_cL9%exm%z8QnM9_
zg6cfD+56hySl#-J#3KKE(cMOfY;!xTKVwMaQ~)8P{$wtWD!>1IgCJW(b~$l(cNd5^
z5hKsY!eDTS4UWKGpb7xqf~lLp+56E%E*Q2_O{KlBJeM95S5bYJ9xsK*!vwGM??>Aers0B7*
zSWEmLxJTLm{1TwSH@F2}2sk(Z@-=iXUH!E=kZlnWYw!)E-x*sL$q$Pb$_fKf&gQ9l!LI!hpPR%I>aGUOjVTKnt
zaG+p#0gEC0wOBnUR$8A(=eiMKk7`ROHsx)S(kRfRM`ppRY{xXdk
z)^WIqIF0Y7bJg<#tO!AD5a|GM*K(b^w`U1>-DEM8U5`kwEtY3kRBigPaBQyDrVmS+
zL=|L3U&u3aSSwwBJuDZiJKj35uh{h5!Y+^J%h&+D5Ut931y@gv@lV2FtfNk#QQAsM
z2;5sJ2rIVSs-Y^AkW&o#`lzZ%(8VD3A42%z6C0aK8o@H&>h=?l&s`y)2vxMtaCcU}
zwpvQ?#oYkJ{Q{Q+1eewlm|So<9g@OUcy_d|i;r~XxyYHBCo{xn~Giu|W
ztyT9R!UL`3M1ht5yhbZ^orvLO0$l>99pW(Nq1WBM)wg8U&?Qu+5w#z}+r!TDbvIt>
z7ttWt2KqfE^NnUW#l`Vjb7_h5Ef=H&eAZ}3`7+C2$I_%Kba!meIKc()TqQEIY`5lR
zHA@TPhmg#@Q0iN8v&lWs46ri^f2`7qj&*;yHhbMD1I}WP2Qi6~`~Rc=xmbOM-yBPa
zNc@tEW>d}M%-VJQGhb+NnX9xwwFdGW#{f;AmKG5KZ(j8fZD)*AYj>O##ipsoBwnO&
z(iaG;>Kj_nW-BYK_*}29{PPS{U2j*ZRk9hm3Q%lU8uGl$%@*aFUGEf26)Z0cXhEi!
z9M)MxPC8u!H@nZ?*Vk8IX(*OH1P%i}*e%O6k(g3h7Ui1FaC8{-*xr2pA**iUhroGD
z7ibP1LfqF5M=NC9^|&m?0ifp30^BFSRv>{<6-%GXdhKgS{aaIe=$@T_ELe
z(+$G$KL?QqdyrEwT4Pmj+73XDifDDZKx(iS9+6As@o#O0frNzSSGvh(5qp=+^Dt&l
zrq?5rRE`6Jg<|+#BAt_I?AnM(stfHa8rma$K*qJl2=bAVY1KH|nS80!8ZND(oMs
zQSV5yxo)*{SfwEI4W^QoXRw+Fac`PQxHjFb+WP?;ABNz
zM1sv~YE)gK+13gIQ*n-A6%~vtt)UBVwdJv1A~3HTVRw
zT_FVz_;v@ef@kYc6~Cq!8=(8+=_%6MPNO8?)N@~Y+CR0Zi*tY5^+zD^0oK`Ut7vi^
z*wiSnDka>6+DtyAHY);HxYohmjVof$x743eu6iY!sURJD-n+w$n5SL^$8F{(H@^9
z|Mo!e
zQN|dbt>0}$y<4_bRj$=3F7N-P$K`S@Der{P8=^s!YJ&=Wtr-A|=8~m1@c1oW0~hGk
zV$%zE8Db*#V;^t+y~wscj@_AMiO-;(Yb<4q`b0*eY>xt)M0k1zIK;{ucKSmrxxjW6jIJ
zHi-RAF|a;D3L7W#M4_!SQ|$G_{0r$k=qq>@6;U@gH#1nQa5s=OYE7wKA4nska2V%a1*E!DzrFMZE570tX0w@H=;ZXpse_lJU$RUl`JD1q>7x1zBe30nfV9?(+{&XzsP5le%e+W&xiO=(7LMvEqw(2iU1Z
zhr?zb0s<3Rk{^9_CGy3&h0{^yp(Cx+dbq#O0tMOW
z<>l{8ZMx|qO^yAE3rR88+p{MaJadk&zk9y7VM~p6YR(pjTeG_bL0SOMzy#j|O)(`b
zj1@@YH!-(0VUghCjtPl$j_*%VsHaQ5i<*OAgFBPa_$NGWqd>r$cTHdBTb=!PSj05_
zn5zY31)BR99wGqpUIWHdccV7dd9Y4wcd4ODnWV6&C>T6T18pv2i+|Wvl~Wxk0Ldjd
z8o{#R+Xpy(BY%#-vVCgoTHD|EYp{_52VckrGEPv$*;2DRA5t2OCZKG%e!j3kk8v~C_@TC45+$~DHZ
z>bUp{;8?k-x3_nPQy`{|60vn>o_%3Dc3Yk=3E
zbjXAMAA44Xp@i;#sQ%F7=lRRp%Y!)?YQi=~lqno6eW>PaF21S`0i#tCcI2
zeCY#UK2act_!>KBlb!VG4eiY^UxrIya>!7}Ws-!zViJ
z?c744E*!Mxjz7x7tU=iyaR(}FZ*0yMMEEYZx{(Yd{R`xI*cMOECT`v=!@LgO(U=Es
z`0?@Gc!3qmo<}Jq&3SgzxKg1iDi-|`s)@yr5+$))SR=c`ECGmXi*AtHnvyB2&aaN9
zQWOCH-?UiKjpz=yU3K}d`<>gfjq(OP@9_v(uwlir`6OUxEG^S95S3D)GDf|~IR`8O
zu-l*J??b}(ZIck_zr@ep-|
zOfC(@Th(ZD56xKB0yDVHD=wW|hF4*SBelc9EIzOy3V6PQTBrFB5!sFCDKBS^6`Dc2
zkgAg)=1_x-J+-{)&ESV;Fq!=tN!cNQT3@7u&!{6rQEmQRFo%vb@?g1%O2g%Dwfz}>
z(&g@5t^L7Pm8kyU=GEjwqDG@RGL1SjUW2uMoLZKE7muEisO$NiS1Ol`_SlLF@szw?
zhrO+97>Ley={XZddRblMZA3-vbqbrFjabQ0XT}6qnBxmNr*+Z*G`F2yUz9$2;h&r(RQy~mr9VHg
z(R#~7_9lHvBTYXFmp@#x_Wao3CCJ}fFBF21`QtI;+s>+BLaGCCma<<#9d}{E`GV#r@8-lIZYmzRNJKYNAyT4KUtv@6xQI<`8G}cvn
z#H3|z?wiQu5?CG=Dfb_olJmm&jlFVWygoZS_`@O9cbA|>rX58A!HxX7<0XzoGJuYa
zrnGvdwIqq@lqyq8dHpPQbl>AC{Pc>Z_{^DDAmFYj#-ye>ew(XQLVfs^g@+676JXQ!
zYI>liN##IAgQ*FvxESn4ydOE_a>r;99gB8{P&wT^iw?L&JoAL&vg?#KG=_xb8sP_)
z?2nwBjo57F&EGzqp6B9%2H=MSdlgiNBsDgMewx6i_j@p|qC)EXG*7D}V}>t*Tn=f(
z7AvMv5egzRe{#E7&0QS;O(m$OI5|0u9%rK0x%oxq68MQI69pw(df;~~#MqL=8NLkq
zf|ZUyUn&e`Km$k$pozC|F_m
ziv%DdAGWB`V~cl_&nGt9(fuT_ECQd1h0nivo#6EprTBmeyzR+Vi1SAvTki`%@mVK3
zClU=Cj@mW(W|y^J_oGPhox-1(&R#^tEJ+>pY9n5&Q1MoG#O~BZI!dC{fIhb}|D>0{=%_H}9fELI!aE&1~vk3s794j>)RXu5#Y5M%i`WD%;Z
z;{_7>pVm}gTV2g@Tn^@IZ7OKn1+wmZUpZ
z`W>6O9+8sp{9GTV+~Xfd2wsIcs#VXDO>ao!?s5|cG!jFX=hIGwPJ^%B9v6`btFNeo
z-u^5t^>(S@=sbeN5d5T5T;^9#^{9HxZojKp6oF^1Je->tP6)n|r(aVKI#D4ca9ICPhLMQYK*9S3676%^4M`kzmAiL2PGEjC
zGn2wAXBMv7W;uOYbdXG1l0aSJWk{UGHCvnsGtf__#S8nQ57$oyWgCW;AE({jf1M>s
zu#FjaA#^W{UpV4AB4oRp5pcl$%bE?HCPDBJ%{FzWh+3W>6%i*!^o9U&3)MGinCwZj
zYfJu+^56H`a`b8w4H$oE&90kogsZ^#WwklXVr#4-Q~I_ucWi0)8KqVwr9%NMf#kP}
zacp^k9Ew)5+&n~dJ%f)i$wtM&M0$W3Lm`&waBRNn&^VA_cR#~?=k0+c_PsBh6J-%O
ztxKve-{TRHLDISlTs8Fu9SW~>bQ&Wc(4`xwf%TSuB;>X2D7*6|$n?Adp4Gal1Qzf-!@CyQo7)H|l>lsRR|y`|(aUomvTl-U8>bH^6|ZyH}w_Y4{E
z|9x+k$u_VMx?#Pt5<`_V28GTiMRUHe*r|tLBn??hY#b0kG4YSndS38m_%cj~uc?$`
zQ#YfJy)^{OS7@{OxFk3|g?D-DdE}Fg#lc;R!|)pH;@_4G+RMnsG7r>!uru{Z+lX5o
zmi}%ukjm27`!vg1UZYMk2-|aTlO9=Tjg=#jP9QCl(({0)IsfXV1c3Nss1`CVD*}-G
z_oOuBCEO+#%e)K%U%9U!AO*r-1#P6WV6mmp!!^D;7fo6cq|4;$wGq!adcMOjis0PH
z{b=`$cNBIR7bRqqRM(&m#?2!pVYAp^e?hpQ>&$)7~
zLT?#ctQ0G=!ZphKyeY=yhAHW5=|7UJSd7&XPUOVHHdYsqX+o_MqQfcerRd(*^<6cM
zCUBKDfl&6{YA`|eFlZdczUHRxa67=q5(5nlQv0we5h#&p=g4VJT3W2eCsFA&nm>4|
z{D=i>PS!xr(P$|rkd(Ip_^`atV1
zjx|n-kOpsIyKDu1=xU15ih!q%j+3kMfxUJ!bV#tM^V>;A
zW-p(GRJocGeb1^g?POyhQa2l6#ktjo_IqvVLwHx
zoMXYt(c8CXs~0_lCD6mYo-DonaVp4C55wJmf98FocpzPJS@*k8I?l2g;!I$$Kr3b&
z2{;X8w?AW5H0KxwnO>S#?Vn_nd|m<3688$Ri=rFRY$TNiCA~digO5L5nI!-WB$d>@
zimKrv|6G=9GFBm~yh&^BhA?C+UNP|fgW>LcfyS)VV!QB((Kd~B#AA277}^iJte0fZ
zmF=WKr}D2NK*aE&&JBc151}XTkOPR$zt=iwJ53_(F+Dy^iu#F}&UZ>1{OIWf9vTuP
zJoeuYQCBzw1mwVv%o7=U?bG`ehx{HS3?>?3m(b6lHsB9N}gp
zM0T2_Z;zLZ{oPn7fua1F#n4#uhqkoHehW2=CsPYPS&R~R2RftEjm^i
zWtMX)w-cP*!RKu~T`*mH6%dl5{|}r}o4w%%nlSJn8CCCL&J=`On732Y$nVLyfXLVr
zcriDgEh_Vx{xc6m0^SC&z(|a8evc-0bNI$KC-mZGbqgHh^u3DsU6zt+tNkXG$X#UI~x3>0i%0NP5LAheQ
zW4>4kzRWw_0~@~YuH=w_E~|GM6Q*Nq6G(r0-l(;7T>U&JwBC@kZb4->3mv7Z(9p>
zn7;JOH$BNvjQ#R0`PxuEw0^SfJBi*XdfR2Z?S}jDvoy^dY-}mhmyXP}`kUK^_fB^v
zdE|rxm2DTNh10)o6Wfvl@xNa@B^cN8sP=^p_6FN)tt3HMOC91h?Y^m-BkX=|r`mSB
z8IU<_YP;&FX<~GMUJ*+zM0AlE%1s~Fr}=A(-9ZFPHQ#Pz{S_Jf
zev~?intGJ*Ox<_g2IyT@16W=moE7)Yb5s(8;&=(tH0@o^Nrbu+9+9TEOjZ}AXuXN%
zUhxjL!3P$Z!&1v4^l>y#g7YS3-rG+GuxI4Q-0p#_5IgIqdqv7E8q6vY>Rr
z?a7N4x4zqei_JWPw^R1)6!WPjyD;-D{<5v5`^RLAM9O<8}B-~MfI|L^^DUQ!e
zeBYJ*{6%Zb_gb+Ds06R*2gSUp_}CeSQk>L*v&HtD^k
zD*j{F1b~9np!sIY>Hzne0aJuLI(cQ!Ay&fP?Vb%S)?7YT8f;(ghRV*f0eJS7)6NEJ
z2P@Mdn&I>%W@q=
z_+G>+og@aO{>vjSyNsRaqB%&hUEVv&Rb1{X7fR+FnLTg5S0D2U?$?iC%<&7se@nBs*X6nKmf}em))s6zvR`QiyW0Q;@D1uAP
zWeJ8$Zr#u=T+BgLXlkaiuDf0Nr2$9NxoRl0(rfRWCUO;)Db?~Z?IO0@UWEKt*Jqk0
zpI?a6TwW@QK4{`hlAh(2C(!e>^x!oP*&yG#U!gfgHq{kUbgH7-(y1noJ|;TPhzF+3
zh<9XD1u)gtO=P)~$&vGlrIT@Bifj*jcVM*3)O%`1DNt*9cU_k58%Ap!{5YrSiqW;f
zcy|5N7ci#W5IO#E2NQH1p8mw(vPE#H!uU4dA$hGmiAuS=SAY~zrRFICfWkh}i#zLw
zrB40HOv$^hV5yUOImU9%N$1DH=iR(tlP?&=t`ykh4X+7wANSN`ow>}5bpyAfYqULb
zWexMdk2vH{YW=7$k29L4nq*QTD$Wb%F3iw9don6kT;k7ubEg{&|uxw#M*>=Gr-hIqg-&4P2(;F-H?^sp~C~;YI
z?%aPui}^i3>0v%yCP%d10MW<_4!y1GA|4Cd(28ROUMOy=ndY&mok1i)n1lYJbmlsm
zfm2M#PT_ZG=tYa?^{M5u4tU%YI-V~`oOE@J(UrUkmgGWHLzlxEqBCKhrYw}(98VGY=-A&ErQR_K!*6CAMz>;P0dZu1dVexCSJHG5&KdUtw4H_T5=W)h`6F!tS0)7}?oHn=FRd(P8!@Un
zc}qx?5zKpkCbG+y@t0nP0jPhz2_7O$W;h7@v&@}vdsCT2k|Oqk6OTawznEi|8ib`P
z`1Fk^z#n;rUE7{=#a6lP#!+BWtVD3FeP|JJOKB&uCVOs~9wJXKpj)yaA{gC<1tWY(
ziHy{XQq=EZM^uoHXC)8W3KcLnVC=f-r^Dx?xK5YSc=bElF=%cq>vzGLwv6@=7-7gjRk6TXbLAfDXoE7vz
zSgb<%+bo&s{G!t@;*8l#Mt?IZ!6o@QwCi5d!!kbGY!0Pd%+fo6f5Bj73DiG17*|(C
zvArEvQIY+dErNzKWOA1IWu8o_ne|2-bHKmxXMWbkDU{>Q8dGB}8H{j7fZnpfUQ$Uh%8Q+fs
zk-_W-n=Ljc;mE`2@3Zw(Wlp#Ehxk8*%JD8#CLLDK&6p3k(D40|i8PN*I}^|O8kPOT
zpxN(4K|*qX4>*M<8tte7#QWYbnHFVzs5@O@v6#XA7<#%$37JAV1vCb_2KeMzQy#s=
zC~uF4!HNL}<2SE4mj7qhKp?c!;!6DiIvM0I`L+9V76Ab?x-*Eb|9S-@q&RHHjT!oF
ziRBGv%HZRAEt*hrPI-{5{jSHvbRLV(kQ$DU!{KVohECama#npo^4#j=4BoNj;&nO4
zl#R@-%{Gh3$bU&{e1AHv18ZH=nt-Hd24)&OI{M~K)K#nx2y8OK(|nzg)
zP=9|Khqt=UkGC)FT9>Ky>ifkiLMob4SAdd?m-yW@Cpxhfgp58ych13f#7EH#mI|c4
z{s3RLmGrY2zBfm>3O-|o9?ee~-=f9VNJFvAJIwm}uvtg&=8oBQCmNsFEgax`gKGnI
zHCnF2l$jPSCO_9VQpvNfIM{c=CNCzm81p4bn5Ft9uX%!E)zMjEXNe~DB9PtC$OyYu
zISuP912%uBY{Ib9n)ygS#Ix5#NJYiMZiV22)TB)~eb0!`miL(3nP?%w$Dg;!b6K^0
zFOiGa
zmx8Fz<>68#=CvyIx&F?m`EL}rg)GNNuLRC!>Ost*a9(_%`nf}ucQb*6_SY#tGX{?w
z1Vj7e4o4xZkdy{#)R#|0n^j6jf=%QsLbJJrT}@iImn{9Uw9%JK_LCg&0XBJ3$(Ui`
z>46-kaBQ5@dNsCbrR+_HM7_wS{%@ld1GMkvPFxubw=|dwzoX>akNEHT@UGAICuS%3
zc+Pqg#tImlm_5yp`wO@h9#uJoaDDY^K7+!vCQ`VDC)~SW9ojcJPkJ(zs4zJ_I
zCbg?hx=QmKCBa&o?b^v&+1kQA|K{_}ku#wAqEk6a$6^dTX)esA~lKGoa)bt%PpU9;f-kF9SG%lnJ}9@?^P8_Qa@
zZ5zvWExVR$S*w+B|>ph;dzmvyR;Ta(&yqwUZjkIQEksD$>SP%W?uz8e&v+=_pJ4UyCX
zn+_sW<2D-vpY{&G@tr?3tLb1<-biG
zm(nD$KBW&L>2ZdxSbO^omwHei6Ij2w{qsGH)G%llckcXtlNM4m
zvC@09EzjfP6R5f(Hu4aFGr`?{bHPrh2k;LbRW2tU6XI67Z@J&%=j+c#ug+)*iqb7G3
zajsE^>y8qq^Du(GNl`)7nH?6tU8IT{=eNH8%3YP|Tn`)TWiKgCX5_Bvtrpz@5ix2C
zOHSXZw>_SXU48E(Nx2OX$L&JKq93}=H{488%8p#e4#oC6ZM503uS-+nV#9E`YRzLB
z%0_z*d6@?nHnGH#UgK)wOp0Bu_|pAUe3*qrf#B|6Hizd3QszVRZxH_=ze~-ccsMKn
zlqo7Y+2b-JzSw5$Z1Tq!r6>aJ{74z8>z^s;!l|q;nQ|OoTE>8SM6d>~iPR@0t7$ID
zN6Yrazcr=1~C0BrW**}5!orvKAV*JTcU0veD
z(7E*W$5l}vYX8+&Ojl>Q@y&x;de|0d)dT~PAL#+N?kv>=rk`%Gifd(ZAK*?FHI8Vx
zntVSx*R%C(j$?rh|Fkn;8_GC27G`^#X6~`TuBgAN`2vh0LE&sA4R~vXiYYis
zBzp=0?~(EnV$J6?RNXRc3SzK}Ahr{)Wo!CVgL?QCHWWntb$-9yak+OGJ%$LL^<$9{
z0s{ZTL)rZNz@TjX?w;P2paEG>ey}>U@z%z+vaR}I@WVMl3bnEt&rfdCktEDm+D6Tu
zIex1Cm&nE@o410*8cqMcy6Zdbj0VGD=Z!bB$mpzd9hp!x?^f+^0SQY^09>RU4)Jgl
zxh1j>OfXu`mr3)3)j`%T7CzsbB2_8VPy}2@fO-BHs*O;r*cPIL
zivJy!opm%6g*-x7`(Z*lKt;4FX|h^QPR<=bQR@U)rHY^LuMdk=OOtgH>Lw2fc|Fly
zt$3~Cvq1ko2SBB|J!7KrxEyob&rzn$kzATJy#JSL0T4)jBep~DYOAZeDs+Ese=pbp
zoSykQOH{yuMjh1(hb2=24%)y%Gc#;G8L9ZY!3#V=Z}rRW?UDJU&S8Gi%aO}m
zjK^9Ltbn-B-h9`V4a{6$&9CEt0&Fw;{;L6A)eQFKoWhzd_+Q`~PPV+V^6DI+=Le<_
z@fwQ}xUNdVl^vEdVnf}2P~rm*jy4A-0I!{w?*Ukz0M4^O?Sj?e`xF7_Q6p@UBY@Jf
zzH^O(k&NL8-}8zTAJyO5K7#XlNQZ)k?gCuiZSdg`o6|eeMjG5)dTTv1;B9B2sU23V
z46$_Qz`VkCfKa88A5ik%e3Fm?bj@mU-xl
zn3|u1kvh<0;hNgoLxsHqYioqb82tL#z0B15{EUo9!1uZCfZyjn(YmK=DBMVE+k7B;
zzRIwzHRU$UGR|gc2PLRy4mjtC`X~tuiCAcnT^t@SHY=|hr}hGv@*OQ;z0+ruCC1Qt
zT;fC+?pIaWE6Q1qXY$7`)*t4GgvaOcm>p?k@HtTog%~3ea6{?pu7p__6pkb!WB7f+
zMeqU4Py1F{C6PEfQedFowlgz=H&?z@IsvT<`vOs)l1-NgJBDCRevvchl{qgxKw*wn(ZCZ
zqEI@Y&f)_ynJjbHC{Psu&lP)iX72!BB$w4a7?)r0!tVkf>E-UZezf3}%3@wYRg}c-
z0N6+a>5X*NI*Pf)Se}GFs16?JDfU;#k
zfQ>Xj2gFnhjN1O2`a{Q9;TuW)=EPY@OEb1|PwD1akn3q3>1A{;Sz(PiNMO|VD
z+QfT?t1AgAe+rGoL$00;EIfsH>frUd`lgLdVYx10J`~j=%B}1Oq7T6JqS~#sBga($
zi}a|5AlH;;PJKz;5d{;+&No*2t?&}wTz2k`pDP7_
zKHNtcdCH?YDpv%`J)YwEA!7aU2bdCtRmN`A1y54|UsS=wux9HNkn?@{8=c~A0s!gx
zGTsTPw9K!@a@z1z+CR$NTOFRn@+OdhgXX=0j0*q(wQj~q;#V1zz6FD1%n=C!e%~N)
z2MyN4^%RH|&-h*y*Uwk#md3c7ymIHhyzy{Tnufn<&t!Iid3=m#M`Q!ssvZ#C&$fq9
z_x4RuMiA0|zlUwW5s)*lH5(UduwFR7ogxL4d0o00U~(w9UnhajC)U=yyB7u!yWvmB
zq+${`L1jk#?1>9k?iHU+!8(jmrBFXv+I}~31HgJzY!cUkrY^)dLj+19A!wmz&vL0Wu&M0OU8o1(+veNg$m}_n}QBs<%)|=a;AG
zHrWrn-v4OAf6@UW*?4;pnS}jewap7?o#YFLB2+arVBED*V|Ym!TqF+7HyOV7yxEWR
zfB$i~o;o%sM_{?lOWk`7JP7^4Pe~&mgBSm>jP+{4+`tjQLf{VY_xDG(n!~<+9g(Mf
zc4PQEnijY}W8OJGuL0ypSn=e-5%P;v4OLJf;`qK;#=0lpze%bQGv8|1(R!Q4TUt4vj1$)T%JI
zwT%H_=s|ku2|)EzD3CyAqVv4?P5rxAPuNybJ~4pzKWuQxpz1hP&Tk-by1B6t0|GEG
z@1bfAR&<-+m-(R4;!N%A;$moSPHVfo94d7;X)ZHEOg6hvW7>1J(X*4>n`_V&fc4px
zuDiRN5nv^u2%#uUHwclwf>#V%%>YP@d483tK>9`d15q__B5A@p$ZPEec;iiSPHHTi
z3Y|K4rr8QzQaZii$)x#wnyZz(7(<|;v%a%KK6|>_UT$w>r1TMFy-1pC6*|>{Sy==O
zI!&E1X4F8@2i<>qonF6P2kIKkeS6RL04Z)qjiT@j=pVr0aUz+)VuUes`KuNy2p&{hT;tA<7Fe1wx)XHbV0G#r
zE`T0)9%^Y}A-T<>qs^BM$nF9;ReJ@6Ubl!4AR?+w$v4pp)!9=FM1W~veCW_VrT)dY
zY@+4lD#m2>jOX+5M8*ovQ=VeZ?9lw4fHvtC!81r5K~TN~Ec0&^
zRaZv}f}YJ$_QBHS`n^0CtCi!3vq*n<4VbKzvTB8mjg2}9p2X(V=H})ZoDT2Er4qkc
zsbaQN$CJy(1RoI5ZHvmK|6~D-jcN)%u<1&sQNQvgTI=xmbUk%jjvTxA
zaXE7c3W6a(G1#4SD&b{vG*kWoT=wpXlz+$w<>2GtDb$+D0$Xikz<=2ED~x%|dvJL8
zDzf7aBt_z*NEl_l1aVJXdBwI)@sj2GIt_bCS>HZ9P!64Hd*FM23eQye94cGwp42U)
z{|)fTjnCG*fQ=aOD;al^COD1GtcDrTjG?`}yogNz$pq5|Fh@`=nWHqOe(l*yN!J
z5D59`Ad*mcsXD%b>`}bdY|OyQiq^=G!*W`rfQ4)iDwE!uVYO2pMX0c$Dr+G)WVvBodX7(E<`oI2{_M-uwA1FBqqe!KIXztB!HjvR;dM{5^X|-vjQ&?AJ=)4vC1CDX
zG9ZVM7}#L5ba0us&Ug+L-3_+dPW1r6jQLJ&TC&65WbPt*=jE$)VjEP7o@dF?%CEVO
zbi)GK;u&0a>&g3-dMkAs*wks1KJ;LWQ~VX9A!=s=-&@0G;F4?F`Soq}$jj5(i)1(o
zIdF2If7X-?b&WYis8i7bvtOS3~a~es<$s>{y8ff?;@1f(+_v3pKfO5+p!T^0%Mb0Z_7p1t!6=J%Z?&@H`JfqQ
zhwz+7J&uR+(K&Mq$cX%T;2xVQ)&145TjtLD@u(D(+9z8X;12Wxrkdxm{XFI~KV;j0
zYt+F#d5X>^m(3hM&L!TTQuR3(^)4QB>_gAG$Fm+pJwPhqrF3osN%p-L>44?qsw~IK
zHrS6tPB79LYfR0Szl)LSa{=+4!7nzB02|1VBnP^!`OK|3eQ|g<;&nO3(Z=I(;SQ3R
ztAn}9Q+Pa1g~P*#(~eWFzFDuoY38lmE(h+rnQmF?H71gF!1YN>Tt3T8Ef
z6o}nl9)vzSF)Qf%-R)rdja)w8bi8-~<&m349=l5*K>vgt*J%#qzy2mR5SC=J+8G+(
z8HG>Z;Ls(}sKz%t?M9$Z|NO(%MF$K#Bez=q;*?1*Z;vAL0Qi*#CM(7s5bxN`@FUq{
z@jLIs+!{3>ON?||aY+G3c7BcY%Mhgf%ipo|LlMkQ*7VVPw9eXJ!_G>JJd3Pa$AO4^
zRKQa*nM5?=R-7c%qXZ-&rOK?Ef~g?hNb)8a7NAiLXpG6Q3fogBL?10R%JW*gXW4c<
zxjGF0tOK0`&wocHx)P&4a!W!;7F_`!>h&@s?iE_y9ULMWTFU$6TJIi$1Z6*Cm3A`KE2)ZmU(TWOus
z_iKZB25TY(m|4HcWd&oz48Wq@+uM_)E`OlJa#iZ
z<&<#W>$rLI`P?;a^w65Io_VwCI791zmDECpCuI}P6wejIipAxS18%A1FqR<6dt%uX
z<`pQR=r;yZ1dLd+BV}VK1kfiw;ifA-iqTIcc^yTOKt>X>2Um
zw%?hRsFit`Lmt!Rz|kET&eVgH7320=LJ#bU6*
zBzi*Njguf31{-`R_*16Bv{yJud-|i(5ou~Zs
zU8MO(ITS{j3IwSTqVan~B9~at&^H4Y{nKI#0;!Jh$9Su(i*RBMNRdL?7y`Ho;E>oX
zVLeL`dRYAWsvzp_q#twM3@VqodHmMUx4m_Y@C+0Fm|q(bn(*D9uIF5~q2HF`i7V#d
z^Wyt#TlzZ_SH(hph!>z;*2yKBgnArKV<`eo7<(mEA7IcXJ8A1iEZ*@|nK(oT2S!Fm
zxRMm-K=kQ)9oOTejn|$qiTp?n)k0bY7nu%&VKFupup=KVx+O0+
zmUfEUbS@NV1V_r%rDgNyXVNVFa!=h`%J;SoEtB@B#@7^uay*x#EMM9_!n%M{@PmfR
zvd6w_h9X$#soztSY74!y_O=U>uvg3ol`#StiJEG?&d=FDIG~Wv`}S=9R1_CQ3J47u
z=T9Qfy2Ef`z7H|)d9r=?nOVl*S2v&gnE`q?6huzN)zI_3GmRdArd!AV?AjY{
z2Mwx`n#cRY%EcHasSl;uhUtK|IWwPWLFe0w01fSa67ERY^w`tp%VVq{4y=2NS7A;9
zs|B9=446rJpT1dQsDI_5blws%qwEKhR339=PjIjz1F_swGoWjI+Q?pcwOPxw;CEY_4fTo3ukWR72y!L^Nh1O;|vyH
zgX2Dx3iQdQsT^((N-KFNn_345yks!>9%?7}W+WDX2ZvX-_eFS?8%da}3;O)XH{w7s
zw^ML&@;kz11@5UAeYIe0O^^s|ULX9C
zULq1Jh6oZ5*Y`f}yVtgV0omuBS8UfcsD$SpW-5+X+m`rI-rGhZR~(%+^xbolExC#_1gc~n(|qNLKv=c^g{C*HVP*JCYJn0JL_S{8Vg
z5*hVdr{_BFc?Z4razh{K<#CxqrgKvd=7I@^OD2z*}lz5KVJtyd?|)pd}IvM!>412?Nb
zJ~0JocWdlu%(2SEsln0N(5Ti0q+HelL+GLl@nQe>kgXu}d4kvf@1}PWXj7bmi|M-2
z$N-KS-hEF)iPXklRSgUpt{r_E?Q`AsCW%3Z?P;|uFtnA+Y1b3zPE7an5!)#iyxxN7
znh-7<-N9|A?GFrk_=a8}Hm{0e>Tau+X)v+x3bE3GdT`(iDFi7oihp!f4=khaqx+K$
zX9s8w0-?2$D^3iy2_CP1)=xSY;`VFkv}(Pm^Wgmj&*ch3s?OfE`*(gk@!CmJKVbUj
zU<=yi#!DgKiBI}iWYd^M2kI-e8^^sahA7*#?b`l$9hNt$QHV15fDl`3wmpQk|+qNo%z7uYye%K0?2yV<4p@&e~!j
zC;U0SbO%ANf=H;c4$cZaP2!ls_&m7xe|3w(R{UoUKG-0wx&
zw#}k2XjD{{trr*7M5URZ3%#2izvFO1H8>u({+Js}pmfW?z9I=OwnCvn#u~0FMitz4
z$`uOk)l9>2z=sFTqB#S_fB`sCB^iSvmN1o~xG;7|uDBABG!#Kxc##_Z=i9AkHD|^G
zv)iyZ<{4`2w_Z2LNgdCRv~oedJ2XkAd+BykKwHUW9uv7*Z1L-jW`Ts_n7=}kJ+Yr%
z_d8s6+vNi2h=0wFj;#=s8;W=0oT*9E6i~=01iAS0Dr(PN;^z>l
zkRSMb$W%mdvL=t!N$UUe_*Ghx)<`y+$ngx{%#b(?3gBltUY<+erD#SCsuI9AO@s1;
zT%P|av4b+CS7-CIjYCZ=3aetPzx=JtXPIn1UBJMm;aKrbaJE=mUBhKTXDH!M!zB8h
z&+PjcI-PHhh|O=C(Lw1*#bLr>(&|@W&vyrN2Jx+#R_RQQNerjaU8xowXFryezUuDF
z_82*rq|mo6y4=UrqW3ja8{f@vHb#FuoCcn33$F7`ex_j{uSajANF^3g&pTMjhz5&z4hH0lg6YZt9({)^*R8HbVM
z=f0;9AJR$zI@{+r5l2fM^0p2xUeg6b4#P~d3=W8p2#-JZIY>W