From 229458043245de432c96f05457dd79330d568a92 Mon Sep 17 00:00:00 2001
From: SAM <60264918+SAM-DEV007@users.noreply.github.com>
Date: Sun, 2 Jun 2024 10:39:37 +0530
Subject: [PATCH] Update transformers.md
Added source for the image
---
contrib/machine-learning/transformers.md | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/contrib/machine-learning/transformers.md b/contrib/machine-learning/transformers.md
index 97346bd..9fbfa6d 100644
--- a/contrib/machine-learning/transformers.md
+++ b/contrib/machine-learning/transformers.md
@@ -6,7 +6,10 @@ mechanism. Before transformers, predecessors of attention mechanism were added t
Transformers are a revolutionary approach to natural language processing (NLP). Unlike older models, they excel at understanding long-range connections between words. This "attention" mechanism lets them grasp the context of a sentence, making them powerful for tasks like machine translation, text summarization, and question answering. Introduced in 2017, transformers are now the backbone of many large language models, including tools you might use every day. Their ability to handle complex relationships in language is fueling advancements in AI across various fields.
## Model Architecture
-
+
+
+Source: Attention Is All You Need - Figure 1
+
### Encoder
The encoder is composed of a stack of identical layers. Each layer has two sub-layers. The first is a multi-head self-attention mechanism, and the second is a simple, positionwise fully connected feed-forward network. Each encoder consists of two major components: a self-attention mechanism and a feed-forward neural network. The self-attention mechanism accepts input encodings from the previous encoder and weights their relevance to each other to generate output encodings. The feed-forward neural network further processes each output encoding individually. These output encodings are then passed to the next encoder as its input, as well as to the decoders.