From fc55d32e87621268fa32e16f0c9bec55464f7ead Mon Sep 17 00:00:00 2001 From: Vrisha Shah <74671946+Vrisha213@users.noreply.github.com> Date: Fri, 17 May 2024 18:44:42 +0530 Subject: [PATCH 1/9] Create confusion-matrix.md Added content on Confusion Matrix with code example and heatmap visualization. --- contrib/machine-learning/confusion-matrix.md | 61 ++++++++++++++++++++ 1 file changed, 61 insertions(+) create mode 100644 contrib/machine-learning/confusion-matrix.md diff --git a/contrib/machine-learning/confusion-matrix.md b/contrib/machine-learning/confusion-matrix.md new file mode 100644 index 0000000..959df9a --- /dev/null +++ b/contrib/machine-learning/confusion-matrix.md @@ -0,0 +1,61 @@ +Confusion Matrix - A confusion matrix is a fundamental performance evaluation tool used in machine learning to assess the accuracy of a classification model. It is an N x N matrix, where N represents the number of target classes. + +For binary classification, it results in a 2 x 2 matrix that outlines four key parameters: +1. True Positive (TP) - The predicted value matches the actual value, or the predicted class matches the actual class. +For example - the actual value was positive, and the model predicted a positive value. +2. True Negative (TN) - The predicted value matches the actual value, or the predicted class matches the actual class. +For example - the actual value was negative, and the model predicted a negative value. +3. False Positive (FP)/Type I Error - The predicted value was falsely predicted. +For example - the actual value was negative, but the model predicted a positive value. +4. False Negative (FN)/Type II Error - The predicted value was falsely predicted. +For example - the actual value was positive, but the model predicted a negative value. + +The confusion matrix enables the calculation of various metrics like accuracy, precision, recall, F1-Score and specificity. +1. Accuracy - It represents the proportion of correctly classified instances out of the total number of instances in the dataset. +2. Precision - It quantifies the accuracy of positive predictions made by the model. +3. Recall - It quantifies the ability of a model to correctly identify all positive instances in the dataset and is also known as sensitivity or true positive rate. +4. F1-Score - It is a single measure that combines precision and recall, offering a balanced evaluation of a classification model's effectiveness. + +To implement the confusion matrix in Python, we can use the confusion_matrix() function from the sklearn.metrics module of the scikit-learn library. +The function returns a 2D array that represents the confusion matrix. +We can also visualize the confusion matrix using a heatmap. + +# Import necessary libraries +import numpy as np +from sklearn.metrics import confusion_matrix,classification_report +import seaborn as sns +import matplotlib.pyplot as plt + +# Create the NumPy array for actual and predicted labels +actual = np.array(['Apple', 'Apple', 'Apple', 'Not Apple', 'Apple', 'Not Apple', 'Apple', 'Apple', 'Not Apple', 'Not Apple']) +predicted = np.array(['Apple', 'Not Apple', 'Apple', 'Not Apple', 'Apple', 'Apple', 'Apple', 'Apple', 'Not Apple', 'Not Apple']) + +# Compute the confusion matrix +cm = confusion_matrix(actual,predicted) + +# Plot the confusion matrix with the help of the seaborn heatmap +sns.heatmap(cm, + annot=True, + fmt='g', + xticklabels=['Apple', 'Not Apple'], + yticklabels=['Apple', 'Not Apple']) +plt.xlabel('Prediction', fontsize=13) +plt.ylabel('Actual', fontsize=13) +plt.title('Confusion Matrix', fontsize=17) +plt.show() + +# Classifications Report based on Confusion Metrics +print(classification_report(actual, predicted)) + +# Results +1. Confusion Matrix: +[[5 1] +[1 3]] +2. Classification Report: + precision recall f1-score support +Apple 0.83 0.83 0.83 6 +Not Apple 0.75 0.75 0.75 4 + +accuracy 0.80 10 +macro avg 0.79 0.79 0.79 10 +weighted avg 0.80 0.80 0.80 10 From 1e543beeab7c458c3a22060e3c68c73b282fa874 Mon Sep 17 00:00:00 2001 From: Vrisha Shah <74671946+Vrisha213@users.noreply.github.com> Date: Fri, 17 May 2024 18:46:04 +0530 Subject: [PATCH 2/9] Update confusion-matrix.md --- contrib/machine-learning/confusion-matrix.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/contrib/machine-learning/confusion-matrix.md b/contrib/machine-learning/confusion-matrix.md index 959df9a..ac5cd31 100644 --- a/contrib/machine-learning/confusion-matrix.md +++ b/contrib/machine-learning/confusion-matrix.md @@ -1,6 +1,6 @@ Confusion Matrix - A confusion matrix is a fundamental performance evaluation tool used in machine learning to assess the accuracy of a classification model. It is an N x N matrix, where N represents the number of target classes. -For binary classification, it results in a 2 x 2 matrix that outlines four key parameters: +For binary classification, it results in a 2 x 2 matrix that outlines four key parameters: 1. True Positive (TP) - The predicted value matches the actual value, or the predicted class matches the actual class. For example - the actual value was positive, and the model predicted a positive value. 2. True Negative (TN) - The predicted value matches the actual value, or the predicted class matches the actual class. From 08c59bff565efbfff5fa8553e50ec5190046f37d Mon Sep 17 00:00:00 2001 From: Vrisha Shah <74671946+Vrisha213@users.noreply.github.com> Date: Fri, 17 May 2024 18:58:33 +0530 Subject: [PATCH 3/9] Update index.md Added link to new content on Confusion Matrix. This includes the title "Confusion Matrix" and the corresponding markdown file link "confusion-matrix.md". --- contrib/machine-learning/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/contrib/machine-learning/index.md b/contrib/machine-learning/index.md index 82596a2..2f27fd5 100644 --- a/contrib/machine-learning/index.md +++ b/contrib/machine-learning/index.md @@ -1,3 +1,3 @@ # List of sections -- [Section title](filename.md) +- [Confusion Matrix](confusion-matrix.md) From 6574b726fecffba739e6fc4d01776944b76333f7 Mon Sep 17 00:00:00 2001 From: Vrisha Shah <74671946+Vrisha213@users.noreply.github.com> Date: Sat, 18 May 2024 12:44:43 +0530 Subject: [PATCH 4/9] Update index.md --- contrib/machine-learning/index.md | 1 + 1 file changed, 1 insertion(+) diff --git a/contrib/machine-learning/index.md b/contrib/machine-learning/index.md index 2f27fd5..40a2237 100644 --- a/contrib/machine-learning/index.md +++ b/contrib/machine-learning/index.md @@ -1,3 +1,4 @@ # List of sections - [Confusion Matrix](confusion-matrix.md) +- [Support Vector Machine Algorithm](support-vector-machine.md) From be8993b6164d12fab5d3ec6e94172b8da5deadef Mon Sep 17 00:00:00 2001 From: Vrisha Shah <74671946+Vrisha213@users.noreply.github.com> Date: Sun, 19 May 2024 09:28:58 +0530 Subject: [PATCH 5/9] Create support-vector-machine.md added conetnt on SVM --- .../support-vector-machine.md | 52 +++++++++++++++++++ 1 file changed, 52 insertions(+) create mode 100644 contrib/machine-learning/support-vector-machine.md diff --git a/contrib/machine-learning/support-vector-machine.md b/contrib/machine-learning/support-vector-machine.md new file mode 100644 index 0000000..90ee922 --- /dev/null +++ b/contrib/machine-learning/support-vector-machine.md @@ -0,0 +1,52 @@ +Support Vector Machine or SVM is one of the most popular Supervised Learning algorithms, which is used for Classification as well as Regression problems. However, primarily, it is used for Classification problems in Machine Learning. +SVM can be of two types - +1. Linear SVM: Linear SVM is used for linearly separable data, which means if a dataset can be classified into two classes by using a single straight line, then such data is termed as linearly separable data, and classifier is used called as Linear SVM classifier. +2. Non-linear SVM: Non-Linear SVM is used for non-linearly separated data, which means if a dataset cannot be classified by using a straight line, then such data is termed as non-linear data and classifier used is called as Non-linear SVM classifier. + +Working of SVM - The goal of SVM is to find a hyperplane that separates the data points into different classes. A hyperplane is a line in 2D space, a plane in 3D space, or a higher-dimensional surface in n-dimensional space. The hyperplane is chosen in such a way that it maximizes the margin, which is the distance between the hyperplane and the closest data points of each class. The closest data points are called the support vectors. +The distance between the hyperplane and a data point "x" can be calculated using the formula − +distance = (w . x + b) / ||w|| +where "w" is the weight vector, "b" is the bias term, and "||w||" is the Euclidean norm of the weight vector. The weight vector "w" is perpendicular to the hyperplane and determines its orientation, while the bias term "b" determines its position. +The optimal hyperplane is found by solving an optimization problem, which is to maximize the margin subject to the constraint that all data points are correctly classified. In other words, we want to find the hyperplane that maximizes the margin between the two classes while ensuring that no data point is misclassified. This is a convex optimization problem that can be solved using quadratic programming. If the data points are not linearly separable, we can use a technique called kernel trick to map the data points into a higher-dimensional space where they become separable. The kernel function computes the inner product between the mapped data points without computing the mapping itself. This allows us to work with the data points in the higherdimensional space without incurring the computational cost of mapping them. + +1. Hyperplane: +There can be multiple lines/decision boundaries to segregate the classes in n-dimensional space, but we need to find out the best decision boundary that helps to classify the data points. This best boundary is known as the hyperplane of SVM. +The dimensions of the hyperplane depend on the features present in the dataset, which means if there are 2 features, then hyperplane will be a straight line. And if there are 3 features, then hyperplane will be a 2-dimension plane. We always create a hyperplane that has a maximum margin, which means the maximum distance between the data points. +2. Support Vectors: +The data points or vectors that are the closest to the hyperplane and which affect the position of the hyperplane are termed as Support Vector. Since these vectors support the hyperplane, hence called a Support vector. +3. Margin: +It may be defined as the gap between two lines on the closet data points of different classes. It can be calculated as the perpendicular distance from the line to the support vectors. Large margin is considered as a good margin and small margin is considered as a bad margin. + +We will use the famous Iris dataset, which contains the sepal length, sepal width, petal length, and petal width of three species of iris flowers: Iris setosa, Iris versicolor, and Iris virginica. The goal is to classify the flowers into their respective species based on these four features. We load the iris dataset using load_iris and split the data into training and testing sets using train_test_split. We use a test size of 0.2, which means that 20% of the data will be used for testing and 80% for training. We set the random state to 42 to ensure reproducibility of the results. + +# Implemetation of SVM in Python + +from sklearn.datasets import load_iris +from sklearn.model_selection import train_test_split +from sklearn.svm import SVC +from sklearn.metrics import accuracy_score + +# load the iris dataset +iris = load_iris() + +# split the data into training and testing sets +X_train, X_test, y_train, y_test = train_test_split(iris.data, +iris.target, test_size=0.2, random_state=42) + +# create an SVM classifier with a linear kernel +svm = SVC(kernel='linear') + +# train the SVM classifier on the training set +svm.fit(X_train, y_train) + +# make predictions on the testing set +y_pred = svm.predict(X_test) + +# calculate the accuracy of the classifier +accuracy = accuracy_score(y_test, y_pred) +print("Accuracy:", accuracy) + +# Output +Accuracy: 1 + + From 4d86a0af83d35990a71144e88c8979e5146529e8 Mon Sep 17 00:00:00 2001 From: Ankit Mahato Date: Thu, 23 May 2024 04:15:56 +0530 Subject: [PATCH 7/9] Update index.md --- contrib/machine-learning/index.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/contrib/machine-learning/index.md b/contrib/machine-learning/index.md index 40a2237..44b39d4 100644 --- a/contrib/machine-learning/index.md +++ b/contrib/machine-learning/index.md @@ -1,4 +1,6 @@ # List of sections +- [Section title](filename.md) - [Confusion Matrix](confusion-matrix.md) - [Support Vector Machine Algorithm](support-vector-machine.md) + From 428bd62cd219830b77dde615f5a0efb3432afbb1 Mon Sep 17 00:00:00 2001 From: Ankit Mahato Date: Thu, 23 May 2024 04:20:12 +0530 Subject: [PATCH 8/9] Update confusion-matrix.md --- contrib/machine-learning/confusion-matrix.md | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/contrib/machine-learning/confusion-matrix.md b/contrib/machine-learning/confusion-matrix.md index ac5cd31..4bedf66 100644 --- a/contrib/machine-learning/confusion-matrix.md +++ b/contrib/machine-learning/confusion-matrix.md @@ -1,4 +1,6 @@ -Confusion Matrix - A confusion matrix is a fundamental performance evaluation tool used in machine learning to assess the accuracy of a classification model. It is an N x N matrix, where N represents the number of target classes. +## Confusion Matrix + +A confusion matrix is a fundamental performance evaluation tool used in machine learning to assess the accuracy of a classification model. It is an N x N matrix, where N represents the number of target classes. For binary classification, it results in a 2 x 2 matrix that outlines four key parameters: 1. True Positive (TP) - The predicted value matches the actual value, or the predicted class matches the actual class. @@ -20,15 +22,18 @@ To implement the confusion matrix in Python, we can use the confusion_matrix() f The function returns a 2D array that represents the confusion matrix. We can also visualize the confusion matrix using a heatmap. +```python # Import necessary libraries import numpy as np -from sklearn.metrics import confusion_matrix,classification_report +from sklearn.metrics import confusion_matrix, classification_report import seaborn as sns import matplotlib.pyplot as plt # Create the NumPy array for actual and predicted labels -actual = np.array(['Apple', 'Apple', 'Apple', 'Not Apple', 'Apple', 'Not Apple', 'Apple', 'Apple', 'Not Apple', 'Not Apple']) -predicted = np.array(['Apple', 'Not Apple', 'Apple', 'Not Apple', 'Apple', 'Apple', 'Apple', 'Apple', 'Not Apple', 'Not Apple']) +actual = np.array(['Apple', 'Apple', 'Apple', 'Not Apple', 'Apple', + 'Not Apple', 'Apple', 'Apple', 'Not Apple', 'Not Apple']) +predicted = np.array(['Apple', 'Not Apple', 'Apple', 'Not Apple', 'Apple', + 'Apple', 'Apple', 'Apple', 'Not Apple', 'Not Apple']) # Compute the confusion matrix cm = confusion_matrix(actual,predicted) @@ -46,8 +51,11 @@ plt.show() # Classifications Report based on Confusion Metrics print(classification_report(actual, predicted)) +``` -# Results +### Results + +``` 1. Confusion Matrix: [[5 1] [1 3]] @@ -59,3 +67,4 @@ Not Apple 0.75 0.75 0.75 4 accuracy 0.80 10 macro avg 0.79 0.79 0.79 10 weighted avg 0.80 0.80 0.80 10 +``` From 34131322a2753f32d44630e1c127a6716b099fca Mon Sep 17 00:00:00 2001 From: Ankit Mahato Date: Thu, 23 May 2024 04:22:17 +0530 Subject: [PATCH 9/9] Update support-vector-machine.md --- .../support-vector-machine.md | 20 ++++++++++++++----- 1 file changed, 15 insertions(+), 5 deletions(-) diff --git a/contrib/machine-learning/support-vector-machine.md b/contrib/machine-learning/support-vector-machine.md index 90ee922..0117e9f 100644 --- a/contrib/machine-learning/support-vector-machine.md +++ b/contrib/machine-learning/support-vector-machine.md @@ -1,12 +1,19 @@ +## Support Vector Machine + Support Vector Machine or SVM is one of the most popular Supervised Learning algorithms, which is used for Classification as well as Regression problems. However, primarily, it is used for Classification problems in Machine Learning. + SVM can be of two types - 1. Linear SVM: Linear SVM is used for linearly separable data, which means if a dataset can be classified into two classes by using a single straight line, then such data is termed as linearly separable data, and classifier is used called as Linear SVM classifier. 2. Non-linear SVM: Non-Linear SVM is used for non-linearly separated data, which means if a dataset cannot be classified by using a straight line, then such data is termed as non-linear data and classifier used is called as Non-linear SVM classifier. -Working of SVM - The goal of SVM is to find a hyperplane that separates the data points into different classes. A hyperplane is a line in 2D space, a plane in 3D space, or a higher-dimensional surface in n-dimensional space. The hyperplane is chosen in such a way that it maximizes the margin, which is the distance between the hyperplane and the closest data points of each class. The closest data points are called the support vectors. -The distance between the hyperplane and a data point "x" can be calculated using the formula − +Working of SVM - The goal of SVM is to find a hyperplane that separates the data points into different classes. A hyperplane is a line in 2D space, a plane in 3D space, or a higher-dimensional surface in n-dimensional space. The hyperplane is chosen in such a way that it maximizes the margin, which is the distance between the hyperplane and the closest data points of each class. The closest data points are called the support vectors. + +The distance between the hyperplane and a data point "x" can be calculated using the formula − +``` distance = (w . x + b) / ||w|| +``` where "w" is the weight vector, "b" is the bias term, and "||w||" is the Euclidean norm of the weight vector. The weight vector "w" is perpendicular to the hyperplane and determines its orientation, while the bias term "b" determines its position. + The optimal hyperplane is found by solving an optimization problem, which is to maximize the margin subject to the constraint that all data points are correctly classified. In other words, we want to find the hyperplane that maximizes the margin between the two classes while ensuring that no data point is misclassified. This is a convex optimization problem that can be solved using quadratic programming. If the data points are not linearly separable, we can use a technique called kernel trick to map the data points into a higher-dimensional space where they become separable. The kernel function computes the inner product between the mapped data points without computing the mapping itself. This allows us to work with the data points in the higherdimensional space without incurring the computational cost of mapping them. 1. Hyperplane: @@ -19,8 +26,9 @@ It may be defined as the gap between two lines on the closet data points of diff We will use the famous Iris dataset, which contains the sepal length, sepal width, petal length, and petal width of three species of iris flowers: Iris setosa, Iris versicolor, and Iris virginica. The goal is to classify the flowers into their respective species based on these four features. We load the iris dataset using load_iris and split the data into training and testing sets using train_test_split. We use a test size of 0.2, which means that 20% of the data will be used for testing and 80% for training. We set the random state to 42 to ensure reproducibility of the results. -# Implemetation of SVM in Python +### Implemetation of SVM in Python +```python from sklearn.datasets import load_iris from sklearn.model_selection import train_test_split from sklearn.svm import SVC @@ -45,8 +53,10 @@ y_pred = svm.predict(X_test) # calculate the accuracy of the classifier accuracy = accuracy_score(y_test, y_pred) print("Accuracy:", accuracy) +``` -# Output +#### Output +``` Accuracy: 1 - +```