Added Neural Network Regression in Python

contrib/machine-learning/Neural network regression.md

@@ -0,0 +1,98 @@
+# Neural Network Regression in Python
+
+## Overview
+
+Neural Network Regression is a type of machine learning algorithm used to predict continuous values. Unlike classification, where the goal is to predict a category or class, regression aims to predict a specific numerical value. Neural networks are particularly powerful for regression tasks when dealing with complex, non-linear relationships in data.
+
+## When to Use Neural Network Regression
+
+### Suitable Scenarios
+
+1. **Complex Relationships**: When the relationship between input features and the target variable is non-linear and complex.
+2. **Large Datasets**: When you have a large dataset that can support the training of a neural network.
+3. **Feature Engineering**: When you can leverage the feature extraction capabilities of neural networks, especially in domains like image or text data.
+
+### Unsuitable Scenarios
+
+1. **Small Datasets**: Neural networks require substantial amounts of data to train effectively. For small datasets, simpler models like linear regression or decision trees might perform better.
+2. **Real-time Predictions**: If low-latency predictions are required and computational resources are limited, simpler models might be more efficient.
+3. **Interpretability**: If model interpretability is crucial, neural networks might not be the best choice due to their "black-box" nature.
+
+## Implementing Neural Network Regression in Python
+
+We'll use TensorFlow and Keras, popular libraries for building and training neural networks in Python.
+
+### Step-by-Step Implementation
+
+1. **Import Libraries**
+
+```python
+import numpy as np
+import pandas as pd
+from sklearn.model_selection import train_test_split
+from sklearn.preprocessing import StandardScaler
+from tensorflow.keras.models import Sequential
+from tensorflow.keras.layers import Dense
+from tensorflow.keras.optimizers import Adam
+```
+
+2. **Load and Prepare Data**
+
+For illustration, let's use a synthetic dataset.
+
+```python
+# Generate synthetic data
+np.random.seed(42)
+X = np.random.rand(1000, 3)
+y = X[:, 0] * 3 + X[:, 1] * -2 + X[:, 2] * 0.5 + np.random.randn(1000) * 0.1
+
+# Split the data
+X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
+
+# Standardize the data
+scaler = StandardScaler()
+X_train = scaler.fit_transform(X_train)
+X_test = scaler.transform(X_test)
+```
+
+3. **Build the Neural Network Model**
+
+```python
+model = Sequential()
+model.add(Dense(64, input_dim=X_train.shape[1], activation='relu'))
+model.add(Dense(64, activation='relu'))
+model.add(Dense(1))  # Output layer for regression
+
+model.compile(optimizer=Adam(learning_rate=0.001), loss='mse', metrics=['mae'])
+```
+
+4. **Train the Model**
+
+```python
+history = model.fit(X_train, y_train, epochs=100, batch_size=32, validation_split=0.2)
+```
+
+5. **Evaluate the Model**
+
+```python
+loss, mae = model.evaluate(X_test, y_test)
+print(f"Test Mean Absolute Error: {mae}")
+```
+
+6. **Make Predictions**
+
+```python
+predictions = model.predict(X_test)
+```
+
+### Explanation
+
+- **Data Generation and Preparation**: Synthetic data is created and split into training and test sets. The data is standardized to improve the neural network's training efficiency.
+- **Model Construction**: A simple feedforward neural network is built using Keras. It consists of two hidden layers with 64 neurons each and ReLU activation functions. The output layer has a single neuron for regression.
+- **Training**: The model is trained for 100 epochs with a batch size of 32. The Adam optimizer is used to adjust the weights.
+- **Evaluation**: The model's performance is evaluated on the test set, using Mean Absolute Error (MAE) as a metric.
+- **Prediction**: Predictions are made on the test data.
+
+## Conclusion
+
+Neural Network Regression is a powerful tool for predicting continuous values, particularly in cases involving complex, non-linear relationships. However, they require large datasets and significant computational resources. For smaller datasets or scenarios requiring model interpretability, simpler models might be preferable. By following the steps outlined, you can build, train, and evaluate a neural network for regression tasks in Python using TensorFlow and Keras.

contrib/machine-learning/index.md

@@ -20,3 +20,4 @@
 - [Grid Search](grid-search.md)
 - [Transformers](transformers.md)
 - [K-nearest neighbor (KNN)](knn.md)
+- [Neural network regression](Neural network regression.md)