A collection or series of simple Python Scripts to train and test MLP Neural Nets, with various synthetic data (equations, such as y = np.sin(np.pi * X1) + np.square(X2)).
Model Hyperparamters are changeable (except the number of inputs)
Some versions use TORCH.NN.
Some Versions use ONLY-NUMPY for easier modification.
Some scripts save/load checkpoint files.
The loss is printed periodically to the CMD console.
All version generally use graphics at end of train and test to display the tranined model's performance.
Python Scripts for Training and Inference/Test of a MLP (using TORCH.NN library features)
y = np.sin(np.pi * X1) + np.square(X2) # Example target function. Hardcoded but equation y= can be user-modified in the script.
Or, you can comment out the second part and use sin() only: y = np.sin(np.pi * X1) #+ np.square(X2) # Example target function
The provided code indeed implements a standard feedforward neural network layer, commonly known as a perceptron layer, rather than a Knot Algebra Network (MLP) layer. Let's clarify the difference and discuss how a MLP layer might be implemented:
Perceptron Layer:
The code defines a single or multi layer in a feedforward neural network. Each hidden layer consists of a linear transformation followed by a non-linear activation function (ReLU in this case).
This type of layer is often used in traditional neural network architectures for tasks such as classification and regression.
Sometimes the loss falls to Epoch [1000/1000], Loss: 0.0006 or Epoch [1000/1000], Loss: 0.0009 or Epoch [1000/1000], Loss: 0.0021
But other times Epoch [1000/1000], Loss: 0.0614 (looks like a horizontal spread).
Having fixed the input train data for evaluation, I infer that the randomness of the model results seems to be caused by the random weights and bias at initialization?
Explanation
MLP Layer Definition:
A custom neural network layer MLPLayer is created, which includes a linear transformation followed by an activation function (ReLU in this example).
The HYPERPARAMETERS are user-specified in the MAIN function such as: input_dim = 2 (must be 2 in this version? because of how the train data and text data is stored, but you can still comment out the second variable part of the y function and use sin(X1) only: y = np.sin(np.pi * X1) #+ np.square(X2) # Example target function) output_dim = 1 hidden_layers = [5, 5] # Example hidden layers X_train, y_train = generate_train_data(200) X_test, y_test = generate_test_data(100)
MLP Model Definition:
The MLP class defines the network architecture with a sequence of MLP_Layer objects. The final layer is a linear layer to produce the output.
Data Preparation:
The generate_data function creates example input data and corresponding target values based on a specified function.
Training Function:
The train_MLP function trains the MLP model using mean squared error loss and the Adam optimizer. Training runs for a specified number of epochs with periodic loss reporting.
Inference Function:
The inference_MLP function generates predictions using the trained MLP model on new test data.
Main Script:
This part initializes the model, criterion, optimizer, and data. It then trains the model and performs inference, finally plotting the true vs. predicted values for visualization.
This script encapsulates the basic workflow for using MLPs in a machine learning context. It can be adapted to different datasets and network configurations depending on the specific requirements of the task.
A custom neural network layer MLP_Layer is created, which includes a linear transformation followed by an activation function (ReLU in this example).
ChatGPT The lines that define the custom neural network layer MLPLayer, which includes a linear transformation followed by an activation function (ReLU in this example), are: class MLPLayer(nn.Module):
Line defining the class: class MLPLayer(nn.Module): Initialization method: def init(self, in_features, out_features): Calling the superclass constructor: super(MLPLayer, self).init() Defining the linear layer: self.linear = nn.Linear(in_features, out_features) Defining the activation function: self.activation = nn.ReLU() Forward method: def forward(self, x): Applying the linear transformation and activation in the forward method: return self.activation(self.linear(x))
Data preparation (example data) function encapsulates the process of generating synthetic example data for training and testing the MLP model, providing input-output pairs that can be used to train and evaluate the network's performance.
def generate_data(n_samples): print("Data preparation (example data)") X = np.random.rand(n_samples, 2) # Example: 2 input features #n_samples samples are generated, each with 2 input features. These features are randomly distributed between 0 and 1. y = np.sin(np.pi * X[:, 0]) + np.square(X[:, 1]) # Example target function # target values are calculated using a specific function. In this case, it's a combination of the sine of the first input feature multiplied by π and the square of the second input feature. This function is an example and can be replaced with any desired function depending on the problem domain. return torch.tensor(X, dtype=torch.float32), torch.tensor(y, dtype=torch.float32) #This line converts the generated NumPy arrays X and y into PyTorch tensors. PyTorch tensors are the preferred data type for working with neural networks in PyTorch. Both X and y are converted to torch.float32 data type for consistency.
The train data and test data will be saved in the text files X1.txt and X2.txt Separating the input data, saving it to file for later re-use (comment out the un-used code lines): def generate_data(n_samples): # Generate random data X1 = np.random.rand(n_samples, 1) # Example: 1st input feature X2 = np.random.rand(n_samples, 1) # Example: 2nd input feature
# Save data to CSV files with commas as delimiters
np.savetxt('X1.txt', X1, delimiter=',')
np.savetxt('X2.txt', X2, delimiter=',')
# Read data back into variables X1 and X2
X1 = np.loadtxt('X1.txt', delimiter=',').reshape(-1, 1)
X2 = np.loadtxt('X2.txt', delimiter=',').reshape(-1, 1)
# Print loaded data for verification
print("Loaded X1:", X1)
print("Loaded X2:", X2)
# Generate y based on X1 and X2
y = np.sin(np.pi * X1) + np.square(X2) # Example target function
return torch.tensor(np.concatenate((X1, X2), axis=1), dtype=torch.float32), torch.tensor(y, dtype=torch.float32)
The train data and test data will be saved in the text files X1.txt and X2.txt in follwing format:
Read good random test data into variables X1 and X2
loaded 100 test data X1: [[0.61322319] [0.11602832] [0.48507701] [0.92841799] ...
Read good random test data into variables X1 and X2
loaded 100 test data X1: [[0.61322319] [0.11602832] [0.48507701] [0.92841799] ...
Read good random test data into variables X1 and X2
loaded 100 test data X1: [[0.61322319] [0.11602832] [0.48507701] [0.92841799] ...