Neural network training library written in C++ (also available in C#) as a learning project. Also available as a nuget package.

Recognizes digits between 0 and 9. Trained on the MNIST handwritten digit dataset.

Uses evolutional selection to evolve the neural network in the racecars, so they learn how to drive around racetracks.

• GPU accelerated Backpropagation (using OpenCL)
• Cost functions:
  • Mean-Squared
  • Cross-entropy
• Activation functions:
  • Sigmoid, ReLU, LeakyReLU, Tanh, Identity, Threshold, SoftPlus, ArcTan
• L1 and L2 regularization
• Xavier and He weight initialization
• Stochastic gradient descent with shuffled mini-batches

var layerConfig = new int[]{ 784, 32, 32, 10 };
var network = Network.CreateNetworkInitRandom( layerConfig, new SigmoidActivation() );

string jsonData = "{ ... }";
var network = Network.CreateNetworkFromJSON( jsonData ); 

string json = network.ExportToJSON();

float[] input = { ... };
ComputeDevice computeDevice = ComputeDeviceFactory.GetFirstOpenCLDevice(); //Get an OpenCL device
float[] results = network.Compute( input, computeDevice );

ComputeDevice computeDevice = ComputeDeviceFactory.GetFirstOpenCLDevice(); //Get an OpenCL device
List<TrainingSuite.TrainingData> trainingData = new List<TrainingSuite.TrainingData>();

//Set up training examples
for (int i = 0; i < 10000; ++i)
{
    float[] trainingInput = ...;
    float[] desiredOutput = ...;
    trainingData.Add( new TrainingSuite.TrainingData( trainingInput, desiredOutput ) );
}

TrainingSuite trainingSuite = new TrainingSuite( trainingData );

//Set up training configuration
trainingSuite.config.epochs = 100;
trainingSuite.config.shuffleTrainingData = true;
trainingSuite.config.miniBatchSize = 50;
trainingSuite.config.learningRate = 0.005f;
trainingSuite.config.costFunction = new CrossEntropyErrorFunction();
trainingSuite.config.regularization = TrainingSuite.TrainingConfig.Regularization.L2;
trainingSuite.config.regularizationLambda = 0.5f;

var trainingPromise = network.Train( trainingSuite, computeDevice );

trainingPromise.Await();

The neural network library

Part of the following library: https://github.com/lecode-official/opencl-dotnet. Used for OpenCL access from .net Core

Tests for the library

A console command line for testing

A sample app that can load the MNIST digit dataset and trains a network.

A test app for development with no specific purpose but to try our some features using a ui.

An HTML web application that uses an already trained network to recognize a drawn number.

An HTML web application that uses neural networks and an evolutional algorithm for training racecars to learn driving on various racetracks.

The following dependencies are set up in nuget:

• Newtonsoft.Json

This project also includes parts of opencl-dotnet: https://github.com/lecode-official/opencl-dotnet

• Optimize memory layout (to reduce number of buffer copies)
• New features: Dropout, Softmax layers
• More activation functions
• Convolutional layer
• LSTM networks

These are some nice resources for learning about Neural Networks:

• Michael A. Nielsen, Neural Networks and Deep Learning
• 3Blue1Brown's But what is a neural network?