DeepLearningJS - Neural Networks

Deep Neural Network Classifier

select file

delimiters

Items: {{Items}} Input features: {{Inputs}} Categories: {{Categories}}

Hidden layers

Layer {{$index + 1}} -> {{nodes}}

Nodes

Learning rate (α)

Epochs

Tolerance

Use L2 Error

Use Optimizer

Training Results

Training Error
{{Network.Cost.toExponential(8)}}

L2 Error
{{Network.L2.toExponential(8)}}

Iterations
{{Network.Iterations}}

select file

Test data

Samples: {{Samples}} Input features: {{Inputs}}

Classification

Threshold

Network Weights

Input Normalization

Minimum

Maximum

Render node labels

Width
Height
Node Size

Width
Height

Data

Data is loaded from a text file. To begin, select the file by clicking on the "Browse" button and choose the file you want to process on the dialog window that appears. Select the delimiter type. Once a file is and the delimiter is selected, click on the "Process Data" button to begin the loading and processing of the training data.

Each line in the file is a training example with one or more features followed by its the classification category. Each of these values are separated by a delimiter, e.g. tabs, spaces, commas, etc.

Example 1

0,1,1

In the above example, the line contains 3 values separated by a comma delimiter. This means that the training example has 2 features (0, 1) and its classification category is 1.

Example 2

8.55 26.3 2

Example 2 also has 2 features (8.55, 26.3) and a classification category of 2, separated by spaces

If there are several examples provided in the text file, it will estimate the number of categories. If the text file loading and processing succeeded, the status indicator will look similar to the one below:

Items: 788 Input features: 2 Categories: 7

A failure in the loading and processing is also similarly indicated:

Items: 0 Input features: 0 Categories: 0

Most loading and processing errors that occur are due to the mismatch between the chosen delimiter and the one used by the file.

Training

The Training page is where the neural network is constructed. A neural network is composed of an input layer, one to several hidden layers, and an output layer. The input and output layers are automatically generated based on the data loaded in the previous page. To add a hidden layer, choose the number of input nodes and click on the "add hidden layer" button.

What is a reasonable number of hidden layers and nodes (per layer)?

There is (still) no definitive or standard way to configure a neural network. A neural network with several hidden layers, e.g. > 3, with lots of neurons will generally take longer to train. Since inputs from previous layers are mapped into succeeding layers up to the final output layer, the number of nodes in each intermediate hidden layer should at least be equal to or greater than the number of input features. One resonabily sized hidden layer is often sufficient for most problems but you can also try experimenting until you find which conguration gives the best performance in terms of run times, classification accuracies, etc.

Epoch, Tolerance, Learning Rate, L2 Error

These are all related to the Training's stop condition. Training stops when it has reached the maximum number of iterations (Epoch) or if the error or cost function is below the Tolerance level. Learning rate sets the speed at which the network "learns" or adapts each iteration.

Two types of errors are computed, the cross-entropy error and the mean square error or simply, the L2 error. This web application uses the L2 by default.

Optimizer

Access to a Nonlinear Conjugate Gradient optimizer is also provided. This is enabled by clicking on the "Use Optimizer" check box. When using an optimizer, the training generally converges much faster. However, at the moment the optimizer uses the cross-entropy error function regardless of whether L2 error is selected or not.

Classify

Test data is also loaded from a text file similarly. Select the file by clicking on the "Browse" button and choose the file you want to process on the dialog window that appears. Click on the "Process data" to load the test data. Similar success or failure indicators are also shown based on the results of the loading process. Set the classification threshold and click on the "Classify" button to classify the loaded test data. The threshold value indicates the minimum prediction score required to classify one sample into a specific category.

Network

Trained neural network parameters are viewable in this page. You can also save or load these trained network parameters to/from a JSON file. This JSON file is fully compatible with the Deep Neural Network Classifier software and can be used interchangeably with that program.

Graph

View the current neural network architecture or save it as a scalable vector graphics (SVG) file.

Plot

Plot the classified test data or save it as a scalable vector graphics (SVG) file.