The Global CONFIG tab is used to configure and view the settings of neural network training.
1 Editing the project comment
Enter any comment you like using text in Project Description.
2 Specifying the number of epochs until the completion of training optimization
Enter the number of epochs in Max Epoch.
In order to simply output an untrained designed network with the parameter properties (e.g., Weight, Bias) specified in all layers that contain some parameters, set Max Epoch to 0.
3 Specifying the conditions for saving a neural network from the training results
In order to save the neural network of the last epoch, clear the Save Best check box. To save the neural network of the Epoch with the smallest error in the validation dataset, check the Save Best check box.
4 Specifying the batch size to use when optimizing using a mini-batch
Enter the number of data samples to use for each parameter update in Batch Size.
Notes
The batch size value must be set to a value less than the number of data samples contained in each dataset used for training.
5 Setting the automatic optimization of network structure to on or off
Click the Enable check box under Structure Search to activate or deactivate the automatic optimization of the network structure.
When training is executed with Structure Search set to Enable, an optimization process is performed, in which the network structure is automatically varied so as to find network structures with a higher accuracy and less amount of needed computations.
Notes
When training is executed with Structure Search set to Enable, training continues indefinitely unless the user explicitly stops it (for example by pressing the Stop Training button)
6 Selecting the automatic network structure optimization algorithm
Set “Method” under Structure Search to one of the following two options:
| Option | Description |
| Random | Automatic network structure search using evolutionary computation. Since several network structures are tried, optimization takes longer time. |
| Network Feature + Gaussian Process | Automatic network structure search using network features and Bayesian optimization based on Gaussian processes. This allows better networks to be found more quickly. |
7 Selecting what to optimize in the automatic network structure optimization
Set “Optimize for” under Structure Search to one of the following two options.
| Option | Description |
| Error | Minimize the error rate when searching for the optimum network. Specify this when you would like to search for the network with the highest accuracy, regardless of the amount of computations. |
| Error and Calculation | A network with low error and low amount of computations is searched for. |
Notes
If “Optimize for” is set to Error, we recommend that you limit the search range of the number of multiplications and additions using the method explained below. If you do not specify a limit, the result might become a network with extremely large amount of computations, and optimization may take a long time.
8 Limiting the search range based on the accuracy or the number of multiplications and additions of the validation dataset during automatic network structure optimization
Under Search Range, set the Min (minimum) and Max (maximum) values for Validation (validation dataset error) and Multiply Add (the number of multiplications and additions). If you would like not to specify the minimum and maximum limit values, delete these values so that the boxes are empty.
For example, if you set Validation Max to 0.05 and Multiply Add Max to 2,000,000, the automatic structure search is performed in a range in which the validation dataset error does not exceed 0.05 and the Multiply Add range does not exceed 2,000,000. (The error on the validation dataset may exceed the range in the final result, but training results that exceed the range are discarded).
9 Stopping searches for which performance improvement cannot be expected during automatic network structure optimization
Select the “Early Stopping” check box. Then, set “Time Limit” to the maximum amount of time that will be spent in a single network evaluation in the days:hours:minutes:seconds format (2-digit numbers each). If you do not want to specify the time limit, delete the Time Limit values so that the boxes are empty.
If the Early Stopping check box is selected, the learning curve is compared to the result of past training networks with a structure containing fewer multiplications and additions than the network structure currently being trained. If the number of errors is twice as large or greater in the same epoch, the training is stopped at that point, and the evaluation of the next network structure begins.
If a time limit is specified, when the time in evaluating a single network exceeds the specified time, the training is stopped at that point, and the evaluation of the next network structure begins.
