Overview of havBpNet-J

havBpNet:J

Intro    Network Classes Overview    Data Classes Overview    Packages

havBpNet:J Home Page

Overview

This document covers the basic steps involved in creating, training, saving, restoring and consulting a network using havBpNet. Also provided are some tips on Data Scaling.

Most code excerpts shown in this section are from the XOR sample application, xortrain.java, which is included with havBpNet. While the examples illustrate only the basic steps involved, there are accompanying discussions covering extensions available with havBpNet.

When using havBpNet, you will be writing programs, applets or callable subroutines/ functions, which function as network trainers or network consulters. As an overview, consider the following two pseudo-code descriptions: one for a trainer and one for a consulter. In both cases, the network involved is a simple 3-layer feed-forward model. All functionality provided by havBpNet is shown as (red).

Trainer:


 if (continuing training of an existing network) then
    restore the network;
 else
    create the layers;
    connect the layers to form the network;
 endif
 initialize various control parameters as needed;
 Read in and scale the training data;
 While (stopping_condition is not met) do // begin the main training loop
    if (this is a test pass) then
        disable weight adjustments to the output layer;
        disable weight adjustments to the hidden layer;
        initialize the running test pass error sum to 0;
    endif
    for i=1 to number_of_training_patterns do
        put the ith input pattern into the input layer;
        cycle forward from the input layer to the hidden layer;
        cycle forward from the hidden layer to the output layer;
        put the ith expected value pattern into the output layer;
        train the output layer’s input weights;
        train the hidden layer’s input weights;
        if (this is a test pass) then 
            add the current output layer error to the running test pass error sum;
        endif
    endfor
    adjust parameters (learning rate, momentum, epoch_size, etc.) as needed;
    if (this is a test pass) then
        report the pass results;
        enable weight adjustments to the output layer;
        enable weight adjustments to the hidden layer;
    endif
 endwhile // end of main training loop
 save the trained network;
 stop // end of Trainer

Consulter:


 Restore the network;
 Read in and scale the consultation data; 
    // this may be done pattern-at-a-time inside the main loop 
    // also (see havTdata aux. class)
 if (expected values will be used to determine network performance) then
    disable weight adjustments to the output layer;
 end if
 for i=1 to number_of_test_patterns do // begin main consultation loop
    put the ith input pattern into the input layer;
    cycle forward from the input layer to the hidden layer;
    cycle forward from the hidden layer to the output layer;
    if (expected values will be used to determine network performance) then
        put the ith expected value pattern into the output layer;
        // only need error from the output layer when consulting
        train the output layer’s input weights; 
        retrieve and report the output layer’s error for the ith input pattern;
    end if
    retrieve and report the output layer’s response to the ith input pattern;
 end for // end of consultation loop
 stop // end of Consulter

While these models can be used for most training or consultation applications built using havBpNet, clearly, you might wish to enhance these models by adding code to ask for user defined layer sizes or parameter values, or you might wish to save intermediate states of the network during training, etc. Also, more complex networks would require additional cycle and train commands. You might even wish to retrieve error from layers other than the output layer. havBpNet provides the freedom and means to easily accomplish all of these and more.

Certain steps shown above may be performed in a different order if it makes sense or is easier to do so. For example, you might wish to put the expected values for a pattern into the output layer just after you put the input pattern into the input layer ... or you might wish to adjust certain parameters at some other point in the model. havBpNet provides you with the tools that you will need to create, train, save, and restore networks. havBpNet allows you to determine when parameters should be adjusted, when training should be stopped, when the current network should be saved, etc.

Assuming that you have a trained and saved network, the consultation module shown above could easily be embedded as a subroutine in some existing application. In this case you would probably wish to put the network restoration code in one subroutine/function and the actual network consultation calls in another.

In the following sections, we will look at the actual coding required to use the tools provided by havBpNet.

Overview || Create || Train || Save || Restore || Consult || Data Scaling

Intro    Network Classes Overview    Data Classes Overview    Packages