Overview of havBpNet-J - Creating a Network

havBpNet:J

Intro    Network Classes Overview    Data Classes Overview    Packages

havBpNet:J Home Page

Creating a Network

Let’s first consider creating a new network for training.

In order to create a network, an application must create a nodelist and several layers, and then connect the layers as required. The following code excerpt illustrates these steps by creating a simple three-layer network with bias and each layer connected as input to the next layer in the net (i.e. IN -> HID -> OUT).


//============== create the nodelist and layers
havBpNodeList net = new havBpNodeList();  
havBpLayer bias = new havBpLayer(net, bias_size, havSys.havINPUT, havSys.havLINEAR);
havBpLayer in = new havBpLayer(net, inputsize, havSys.havINPUT, havSys.havLINEAR);
havBpLayer hid = new havBpLayer(net, hidsize, havSys.havHIDDEN, havSys.havHTAN);
havBpLayer out = new havBpLayer(net, outsize, havSys.havOUTPUT, havSys.havLINEAR);

//============== connect the layers
bias.Connect(hid);   // bias both the hidden and output layers
bias.Connect(out);
in.Connect(hid);
hid.Connect(out);

Our example network now exists.

Notice in particular the definition of the bias layer. havBpNet does not automatically create bias nodes for layers, so, if you want a bias node, you are responsible for creating it. Since havBpNet encourages the use of layers, a bias node is most easily implemented as a bias layer with one node.

We still need to set the value of the bias layer’s node (node 0). In this case we will use a bias value of 1.0 as shown below...


bias.PutValue(0, 1.0);  // establish the bias value (ie node 0 of the bias layer)

Saving layers (and nodes) is discussed later in this chapter, but it is worth noting here that a node’s value is not saved when the node is saved; so, when restoring a network’s bias layer, you must reassign its value by reissuing the PutValue(...) message to the bias layer.

Notice that both the in and bias layers have been defined as being input layers. Any number of layers may be defined as type input, hidden, or output.

Also, notice that the transfer-function of the hidden layer has been defined as the non-linear hyperbolic-tangent which will produce values in the range -1 to 1.

When a layer is created, its nodes are also created. Node creation is performed automatically by havBpNet with the type of node based on the indicated transfer function of the layer. In the example above, the nodes created for the bias, input and output layers will be of type havBpLinearNode, whereas, the nodes created for the hidden layer will be of type havBpHtanNode.

Finally, notice that no error function is specifically defined for the output layer. This form of the layer creator will use the default layer error function: squared error. An alternate constructor is defined which allows you to use one of the other error functions: havSys.havCUBE or havSys.havQUAD.

Once the layers exist, they may be manipulated as needed. For example, we can establish the learning rate and momentum factor of the hidden and output layers as follows...


hid.PutBeta(hid_beta);    // assign learning rates
out.PutBeta(out_beta);
hid.PutMomentum(hid_mu);  // assign learning momentum
out.PutMomentum(out_mu);

Since the input and bias layers do not have any input weights to be trained, there is no need to set the learning rate or momentum of these layers.

Before examining a training example, let’s cover several details and extensions not covered above.

First, notice that layer creation requires that an instance of havBpNodeList be specified for the layer. The nodelist specified must exist prior to layer creation. All layers that will be explicitly connected using the Connect(...) method must specify the same instance of havBpNodeList in their creation.

Notice that the Connect(...) method tells a layer to connect itself as input to a layer that it will feed. A layer may feed more than one layer, in which case there would be several Connect(...) messages sent to the layer as shown for the bias layer above. Also, a layer may be fed by more than one layer.

A layer may be connected to itself in order to accomplish recurrency; however, we suggest that you use a network of the form described by Jordan(1986) and Elman (1988).

User applications need not specify backward connections used for error back-propagation. These are assumed to be the reverse of the feed-forward connections.

Overview || Create || Train || Save || Restore || Consult || Data Scaling

Intro    Network Classes Overview    Data Classes Overview    Packages