havBpNet:J

Intro    Network Classes Overview    Data Classes Overview    Packages

havBpNet:J Home Page

Data Scaling

As a final note on using havBpNet, we would like to briefly cover the scaling and un-scaling of input-, expected-, and output-data values.

Neural networks are used for two basic types of problems: pattern recognition/matching and function approximation. In the case of pattern recognition/matching, the network’s output is most often constrained to a predetermined range or set of finite values. In the case of function approximation, the network’s response is often continuous (perhaps over a finite range). In either case, it is often desirable to scale the input pattern’s, expected value pattern’s, and output pattern’s individual data values to some smaller range.

Since most data scaling required can easily be implemented using standard run-time library routines supplied with all compilers, havBpNet does not provide integrated data-scaling functionality. It is left to the user application to perform any scaling deemed appropriate for the application. As an efficiency consideration, we do suggest that all input and expected values be scaled prior to entering the training loop of a neural network trainer application.

havBpNet does provide two functions, havlog(x) and havexp(x), that can be used when data ranges include both positive and negative values.

Suggested Approaches to Data Scaling

Since we have not found any definitive text covering the topic of data scaling for neural networks, we offer the following suggestions concerning the scaling of input, expected, and output values. These are several approaches that we have used successfully in our own applications.

If an application is not required to deal with high resolution data, and if the range of data values is not too large, simple linear scaling will often suffice. Input and expected values are scaled to some relatively small range (for example: (0 to 1) or (-1 to 1)) and output values are scaled from this “squashed” range up to the original data range. The run-time libraries supplied with all compilers should include a scale(...) function that satisfactorily performs linear scaling.

Other applications involve the use of high resolution and continuously-valued pattern element values. Often, linear scaling is not sufficient for these applications due to loss of resolution. For these types of applications, we have used several approaches to data value scaling.

Whenever possible, we use the log(x) function to squash our input and expected values and the exp(x) function to un-squash our output values. The log(x) and exp(x) functions are commonly provided with all run-time libraries. This usually works well where the data values are all greater than zero.

When an application’s value range includes both positive and negative values, we have used a variation of the mu-law. For your convenience, havBpNet includes implementations of two functions, havlog(x,f=1.0) and havexp(x,f=1.0), that you may use for squashing and un-squashing data. These functions are described below ...

Another approach that we have used in dealing with ranges that include both positive and negative values is to simply squash with the tanh(x) and un-squash with the atanh(x) functions.

Auxiliary classes - havTdata & havTpat

The auxiliary data-handling classes supplied with havBpNet offer several additional capabilities in terms of data scaling.

One method will calculate a mean & standard-deviation for each input-pattern and/or expected pattern column (imagine the collection of patterns arranged as a table with each row being a pattern and columns representing elements of the patterns). Each pattern element is squashed by subtracting the appropriate mean and dividing by the appropriate standard-deviation. This approach yields a distribution centered about zero.

The second method will perform vector-normalization on each pattern individually, thus scaling all patterns to be unit vectors. This is accomplished by dividing each pattern element by the square-root of the sum of the squared elements in the pattern. That is, for a pattern with elements (e1, e2, e3), each element en(=e1,e2,e3) is squashed by...

Additional facility is provided which allows either of these two methods to be combined with use of havlog.

As an additional convenience, these auxiliary classes also provide a mechanism for calculating and (optionally) printing the Percent Absolute Error using un-scaled pattern values.

Intro    Network Classes Overview    Data Classes Overview    Packages

Copyright © 1998 by hav.Software. All Rights Reserved.