havBpNet-J Data Scaling

havBpNet:J

Intro    Network Classes Overview    Data Classes Overview    Packages

havBpNet:J Home Page

Normalization Methods available in the havTdata class

Now that the training and test patterns have all been read into our application and optionally scaled with havlog, we must decide whether or not any additional normalization is required/desired. The havTdata class offers two methods of normalizing data.

Normalization using Mean & Standard-deviation

The first method of normalization will calculate the mean and std-dev for each element column and use these to perform additional scaling. To understand this normalization, imagine that your collection of patterns exists as a 2D table where each row is one pattern (input- and expected-elements) and each column is an element in either an input- or expected- pattern. The program will calculate one mean and one std-dev for each column. Once these pairs have been calculated, each element in each pattern will be modified by subtracting its related mean and dividing the resulting value by its related std-dev.

This sort of scaling may be accomplished as shown below.


//============= apply M&S normalization

    train_data.CalculateParms();      // calculate M&S for train Pat & Expect
    train_data.Normalize();           // apply train M&S to pat * expect

    test_data.CopyParms(train_data);  // copy M&S from taining data set
    test_data.Normalize();            // apply M&S for test data set

Notice that we have normalized the test data set with the mean & std-dev values calculated for the training set.

Also, the above example illustrates how we would perform normalization of both input- and expected-pattern parts of out data sets. When a data set is created, it creates a default set of M&S values. All default means are 0.0 and all default std-dev’s are 1.0. In this way, we can issue Normalize() messages even if we have not calculated new M&S values. Furthermore, havTdata provides several additional methods for calculation and application of M&S values for only the input-pattern part or the expected-pattern part of a data set; however, these abilities will likely not be used very often.

Normalization using standard Vector Normalization

havTdata allows you to normalize each pattern individually by scaling the elements of the input-pattern and/or expected-patterns parts as if they were vectors. Each pattern part is scaled, as a vector, to unit-space. This allows each pattern to retain its shape with respect to other patterns in the data set. Vector normalization is accomplished by calculating a normalization factor for the vector as the square-root of the sum of the squared vector elements - and then dividing each vector element by this factor - as in..

F = sqrt(sum(e[i], i=1,n))

... then, for all elements i=1,n in the pattern ...

e[i] /= F

It is important to consider how the trained network will be used for consultation when choosing to apply Vector normalization. If expected patterns will not be used during consultation, then you should not apply Vector normalization to expected-pattern parts during training. This is because, later, during consultation, there is no way to calculate the factor for un-normalizing the network’s response.

To apply Vector normalization to only the input-pattern parts of the training and test data sets we would use the following code excerpt.


//============= apply vector normalization to input and expected patterns

    train_data.Normalize();  // vector normalize training data
    test_data.Normalize();   // vector normalize test data

havTdata provides additional methods for application of Vector normalization to both the input-pattern and expected-pattern parts or to the expected-pattern part only - but, again, these are not expected to be used very often.

Data Overview || Data Create || Data Scaling || Data Performance
Data Save & Restore || Data Presentation Order

Intro    Network Classes Overview    Data Classes Overview    Packages