havBpNet-J Data Error Calculation

havBpNet:J

Intro    Network Classes Overview    Data Classes Overview    Packages

havBpNet:J Home Page

Performance Determination & Reporting using havTpat

Whereas the havBpLayer and havBpNode classes provide mechanisms for calculating various error values (such as squared error for a layer, etc.), the havTpat class offers an additional mechanism for calculating the Percent Absolute Error for a training pattern with an option to print a report for the pattern at the same time. Two methods are provided: PatError(...) and SignPatError(...). Generally speaking, these methods are the same except that SignPatError(...) compares only the sign of expected and actual-response values; whereas, PatError(...) compares their actual values. The SignPatError(...) is provided for those cases where only binary response (on/off) is required.

When called, these methods calculate the Absolute Error as a percent of the expected value. This percent is the (return) value of the method(s). Error is calculated as the sum of the individual component element-errors for a pattern, as in...


Ek = sum(abs((e[k][i] - a[k][i])/e[k][i]) * 100,  i=1,n)

...where   Ek       is the error returned for the kth pattern
           e[k][i]  is the expected value of the ith element in the kth pattern
           a[k][i]  is the actual NN response for the ith element in the kth pattern
           n        is the number of elements in the pattern (i.e. size of output layer)

Error is calculated using un-scaled and un-normalized values so that it is a valid representation of the error in the original pattern-value range. That is, if havlog scaling or M&S normalization has been applied to the training patterns’ expected values, then the expected values and NN responses are un-normalized and un-scaled prior to calculating the error. NOTE: Vector Normalization is not un-applied for error calculation since it would only be a linear transformation.

These methods will most often be used in the test loop of a trainer or in a consultation application. The following excerpt illustrates use.


 static boolean show_training_progress = false; // controls test pass reporting
   .  .  .

 double tot_err;    // various error variables (sums etc.)
 double RMS_err;
   .  .  .

 //============ performance test loop section of main training loop
 tot_err = 0.0;
 RMS_err = 0.0;
 out.SetTraining(havSys.havOFF);        // disable weight updates while testing
 out.PutEpochSize(0);                   // don't use batch error while testing

 for (i=0; i<number_of_patterns; i++) { // begin test loop through test patterns
    pat = test_data.GetPat(i);          // get pnt to ith  test pattern in test data set
    in.PutValue(pat.GetPat());          // put the next pattern in the input layer
    hid.Cycle();                        // cycle forward through network
    out.Cycle();
    out.PutExpected(pat.GetExp());      // Put expected reaponse
    out.CalculateUpdates();             // Calculate overall errors
    if (signError) {
        tot_err += pat.PrintSignPatError(out, show_training_progress);
    } else {
        tot_err += pat.PrintPatError(out, show_training_progress);
    }
    RMS_err += out.GetSquaredError();
 }  

 out.SetTraining(havSys.havON);     // re-enable weight updates 
 out.PutEpochSize(epoch_size);      // and batch error after testing
 //=============== end of test loop through test patterns
   .  .  .

The report option is selected by an optional argument in the call. If selected, then a report line will be printed on stdout showing:

the expected values (un-scaled & un-normalized)
the NN response values (un-scaled & un-normalized)
the individual element errors (as % Abs Error)
the total % Abs Error for the pattern
the RMS error for the pattern - as the sqrt(squared pattern error) in internal value range

Data Overview || Data Create || Data Scaling || Data Performance
Data Save & Restore || Data Presentation Order

Intro    Network Classes Overview    Data Classes Overview    Packages