Latest version 

How to manage scores

Introduction

There are many potential scoring algorithms, and many things we would like to do with the scores they output. This page looks into the question of how to represent scores, in order that they flow easily into later stages of analysis.

General character of scores

Scores can be trivial — average heart rate within an epoch is a single simple value — but that is the exception. Generally the output from a scoring algorithm will comprise multiple elements:

• point scores: e.g. latency and amplitude of a component
• band scores: e.g. alpha power within a given frequency range
• non-localized values: e.g. χ² for a fit
• control parameters: e.g. threshold for EDA responses
• series: e.g. ICA components
• matrices: e.g. cross correlations

Requirements of a good representation

The principal requirement is that the score algorithm pass everything salient to subsequent analysis stages. This sounds easy: just bundle all values into an object. However this could easily appear as a jumble of anonymous values. Instead we would like the scores to be usable by later visualization and analysis stages. We want those later processes to be able to digest the scores, without having any specific knowledge of the algorithm that generated the values. This is the central problem.

A partial solution is to bundle tailored methods with the scores, which explicate the relationship between the values. For example, the object representing a traditional set of peak amplitudes and latencies might store the values and have methods that return the labels, latencies and amplitudes as (a) a simple tabulation, (b) a JTree, (c) an XML version, and (d) a set of SQL INSERT commands. This is the natural OO way to encapsulate data and function, and largely allows later anaysis stages to delegate score-specific options to the score package itself.

However this is not always enough.

Visualizing of the data

If we wish to show ERP peak scores on a timeseries plot, given the score object generated by the scoring algorithm, then we need to identify and extract all relevant labels, latencies and amplitudes, and we need to do this in a way that is generic to all scores.

It is easy with reflection to access all score value, but it is hard to infer their relations, wherein resides their full meaning. A list of latencies is no use unless matched with labels ('N1', 'P2' etc). It is even harder (to labour the point) to establish these relations in a way that is generic.

To deal with this problem it is helpful to tabulate typical relationships amongst variable.

Note that those combinations that are relevant to visualization typically involve a latency (or frequency) coupled with a label (or numerical value), which together define what should be added to a plot. [A range of t or f is also possible, as is an array of values (as in waveform fits or decompositions).]

If scores matching particular combinations of values in the table can be flagged somehow, then a plotting option has a good chance of adding them to the plots; and in a way that demands no understanding of their meaning.

How to 'flag' scores as being potentially plottable?

1. Define interfaces corresponding to each valid combination involving latencies or frequencies. The score object can then be searched for any of the 14 or so possible interfaces. If any are encountered, then the corresponding values can be extracted by predefined get methods, and passed to a suitable graphical method for rendering.
2. Annotations can be defined, and attached to all variables representing a latency (or frequency) point (or range). Just four annotations would be required. Each would point to the associated label or numeric value.

Common data structures

Here are various score types, and a possible representation of the values. They are intended as a reminder of the degree of generality that is required when designing a generic representation for scores.

Actually there are elaborations to consider. For a proper audit trail each score should include the version number of the scoring algorithm. Some scores may contain manually-determined values as well as values determined by the algorithm.

Reaction time

    String label = "RT";
    float latency = 0.334f;     // s

Electrodermal response

    float onset = 0.743f;       // s
    float peakLatency = 5.334f; // s
    float amplitude = 0.423f;   // μS

Heart rate

    float rate = 78.2f;     // bpm

Traditional ERP components

    LabelLatVal[4] components = {//    label    t    amp
                        new LabelLatVal("N1", 0.12, -3.2),
                        new LabelLatVal("P2", 0.18,  1.2),
                        new LabelLatVal("N2", 0.22, -1.0),
                        new LabelLatVal("P3", 0.35,  9.7)};

ICA decomposition

    Component[3] components = {//   t0   t1     f(t)       wt
                     new Component(0.0, 0.6, waveform0, weight0),
                     new Component(0.0, 0.6, waveform1, weight1),
                     new Component(0.0, 0.6, waveform2, weight2)};

Traditional band powers

    LabelFreqRangeVal[4] components = {//        f0    f1   pow
                 new LabelFreqRangeVal("Delta", 0.5,  3.0, 122.2),
                 new LabelFreqRangeVal("Theta", 3.0,  8.0,  62.3),
                 new LabelFreqRangeVal("Alpha", 8.0, 13.0,  10.7),
                 new LabelFreqRangeVal("Beta", 13.0, 30.0,   2.4)};

EEG fit

    FitParam[4] components = {//        init   fit    SD  
                 new FitParam("Gamma", 100.0, 165.3, 23.3),
                 new FitParam("Alpha",  50.0,  65.3, 37.2),
                 new FitParam("Gesre",  -8.0,  -6.3,  2.7),
                 new FitParam("Gii",    -5.0,  -3.4,  1.4)};
    float chisq = 233.2;
    int nIterations = 43;
    float[] fittedFreqs = modelFreqs;
    float[] fittedSpectrum = modelSpect;

Implementation

The data structures shown in Common data structures exemplify the rich mix of interrelated values. Some are a simple latencies with a label (e.g. reaction time), others are a value associated with a label and range of frequencies (e.g. band powers), others involve a function (e.g. EEG fits). Such values are potentially plottable, so they must be flagged in some way. [Other values in the above data structures (e.g. chisq) are not of relevance to plotting, so can be ignored here.]

We use annotations to deal with all combinations of values, that are likely to be plotted. Annotations are (in this context) applied to classes. As a result, a score object can be inspected via reflection for any annotated objects, and if found, those annotated objects can yield the plottable points, ranges of values, and labels. The annotations simply list the names of all relevant variables within the annotated object. An example follows.

import java.io.*;
import java.util.*;
import scoreClasses.annotations.*;

@LatencyPoint(xValue = "latency", yString = "sValue")
public class LatPointSValue implements Cloneable
{
    private float lat;
    private String label;
    private String whichSide;
    
    public LatPointSValue(float lat, String label) {
        this.lat = lat;
        this.label = label;
    } // LatPointSValue

    public float getLatency() {return lat;}
    public String getSValue() {return label;}
    public String toString() {return label+": "+lat+" s ("+whichSide+")";
}

Note how the name LatencyPoint implies that that this is a time-domain object, so that there is a latency value, and that value may have some associated float and/or String. What is actually specifed in the annotation is that the x-value is identified by "latency" so is accessible by "getLatency()"; and calling that function returns the variable "lat". The process is indirect — but has great generality, and doesn't interfere with the main task of the class.

See the source code for more complex and interesting examples of annotations applied to scores.