The validation approach depends on distinguishing measurement from prediction:
Informally we say that a measure is valid if it accurately characterizes the attribute it claims to measure.
- Measures or measurement systems are used to assess an existing entity by numerically characterizing one or more of its attributes.
- Prediction systems are used to predict some attribute of a future entity, involving a mathematical model with associated prediction procedures.
On the other hand, a prediction system is valid if it makes accurate predictions.
Validating a prediction system in a given environment is the process of establishing the accuracy of the prediction system by empirical means; that is by comparing model performance with known data in the given environment, with respect to some specified acceptance range.
Validating a software measure is the process of ensuring that the measure is a proper numerical characterization of the claimed attribute by showing that the representation condition is satisfied. The measure must not contradict any intuitive notions about the entity.
If a measure for assessment is valid, then we say that it is valid in the narrow sense or is internally valid.
If a measure is internally valid and is a component of a valid prediction system, then we say that the mesure is valid in the wide sense.