Cross-validation

The cross-validation stopping strategy works as follows.

First of all, the input experimental measurements are divided into two categories at random, the training sample and the validation sample, typically with equal probability.

Only the former is used in the fit, while the latter plays the role of a control sample used to monitor and validate the training progress.

The optimal stopping point is defined as the global minimum of the \chi^2 of the validation sample, computed over a large fixed number of iterations (hence the name “look-back”).

As shown in the figure above, a shorter fit would result in under-learning (where the NN has not properly learned yet the underlying law) while a longer fit instead leads to over-learning (where the NN ends up fitting statistical fluctuations).

The tell-tale sign of the latter is the increase of the validation \chi^2 increases (rather than the decrease) as the number of iterations increases, indicating that what is being learned in the training sample is not present in the validation one (namely the fluctuations).