mathnet-numerics
mathnet-numerics copied to clipboard
Incorrect Values for MovingStatistics StandardDeviation
I created a simple test case to get comfortable with the MovingStatistics class and surprisingly stubled into a calculation error:
public void TryingOutMathNetStatisticsOptions()
{
var dataList = new List<double>()
{
1, 1, 1, 1, 1,
2, 2, 2, 2, 2,
3, 3, 3, 3, 3,
4, 4, 4, 4, 4,
3, 3, 3, 3, 3,
2, 2, 2, 2, 2,
1, 1, 1, 1, 1
};
var statistics = new MathNet.Numerics.Statistics.MovingStatistics(5);
var msCount = new List<double>();
var msMean = new List<double>();
var msStdDev = new List<double>();
var msPopStdDev = new List<double>();
foreach (var data in dataList)
{
statistics.Push(data);
msCount.Add(statistics.Count);
msMean.Add(statistics.Mean);
msStdDev.Add(statistics.StandardDeviation);
msPopStdDev.Add(statistics.PopulationStandardDeviation);
Trace.WriteLine($"{data}\t{statistics.Count}\t{statistics.Mean}\t"+
$"{statistics.StandardDeviation}\t{statistics.PopulationStandardDeviation}");
}
for (int i = 4; i < dataList.Count; i += 5)
{
Assert.That(msStdDev[i], Is.Not.NaN);
Assert.That(msPopStdDev[i], Is.Not.NaN);
}
}
The data set was designed to be used with a window width of 5 with 5 consecutive identical values followed by the next set of 5. As both standard deviations step down to zero, for the later data the calculation behind the statistics.Push(data) call will accumulate a small rounding error in the magnitude of 1E-15 that will finally yield in a negative variance.