Keras.NET
Keras.NET copied to clipboard
CPU usage decrease over time
When I use a lot the model.Prediction() the usage of CPU decrease over time, and the time to wait between each prediction increase over time.
===================================================
average millisecond for 1000 predict: 56,54410229999994
===================================================
average millisecond for 1000 predict: 63,12716950000007
===================================================
average millisecond for 1000 predict: 64,76692500000009
===================================================
average millisecond for 1000 predict: 71,62019349999997
===================================================
average millisecond for 1000 predict: 78,61800759999998
This is a critical issue for me because I need to compute a lot of predictions for my training. Any idea on how to fix this?
Here is the code I made to test this problem. I use the same input each time, so it's related to the predict method.
List<double> elapsedList = new List<double>();
for (int j = 0; j < 1_000; j++)
{
elapsedList.Clear();
for (int i = 0; i < 1_000; i++)
{
var ss = DateTime.Now;
model.Predict(input, verbose: 0);
elapsedList.Add((DateTime.Now - ss).TotalMilliseconds);
}
var average = elapsedList.Average();
Console.WriteLine($"===================================================");
Console.WriteLine($"average millisecond for 1000 predict: {average}");
}