math icon indicating copy to clipboard operation
math copied to clipboard

Investigate tests with very long run times

Open rok-cesnovar opened this issue 4 years ago • 2 comments

Description

A few tests in our non-prob unit tests take an extraordinarily long amount of time to run.

We currently run 5005 tests in the Full Unit tests on Jenkins. 3343 of those run for less than a millisecond, 4418 less than 10 ms, 4760 less than 100ms and 4902 less than a second.

In total, the tests run for around 13-14 minutes, with the 100 tests that individually run for 1 or longer taking 12 minutes of the 13-14. This seems a bit surprising to me and I think we should investigate this. 13 minutes isn't a lot, but if we are wasting 10 minutes of those, I think it's worth taking a bit to not waste computationally cycles.

We might as well discover that all of these are needed.

Listing the top 15 tests that take almost 8 minutes to run in total:

Test Links Time [s]
choleskyDecomposeGeneral link 68
choleskyDecomposeGeneralBig link 62
lub_stdvec_lb_vec_ub_mat_constrain link 51
offset_multiplier_stdvec_mu_vec_sigma_mat_constrain link 37
complexPow link 35
lub_stdvec_lb_mat_ub_vec_constrain link 31
lub_stdvec_mat_scalar1_constrain link 27
~~lorenz_test, param_and_data_finite_diff (ODE)~~ ~~link~~ 25
fallingFactorial_vec link 23
offset_multiplier_stdvec_mu_mat_sigma_vec_constrain link 22
offset_multiplier_stdvec_mat_scalar1_constrain link 19
lub_stdvec_mat_scalar_constrain link 17
matrixExpMultiply link 14
~~harmonic_oscillator_test, param_and_data_finite_diff~~ ~~link~~ 13
offset_multiplier_stdvec_mat_scalar_constrain link 12

All of these are tests in mix/.My feeling is that the constrain tests and Cholesky decomposition tests are unnecessarily big and could be easily reduced. ODEs are more difficult problems and my gut feeling there is that is expected.

Current Version:

v4.2.1

rok-cesnovar avatar Nov 07 '21 10:11 rok-cesnovar

From the ode tests it is the djoint solver which is eating up most of the execution time. The adjoint solver must be run with high accuracy to get the precision required by the tests.

wds15 avatar Nov 07 '21 19:11 wds15

Thanks @wds15! Will strike those out from the list then.

rok-cesnovar avatar Nov 07 '21 19:11 rok-cesnovar