calibr8
calibr8 copied to clipboard
Optimization algorithm to determine HDIs does not converge well
While working on #12 I added tests and came up with one to check if calibr8.core._get_hdi is doing a good job.
It does not.
I believe the scipy.optimize.fmin doesn't cope well with the switching inside the hdi_objective, or at least not in combination with the step-function like area calculation by _interval_prob.
Some starting points to fix this:
- Make a detailed heatmap visualization of the objective function that's being optimized, based on a coarse
x_cdfandcdfinput. (Hypothesis: It's not smooth, but has step functions in one dimension.) - Adapt
test_interval_probto an interpolation that should give much more accurate results compared to the step-function. (Also nicer conceptually!) - Use the triangular example from
test_get_hdias a starting point, because it's HDI is easy to determine analytically (centerline of the triangle to its right flank).