Using the hessian matrix in optimization

[andrewfowlie]

Summary

Some optimization algorithms can make use of the hessian matrix, e.g., in scipy the methods Newton-CG, dogleg, trust-ncg, trust-krylov, trust-exact and trust-constr. The scipy API is like this:

scipy.optimize.minimize(fun, x0, args=(), method=None, jac=None, hess=None, hessp=None, bounds=None, constraints=(), tol=None, callback=None, options=None)

The hessian is not passed through the options dict; it has it's own kwarg hess.

It would be nice if pyhf passed the hessian automatically, since it is available cheaply with autodiff. I have tried passing it manually, but I can't get it to work. The interface with scipy minimize is like this

https://github.com/scikit-hep/pyhf/blob/5b632519838f6ad2c73a8c8f8b53a09c1d8c678b/src/pyhf/optimize/opt_scipy.py#L93-L102

so you can't input an extra hess kwarg argument by passing extra solver_options.

Additional Information

Code of Conduct

• [X] I agree to follow the Code of Conduct

[kratsg]

One thought at the moment is you can subclass the scipy_optimizer via something like this for now:

class scipy_optimizer_hess(pyhf.optimize.scipy_optimizer):
    def _get_minimizer(self, *args, hess=None, **kwargs):
        return lambda *a, **k: scipy.optimize.minimize(*a, **k, hess=hess)

and then you can use it like so:

pyhf.set_backend(pyhf.tensorlib, scipy_optimizer_hess())

just to get a quick way of having it working for right now. This is probably something that needs a bit more thought as minuit doesn't support the hess, so we'd need a way to handle it seamlessly.

[andrewfowlie]

Thanks, yes, that can work. It's trickier than I first thought, though, because of how it must interact with fixed_params.