Support `sensi_orders` in gradient checks
While debugging another model, the issue arose again that sometimes Objective(x, sensi_orders=(0,)) provides a different fval than Objective(x, sensi_orders=(0,1)). Tolerances etc. can be adjusted until these two are similar; however, pyPESTO's check_grad might incorrectly report that gradients are not close, because it computes fval with sensis
https://github.com/ICB-DCM/pyPESTO/blob/ead29b36db80d2eeee91b5d29bfbf4430137f8aa/pypesto/objective/base.py#L535
but fval_p and fval_m (for the plus/minus steps used in finite difference approximations) without sensis
https://github.com/ICB-DCM/pyPESTO/blob/ead29b36db80d2eeee91b5d29bfbf4430137f8aa/pypesto/objective/base.py#L556
https://github.com/ICB-DCM/pyPESTO/blob/ead29b36db80d2eeee91b5d29bfbf4430137f8aa/pypesto/objective/base.py#L561
This could be resolved by using the same sensi_orders in all function calls, or otherwise providing some way for the user to specify this.
This could be resolved by using the same
sensi_ordersin all function calls, or otherwise providing some way for the user to specify this.
It might make sense to address that independently of gradient checks, since this will also be a problem for any optimizer that calls the objective with varying sensi_orders. Maybe a simple wrapper around Objective that always passes a certain sensi_orders internally and then filters the result to match the externally requested sensi_orders.