Uncaught C exceptions

[FFroehlich]

2024-07-05 03:23:14.990 - amici.swig_wrappers - WARNING - [cEFM192A__EGF+cEFM192A][AMICI:Inf] AMICI encountered a Inf value for sxdot[0] (0) at t=0.000000 
2024-07-05 03:23:14.990 - amici.swig_wrappers - WARNING - [cEFM192A__EGF+cEFM192A][AMICI:Inf] AMICI encountered a Inf value for sxdot[0] (0) at t=0.000000 
2024-07-05 03:23:14.990 - amici.swig_wrappers - WARNING - [cEFM192A__EGF+cEFM192A][AMICI:Inf] AMICI encountered a Inf value for sxdot[0] (0) at t=0.000000 
2024-07-05 03:23:14.990 - amici.swig_wrappers - WARNING - [cEFM192A__EGF+cEFM192A][AMICI:Inf] AMICI encountered a Inf value for sxdot[0] (0) at t=0.000000 
2024-07-05 03:23:14.990 - amici.swig_wrappers - WARNING - [cEFM192A__EGF+cEFM192A][AMICI:Inf] AMICI encountered a Inf value for sxdot[0] (0) at t=0.000000 
2024-07-05 03:23:14.990 - amici.swig_wrappers - WARNING - [cEFM192A__EGF+cEFM192A][AMICI:Inf] AMICI encountered a Inf value for sxdot[0] (0) at t=0.000000 
2024-07-05 03:23:14.990 - amici.swig_wrappers - WARNING - [cEFM192A__EGF+cEFM192A][AMICI:Inf] AMICI encountered a Inf value for sxdot[0] (0) at t=0.000000 
2024-07-05 03:23:14.991 - amici.swig_wrappers - WARNING - [cEFM192A__EGF+cEFM192A][AMICI:Inf] AMICI encountered a Inf value for sxdot[0] (0) at t=0.000000 
2024-07-05 03:23:14.991 - amici.swig_wrappers - WARNING - [cEFM192A__EGF+cEFM192A][AMICI:Inf] AMICI encountered a Inf value for sxdot[0] (0) at t=0.000000 
2024-07-05 03:23:14.991 - amici.swig_wrappers - WARNING - [cEFM192A__EGF+cEFM192A][AMICI:Inf] AMICI encountered a Inf value for sxdot[0] (0) at t=0.000000 
2024-07-05 03:23:14.991 - amici.swig_wrappers - DEBUG - [cEFM192A__EGF+cEFM192A][CVODES:CVode:OTHER] AMICI ERROR: in module CVODES in function CVode : At t = 0 repeated recoverable sensitivity right-hand side function errors. 
2024-07-05 03:23:14.991 - amici.swig_wrappers - ERROR - [cEFM192A__EGF+cEFM192A][OTHER] AMICI simulation failed: CVODE routine reInitPostProcess failed with error code -43. tout: 5.5, t: 0.
2024-07-05 03:23:15.032 - amici.swig_wrappers - DEBUG - [cEFM192A__EGF+cEFM192A][CVODES:CVode:OTHER] AMICI ERROR: in module CVODES in function CVode : At t = 0, the setup routine failed in an unrecoverable manner. 
2024-07-05 03:23:15.033 - amici.swig_wrappers - ERROR - [cEFM192A__EGF+cEFM192A][OTHER] AMICI simulation failed: CVODE routine reInitPostProcess failed with error code -6. tout: 5.5, t: 0.
terminate called after throwing an instance of 'amici::CvodeException'
  what():  CVODE routine CVodeGetSens failed with error code -25. 

While the underlying issue is Inf in some of the equations, this shouldn't lead to an uncaught c++ exception that terminates python.

[dweindl]

Do you have more context where that happened? runAmiciSimulation / runAmiciSimulations should catch everything. Might be again some linking issue such that exceptions in the base extension and model extension don't match?

[FFroehlich]

this was in the context of runAmiciSimulations, but we were unable to reproduce the issues locally. In principle the models should be automatically compiled, we will investigate this further.

[dilpath]

I can reproducibly produce the error below. The error is not unexpected since I am setting up a custom simulator that builds on simulate_petab, and this custom simulator isn't really working yet, but it looks like an uncaught exception too. Let me know if you'd like an MWE.

terminate called after throwing an instance of 'amici::AmiException'
  what():  Number of initial conditions sensitivities (40) in model does not match ExpData (35).
Aborted (core dumped)

[dweindl]

terminate called after throwing an instance of 'amici::AmiException'
  what():  Number of initial conditions sensitivities (40) in model does not match ExpData (35).
Aborted (core dumped)

Thanks. This shouldn't happen. There should be a Python RuntimeError instead. On my computer, this works as excepted. I added a test case in #2484, does this pass for you?

Potential issues:

• different swig versions
• updating amici without re-installation
• re-installing amici without re-importing the model(?)
• linking issues

[dilpath]

I added a test case in https://github.com/AMICI-dev/AMICI/pull/2484, does this pass for you?

The first version of this PR passed for me. The current version fails with

FAILED test_swig_interface.py::test_python_exceptions - RuntimeError: Dimension mismatch. Size of parameters does not match number of model parameters.

but I guess this still counts as a pass, since there is no uncaught exception.

Potential issues:

* different swig versions

* updating amici without re-installation

* re-installing amici without re-importing the model(?)

* linking issues

The 2nd and 3rd aren't the case for me I think. My SWIG is 4.0.2. Not sure how to check the linking issue. I'll try to create an MWE then.

[dweindl]

The first version of this PR passed for me. The current version fails with

Good. Sorry, fixed the other issue. RuntimeError is good though.

I'll try to create an MWE then.

Yes, that would be interesting then.

[dweindl]

I'll try to create an MWE then.

Thanks. So in @dilpath's case, the problem is that the error occurs already here, outside try:

https://github.com/AMICI-dev/AMICI/blob/b958cce6a08683104b570b6aaa672d83c2c3fed6/src/amici.cpp#L94

When using runAmiciSimulation directly, this exception will result in a Python RuntimeError which is fine. However, for runAmiciSimulations (used by simulate_petab), this exception must be caught within the simulation thread, otherwise the program will terminate as observed here.

The same issue is lurking here: https://github.com/AMICI-dev/AMICI/blob/b958cce6a08683104b570b6aaa672d83c2c3fed6/src/amici.cpp#L291

The original issue is probably slightly different, but I assume it's also related to using runAmiciSimulations with some exception in runAmiciSimulation not being caught.