Benchmark-Models-PEtab icon indicating copy to clipboard operation
Benchmark-Models-PEtab copied to clipboard

Validate simulation tables

Open dweindl opened this issue 3 months ago • 1 comments

Re: #276, #275, #277

Implement checks for simulation tables. Make sure that petab.v1.calculate.calculate_llh succeeds, and ideally, confirm that value with a simulator.

dweindl avatar Oct 20 '25 19:10 dweindl

There are issues with several other simulation tables.

From #278:

FAILED src/python/test/test_base.py::test_can_calculate_llh_from_simulation_df[Armistead_CellDeathDis2024] - ValueError: Cannot replace all parameters in noise formula 0.088129278623526*Sphinga for observable Sphinga_obs. Missing {noiseParameter1_Sphinga_obs, Sphinga}. Note that model states are currently not supp...
FAILED src/python/test/test_base.py::test_can_calculate_llh_from_simulation_df[Beer_MolBioSystems2014] - IndexError: single positional indexer is out-of-bounds
FAILED src/python/test/test_base.py::test_can_calculate_llh_from_simulation_df[Boehm_JProteomeRes2014] - IndexError: single positional indexer is out-of-bounds
FAILED src/python/test/test_base.py::test_can_calculate_llh_from_simulation_df[Borghans_BiophysChem1997] - IndexError: single positional indexer is out-of-bounds
FAILED src/python/test/test_base.py::test_can_calculate_llh_from_simulation_df[Elowitz_Nature2000] - IndexError: single positional indexer is out-of-bounds
FAILED src/python/test/test_base.py::test_can_calculate_llh_from_simulation_df[Froehlich_CellSystems2018] - NotImplementedError: Unsupported combination of noise_distribution and scale specified: normal, nan.
FAILED src/python/test/test_base.py::test_can_calculate_llh_from_simulation_df[Fujita_SciSignal2010] - IndexError: single positional indexer is out-of-bounds
FAILED src/python/test/test_base.py::test_can_calculate_llh_from_simulation_df[Isensee_JCB2018] - IndexError: single positional indexer is out-of-bounds
FAILED src/python/test/test_base.py::test_can_calculate_llh_from_simulation_df[Lang_PLOSComputBiol2024] - ValueError: Cannot replace all parameters in noise formula 1.9343988272e-11*_Nuc__CCNA_CDKN1A_B_ + 1.9343988272e-11*_Nuc__CCNA_CDKN1A_B_1__CDKN1A_CCNE_A_B_1_Ser130u_ + 1.9343988272e-11*_Nuc__CCNA_CDKN1A_B_1...
FAILED src/python/test/test_base.py::test_can_calculate_llh_from_simulation_df[Lucarelli_CellSystems2018] - IndexError: single positional indexer is out-of-bounds
FAILED src/python/test/test_base.py::test_can_calculate_llh_from_simulation_df[Perelson_Science1996] - IndexError: single positional indexer is out-of-bounds
FAILED src/python/test/test_base.py::test_can_calculate_llh_from_simulation_df[Raia_CancerResearch2011] - IndexError: single positional indexer is out-of-bounds
FAILED src/python/test/test_base.py::test_can_calculate_llh_from_simulation_df[Sneyd_PNAS2002] - IndexError: single positional indexer is out-of-bounds
FAILED src/python/test/test_base.py::test_can_calculate_llh_from_simulation_df[Weber_BMC2015] - IndexError: single positional indexer is out-of-bounds
FAILED src/python/test/test_base.py::test_can_calculate_llh_from_simulation_df[Zheng_PNAS2012] - IndexError: single positional indexer is out-of-bounds

All IndexErrors are probably incorrect simulation tables.

The NotImplementedError is likely a bug in libpetab.

That ValueErrors are probably due to state-dependent observation models, that are not adequately addressed in the current simulation table setup. (See also #280.)

dweindl avatar Oct 20 '25 20:10 dweindl