scipy python version dependency errors

[ajpotts]

Hi Amanda, I've got some notes on what I'm seeing here: This seems to be causing failures for me, both in CI and locally, for some of the SciPy tests. Oddly, it seems to behave differently for Python 3.10 and 3.11. Here is what I'm seeing (output truncated): Python 3.11

$ python3 -m pytest tests/scipy/scipy_test.py -k "test_power_divergence"
platform darwin -- Python 3.11.7, pytest-7.2.1, pluggy-1.0.0


tests/scipy/scipy_test.py .                        [100%]

============ 1 passed, 1 deselected in 6.21s =============

Python 3.10

$ python3.10 -m pytest tests/scipy/scipy_test.py -k "test_power_divergence"
============================= test session starts ==============================

tests/scipy/scipy_test.py F                                              [100%]

=================================== FAILURES ===================================
_______________________ StatsTest.test_power_divergence ________________________
>                   assert np.allclose(ak_power_div, scipy_power_div, equal_nan=True)
E                   assert False
E                    +  where False = <function allclose at 0x106e1bbf0>(Power_divergenceResult(statistic=67.8313215\
1610018, pvalue=nan), Power_divergenceResult(statistic=67.83132151610016, pvalue=0.0), equal_nan=True)
E                    +    where <function allclose at 0x106e1bbf0> = np.allclose

tests/scipy/scipy_test.py:55: AssertionError
=========================== short test summary info ============================
FAILED tests/scipy/scipy_test.py::StatsTest::test_power_divergence - assert F...
================= 1 failed, 1 deselected, 7 warnings in 5.64s ==================

I am inclined to believe that this is a dependency versioning issue, but it doesn't make a lot of sense to me and I'm not quite sure how to proceed from here (edited)

[ajpotts]

I researched this a bit and the problem is that the power_divergence function in scipy underwent a non-trivial refactor in scipy 1.14.0. We would need to refactor that function to match the new one. In the new version, sometimes a p-values of nan becomes a value of 0.