lbann icon indicating copy to clipboard operation
lbann copied to clipboard

Test Python frontend

Open ndryden opened this issue 6 years ago • 1 comments

Based on #1357, it seems we're not sufficiently aggressively testing the Python frontend in the integration tests. Can we at least run --setup-only on all the scripts in the applications and ensure they don't fail?

ndryden avatar Nov 27 '19 15:11 ndryden

This is a good idea. There's a bit of hairiness since the Bamboo tests are intended to switch between frontends (although this is broken, see #1289). This makes it hard to write tests naturally (see the create_tests function in lbann/bamboo/common_python for a convoluted function that takes functions as inputs and outputs). Things would be simpler if we made the Bamboo tests build-specific (i.e. instead of running a monolithic set of tests for all builds, run it once for GCC, then run it again for Clang, then run it again for Intel, etc.)

I think a good approach would be to create a test directory inside lbann/python/lbann and to put some PyTest tests there. This is similar to how our Catch2 tests are located close to their corresponding source files (e.g. lbann/src/utils/unit_test). The Bamboo infrastructure can handle to messiness of dispatching to the correct build.

timmoon10 avatar Dec 02 '19 18:12 timmoon10