easybuild icon indicating copy to clipboard operation
easybuild copied to clipboard

add default moduleclass for AI/ML software?

Open migueldiascosta opened this issue 3 years ago • 3 comments

currently, AI/ML software is categorized rather loosely: scikit-learn is data, pytorch is devel, tensorflow is lib, keras is math...

there's nothing about AI/ML in the list of default moduleclasses, https://github.com/easybuilders/easybuild-framework/blob/454fe5e024b443b189f8e0f9f8727ed54c28ff56/easybuild/tools/config.py#L400-L424 (but it's not that they are not specific, there's even quantum :) )

should we do something about this?

migueldiascosta avatar Aug 03 '22 13:08 migueldiascosta

Yeah, the time is right to add a new default module class and start using it. Do we want to go with ai, or ai_ml, etc.?

I guess ai is broad enough to also cover machine learning?

boegel avatar Aug 03 '22 14:08 boegel

Another question is if we would reclassify existing easyconfigs or just new ones

migueldiascosta avatar Aug 04 '22 06:08 migueldiascosta

I think re-classifying makes sense. PRs like that are trivial enough to merge without requiring test reports for all easyconfigs we touch (which can be brutally expensive for software like TensorFlow and PyTorch...).

The downside of that would be that trying to use an easyconfig that has been updated to use ai as module class would result in the following unless a sufficiently recent version of EasyBuild framework (develop branch or the to-be-released EasyBuild v4.6.1) is used:

ERROR: Failed to process easyconfig /tmp/example.eb: moduleclass provided 'ai' is not valid: ['base', 'astro', 'bio', 'cae', 'chem', 'compiler', 'data', 'debugger', 'devel', 'geo', 'ide', 'lang', 'lib', 'math', 'mpi', 'numlib', 'perf', 'quantum', 'phys', 'system', 'toolchain', 'tools', 'vis']

boegel avatar Aug 05 '22 15:08 boegel