Add DeepSpeed easyblock

[VRehnberg]

Adds an EasyBlock for building DeepSpeed. Main reason for not handling it in the easyconfig is to access cuda_compute_capabilities.

Also expands PythonPackage easyblock so that PATH and LD_LIBRARY_PATH is exported for runtest with testinstall as well (compared to only PYTHONPATH originally).

[VRehnberg]

Test report by @VRehnberg

Overview of tested easyconfigs (in order)

• FAIL (build issue) DeepSpeed-0.14.5-foss-2023a-CUDA-12.1.1.eb (partial log available at https://gist.github.com/VRehnberg/3537c6b4776752b8e1cf3abbb8f9e06d)

Build succeeded for 0 out of 1 (1 easyconfigs in total) alvis1-03 - Linux Rocky Linux 8.9, x86_64, Intel(R) Xeon(R) Gold 6244 CPU @ 3.60GHz, 1 x NVIDIA Tesla V100-SXM2-32GB, 560.35.03, Python 3.6.8 See https://gist.github.com/VRehnberg/0ee552005c7319653c59e7beb3a63b7e for a full test report.

[VRehnberg]

Test report by @VRehnberg

Overview of tested easyconfigs (in order)

• FAIL (build issue) DeepSpeed-0.14.5-foss-2023a-CUDA-12.1.1.eb (partial log available at https://gist.github.com/VRehnberg/2773836659824c214e275a5f2ec29721)

Build succeeded for 0 out of 1 (1 easyconfigs in total) alvis1-02 - Linux Rocky Linux 8.9, x86_64, Intel(R) Xeon(R) Gold 6244 CPU @ 3.60GHz, 1 x NVIDIA Tesla V100-SXM2-32GB, 560.35.03, Python 3.6.8 See https://gist.github.com/VRehnberg/62dc55136208c4601668fbff0a174866 for a full test report.

[VRehnberg]

Test report by @VRehnberg

Overview of tested easyconfigs (in order)

• FAIL (build issue) DeepSpeed-0.14.5-foss-2023a-CUDA-12.1.1.eb (partial log available at https://gist.github.com/VRehnberg/a82a4faf68919c4a5aaee509bfe9eaa9)

Build succeeded for 0 out of 1 (1 easyconfigs in total) alvis1-02 - Linux Rocky Linux 8.9, x86_64, Intel(R) Xeon(R) Gold 6244 CPU @ 3.60GHz, 1 x NVIDIA Tesla V100-SXM2-32GB, 560.35.03, Python 3.6.8 See https://gist.github.com/VRehnberg/00bb6c0b928ef06dc1c7f9ab2a23b3c2 for a full test report.

[jfgrimm]

Adds an EasyBlock for building DeepSpeed. Main reason for not handling it in the easyconfig is to access cuda_compute_capabilities.

for reference, in the easyconfig you could have used one of the templates, e.g. %(cuda_cc_semicolon_sep)s

[VRehnberg]

for reference, in the easyconfig you could have used one of the templates, e.g. %(cuda_cc_semicolon_sep)s

Thx, I'll remember that for the future.