COSMA icon indicating copy to clipboard operation
COSMA copied to clipboard
verified publisher icon eth-cscs

cosma conda package on conda-forge?

Open ltalirz opened this issue 4 years ago • 7 comments

For a related discussion see https://github.com/electronic-structure/SIRIUS/issues/621

Over the past few weeks we've been adding conda packages for spla, spfft and sirius and I'm now in the process of integrating them into the cp2k one.

COSMA may be somewhat of a fringe case for conda since conda binaries are unlikely to be run on really large-scale machines (for maximum performance you would compile it yourself) but it could still be useful for reproducibility and quick testing.

Once could start with a MPI-only build which I think would be straightforward; it is also possible to create GPU-enabled conda packages but there is little documentation, so this would probably better be left to a later stage.

Let me know what you think!

ltalirz avatar May 28 '21 16:05 ltalirz

Hi Leopold, I think this is a great idea, would be certainly nice to have it.

Although I don't have much experience with writing conda packages. Would you maybe like to help us do it for COSMA? I can add you as a collaborator to the repo in that case.

Let me know how we should proceed.

kabicm avatar May 28 '21 16:05 kabicm

Would we have to make a conda package for each COSMA dependency (?), which are basically the following:

  • COSTA: the algorithm for transposing and transforming distributed matrices between different distributed layouts (e.g. from the block-cyclic layout to an arbitrary grid-like layout). It is highly optimized and is used to provide SCALAPACK wrappers to COSMA.
  • Tiled-MM: this is the GPU backend of COSMA, which implements a cublasXt alternative that is also ported to AMD.

kabicm avatar May 28 '21 16:05 kabicm

Thanks for the quick reply @kabicm ! Indeed, we should probably start from the dependencies. I'll start by creating a PR for a conda package for COSTA and ping you there.

ltalirz avatar May 29 '21 12:05 ltalirz

Hi @ltalirz , sorry for keeping you wait -- I cannot find the PR for a conda package, can you please send me this again?

kabicm avatar Jun 15 '21 12:06 kabicm

Hi @ltalirz, do you maybe know what's the problem with: https://github.com/conda-forge/staged-recipes/pull/15073 ?

kabicm avatar Oct 04 '21 14:10 kabicm

I think it was not clear to me and so I stopped. Now the logs are gone, so I've just restarted the CI.

ltalirz avatar Oct 04 '21 16:10 ltalirz

So, costa is taken care of https://anaconda.org/conda-forge/costa Maybe mention in the README of the repo that you can now conda install -c conda-forge costa and conda install -c conda-forge costa-scalapack on Linux and MacOS.

Anyone interested in opening a PR for tiled-mm?

ltalirz avatar Oct 09 '21 15:10 ltalirz