abacus-develop icon indicating copy to clipboard operation
abacus-develop copied to clipboard

Refactor: design new 2d block cyclic distribution related functions with scalapack interface

Open kirk0830 opened this issue 1 year ago • 0 comments

Describe the Code Quality Issue

Top-down viewpoint

In ABACUS, there are high dimensional data organized in various ways, such as raw pointers (and nested), std::vector's (and nested), ct::Tensor, HContainer. However, their use are relatively still confined in some classes, rather than widely used containers in ABACUS. This will be risky especially when there are data passing through/across classes: the most common choice of data container has to be raw pointer, and many linear algebra operations (interfaces) are not highly encapsulated, developers should still explicitly pass raw pointer to them.

Bottom-up viewpoint

There are two out-dated classes, Local_Orbital_Wfc and Local_Orbital_Charge. In the former, as far as what I investigated, it contains some 2-dimensional block cyclic distribution indexing related functions but they should not be in a class not directly relevant with scalapack, this will cause redundant coupling between classes.

Todo's

  • [ ] Add enough annotations in Local_Orbital_Wfc class
  • [ ] Add enough annotations in scalapack_connector.h file
  • [ ] Publish a design note of new scalapack interface (as emphasized by scalapack manual, any practical linear algebra operation should be later than data distribution, therefore 2D block cyclic distribution (2D-BCD) management will be pre-requisite)
  • [ ] Code new scalapack interface, incoporating 2D-BCD
  • [ ] Connect functions directly using 2D-BCD
  • [ ] Add interfaces towards HContainer, ct::Tensor and something others

Reference

Basic

BLACS quick reference: https://netlib.org/blacs/BLACS/QRef.html SCALAPACK quick reference: https://netlib.org/scalapack/slug/node183.html

Future consideration (interfaces and heterogeneous computation)

SLATE library: https://dl.acm.org/doi/pdf/10.1145/3295500.3356223

Possible to take into consideration

Note that 2D-BCD is claimed for dense matrix distribution, DBCSR library for sparse matrix: https://github.com/cp2k/dbcsr

kirk0830 avatar May 06 '24 02:05 kirk0830