Refactor: design new 2d block cyclic distribution related functions with scalapack interface
Describe the Code Quality Issue
Top-down viewpoint
In ABACUS, there are high dimensional data organized in various ways, such as raw pointers (and nested), std::vector's (and nested), ct::Tensor, HContainer. However, their use are relatively still confined in some classes, rather than widely used containers in ABACUS. This will be risky especially when there are data passing through/across classes: the most common choice of data container has to be raw pointer, and many linear algebra operations (interfaces) are not highly encapsulated, developers should still explicitly pass raw pointer to them.
Bottom-up viewpoint
There are two out-dated classes, Local_Orbital_Wfc and Local_Orbital_Charge. In the former, as far as what I investigated, it contains some 2-dimensional block cyclic distribution indexing related functions but they should not be in a class not directly relevant with scalapack, this will cause redundant coupling between classes.
Todo's
- [ ] Add enough annotations in
Local_Orbital_Wfcclass - [ ] Add enough annotations in
scalapack_connector.hfile - [ ] Publish a design note of new scalapack interface (as emphasized by scalapack manual, any practical linear algebra operation should be later than data distribution, therefore 2D block cyclic distribution (2D-BCD) management will be pre-requisite)
- [ ] Code new scalapack interface, incoporating 2D-BCD
- [ ] Connect functions directly using 2D-BCD
- [ ] Add interfaces towards
HContainer,ct::Tensorand something others
Reference
Basic
BLACS quick reference: https://netlib.org/blacs/BLACS/QRef.html SCALAPACK quick reference: https://netlib.org/scalapack/slug/node183.html
Future consideration (interfaces and heterogeneous computation)
SLATE library: https://dl.acm.org/doi/pdf/10.1145/3295500.3356223
Possible to take into consideration
Note that 2D-BCD is claimed for dense matrix distribution, DBCSR library for sparse matrix: https://github.com/cp2k/dbcsr