charmplusplus
improved topology-aware partitioner
Original issue: https://charm.cs.illinois.edu/redmine/issues/1677
The current topology-aware recursive-bisection partitioner (described in Phillips et al., SC14) was based on an algorithm designed to map the patch grid to the overall torus, and is therefore sub-optimal for partitioning where communication between neighboring partitions is irrelevant. This results in large outlier partitions when there are gaps in the torus, as is increasingly the case on Blue Waters as "communication transparent" jobs may be scattered throughout the allocation. A better solution may be to use a variant of k-means clustering that guarantees an equal numbers of elements in each cluster (i.e., find k clusters of n elements each that minimize the maximum cluster Manhattan metric diameter). This seems like a good researchy CS topic.
Original date: 2017-09-18 14:37:21
I assume this issue refers to code inside Charm++?
I don't quite understand your description of the issue with the previous code, but the new ST_RecursivePartition code can be used to do recursive partitioning on Blue Waters that "makes sense". It might be what you are looking for.
Original date: 2017-09-18 18:29:53
Yes, the code in Charm++ that handles the +partitions (+replicas) setup.
Original date: 2017-10-11 20:11:46
Juan, please follow up with more details/discussion, and decide on scheduling this.
