GEOS-DEV
feat: hypre improvements
- Replace internal GEOS SDC implementation with hypre's
- Remove tracking of certain SDC times
- Add hypre log level support
- Update hypre umpire pools naming
- Add missing coarse solver options in AMG
- Remove unused SDC variables in GEOS data structures
- Add
separateComponentsoption to mechanics solver setup in MGR - Move
addCommaSeparatorsto StringUtilities - Add unknowns and nonzeros info to linear solver message, e.g.:
Linear Solver | Success | Unknowns: 645,162 | Nonzeros: 42,898,950 | Iterations: 39 | Final Rel Res: 9.4042e-05 | Setup Time: 0.538 s | Solve Time: 0.933 s
Requires https://github.com/GEOS-DEV/thirdPartyLibs/pull/286
Here is a quick test I ran on a version of SPE10 with burdens, compositional poromechanics, though with homogeneous properties, (so its just a high perm slab between two low perm boxes). Should be about 40 million DoF, and I ran on Dane.
Here were the results with develop a few weeks ago
| Mpi ranks | Total Run Time (s) | Linear Solver Setup (s) | Linear Solver Solve (s) | Total GMRES Iterations (always 43 Total Newton iters) |
|---|---|---|---|---|
| 64 | 2087 | 523 | 1966 | 1850 |
| 128 | 1451 | 343 | 1381 | 1833 |
| 256 | 963 | 234 | 662 | 1890 |
| 512 | 624 | 182 | 404 | 1974 |
Here are the results with the new Hypre
| Mpi ranks | Total Run Time (s) | Linear Solver Setup (s) | Linear Solver Solve (s) | Total GMRES Iterations (always 43 Total Newton iters) |
|---|---|---|---|---|
| 64 | 1568 | 233 | 1162 | 2281 |
| 128 | 1115 | 139 | 865 | 2229 |
| 256 | 723 | 85 | 572 | 2293 |
| 512 | 470 | 85 | 347 | 2366 |
Note that this does not double the number of nodes for each test, I will do that now
Old results for heterogeneous SPE10 compositional poromechanics on CPU (dane)
| Dane cores (Mpi ranks) | Total Run Time (s) | Linear Solver Setup (s) | Linear Solver Solve (s) | Time steps/Nonlin iters / Lin iters |
|---|---|---|---|---|
| 2 (224) | 2409 | 488 | 1764 | 24/76/3464 |
| 4 (448) | 1440 | 380 | 977 | 24/76/3862 |
| 8 (896) | 1075 | 447 | 583 | 24/76/4286 |
New results for heterogeneous SPE10 compositional poromechanics on CPU (dane)
| Dane cores (Mpi ranks) | Total Run Time (s) | Linear Solver Setup (s) | Linear Solver Solve (s) | Time steps/Nonlin iters / Lin iters |
|---|---|---|---|---|
| 2 (224) | 1770 | 185 | 1425 | 24/76/3882 |
| 4 (448) | 952 | 152 | 719 | 24/76/3960 |
| 8 (896) | 686 | 225 | 417 | 24/76/4228 |
We are investigating a few convergence issues with Poromechanics runs using this branch. @drmichaeltcvx, please you can put your tests on hold until the issue is sorted out
[like] Thomadakis, Michael reacted to your message:
From: Victor A. P. Magri @.> Sent: Thursday, October 10, 2024 9:29:13 PM To: GEOS-DEV/GEOS @.> Cc: Thomadakis, Michael @.>; Mention @.> Subject: [EXTERNAL] Re: [GEOS-DEV/GEOS] feat: hypre improvements (PR #3339)
Be aware this external email contains an attachment and/or link. Ensure the email and contents are expected. If there are concerns, please submit suspicious messages to the Cyber Intelligence Center using the Report Phishing button.
We are investigating a few convergence issues with Poromechanics runs using this branch. @drmichaeltcvxhttps://github.com/drmichaeltcvx, please you can put your tests on hold until the issue is sorted out
— Reply to this email directly, view it on GitHubhttps://github.com/GEOS-DEV/GEOS/pull/3339#issuecomment-2406069968, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AS6ZG2RPWTBYKN45GHRDVELZ23WSTAVCNFSM6AAAAABN4TL2HKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMBWGA3DSOJWHA. You are receiving this because you were mentioned.Message ID: @.***>
[like] Thomadakis, Michael reacted to your message:
From: Victor A. P. Magri @.> Sent: Tuesday, October 22, 2024 5:23:58 PM To: GEOS-DEV/GEOS @.> Cc: Thomadakis, Michael @.>; Mention @.> Subject: [EXTERNAL] Re: [GEOS-DEV/GEOS] feat: hypre improvements (PR #3339)
Be aware this external email contains an attachment and/or link. Ensure the email and contents are expected. If there are concerns, please submit suspicious messages to the Cyber Intelligence Center using the Report Phishing button.
@paveltominhttps://github.com/paveltomin this should work for SEQ runs now
— Reply to this email directly, view it on GitHubhttps://github.com/GEOS-DEV/GEOS/pull/3339#issuecomment-2429848059, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AS6ZG2S5RJU3QN4QOIMEZB3Z42C25AVCNFSM6AAAAABN4TL2HKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMRZHA2DQMBVHE. You are receiving this because you were mentioned.Message ID: @.***>
Codecov Report
Attention: Patch coverage is 19.04762% with 68 lines in your changes missing coverage. Please review.
Project coverage is 57.50%. Comparing base (
4bcf7df) to head (bea64ed). Report is 64 commits behind head on develop.
Additional details and impacted files
@@ Coverage Diff @@
## develop #3339 +/- ##
===========================================
- Coverage 57.55% 57.50% -0.05%
===========================================
Files 1135 1134 -1
Lines 97850 97830 -20
===========================================
- Hits 56316 56257 -59
- Misses 41534 41573 +39
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
![codecov[bot] avatar](https://avatars.githubusercontent.com/in/254?v=4 "Published by a pub.dev verified publisher"){kind=small}
@CusiniM @rrsettgast Could you do a new LC build and create the LvArray PR?
Can we put this on the merge queue?
Thanks!
@castelletto1 @CusiniM @rrsettgast Can you guys take it from here? It seems there's nothing else I can do. Thanks!
@castelletto1 @CusiniM @rrsettgast Can you guys take it from here? It seems there's nothing else I can do. Thanks!
is this basically ready? I mean it clearly needs an LvArray PR and a rebaseline but is it ready otherwise?
Right, it's ready. Needs a LvArray PR and LC build, which I can't do. Rebaseline is due to field value change (amgNumFunctions)