pecan
pecan copied to clipboard
Improved the current SDA workflow to reach the North American runs with 6400 sites.
Description
Motivation and Context
- The
foreachpackage seems to be better compared to thefurrrpackage concerning memory allocation. Thus, in this PR, I replaced everyfurrrwithforeachduring the general SDA workflow. - The computational power and memory are limited when executing certain SDA procedures locally (e.g., splitting meteorology files, writing configuration files, reading SDA outputs, running Bayesian MCMC analysis part, and removing files (e.g., removing NC files after the first SDA run)). Therefore, in the PR, I developed features of
qsubsubmissions (specified by thebatch.settingssection in the XML file, which is also documented in the pecan book) during the SDA workflow. - To avoid the complex
if-elseusage within the currentsda.enkf_Multisitefunction for the abovebatchjob submissions, I developed a newsda.enkf_NorthAmericafunction (although most of them are just copy-and-paste from thesda.enkf_Multisitefunction), which is cleaner and will only be used if thebatch.settingsis not empty.
Review Time Estimate
- [ ] Immediately
- [ ] Within one week
- [x] When possible
Types of changes
- [x] Bug fix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
Checklist:
- [ ] My change requires a change to the documentation.
- [ ] My name is in the list of CITATION.cff
- [ ] I have updated the CHANGELOG.md.
- [ ] I have updated the documentation accordingly.
- [ ] I have read the CONTRIBUTING document.
- [ ] I have added tests to cover my changes.
- [ ] All new and existing tests passed.