ESCOMP
Adding excess ground ice to CTSM
Description of changes
Add parameterization to allow excess ice in soil and subsidence
Specific notes
Parameterization for excess ice is described in Lee et al., 2014.
This code is a modified version of code provided by Lei Cai.
Works only for the nuopc driver.
Files changed: bld/CLMBuildNamelist.pm, bld/namelist_files/namelist_defaults_ctsm.xml, bld/namelist_files/namelist_definitionss_ctsm.xml -- added namelist options; src/main/clm_varctl.F90, src/main/controlMod.F90 -- added option to switch excess ice physics and read namelist option; src/biogeophys/WaterStateType.F90 -- added prognostic excess ice variable and a history field; src/biogeophys/WaterStateBulkType.F90, src/main/clm_instMod.F90, -- added arguments to soubrutiens for proper initialization; src/biogeophys/TemperatureType.F90 -- initial soil temperature set to 268.15 K at the cold start (might be redundant if https://github.com/ESCOMP/CTSM/issues/1460 is added) src/biogeophys/WaterDiagnosticBulkType.F90 -- added two diagnostic excess ice variables and two history fields; src/biogeophys/SoilTemperatureMod.F90 -- added main excess ice calculations; src/biogeophys/TotalWaterAndHeatMod.F90 -- added excess ice to total water for balance checks; src/biogeophys/SoilHydrologyMod.F90 -- added excess ice for ice fraction calculation;
New files: src/cpl/share_esmf/ExcessIceStreamType.F90 -- routines to read dataset with initial excess ice concentration
CTSM Issues Fixed (include github issue #): https://github.com/ESCOMP/CTSM/issues/1229
Are answers expected to change (and if so in what way)?
Yes, when excess ice parameterization is used.
Any User Interface Changes (namelist or namelist defaults changes)?
New namelist options added:
use_excess_ice (logical, in clm_inparm) default - .false.; turns on excess ice physics
stream_meshfile_exice, stream_fldfilename_exice and stream_mapalgo_exice (char, in exice_streams) defaults - lnd/clm2/paramdata/exice_init_0.125x0.125_ESMFmesh_c20220516.nc, lnd/clm2/paramdata/exice_init_0.125x0.125_c20220516.nc andbilinear; meshfile, stream file and spatial interpolation algorithm for initial values of excess ice. Dataset is interpolated to 0.125x0.125 degrees grid from Brown et al., 1997, can be found here. Dataset is only used during the cold start or for hybrid runs (or start from finidat) that do not have excess ice.
Testing performed, if any: On sigma2 machines: comparisons with the upstream/master version and switched-off excess ice physics give the same results for soil state variables. More information and figures on changes and testing can be found here
@ka7eh
It looks like this is up to date with ctsm5.1.dev071 and it should be update to the very latest. I tried doing that and was able to do so without conflicts.
@ekluzek I was able to build and run the model with the commits above (both with use_excess_ice=.true. and .false., cold start and from finidat, resubmits also worked). I haven't resolved anything above, only commented since I am not sure if I am supposed to resolve it. Not sure if unit_test code is built when normal cases are built though (regarding NLFilename in WaterType initialization routines.
This is a list of things we need to do here. We will talk about this at an upcoming Zoom meeting.
List for Matvey:
- [x] Excess ice and Hillslope, will put excess ice equally on all columns. This is OK as a starting point, but likely should be weighted by elevation. This will be a research project.
- [x] Make sure excess ice is changing TWS as expected.
- [x] Change wording of comment here, and go through all of the code to make sure this doesn't apply #1819
- [x] Decide about output of history variables. Figured out a plan.
- [x] Make the 3d history field inactive, unless intentionally added in, and make the 2d field active with excess_ice by default.
- [x] Update technical note for excess ice. As a longer term issue. Erik will make this a separate issue.
Questions we answered:
- [x] Also how does the spinup need to happen with excess ice? The normal process is to start from a cold start or use initial dataset and run of fifty to a hundred years. This could be the post-AD step. The stream file is for current conditions.
- [x] Are there any code configurations that interact with excess ice that we should make sure we test? We don't think so.
- [x] Decide if excess ice should be default on for clm5_1 physics (or if it should be turned on by default later). This will be decided later.
List for Erik and Kaveh:
- [x] Erik will examine #1820 Can we use the nearest tint_algo method and get the correct behavior?
- [x] Talk about general flow of use of excess ice. Are excess ice streams only used if fields aren't on restart files? If so we should probably make this more explicit so the user knows what's being used. So it would die with an error if you aren't using streams and its not on the restart file. And it would die if you are using streams and you have fields on the restart file.
- [x] Erik will look into this both in the model and the namelist level
- [x] Make sure we have at least one test in the aux_clm testlist that runs with excess ice -- Erik will check
- [x] Add an excess ice test to the science list -- Erik
- [x] Erik will work on the running of the test suite and adding tests.
- [x] Run the complete test list and make sure it works. Erik
- [x] Update to latest, Erik
- [x] Erik make sure Kaveh has access to izumi
- [x] Erik will run the tests on izumi
- [x] Kavey will check about giving us access to some of the Norway machines
- [x] Kavey send Erik the quickstart guide and access to the data folder (https://metos-uio.github.io/CTSM-Norway-Documentation/quick-start/)
Things we thought we should do that we didn't get to:
- Use filter in excess_iceStreams.
- Erik will login and check out the Norway machine
- Dave and Sean: BGC does soil Carbon interact with excess ice? Does it modify the mixing process? We should evaluate the soil carbon with excess ice Dave and Sean need to look into. There are differences.
I've checked TWS variable. It gets changed with the amount of excess ice, as expected. So, this one passes.
@ekluzek added log and fixed history fields. Also, maybe #1819 should be closed.
When the initial data interpolation is used you automatically get logs like:
Skipping : VARNAME is NOT on input file
So I just added a reminder in the stream output. The restart without the variable gets an additional log line.
I found a simple problem in DEBUG mode that I'll fix:
1:forrtl: severe (194): Run-Time Check Failure. The variable 'waterstatetype_mp_restart_$NBEDROCK' is being used in '/glade/work/erik/ctsm_worktrees/excess_ice/src/biogeophys/WaterStateType.F90(740,33)' without being defined
1:Image PC Routine Line Source
1:cesm.exe 0000000002FECE06 waterstatetype_mp 740 WaterStateType.F90
1:cesm.exe 0000000002FBE99B waterstatebulktyp 218 WaterStateBulkType.F90
1:cesm.exe 000000000303685F watertype_mp_rest 752 WaterType.F90
1:cesm.exe 000000000097EA90 clm_instmod_mp_cl 533 clm_instMod.F90
1:cesm.exe
@ekluzek Is this because excess ice is read before nbedrock is read from a restart? I do not remember encountering this when I was testing running from finidat or hybrid run.
@mvdebolskiy the fix is straightforward, see what I did in 212e10476a64911da1ca9dff9f1a172515c4707a which I just pushed. Let me know if you see any issues with it...
@mvdebolskiy the fix is straightforward, see what I did in 212e104 which I just pushed. Let me know if you see any issues with it...
No issues. Nice catch, Thanks.
@mvdebolskiy I've ran most of the testing and the only issue I'm seeing is a difference in answers for some cases without excess_ice. This might be something simple where an order of operations was changed. We do like to preserve the answers when a new feature is added and it's turned off, because it makes evaluation much simpler. So if we can find it and fix it, we'd like to. Let me know if you know what this might be.
For example, this list shows changes in answers
ERI_D_Ld9.f10_f10_mg37.I1850Clm45Bgc.cheyenne_gnu.clm-default ERI_D_Ld9.ne30_g17.I2000Clm50BgcCru.cheyenne_intel.clm-vrtlay ERP_D_Ld5.f09_g17.I2000Clm50Vic.cheyenne_intel.clm-vrtlay ERP_D_Ld5.f10_f10_mg37.I2000Clm50Vic.cheyenne_intel.clm-vrtlay ERP_D_Ld5.f10_f10_mg37.IHistClm45Sp.cheyenne_intel.clm-decStart ERP_D_P36x2_Ld3.f10_f10_mg37.I2000Clm45BgcCrop.cheyenne_gnu.clm-no_subgrid_fluxes ERP_Ld5.f09_g17.I2000Clm50Vic.cheyenne_intel.clm-vrtlay ERP_P36x2_D_Ld5.f10_f10_mg37.I1850Clm45BgcCrop.cheyenne_intel.clm-crop ERP_P36x2_D_Ld5.f10_f10_mg37.I1850Clm45BgcCru.cheyenne_intel.clm-default ERP_P36x2_D_Ld5.f10_f10_mg37.I2000Clm45Sp.cheyenne_intel.clm-default ERP_P36x2_D_Ld5.f10_f10_mg37.IHistClm45BgcCru.cheyenne_intel.clm-decStart ERS_D_Ld6.f10_f10_mg37.I1850Clm45BgcCrop.cheyenne_gnu.clm-clm50CMIP6frc ERS_D_Ld6.f10_f10_mg37.I1850Clm45BgcCrop.cheyenne_intel.clm-clm50CMIP6frc ERS_Ly5_P72x1.f10_f10_mg37.IHistClm45BgcCrop.cheyenne_intel.clm-cropMonthOutput SMS_D_Ld5.f10_f10_mg37.I2000Clm45Fates.cheyenne_intel.clm-FatesColdDef SMS_D_Ly6_Mmpi-serial.1x1_smallvilleIA.IHistClm45BgcCropQianRs.cheyenne_intel.clm-cropMonthOutput SMS_Ld5.f10_f10_mg37.I1850Clm45BgcCrop.cheyenne_gnu.clm-crop SMS_Ld5.f10_f10_mg37.I1850Clm45BgcCrop.cheyenne_intel.clm-crop SMS_Ld5.f10_f10_mg37.I2000Clm45Fates.cheyenne_intel.clm-FatesColdDef
Edit: And on izumi these ones...
SMS_D_Ld1_P48x1.f10_f10_mg37.I2000Clm45BgcCrop.izumi_nag.clm-oldhyd SMS_D_Ld5.f10_f10_mg37.I1850Clm45BgcCrop.izumi_nag.clm-crop SMS_Ld5.f10_f10_mg37.I2000Clm45Fates.izumi_intel.clm-FatesColdDef
Some of the above are cold-starts and some are not.
Another thing I'm seeing that izumi helps us with is that some arrays need to be dimensioned by nlevmaxurbgrnd rathn than nlevgrnd. I think that will be easy to fix as well.
@mvdebolskiy it looks like the answer changing thing was the change in the soil depth for excess ice in SoilTemperature. It shouldn't matter, but turning it off when excess ice isn't on, seems to resolve the change in answers. Look at 782fc1923e8e050358588937d112cd20f6ed7712. I think we probably want it this way anyway, although it is also good to know that in most cases this change didn't matter in most instances.
@ekluzek I have a suspicion that the fails were caused because some compilers did not like when you add anything to the spval. In initVerticalMod we have loops like this one where depth between nlevgrnd and nlevurb are assigned spval and in SoilTemperature we adjust depths for 1:nlevmaxurbgrnd.
@mvdebolskiy this is looking almost ready right now. Do you want to try running the latest on Fram and make sure results look as you expect? Also please assign yourself to #1923
- [x] Matvey will run on Fram and make sure it looks as he expects with the latest version
@ekluzek I will run the latest on fram during the weekend. Is it even possible for me to assign myself to the issue on this repository? I don't see a settings button on the issue or anything.
@mvdebolskiy make sure you have set a watch level to the CTSM repository, and then I should be able to assign you to the issue. Watch level is at the very top on the right side with other settings. You are right you might not have permissions to assign people, I thought you could assign yourself, but looks like not.
Also be sure to "star" CTSM, just so we can say we are winning the repository popularity contest. :-)
@ekluzek I've just commented on the issue and you should be able to assign me. I already starred and watched the repository long ago.
- [x] Matvey will run on Fram and make sure it looks as he expects with the latest version
Just ran two 2 year global cases form with CLM_FORCE_COLDSTART=off and RESUBMIT=1 on fram with and without excess ice. Both worked.
Running a couple of single point ones made with subset_data, just to make sure.
Awesome @mvdebolskiy. There's one subtle aspect to this work that I wanted to ask about. An important aspect of this code is that if conditions warrant -- a gridcell and level of excess ice can melt. But, since this isn't usually the case, we can't guarantee this will happen for a short simulation of the model. So how often does excess ice points melt? Or another way of asking is ow long would we have to run to show that excess ice can melt? Another way to do it would be to do something like start at 2100 which should be warmer and likely to have some points that melt. Since, this isn't a standard option of the model it might be OK, to not test this. But, to fully exercise all of the code we do need to run a test where there is excess ice that melts. What would be the best way to do that?
@ekluzek even if you run 2100 all of the excess ice won't melt, since there is a lot of cold cells left with permafrost. To melt everything, one can adjust any forcing f.e. 1901-1951 with a rediculous uniform temperature change (like 20 degrees per decade or more). Or just run a small spinup 1901-1921 and then put a line in the atm2lnd to adjust air temperature by 30 degrees or so and run another 20-30 year case.
And then just look for if the model falls apart or not what difference in timing of getting rid of the permafrost inclusion of excess ice makes.
I usually try to test some ridiculous extreme cases to see if my model can withstand it.
To test the code we don't need ALL the excess ice to melt, but we want to know that at least one point melted so that code was exercised. You have a good point that it can be useful to stress test a parameterization with a ridiculous case, so that you know it functions under unusual conditions. For now I'm just musing about what starting conditions or length of simulation would guarantee that at least one point melts. You are right we could do a crazy case using anomaly forcing to apply a ridiculous warming to a short simualtion and see if that would work.
Oh, in that case -- we need to run a 20-50 years case from finidat and just look for cells that had TOTEXICE_VOL > 0 at the first time step and TOTEXICE_VOL = 0. Then just pick one cell that melts the fastest. I can't run it on fram -- too little allocated hours left on the project I am on right now. I can try on betzy but that will take some time to set up.
TODOs
- Update to latest tag: Resorted to
git merge escomp/masterbecause escomp/ctsm5.1.dev129 said "not something that we can merge" - Resolved conflicts
- Test-suites OK
- Go through this check list
@mvdebolskiy @ekluzek I'm ready to push the update to escomp/master and need permission to do so to this PR. If I remember right, I need to be made collaborator on this PR by one of you.
That's awesome @slevis-lmwg. @mvdebolskiy can you make @slevis-lmwg a collaborator on your fork of CTSM? On your fork go to the "Settings" tab and then "Collaborators" and they'll be a way to add a collaborator and you can invite Sam and give him permission to work on your fork.
@swensosc @mvdebolskiy @ekluzek I made a branch tag as follows:
git tag -a branch_tags/exice_wrpmip.n01_ctsm5.1.dev129
git push escomp branch_tags/exice_wrpmip.n01_ctsm5.1.dev129
Test suites OK and generated baselines using the branch tag's name exice_wrpmip.n01_ctsm5.1.dev129
- [x] Add a note to the issue handling that this is a permanent feature as opposed to others that might be removed.
- [x] Add issue as a parameter to restUtils and use in restFileMod so it's defined in one place.
- [x] Shorten the 12 month test, but add a note that we aren't really testing excess ice unless there's the rare chance that a gridcell will have excess ice melt. Since, this isn't a standard feature that's always on, we think this might be OK.
- [x] Only have one test for this since this isn't currently a standard feature right now.
To address these pending TODOs, I followed up with @ekluzek :
- Not shortening the 12-m test because it only runs in the ctsm_sci test-suite. However, Erik requested that I make sure to run this test because it probably didn't run in the aux_clm test_suite
SMS_Lm12.f09_f09_mg17.I1850Clm51Sp.cheyenne_intel.clm-ExcessIceStartup_output_sp_exice - Of the other excess ice test (that runs in the aux_clm test-suite), we will keep the izumi one and remove the cheyenne one.
PASS: SMS_Lm12.f09_f09_mg17.I1850Clm51Sp.cheyenne_intel.clm-ExcessIceStartup_output_sp_exice
I think this PR is ready. When it's next in line to merge, I will need to:
- Rerun the test-suites
- Update ChangeLog/ChangeSum