MintPy icon indicating copy to clipboard operation
MintPy copied to clipboard

Issue after ISCE2 Stitching of HyP3 Sentinel-1 Burst Products

Open JianBo17 opened this issue 11 months ago • 3 comments

Description of the problem Recently, HyP3 started providing Sentinel-1 Burst InSAR products, and MintPy has updated support for Multi-Burst products. However, HyP3 currently only supports up to 15 bursts per Multi-Burst product. To process more than 15 bursts, I downloaded multiple individual burst products (not Multi-Burst) from HyP3 and stitched them using ISCE2, which is capable of handling a larger number of bursts. The stitching process completed successfully, and the merged products are shown below.

Image

However, when I tried to run smallbaselineApp.py in MintPy, I encountered a few issues:

  1. MintPy could not recognize the ISCE2-stitched product names.

******************** step - load_data ********************

load_data.py --template /home/jianbo/hyp3/test/burst_data/SBAS/smallbaselineApp.cfg processor : hyp3 SAR platform/sensor : unknown from project name "None"

prepare metadata files for hyp3 products prep_hyp3.py "./data//unw_phase.tif" /home/jianbo/anaconda3/envs/InSAR/lib/python3.11/site-packages/osgeo/gdal.py:311: FutureWarning: Neither gdal.UseExceptions() nor gdal.DontUseExceptions() has been explicitly called. In GDAL 4.0, exceptions will be enabled by default. warnings.warn( Traceback (most recent call last): File "/home/jianbo/anaconda3/envs/InSAR/bin/smallbaselineApp.py", line 8, in sys.exit(main()) ^^^^^^ File "/home/jianbo/InSAR/MintPy/src/mintpy/cli/smallbaselineApp.py", line 209, in main run_smallbaselineApp(inps) File "/home/jianbo/InSAR/MintPy/src/mintpy/smallbaselineApp.py", line 1155, in run_smallbaselineApp app.run(steps=inps.runSteps) File "/home/jianbo/InSAR/MintPy/src/mintpy/smallbaselineApp.py", line 908, in run self.run_load_data(sname) File "/home/jianbo/InSAR/MintPy/src/mintpy/smallbaselineApp.py", line 176, in run_load_data mintpy.cli.load_data.main(iargs) File "/home/jianbo/InSAR/MintPy/src/mintpy/cli/load_data.py", line 123, in main load_data(inps) File "/home/jianbo/InSAR/MintPy/src/mintpy/load_data.py", line 812, in load_data prepare_metadata(iDict) File "/home/jianbo/InSAR/MintPy/src/mintpy/load_data.py", line 636, in prepare_metadata prep_module.main(iargs) File "/home/jianbo/InSAR/MintPy/src/mintpy/cli/prep_hyp3.py", line 94, in main prep_hyp3(inps) File "/home/jianbo/InSAR/MintPy/src/mintpy/prep_hyp3.py", line 164, in prep_hyp3 meta = add_hyp3_metadata(fname, meta, is_ifg=is_ifg) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/jianbo/InSAR/MintPy/src/mintpy/prep_hyp3.py", line 49, in add_hyp3_metadata date1, date2 = (dt.datetime.strptime(x,'%Y%m%dT%H%M%S') for x in job_id.split('')[1:3]) ^^^^^^^^^^^^ File "/home/jianbo/InSAR/MintPy/src/mintpy/prep_hyp3.py", line 49, in date1, date2 = (dt.datetime.strptime(x,'%Y%m%dT%H%M%S') for x in job_id.split('_')[1:3]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/jianbo/anaconda3/envs/InSAR/lib/python3.11/_strptime.py", line 567, in _strptime_datetime tt, fraction, gmtoff_fraction = _strptime(data_string, format) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/jianbo/anaconda3/envs/InSAR/lib/python3.11/_strptime.py", line 349, in _strptime raise ValueError("time data %r does not match format %r" % ValueError: time data '142' does not match format '%Y%m%dT%H%M%S'

The naming convention did not match MintPy’s expected format. The format of ISCE stitched products and txt file as follows:

Image

Image

I fixed this by:

Renaming the product files to match the HyP3 SLC InSAR naming convention.

Editing the associated .txt files to update the "Reference Granule" and "Secondary Granule" fields. Since ISCE2's merged .txt contains multiple burst timestamps, I extracted and retained the earliest and latest timestamps.The file name and txt file after editing as follows:

Image

Image

After renaming and editing, MintPy was able to recognize and load the data.

  1. However, in the final velocity map, the individual bursts were not stitched together — there were visible gaps between the bursts (as shown in the attached figure).

Image Interestingly, this issue does not occur in the timeseries.h5 file.The time-series outputs show properly connected bursts, indicating that the spatial stitching worked correctly at the time series level.

Image

Although the time series file appears to be connected across bursts, it still shows unusual behavior, with deformation values from different dates being nearly identical in some cases.

System information

  • Operating system: Ubuntu 22.04 LTS
  • Python environment: conda
  • MintPy version: MintPy version 1.6.1.post26, date 2025-04-01
  • InSAR processor/product: hyp3, isce2
  • Your custom / default template file (if the bug is related to a specific dataset):

smallbaselineApp.txt

JianBo17 avatar May 28 '25 12:05 JianBo17

Verify the hd5 files meta data and paste it here.

you can use command h5ls

https://mintpy.readthedocs.io/en/latest/api/data_structure/

barely-sad-one avatar Jun 05 '25 10:06 barely-sad-one

The detail information of hd5 file by h5ls as follows:

~/hyp3/test/burst_data/SBAS$ h5ls velocity.h5 
intercept                Dataset {2805, 2688}
interceptStd             Dataset {2805, 2688}
residue                  Dataset {2805, 2688}
velocity                 Dataset {2805, 2688}
velocityStd              Dataset {2805, 2688}
~/hyp3/test/burst_data/SBAS$ h5ls velocity.h5 timeseries.h5 
intercept                Dataset {2805, 2688}
interceptStd             Dataset {2805, 2688}
residue                  Dataset {2805, 2688}
velocity                 Dataset {2805, 2688}
velocityStd              Dataset {2805, 2688}
bperp                    Dataset {22}
date                     Dataset {22}
timeseries               Dataset {22/Inf, 2805, 2688}

JianBo17 avatar Jun 09 '25 15:06 JianBo17