sunpy
No map source to handle GONG H-alpha data
Describe the bug
Originally raised by nawinnga in the sunpy chat room.
There is no map source to do the necessary translation from the GONG H-alpha headers to standard meta data Map can use related to #6653 and #6655.
It doesn't look like its trivial to find/extract/calculate the necessary WCS information (e.g. PCi_j + CDELT or CDi_j ...) without some reference to what the fits headers mean some info on this NSO page
To Reproduce
from astropy.io import fits
import sunpy
# to get around issue #6655
hdul = fits.open('ftp://gong2.nso.edu/HA/haf/202203/20220318/20220318000050Bh.fits.fz')
sunpy.map.Map((hdul[1].data, hdul[1].header))
Map((hdul[1].data, hdul[1].header))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/shane/.virtualenvs/temp/lib/python3.9/site-packages/sunpy/map/map_factory.py", line 331, in __call__
new_map = self._check_registered_widgets(data, meta, **kwargs)
File "/Users/shane/.virtualenvs/temp/lib/python3.9/site-packages/sunpy/map/map_factory.py", line 381, in _check_registered_widgets
return WidgetType(data, meta, **kwargs)
File "/Users/shane/.virtualenvs/temp/lib/python3.9/site-packages/sunpy/map/mapbase.py", line 232, in __init__
self._validate_meta()
File "/Users/shane/.virtualenvs/temp/lib/python3.9/site-packages/sunpy/map/mapbase.py", line 1409, in _validate_meta
raise MapMetaValidationError('\n'.join(err_message))
sunpy.map.mapbase.MapMetaValidationError: Image coordinate units for axis 1 not present in metadata.
Image coordinate units for axis 2 not present in metadata.
See https://docs.sunpy.org/en/stable/code_ref/map.html#fixing-map-metadata for instructions on how to add missing metadata.
Setting the cdelt and cunit at least get lets the map be loaded but the WCS is still non-standard
hdul[1].header['cdelt1'] = 1
hdul[1].header['cdelt2'] = 1
hdul[1].header['cunit1'] = 'arcsec'
hdul[1].header['cunit2'] = 'arcsec'
mm = Map((hdul[1].data, hdul[1].header))
WARNING: SunpyMetadataWarning: Missing metadata for observer: assuming Earth-based observer.
For frame 'heliographic_stonyhurst' the following metadata is missing: hgln_obs,hglt_obs,dsun_obs
For frame 'heliographic_carrington' the following metadata is missing: crlt_obs,dsun_obs,crln_obs
[sunpy.map.mapbase]
WARNING: SunpyUserWarning: Could not determine coordinate frame from map metadata.
Could not determine celestial frame corresponding to the specified WCS object [sunpy.map.mapbase]
<sunpy.map.mapbase.GenericMap object at 0x1297eb160>
SunPy Map
---------
Observatory: NSO-GONG
Instrument:
Detector:
Measurement: 6562.808
Wavelength: 6562.808
Observation Date: 2022-03-18 00:00:50
Exposure Time: Unknown
Dimension: [2048. 2048.] pix
Coordinate System: Unknown
Scale: [1. 1.] arcsec / pix
Reference Pixel: [1023. 1023.] pix
Reference Coord: [0. 0.] arcsec
array([[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
...,
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0]], dtype=int16)
Screenshots
No response
System Details
OS: Mac OS 10.16 Arch: 64bit, (i386) python3.9 sunpy: 4.1.0 astropy: 5.1.1 numpy: 1.23.5
Installation method
pip
CDELT1 and CDELT2 are at least in the ROB-generated JP2 files, with values of 1.082371820584221. Helioviewer FITS to JP2 processing usually just copies the FITS header into the JP2 header.
I still had this header in my terminal might be useful ...
SIMPLE = T / Fits standard
BITPIX = 16 / Bits per pixel
NAXIS = 2 / Number of axes
NAXIS1 = 2048 / Axis length
NAXIS2 = 2048 / Axis length
GONGKEY1= '--------- Gong Keywords Section 1: Solarsoft ----------------'
ORIGIN = 'National Solar Observatory -- GONG' / FITS file originator
OBS-SITE= 'NSO/GONG NETWORK' / Instrument Site location
TELESCOP= 'NSO-GONG' / NSO/GONG Network
OBS-URL = 'http://gong.nso.edu' / The GONG website
DATE = '2022-03-18T00:01:24' / Date FITS file was generated
DATE-OBS= '2022-03-18T00:00:50' / Observation start date and time (UTC)
TIME-OBS= '00:00:50' / Observation start time (UTC)
WAVELNTH= 6562.808 / [A] exact wavelength of obs
WCSNAME = 'Heliocentric-cartesian (approximate)'
CTYPE1 = 'GONGX ' / Coordinate Axis Label
CTYPE2 = 'GONGY ' / Coordinate Axis Label
CRVAL1 = 0.00000 / Reference Pixel value
CRVAL2 = 0.00000 / Reference Pixel value
CRPIX1 = 1024.0
CRPIX2 = 1024.0
EPH_RA = -0.042049004 / Right Acension (radians)
EPH_DEC = -0.018226864 / Declination (radians)
IMTYPE = 'H-ALPHA '
IMGMN01 = 1828.983276367 / Image mean
IMGRMS01= 1389.129638672 / Image RMS (Standard Deviation)
IMGSKW01= -0.362854451 / Image skewness
IMGMIN01= 0 / Image Min
IMGMAX01= 3813 / Image Max
IMGADV01= 1347.447021484 / Image Average Deviation
IMGVAR01= 1929681.125000000 / Image Variance
IMGKUR01= -1.758516669 / Image Kurtosis
GONGKEY2= '--------- Gong Keywords Section 2: Raw ----------------'
IRAF-TLM= '2022-03-18T00:01:24' / Time of last modification
SITENAME= 'BB '
SITE = 'BB '
J2000 = 8111.500521269 / Julian 2000 date
LST = 1.026294883 / Local apparent sidereal time (radians)
ZENITH = 1.172003679 / Zenith distance (radians)
RA = -0.042049004 / Right ascension (radians)
DEC = -0.018226864 / Declination (radians)
PA = -0.433714898 / Solar position angle (radians)
HA = 1.068343888 / Hour angle (radians)
BETA = 0.540893145 / Parallactic angle (radians)
RADIUS = 900
EROLL = 1.15355695 / Ephemeris roll angle (radians)
EPITCH = -0.290095055 / Ephemeris pitch angle (radians)
EROTATOR= 1.337248826 / Ephemeris rotator angle (radians)
ROLL = 1.158456442 / Roll position (radians)
PITCH = -0.290830509 / Pitch position (radians)
ROTATOR = 1.33718651 / Rotator position (radians)
SMEAR = 0.016 / Camera smear correction parameter
CAMERA = 'DVC 4000M-CL' / Camera Model
CAMERASN= 3344 / Camera serial number
SHARPNSS= 0.00868893836065888 / Image sharpness quantifier (0=Worst,1=Best)
SKYBRITE= 194 / Average of square regions in four corners
ECLIPSE = 0 / Eclipse Flag (1=TRUE, 0=FALSE)
HASERNUM= 'QPE101409DH' / H-Alpha filter Serial Number
HAVERSN = 1.6 / H-Alpha filter Firmware Version
HAERRCOD= 0 / H-Alpha filter Error Code (0=Good)
HAONBAND= 1 / Filter On-Band status (0=Controlling,1=Setpt)
HACNTRWL= 6562.7 / H-Alpha filter current Center Wavelength
HAWNGSHF= -0.1 / H-Alpha filter Wavelength Shift setting
HAHT1PWM= 20.23 / Filter Heater 1 Power Module setting (0-100%)
HALMTPWM= 1023 / Filter Power Modules Limit setting (<=1023)
HATEMP1 = 137.63 / Filter Heater 1 current Temperature (F)
HAVOLTS = 11.02 / H-Alpha filter Current Voltage
HAPOTPOS= 0 / Filter calibration Pot Postion (0=Normal)
HATEMP2 = 156.48 / Filter Heater 2 current Temperature (F)
HAHT2PWM= 55.62 / Filter Heater 2 Power Module setting (0-100%)
HATURNON= 287 / H-Alpha filter Turn On count
HA-AGE = 6452150 / H-Alpha filter power-on Age in minutes
RMS_XROI= 960 / X starting position of RMS ROI box (GONG)
RMS_YROI= 960 / Y starting position of RMS ROI box (GONG)
RMS_SIZE= 181 / Axis length of RMS ROI box (GONG)
RMS = 3354.7450289537 / RMS of pixel values in RMS ROI box (GONG)
GTCLMBXC= 1039.99 / Saved getctr values
GTCLMBYC= 1010.36
GTCLMBRA= 900.38
GTCLMBSX= 0.25
GTCLMBSY= 0.367
GTCLMBSR= 0.267
GTC_FLAG= 0
GEOLMBXC= 1040.322 / Saved geom values
GEOLMBYC= 1010.009
GEOLMBMA= 901.151
GEOLMBMI= 899.286
GEOLMBAN= 67.419
GONGKEY3= '--------- Gong Keywords Section 3: Processing History ------------'
HALKEY10= 'H Alpha Site Paramters'
HA-OBJ = 61.2115958509971 / [arcdeg] sun's local hour angle (+=West)
DEC-OBJ = -1.04432238095894 / [arcdeg] sun's declination (+=North)
AZ-OBJ = 0.0 / [arcdeg] sun's azimuth
EL-OBJ = 22.8491356194947 / [arcdeg] sun's elevation
ETA-OBJ = 30.9908943760577 / [arcdeg] sun's parallactic angle (+=North)
SOLAR-P = -24.8430963822425 / [arcdeg] solar P angle
SOLAR-B0= -7.10886621520643 / [arcdeg] solar B0 angle
SOLAR-L0= 221.646911730488 / [arcdeg] Carrington Longitude
SOLAR-SR= 2255 / Synodic Rotation Number
SOLAR-R = 964.304264124869 / [arcsec] solar semi-diameter
SITE-LON= -116.921419993861 / [arcdeg] Site's longitude (+=East)
SITE-LAT= 34.2603299816775 / [arcdeg] Site's latitude (+=North)
CCD-ROT = 76.615143444828 / [arcdeg] CCD camera angle
BC-METHD= 'radial ' / Calibration method
BC-SETPT= 3200.0 / [a/d cts] ROI setpoint (GONG)
BC-VALUE= 3367.0 / [a/d cts] central intensity (GONG)
BC-SCALE= 0.95040095040095 / scale factor applied (GONG)
BC-TILT1= 0.0 / plane tilt removed:
BC-TILT2= 0.0 / plane = t[1] + t[2]*x + t[3]*y
BC-TILT3= 0.0
DISKPRO1= 0.0 / Disk Profile Coefs.: p[]
DISKPRO2= 0.0 / phi = 1 - cosT*ln(1 + 1/cosT)
DISKPRO3= 0.0 / profile = p[1] + p[2]*cosT + p[3]*phi
HALKEY11= 'HAL Processing History'
GHISTSEQ= 9
GHIST000= 'Task GEOM (22/03/18 00:01:21, V2.14)'
GHIST001= 'edge_type=gradient,opt_type=ols,kern_width=3,usehdr=no,auto_type'
GHIST002= '=none,xcenter=1039.99,ycenter=1010.36,majorax=900.38,minorax=900'
GHIST003= '.38,angle=0.,nfit=5,width=30.,pcfit=1.,pixlenx=1.,pixleny=1.,fco'
GHIST004= 'l=1,lcol=2048,frow=1,lrow=2048,niter=1,finterp_fact=0,output=no,'
GHIST005= 'resout=no,lmbout=no,updhdr=yes,err_action=skip'
GHIST006= 'Task LMBCOR (22/03/18 00:01:23, V2.14)'
GHIST007= 'sw_dim=yes,n1_ot=0,n2_ot=0,sw_pix=no,pxlx_ot=1.,pxly_ot=1.,sw_li'
GHIST008= 'mb=no,xcenter=1024.,ycenter=1024.,minorax=900.,majorax=900.,angl'
GHIST009= 'e=0.,pcr=115.,rot=81.2151,interp=4'
FNDLMBXC= 1024.0
FNDLMBYC= 1024.0
FNDLMBMI= 900.0
FNDLMBMA= 900.0
FNDLMBAN= 0.0
PIXLENX = 1.0
PIXLENY = 1.0
OFFSET = 4.6 / Per site camera offset angle (degrees,+=CCW)
COMMENT
COMMENT This GONG product is produced by the H Alpha Laminator (HAL).
COMMENT
COMMENT See http://gong.nso.edu for more information about GONG.
COMMENT
PLATFORM= 'Linux ncs-gong-proc3-lx'
FITSWASH= '$RCSfile: halpha_header.kw,v $ $Revision: 1.15 $ $Date: 2011/06/29 1'
CHECKSUM= 'jJIpj99njGGnj99n' / HDU checksum updated 2022-03-18T00:01:24
DATASUM = '1012793542' / data unit checksum updated 2022-03-18T00:01:24
EXTEND = T / File may contain extensions
I'd be interested in seeing this resolved and would be willing to help write the source as it would make one of my current projects easier. Who would be good to contact about the metadata clarification needed?
I manually created a map by editing the downloaded GONG h-alpha header and adding the CDELT values @wafels suggested. I was able to reproject and get a decent correspondence between a SUVI 195 angstrom image and the GONG h-alpha. But it would be great to avoid the hacking and get this into SunPy.
What exactly is missing in the GONG FITS files? Rotation information, unit and equivalent pixel size at least. Anything else? I ran it through the online FITS verifier and got the following error report:
File name: 20230622000042Lh.fits.fz
Run Number 34344
fitsverify 4.20 (CFITSIO V3.470)
--------------------------------
*** Error: (from CFITSIO error stack:)
Couldn't find the string 'SIMPLE' in the stdin stream.
This does not look like a FITS file.
failed to copy stdin into memory (stdin_open)
failed to find or open the following file: (ffopen)
-
**** Abort Verification: Fatal Error. ****
From the NISP Data Center Coordinator ([email protected])
Meanwhile, you could use the following GONG H-alpha keywords to get CDELT1, CDELT2 and CROTA values.
The pixel size CDELT1 and CDELT2 can be computed using two other magnitudes that are included in the GONG H-alpha file headers:
- Radius of the solar disk in pixels in x and y directions: FNDLMBMI and FNDLMBMA. H-alpha images have been sized to circular-sun shape with a fixed solar-image radius of 900 pixels.
- Semidiameter of the solar disk in [arcsec]: SOLAR-R.
~~CROTA2 is a solar P-angle. In GONG H-alpha headers it corresponds to the SOLAR-P keyword.~~
I need to make a correction for what I said the GONG H-alpha CROTA2 keyword is. For raw GONG H-alpha images CROTA2 = ROTATOR + OFFSET, but for the H-alpha data available for download CROTA2 = 0.0 degrees. So, please, use CROTA2=0.0 degrees for the GONG H-alpha CROTA keyword.
This DOI 10.25668/as28-7p13 points to some documentation and if you follow the links (file you can get to this page which is the best "reference" I can find.