georinex icon indicating copy to clipboard operation
georinex copied to clipboard
verified publisher icon geospace-code

Error loading BRDC00IGS_R_20240840000_01D_MN.rnx

Open kstamatiou opened this issue 2 years ago • 2 comments

  1. Said Rinex file is the daily IGS multi-constellation broadcast ephemeris in rinex v3 format. File format is as follows:
     3.04           N: GNSS NAV DATA    M: MIXED            RINEX VERSION / TYPE
MergeMNfile.tcl     IGS                 20240325 193506 GMT PGM / RUN BY / DATE
BDSA   3.8184E-08  4.4703E-07 -3.8743E-06  7.0930E-06 Q 02  IONOSPHERIC CORR
BDSA   3.4459E-08  6.1095E-07 -4.4703E-06  7.3910E-06 W 05  IONOSPHERIC CORR
BDSA   3.4459E-08  6.1095E-07 -4.4703E-06  7.3910E-06 W 06  IONOSPHERIC CORR
BDSA   3.6322E-08  5.3644E-07 -4.2915E-06  7.5102E-06 G 07  IONOSPHERIC CORR
BDSA   3.4459E-08  6.1095E-07 -4.4703E-06  7.3910E-06 Q 08  IONOSPHERIC CORR
BDSA   3.4459E-08  6.1095E-07 -4.4703E-06  7.3910E-06 W 09  IONOSPHERIC CORR
BDSA   3.6322E-08  5.5879E-07 -4.2319E-06  6.9737E-06 K 10  IONOSPHERIC CORR
BDSA   3.4459E-08  6.0350E-07 -4.4703E-06  7.3314E-06 U 11  IONOSPHERIC CORR
BDSA   3.4459E-08  6.0350E-07 -4.4703E-06  7.3314E-06 S 12  IONOSPHERIC CORR
BDSA   3.4459E-08  6.0350E-07 -4.4703E-06  7.3314E-06 S 13  IONOSPHERIC CORR
BDSA   3.4459E-08  6.1095E-07 -4.4703E-06  7.3910E-06 W 14  IONOSPHERIC CORR
BDSA   3.4459E-08  6.1095E-07 -4.4703E-06  7.3910E-06 W 16  IONOSPHERIC CORR
BDSA   3.5390E-08  5.9605E-07 -4.3511E-06  7.1526E-06 O 19  IONOSPHERIC CORR
BDSA   3.2596E-08  5.5879E-07 -3.9935E-06  6.3181E-06 A 20  IONOSPHERIC CORR
BDSA   3.4459E-08  6.0350E-07 -4.4703E-06  7.3314E-06 S 21  IONOSPHERIC CORR
BDSA   3.4459E-08  6.1095E-07 -4.4703E-06  7.3910E-06 Q 22  IONOSPHERIC CORR
BDSA   3.4459E-08  6.1095E-07 -4.4703E-06  7.3910E-06 W 23  IONOSPHERIC CORR
BDSA   3.4459E-08  6.1095E-07 -4.4703E-06  7.3910E-06 W 24  IONOSPHERIC CORR
BDSA   3.4459E-08  6.1095E-07 -4.4703E-06  7.3910E-06 W 25  IONOSPHERIC CORR
BDSA   3.4459E-08  6.1095E-07 -4.4703E-06  7.3910E-06 W 26  IONOSPHERIC CORR
BDSA   3.4459E-08  6.1095E-07 -4.4703E-06  7.3910E-06 W 27  IONOSPHERIC CORR
BDSA   3.4459E-08  6.1095E-07 -4.4703E-06  7.3910E-06 W 28  IONOSPHERIC CORR
BDSA   3.4459E-08  6.1095E-07 -4.4703E-06  7.3910E-06 X 29  IONOSPHERIC CORR
BDSA   3.4459E-08  6.1095E-07 -4.4703E-06  7.3910E-06 W 30  IONOSPHERIC CORR
BDSA   3.4459E-08  6.1095E-07 -4.4703E-06  7.3910E-06 W 32  IONOSPHERIC CORR
BDSA   3.4459E-08  6.1095E-07 -4.4703E-06  7.3910E-06 W 33  IONOSPHERIC CORR
BDSA   3.4459E-08  6.0350E-07 -4.4703E-06  7.3314E-06 S 34  IONOSPHERIC CORR
BDSA   4.0978E-08  2.6077E-07 -2.1458E-06  3.1590E-06 Q 35  IONOSPHERIC CORR
BDSA   3.6322E-08  5.5879E-07 -4.2319E-06  6.9737E-06 K 36  IONOSPHERIC CORR
BDSA   3.4459E-08  6.0350E-07 -4.4703E-06  7.3314E-06 S 37  IONOSPHERIC CORR
BDSA   3.5390E-08  5.9605E-07 -4.3511E-06  7.1526E-06 O 38  IONOSPHERIC CORR
BDSA   3.4459E-08  6.1095E-07 -4.4703E-06  7.3910E-06 W 39  IONOSPHERIC CORR
BDSA   3.6322E-08  5.3644E-07 -4.2915E-06  7.5102E-06 G 40  IONOSPHERIC CORR
BDSA   3.4459E-08  6.1095E-07 -4.4703E-06  7.3910E-06 W 41  IONOSPHERIC CORR
BDSA   3.4459E-08  6.1095E-07 -4.4703E-06  7.3910E-06 W 42  IONOSPHERIC CORR
BDSA   3.4459E-08  6.0350E-07 -4.4703E-06  7.3314E-06 U 43  IONOSPHERIC CORR
BDSA   3.4459E-08  6.1095E-07 -4.4703E-06  7.3910E-06 Q 44  IONOSPHERIC CORR
BDSA   3.6322E-08  5.6624E-07 -4.2319E-06  6.8545E-06 M 45  IONOSPHERIC CORR
BDSA   3.6322E-08  5.3644E-07 -4.2915E-06  7.5102E-06 G 46  IONOSPHERIC CORR
BDSA  -9.3132E-09 -6.8545E-07  1.1921E-07  3.3379E-06 D 50  IONOSPHERIC CORR
BDSB   8.8064E+04  7.2090E+05 -5.3084E+06  5.8327E+06 Q 02  IONOSPHERIC CORR
BDSB   1.1469E+05  5.2429E+05 -5.1773E+06  6.0293E+06 W 05  IONOSPHERIC CORR
BDSB   1.1469E+05  5.2429E+05 -5.1773E+06  6.0293E+06 W 06  IONOSPHERIC CORR
BDSB   9.2160E+04  1.0322E+06 -7.7332E+06  7.9299E+06 G 07  IONOSPHERIC CORR
BDSB   1.1674E+05  5.5706E+05 -5.6361E+06  6.6191E+06 Q 08  IONOSPHERIC CORR
BDSB   1.1469E+05  5.2429E+05 -5.1773E+06  6.0293E+06 W 09  IONOSPHERIC CORR
BDSB   8.8064E+04  1.1305E+06 -8.3231E+06  8.3231E+06 K 10  IONOSPHERIC CORR
BDSB   1.1674E+05  5.2429E+05 -5.2429E+06  6.0948E+06 U 11  IONOSPHERIC CORR
BDSB   1.1059E+05  6.5536E+05 -6.0293E+06  6.8157E+06 S 12  IONOSPHERIC CORR
BDSB   1.1059E+05  6.5536E+05 -6.0293E+06  6.8157E+06 S 13  IONOSPHERIC CORR
BDSB   1.1469E+05  5.2429E+05 -5.1773E+06  6.0293E+06 W 14  IONOSPHERIC CORR
BDSB   1.1469E+05  5.2429E+05 -5.1773E+06  6.0293E+06 W 16  IONOSPHERIC CORR
BDSB   1.0445E+05  7.7005E+05 -6.8157E+06  7.5366E+06 O 19  IONOSPHERIC CORR
BDSB   9.2160E+04  1.1469E+06 -8.3231E+06  8.3231E+06 A 20  IONOSPHERIC CORR
BDSB   1.1059E+05  6.5536E+05 -6.0293E+06  6.8157E+06 S 21  IONOSPHERIC CORR
BDSB   1.1674E+05  5.5706E+05 -5.6361E+06  6.6191E+06 Q 22  IONOSPHERIC CORR
BDSB   1.1469E+05  5.2429E+05 -5.1773E+06  6.0293E+06 W 23  IONOSPHERIC CORR
BDSB   1.1469E+05  5.2429E+05 -5.1773E+06  6.0293E+06 W 24  IONOSPHERIC CORR
BDSB   1.1469E+05  5.2429E+05 -5.1773E+06  6.0293E+06 W 25  IONOSPHERIC CORR
BDSB   1.1469E+05  5.2429E+05 -5.1773E+06  6.0293E+06 W 26  IONOSPHERIC CORR
BDSB   1.1469E+05  5.2429E+05 -5.1773E+06  6.0293E+06 W 27  IONOSPHERIC CORR
BDSB   1.1469E+05  5.2429E+05 -5.1773E+06  6.0293E+06 W 28  IONOSPHERIC CORR
BDSB   1.1469E+05  5.2429E+05 -5.1773E+06  6.0293E+06 X 29  IONOSPHERIC CORR
BDSB   1.1469E+05  5.2429E+05 -5.1773E+06  6.0293E+06 W 30  IONOSPHERIC CORR
BDSB   1.1469E+05  5.2429E+05 -5.1773E+06  6.0293E+06 W 32  IONOSPHERIC CORR
BDSB   1.1469E+05  5.2429E+05 -5.1773E+06  6.0293E+06 W 33  IONOSPHERIC CORR
BDSB   1.1059E+05  6.5536E+05 -6.0293E+06  6.8157E+06 S 34  IONOSPHERIC CORR
BDSB   1.2902E+05 -3.1130E+05  1.3107E+06 -3.2768E+05 Q 35  IONOSPHERIC CORR
BDSB   8.8064E+04  1.1305E+06 -8.3231E+06  8.3231E+06 K 36  IONOSPHERIC CORR
BDSB   1.1059E+05  6.5536E+05 -6.0293E+06  6.8157E+06 S 37  IONOSPHERIC CORR
BDSB   1.0445E+05  7.7005E+05 -6.8157E+06  7.5366E+06 O 38  IONOSPHERIC CORR
BDSB   1.1469E+05  5.2429E+05 -5.1773E+06  6.0293E+06 W 39  IONOSPHERIC CORR
BDSB   9.2160E+04  1.0322E+06 -7.7332E+06  7.9299E+06 G 40  IONOSPHERIC CORR
BDSB   1.1469E+05  5.2429E+05 -5.1773E+06  6.0293E+06 W 41  IONOSPHERIC CORR
BDSB   1.1469E+05  5.2429E+05 -5.1773E+06  6.0293E+06 W 42  IONOSPHERIC CORR
BDSB   1.1674E+05  5.2429E+05 -5.2429E+06  6.0948E+06 U 43  IONOSPHERIC CORR
BDSB   1.1674E+05  5.5706E+05 -5.6361E+06  6.6191E+06 Q 44  IONOSPHERIC CORR
BDSB   8.8064E+04  1.1305E+06 -8.3231E+06  8.3231E+06 M 45  IONOSPHERIC CORR
BDSB   9.2160E+04  1.0322E+06 -7.7332E+06  7.9299E+06 G 46  IONOSPHERIC CORR
BDSB  -1.8842E+05 -7.5366E+05 -7.3400E+06  4.1288E+06 D 50  IONOSPHERIC CORR
GAGP  4.8312358558E-09-2.664535259E-15      0 2307          TIME SYSTEM CORR
GAL    1.6725E+02 -1.0547E-01  7.2937E-03                   IONOSPHERIC CORR
GPSA   3.9116E-08  1.4901E-08 -1.7881E-07  0.0000E+00       IONOSPHERIC CORR
GPSB   1.4336E+05  0.0000E+00 -3.2768E+05  2.6214E+05       IONOSPHERIC CORR
QZSA   1.0245E-08 -1.4901E-08 -4.1723E-07  1.1921E-06       IONOSPHERIC CORR
QZSB   1.0240E+05  0.0000E+00 -8.5197E+05  8.3231E+06       IONOSPHERIC CORR
BDSA   3.4459E-08  6.1095E-07 -4.4703E-06  7.3910E-06 W 60  IONOSPHERIC CORR
BDSB   1.1469E+05  5.2429E+05 -5.1773E+06  6.0293E+06 W 60  IONOSPHERIC CORR
GAUT  1.8626451492E-09-8.881784197E-16 518400 2306          TIME SYSTEM CORR
GPUT -2.7939677238E-09-1.776356839E-15 233472 2307          TIME SYSTEM CORR
GLUT  3.2596290112E-09 0.000000000E+00      0    0          TIME SYSTEM CORR
QZUT -1.8626451492E-09 0.000000000E+00 356352 2158          TIME SYSTEM CORR
BDUT -9.3132257462E-10 2.042810365E-14 604745  950          TIME SYSTEM CORR
    18    18  1929     7                                    LEAP SECONDS
                                                            END OF HEADER
G01 2024 03 23 23 59 44 1.756171695888E-04 2.160049916711E-12 0.000000000000E+00
     2.000000000000E+01 4.296875000000E+01 3.556219559390E-09 2.959257184757E+00
     2.056360244751E-06 1.293790643103E-02 1.289322972298E-05 5.154013608932E+03
     6.047840000000E+05 4.656612873077E-08-2.990394506620E+00-1.918524503708E-07
     9.909930100366E-01 1.444062500000E+02 1.002905023843E+00-7.299589771500E-09
     1.003613233091E-10 1.000000000000E+00 2.306000000000E+03 0.000000000000E+00
     2.000000000000E+00 6.300000000000E+01 5.122274160385E-09 2.000000000000E+01
     6.012180000000E+05 4.000000000000E+00
  1. The error that georinex.load() returns is:
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[2], line 2
      1 filepath = "/home/kostas/Downloads/BRDC00IGS_R_20240840000_01D_MN.rnx"
----> 2 dataset = gr.load(filepath)

File ~/zephr-xyz/zephyrus/.venv/lib/python3.10/site-packages/georinex/base.py:65, in load(rinexfn, out, use, tlim, useindicators, meas, verbose, overwrite, fast, interval)
     62 info = rinexinfo(rinexfn)
     64 if info["rinextype"] == "nav":
---> 65     return rinexnav(rinexfn, outfn, use=use, tlim=tlim, overwrite=overwrite)
     66 elif info["rinextype"] == "obs":
     67     return rinexobs(
     68         rinexfn,
     69         outfn,
   (...)
     77         interval=interval,
     78     )

File ~/zephr-xyz/zephyrus/.venv/lib/python3.10/site-packages/georinex/base.py:166, in rinexnav(fn, outfn, use, group, tlim, overwrite)
    164     nav = rinexnav2(fn, tlim=tlim)
    165 elif int(info["version"]) == 3:
--> 166     nav = rinexnav3(fn, use=use, tlim=tlim)
    167 else:
    168     raise LookupError(f"unknown RINEX  {info}  {fn}")

File ~/zephr-xyz/zephyrus/.venv/lib/python3.10/site-packages/georinex/nav3.py:98, in rinexnav3(fn, use, tlim)
     95 duplicate = tu.size != t[svi][check].size
     97 full_fields = fields[sv[0]]
---> 98 compact_fields = _sparefields(full_fields, sys=sv[0], N=_num_fields(raws[svi[0]]))
     99 compact_index = [i for i, c in enumerate(full_fields) if c in compact_fields]
    101 darr = np.empty((svi.size, len(full_fields)))

File ~/zephr-xyz/zephyrus/.venv/lib/python3.10/site-packages/georinex/nav3.py:234, in _sparefields(cf, sys, N)
    231         cf = cf[:28]
    233 if N != len(cf):
--> 234     raise ValueError(f"System {sys} NAV data is not the same length as the number of fields.")
    236 return cf

ValueError: System I NAV data is not the same length as the number of fields.
  1. Operating system is Ubuntu 22.04.4. LTS, Python version is 3.10.10, and I have installed the latest georinex release.

kstamatiou avatar Mar 25 '24 23:03 kstamatiou

This is representative of a larger issue.

  1. For a temporary fix @kstamatiou, you could clone the repository and install as a development version. Then, change these lines to be
elif sys == "I":
    if N < 28 and len(cf) >= 28:
        N = 28
        cf = cf[:28]
    elif N == 28:
        cf = cf[:28]

Also, the cropped BRDC00IGS_R_20240840000_01D_MN.rnx file you posted in this issue should work with or without this fix since the issue is with IRNSS (system I) and not GPS (system G). I downloaded the full rinex file from IGS and have attached here a cropped version of the rinex that includes an "I02" entry.

BRDC00IGS_R_20240840000_01D_MN.zip

  1. I think the larger issue at hand is that #98 changed the logic of the _sparefields check. Now there are many possibilities of N being smaller than the standard value for its constellation since missing values that are treated as zero may be present at the end of the entry. In a previous commit, I made a patch to fix the same issue with QZSS, but in this issue the IRNSS entries have a similar problem.

@scivision, I'm happy to create a pull request to fix this specific IRNSS issue, but I don't know if you'd rather have me reevaluate the _sparefields function or even the _num_fields function to fix the overall problem and not just patch by constellation. Let me know what you'd think is best.

betaBison avatar Mar 27 '24 18:03 betaBison

Thanks a lot for the reply @betaBison.

Indeed, the problem is due to IRNSS, so the interim solution that I came up with is to simply truncate the file in order to exclude these entries; then load() works just fine.

kstamatiou avatar Mar 27 '24 19:03 kstamatiou