nnpdf icon indicating copy to clipboard operation
nnpdf copied to clipboard

Remove HepData raw tables from the repository

Open Radonirinaunimi opened this issue 1 year ago • 2 comments

Addresses https://github.com/NNPDF/nnpdf/issues/2199 by removing the HepData raw tables from the repository. This adds a module that downloads the raw tables from the server and from the HepData when the commondata need to be regenerated.

TO DO:

  • [ ] download the raw tables from the NNPDF server
  • [ ] propagate across all the datasets
    • [ ] fixed-target DIS
    • [ ] collider DIS
    • [ ] fixed-target inclusive DY
    • [ ] Tevatron inclusive DY
    • [ ] ATLAS inclusive DY
    • [ ] ATLAS DY+jet
    • [ ] ATLAS single-inclusive jet and dijet production
    • [ ] ATLAS top pair production
    • [ ] ATLAS single top production
    • [ ] ATLAS pormpt photon production
    • [ ] CMS inclusive DY
    • [ ] CMS DY+jet
    • [ ] CMS single-inclusive jet and dijet production
    • [ ] CMS top pair production
    • [ ] CMS single top production
    • [ ] LHCb inclusive DY
  • [ ] fail commondata tests only when kinematics.yaml, data.yaml, and uncertainties*.yaml are modified

Radonirinaunimi avatar Nov 27 '24 02:11 Radonirinaunimi

I understood @scarlehoff was going to purge the rawdata (that already got merged, which shouldn't be much) from the history entirely, not just remove it from the repo but keep it in the history?

RoyStegeman avatar Nov 27 '24 11:11 RoyStegeman

the raw tables from the server and from the HepData when the commondata need to be regenerated.

And, after regeneration, the blob of data in the server gets updated.

I understood @scarlehoff was going to purge the rawdata (that already got merged, which shouldn't be much) from the history entirely, not just remove it from the repo but keep it in the history?

First we need the infrastructure so that it is truly not needed, then we change the history of the repo.

scarlehoff avatar Nov 27 '24 12:11 scarlehoff