Build packages from branches

[Zaharid]

We should try to have packages from non-master branches that can be trivially used to e.g. run fits.

At the moment we can upload the packages that get built, but those are indistinguishable from nnpdf-master so that is not very good. Instead we should have:

• Different package names.
• Packages that are mutually incompatible (you can only have one NNPDF package from any branch installed).

Conda used to make that very complicated but now there is a feature that may be just right: https://docs.conda.io/projects/conda/en/latest/user-guide/concepts/packages.html#mutex-metapackages

The idea would be that you can just do conda install nnpdf-flbasis and run that out of the box.

[scarlehoff]

Why not different package versions? Why would nnpdf=flbasis not be ok and easier (as automatically ensures that only one nnpdf package is installed) Also, wouldn't also be easier to create a package nnpdf-flbasis that has nnpdf as an anti-dependency (I'm guessing it can be done, maybe not)?

Asking because I'm not sure what do you have in mind for this mutex feature to make things easier or "just right".

In any case: Won't we have too many useless packages that way? Do we want to do it for all branches or rather do something like:

if [PACKAGED] in PR name then upload package

I know how to do this for github actions and it is reasonably easy but they would only be for Linux because 1) the known problem with the minutes limit 2) I don't have a Mac.

I can check how to do it with Travis, it might also be possible.

[Zaharid]

Why not different package versions? Why would nnpdf=flbasis not be ok and easier (as automatically ensures that only one nnpdf package is installed)

Versions imply a sort order. Here we want incompatible branches as you say below.

Also, wouldn't also be easier to create a package nnpdf-flbasis that has nnpdf as an anti-dependency (I'm guessing it can be done, maybe not)?

See about mutex packages in the message above.

Asking because I'm not sure what do you have in mind for this mutex feature to make things easier or "just right".

AFAICT it is a thing that says you can have "nnpdf" or "nnpdf-flbasis" but not both, which is just what we want. This is a relatively new thing: conda used to have something called "features" which was utterly broken. I guess this is the right way(tm).

In any case: Won't we have too many useless packages that way?

I'd say we would enable this manually for the branches we want, like we can do now.

Do we want to do it for all branches or rather do something like:

if [PACKAGED] in PR name then upload package

I know how to do this for github actions and it is reasonably easy but they would only be for Linux because 1) the known problem with the minutes limit 2) I don't have a Mac.

I can check how to do it with Travis, it might also be possible.

I'd say we mainly want to do it in travis. It "should" be a "minor" modification of the recipe, to build a different name and make the mutex thing.

[scarlehoff]

Versions imply a sort order. Here we want incompatible branches as you say below.

I said version but I realise now I meant builds https://docs.conda.io/projects/conda-build/en/latest/resources/variants.html

See about mutex packages in the message above.

I clearly didn't understand the thing about mutex packages in the message above as I don't see how they can be used to build a package per branch like you say. Actually reading the documentation I understand it does the = thing:

The metapackage is what allows us to achieve the mutual exclusivity. It unifies the options on a single package name, but with a different build string

Also reading the conda documentation is giving me a headache. Who wrote that. The examples are incredibly bad.

What I understand that I can do is to have two packages nnpdf and nnpdf-some_branch which has track_feature: nnpdf

And then nnpdf will be default but I can only have one of the two installed.

Now, this is what I understand knowing what you want to do but that's not what the documentation is telling me and 100% not what those specific examples are pointing me to think.

[scarlehoff]

since we can install now directly with pip from the repository, running from a branch in e.g., a cluster is trivial

[scarlehoff]

But actually, this is now closed by #2305 :)