Build and store wheels in CI runs

[altendky]

🚀 Feature

Build and store wheels in CI runs such that they are available via the Jenkins artifacts page.

Motivation

A user of DGL came into #python on Freenode IRC asking for help dealing with local building of the unreleased DGL version for some included bugfixes. This effort could be avoided if the existing CI system made wheels of development versions available for download.

Pitch

I started a basic PR in #2574 but am unable to test it fully since Jenkins appears to not be using the Jenkinsfile from the PR. I proved this to myself by deleting many of the test jobs and observing that they were still running. Also, the added artifact archiving step doesn't seem to do anything.

I'll note that it would be even better to actually test based on the built wheels than to just store them on the side. Build the wheel, then install for test using just that and no access to the source nor any cached editable installation as seems to maybe be what is used now, perhaps? This would validate the wheels are actually built properly which is an important step in the build and release process.

[VoVAllen]

Can our nightly build solve your issue? We have nightly build for linux, which is available through pip install --pre dgl-cuXX

[jermainewang]

I started a basic PR in #2574 but am unable to test it fully since Jenkins appears to not be using the Jenkinsfile from the PR. I proved this to myself by deleting many of the test jobs and observing that they were still running. Also, the added artifact archiving step doesn't seem to do anything.

We only allow changes on Jenkinsfile from permissioned users to protect the server from abuse. That's why your change does not work. Sorry for that!

I'll note that it would be even better to actually test based on the built wheels than to just store them on the side. Build the wheel, then install for test using just that and no access to the source nor any cached editable installation as seems to maybe be what is used now, perhaps? This would validate the wheels are actually built properly which is an important step in the build and release process.

This is actually how CI works right now. Could you let the DGL user try the nightly build as @VoVAllen suggested? You could also point them to our slack channel so we could handle the question for you. Thanks!

[altendky]

No problem, my PR was half hijacking your CI system so I didn't really expect it to work. :] Anyways, the user is on Windows so if the nightlies are Linux only, that won't help. I shared the link to this issue with them but will try to follow up as well to make sure they see your slack suggestion.

As to the present CI setup, i saw a -e install and stashing of DLLs or somesuch. Maybe I didn't look hard enough but I didn't see any building of an sdist and/or a wheel and stashing or archiving of it.

Anyways, I guess that within the existing approach the request here would be to include Windows wheels in the nightlies.

Side note, you don't have a long description on PyPI. That would probably be a nice addition (and a separate ticket really)

Thanks for the follow up and explanations.