pyupio
Generate new hashes if additional files/platforms uploaded
Hi!
For projects that use hashes in their requirements files, pyup-bot's PR includes the hashes for all available files at the time the PR was open. However if additional package variants are uploaded later (for example a delay between different platforms as seen in aio-libs/aiohttp#2347, or else say a package retrospectively adding wheels for the first time), the PR is not updated.
This means that the user has to close the PR and delete the branch to force pyup-bot to open a new PR. (Compare to people who are not using hashes, who can just retrigger the Travis job.)
It would be great if instead the existing PR was updated any time additional files were detected. (Though I guess this would make the current "skip if branch exists" optimisation not possible and potentially increase load, unless some other caching of versions/files available was performed).
Many thanks :-)
Unfortunately, there's no good way to support something like this.
The only way to get notified on new package releases (not uploads) is to subscribe to PyPi's RSS feed and fetch the meta data once the release is published. Other than that, there's no system in place that could be used to get notified on any package related events.
Ah I didn't realise that was not currently possible.
Looking at the warehouse (PyPI replacement) tracker I see there is an issue open for a related feature: pypa/warehouse#1683
Until that's resolved I guess this could be closed?
The aiohttp 2.3.3 update broke us again. I think perhaps #262 is a more viable alternative to this idea for now?
What about a ~2 hour delay on pyup.io's side for aiohttp until all the hashes are there?
Okay, I've added a 120 minutes delay for aiohttp and multidict.
Caveat: You might get pull requests sooner than that if there's an update for one of your other dependencies available.
Thank you for adding the delay, though I think sadly 2 hours might not be enough for aiohttp.
For example, this PR doesn't have the manylinux1 hash: https://github.com/mozilla/treeherder/pull/3003
...so now fails after a retrigger: https://travis-ci.org/mozilla/treeherder/jobs/309306993#L705
For the 2.3.5 release, the first PyPI upload appears to be the one here (ignore the error, it still seems to have uploaded): https://ci.appveyor.com/project/asvetlov/aiohttp/build/1.0.704/job/pcfvdkt0bvakwwjh#L734 ...at ~00:30 UTC.
Whereas the manylinux1 build was uploaded here: https://travis-ci.org/aio-libs/aiohttp/jobs/309248910#L2488 ...at ~07:40 UTC.
(The OS X wheels were uploaded even later, at ~09:00 UTC due to the usual Travis OS X infra backlogs - though I'm less fussed about these)
Could the delay be increased to say 8-10 hours?
@edmorley @tomprince I will add pycryptodome and hypothesis to this list, as well as increase the delay to 10 hours.
Great idea to add hypothesis to the delay list, but something went wrong in https://github.com/mozilla/balrog/pull/731/commits/bb32f138acad6b89bc30f6b1c444e66010c28b67. The wheels had Last-Modified headers about 4 hours after the tarball, if that's a fair indication.
@nthomas-mozilla I haven't merged adding hypothesis into the delayed package list as well as increasing the delay to 10 hours yet. Thanks for the bump 👍