Rewrite updater.sh -> .py. Remove redundant info in updater.yml.

[Salamandar]

I rewrote the updater.sh into updater.py. Those are the upsides:

• More structured code :
  • functions modifiable by maintainer
  • core generic code, should not be modified
• We calculate the shasum as we download the file (no temporary file)

I also changed some things in the updater.yml:

• move the "exit 1" from the script to a "if false" in the yml. That allows to put the update.yml in the repository without triggering failed actions.
• Remove the commit step, because peter-evans/create-pull-requestalready does this
• Same for the "setting up git user"

[tituspijean]

I have tried your script (much easier than Bash 😅), but it seems quite slow (between 2 and 3 minutes).

https://github.com/tituspijean/wordpress_ynh/actions/workflows/updater.yml

Did you experience the same slowness?

[yalh76]

My opinion is a python script is less easier to understand, less easier to tweak, can't be executed manually line by line for the one who doesn't know python. and others scripts "install, remove, etc ..." are in bash so the step to package YunoHost apps will increase ...

[alexAubin]

Personally I'm fine with Python which is more relevant when the code complexity increases ... but that's also because I'm pretty fluent in python ;) I do kinda understand the points raised by yalh

However, I'm curious why that script should be included in every app .. Or to put it another way : where / what is the app-specific part, what does actually define how we check for update and find the new url for the source etc ? Shouldnt we have a "base" code in https://github.com/YunoHost/apps/tree/master/tools for example, and then have a sort of declarative yaml/whatever file that contains the info on how to check for that app update exactly ? (I never digged in the auto-update mechanism before)

[alexAubin]

Note that having a common piece of code would also :

• be easier to maintain (no need to update every app each time the code changes)
• we could imagine a single, centralized "CI" job that checks for the updates and creates the appropriate PR ... especially since recently where all the github auto-update jobs seems to trigger failures etc ...

[tituspijean]

I wholeheartedly agree that centralizing the auto-update mechanism would be easier. As for the declarative format, you're the expert. 😛

The main differences between apps are:

• where to find the latest available version (Github release, external git forge, API, json...)
• where to find the sources

I will try to review whether or not I needed to do some blackmagic in some apps, though overall that's it.

[Salamandar]

Shouldnt we have a "base" code in https://github.com/YunoHost/apps/tree/master/tools

That's a nice idea. Indeed, you can see in my script that there are only 3 functions that are "app specific".

The difficult part with a descriptive approach would be to support the multiple ways apps might be released. For example, for Mediawiki, I check github for the releases/tags, then download the tarballs from the HTTP mirror. That's really specific and it would make my head hurt to think about how to describe it.

Though, the generic functions might be imported from a provided "python module" (could be a simple file copied here).

[Salamandar]

My opinion is a python script is less easier to understand, less easier to tweak

Yes, I fully understand this. At first the bash script was OK for me. But python comes with nice abstractions, TYPES AND LISTS OF LISTS, and with proper logging (I recon there is almost none here) it's really easy to use.

[Salamandar]

I have tried your script (much easier than Bash sweat_smile), but it seems quite slow (between 2 and 3 minutes).

Okkkk after some debugging, I found what's up : requests.get(url, stream=True).iter_content() iterates BYTE PER BYTE. The fix : iter_content(some_sane_size). as sane size, I tested 10*1024 and the script now runs in less than 4 seconds.

[tituspijean]

General discussion on the auto-updater infrastructure: https://github.com/YunoHost/issues/discussions/2164

[alexAubin]

Sooo FYI I started drafting https://github.com/YunoHost/apps/pull/1655 following various discussions these past weeks

[Salamandar]

Another idea I had : this "common" code could very well be in a custom github action hosted by the yunohost org. That way, it's even easier to manage this code and migrate for newer manifest formats etc.

[ericgaspar]

is this PR not superseded by #206?

[Salamandar]

What's the point of leaving the Github Actions ?

I really like being able to manually run the update merge request github action , when I know an app has a new release.

If the goal is to centralize the common code, it could be via a custom github action (Same for the readme generator, actually)

[tituspijean]

What's the point of leaving the Github Actions ?

ASAIK:

1. Be Github-independent
2. Factorize the code

I really like being able to manually run the update merge request github action , when I know an app has a new release.

I guess we could imagine adding an API or a !upgrademe <app> trigger for the yunobot. But as we say in slang French, yakafokon.

If the goal is to centralize the common code, it could be via a custom github action (Same for the readme generator, actually)

The README generator is triggered whenever a new commit it pushed to any branch of any app of the organization, through a webhook. It is not a custom Github Action.

[Salamandar]

The README generator is triggered whenever a new commit it pushed to any branch of any app of the organization, through a webhook. It is not a custom Github Action.

I meant, the readme generator could be a github action ^^

Anyways. I understand the will of being github independant. I'm just wondering if that independance doesn't come with a too big dev/effort cost. But if you made that decision, i think it's the good one :)

[Salamandar]

I guess we could imagine adding an API or a !upgrademe trigger for the yunobot. But as we say in slang French, yakafokon.

Very "meh". I'd prefer a script that can be run for a single app, even if it's in another repo (like the apps repo).

[alexAubin]

Closing because the new workflow has been merged ... If you really want to be able to manually trigger the PR, then it should be possible to tweak https://github.com/YunoHost/apps/blob/master/tools/autoupdate_app_sources/autoupdate_app_sources.py accordingly, but considering the current script is ran in a daily cron job I'm not sure to see the point @_@

[Salamandar]

Yeah i'm already on working on splitting it into 2 scripts :

• one that can be run locally, without github.com/yunohost-apps dependency.
• one that lists the apps, git clones the repos, and makes the MRs.