[IMP] queue_job_cron_jobrunner: channel

[Koudja]

Implement channel on queue_job_cron_jobrunner

[OCA-git-bot]

Hi @ivantodorovich, some modules you are maintaining are being modified, check this out!

[petrus-v]

This PR require https://github.com/OCA/queue/pull/754 to avoid opened file descriptor leak !

[OCA-git-bot]

This PR has the approved label and has been created more than 5 days ago. It should therefore be ready to merge by a maintainer (or a PSC member if the concerned addon has no declared maintainer). 🤖

[hbrunn]

@Koudja this is still in draft state, is this intentional?

[petrus-v]

@Koudja this is still in draft state, is this intentional?

In fact we are still supervising the system and keep improving it as long we met difficulties ! yesterday we've implement the stop_processing feature to avoid to be kill by the cpu time limit in case there is a lot of job in the queue, this is not yet deployed to our production (and I'm going to somme logging).

So I suppose keeping it in draft state as long we have not get some stabilities is probably better ! But review and feedback are welcome otherwise so not sure what suits most OCA conventions in such case ?

[hbrunn]

draft is good I think, because of the label I nearly merged it

[sbidoul]

I'm wondering... now that the job scheduler (misnamed jobrunner) can run on odoo.sh (#668), queue_job_cron_jobrunner could maybe be simplified to only run jobs that are in state enqueued, and keep all the scheduling logic (channels, priorities, eta, etc) in the regular jobrunner ?

[petrus-v]

I'm wondering... now that the job scheduler (misnamed jobrunner) can run on odoo.sh (#668), queue_job_cron_jobrunner could maybe be simplified to only run jobs that are in state enqueued, and keep all the scheduling logic (channels, priorities, eta, etc) in the regular jobrunner ?

As of today, we are not using the mentioned patch and have configured our setup to avoid spawning the job scheduler/runner, as suggested in the module’s documentation.

To be honest, we haven’t spent much time evaluating this patch, so we might have missed some key information. However, we do have some concerns:

• A preference for using a cron worker to process queue jobs.
• Uncertainty about how Odoo.sh manages process lifecycles, especially regarding jobrunner. If it gets killed (due to inactivity, lack of HTTP requests, or other obscure platform limits), how does the system handle it? Would it be properly restarted by Odoo.sh?

If I understand correctly, your idea is to have the queue jobrunner focus on managing job states—ensuring that tasks are enqueued while respecting channel capacities—while letting the ir.cron execute them based on priority. This approach would remove the need for HTTP workers to process jobs.

That sounds like an interesting direction and avoid to duplicate the channel implementation logic. Would you be open to further designing this together or discussing the long-term vision for the module?

[sbidoul]

If I understand correctly, your idea is to have the queue jobrunner focus on managing job states—ensuring that tasks are enqueued while respecting channel capacities—while letting the ir.cron execute them based on priority. This approach would remove the need for HTTP workers to process jobs.

Yes, that's the idea.

Uncertainty about how Odoo.sh manages process lifecycles, especially regarding jobrunner. If it gets killed (due to inactivity, lack of HTTP requests, or other obscure platform limits), how does the system handle it? Would it be properly restarted by Odoo.sh?

I'm not sure either. On the project where we use the HA jobrunner on odoo.sh, we don't observe issues with this, but yeah, "obscure platform limits" are the key words. But I tend to think this is not a blocker for that approach.

That sounds like an interesting direction and avoid to duplicate the channel implementation logic. Would you be open to further designing this together or discussing the long-term vision for the module?

We don't need this so far :crossed_fingers:, but I'd be happy to discuss.

[Koudja]

I updated PR with latest comments. It's been on our production for few weeks now without any issue (with thousands jobs per day). I updated this PR state to ready for merge based on this.

[github-actions[bot]]

There hasn't been any activity on this pull request in the past 4 months, so it has been marked as stale and it will be closed automatically if no further activity occurs in the next 30 days. If you want this PR to never become stale, please ask a PSC member to apply the "no stale" label.

[Koudja]

We've migrated on HTTP worker, so I am closing this PR.