Flowpack
FEATURE: Allow jobManager to be interrupted by system SIGINT
This is a first draft, to checkout if this is something you are even going to accept. (It is fully functionally though - but it needs to be documented - which I'd love to do, if you accept)
This change allows the JobWorker to be interrupted on the Command-Line by pressing CTRL+C or sending a SIGINT-Signal at a safe state (it will only be interrupted if the last job had been completed successfully or a timeout occured)
Additionally the JobManager provides a new Message queues might use to signal, they are now safe to interrupt (ideally before they go sleeping in their own "waiting for timeout" handling)
As I had no Idea how this kind of behavior can be tested I wrote some very stupid application to ensure my assumptions. If you would like to see this, or have an idea how this could be tested (the JobCommand is currently not tested at all), I am thankful for any suggestion
Any feedback is highly welcome. Otherwise I would need to maintain an own fork, which I would love to avoid.
What do you mean with "minimal example"? My struggle here is: On the command line it is quite easily testable. Just press CTRL+C after starting a worker command. In the current state it will interrupt anything immediately. But thats not that nice, if the job was already running for several hours and is near its completion. So my example project has a simple sleep job.
As said, if have no idea how to test this with a single threaded php-unit test. As this would run unnecessary long, call a command worker in unknown environment loosing the test scope, etc... But maybe at this place its just me being unexperienced.
What do you mean with "minimal example"?
On the command line it is quite easily testable
Right. But for that scenario we wouldn't need to adjust the JobManager, would we?
So my example project has a simple sleep job.
That's exactly the kind of example I was thinking of :) Is that project public somewhere and/or can we provide an example of such a job/queue that makes use of the signals? I couldn't wrap my head around that part yet
No problem. I build specifically for this part an own test project.
So no hassle to share it. To get the perfect view, a patch in Flowpack/jobqueue-doctrine makes sense. (only the one line calling interruptMe)
Do you want it with, or without?
I would attach my Test scenarios to the readme.md so you could repeat, what I played with
I just would like to understand this feature better. So a little code snippet would be enough:
- What line is needed in
jobqueue-doctrineand would we have to add it in other implementations, too? - How to use this from a custom job/queue?
Understood. And completely feel the urge. That asynchronous stuff is hard to wrap ones head around. Especially with the single threaded PHP processes. I had to learn about pcntl and the way it handles interruptions, too. The usual advice is to set declare(ticks=1) which I absolutely disliked. I used the symfony/messenger component as source of inspirtation. ;)
I will publish the test project and with it some lines documenting, in which way the proposed changes changes the game. Cannot promise to do it today. But at latest tomorrow.
Wow... sorry. Got unexpectedly a bit busy. Will deliver the next days, if not today (bringing some pressure on me in this topic ;) )
Here is the example Project: https://github.com/fcool/JobQueueSigIntDemonstration
@fcool thanks a lot for putting so much effort into an example project. But do I get it right that it actually doesn't do anything special but providing a patch for the doctrine implementation? So, to respond to my question from above: Queues that use sleep to wait for new jobs can now invoke
JobManager::interruptMe();
in order to interrupt processing if requested by the user. Correct?
Now you lost me completely. No the example project is only there to demonstrate the behvior. The doctrine patch is only "sugar" and not needed to observe the game changing behavior. Ant that is, that you are able to send SIGINT (by pressing CTRL+C) to your worker, WITHOUT interrupting the working task. So if you go back to master of jobqueue and make the same (start the dummy task and press ctrl+c) you will have a "reserved" job in the database, staing there forewer. In the "changed" version (the one from this PR), used in the example process, a current task will be worked on until it is completed and THEN the worker stops.
By patching doctrine you allow to the doctrine queue to be interrupted during its "waiting" phases. As in default this is 60 seconds it would be only be interruptable once all 60 seconds, as long as there are no short tasks. But it is completely "benificial" - the game changer is the jobque itself.
Of course the PCNTL extensions have to be installed.
Jast saw this, and even thought it is old by now… it still seems a great addition. @fcool, would you bring this up to date as needed?
