Ruby Daemons: Can monitor + multiple keep several worker children up?

[thuehlinger]

We are using daemons via active_messaging to keep our background pollers (message queue consumers) running. We now need to increase the amount of work done, and are looking into the multiple option. How can we keep 4 (or any consistent number) of workers up and running. We typically monitor the pid file with monit and have it call 'start' if the process dies, but with multiple there is no way to tie the process that is running to a specific pid file. Rephrased: If worker 3 dies, and monit calls start again because poller2.pid is empty, the multiple option just cases poller4.pid to be created. Monit then doesn't see a process for poller2.pid and calls start again ad infinitum.

We also experimented with the 'monitor' option, but it only keeps one worker running at all times. Calling start 4 times does spawn 4 workers and the monitor worker, but when I kill any of the workers monitor doesn't restart it (how could it know about them it only knows about the first worker created in the first start call right?).

[sodabrew]

I've been thinking about exactly this use case, too. Here's what I'm thinking:

main program
  load its configuration
  uses daemons to daemonize itself
  uses daemons to start child processes 1, 2, 3 .. n
  main loop do
    check state, handle signals, reload config files, etc.
    daemons restarts child processes as needed to keep 'n' running
  end
  child loop do
    work, work, work
  end
  term signal handler do
    signal child processes to exit
    wait for child processes to exit, i.e. Process.waitpid
    exit main process
  end

[jufemaiz]

Hi guys, curious if this has progressed.

[2called-chaos]

I don't know if that helps anyone but I had a similar issue but with named instances (one name can only have one instance). I had(still have) a daemon with multiple: false but I wanted to enable "multi instances support".

I solved it by tampering with the app_name option. I guess one could adapt that to just number them? As I said I am not sure if that helps you but here you go:

https://github.com/2called-chaos/mcl/blob/master/lib/mcld.rb#L42