Specify number of processes

[aeldaly]

I'm trying to build a daemon that accepts the number of processes to spawn as an argument.

Here's the runner:

#!/usr/bin/env ruby

require 'daemons'

Daemons.run(
  './script/startup/jobs',
  dir_mode: :normal,
  dir: '/var/app/support/pids',
  log_dir: '/var/app/support/logs/',
  logfilename: 'async_runner.log',
  backtrace: '/var/app/support/logs/async_runner.error.log',
  multiple: true,
  monitor: true,
  monitor_interval: 180 # seconds
)

And this is script/startup/jobs:

#!/usr/bin/env ruby

require 'optparse'
require 'ostruct'
require File.expand_path('../../../config/environment',  __FILE__)

lib_path = Dir["#{Rails.root}/lib/async/*.rb", "#{Rails.root}/lib/*.rb"]
lib_path.each { |f| require f }

options = OpenStruct.new(worker_count: ENV['ASYNC_SUBSCRIBER_COUNT'].to_i || 1)

opts = OptionParser.new do |opts|
  opts.banner = 'Usage: jobs [options]'
  opts.separator ''
  opts.on('-n', '--worker_count count', 'Number of worker process to spawn') do |count|
    options.worker_count = count.to_i
  end
  opts.on('-c', '--task_classes classes', 'classes to run') do |classes|
    options.task_classes = classes
  end
end

opts.parse!

runner = AsyncRunner.new({
  task_classes: options.task_classes
})

options.worker_count.times do |x|
  Rails.logger.info "Starting async runner daemon...."
  runner.run
end

This is not working. It ends up running runner.run only once.

I also tried wrapping runner.run inside a fork but that is not working either. Any ideas on how to do this?

Thanks.

[aeldaly]

Also, it's not really logging anything to the log file

[aeldaly]

Another weirdness...

Changed runner to this:

#!/usr/bin/env ruby

require 'daemons'

Daemons.run(
  './script/startup/jobs',
  dir_mode: :normal,
  dir: '/var/app/support/pids',
  log_dir: '/var/app/support/logs/',
  logfilename: 'async_runner.log',
  backtrace: '/var/app/support/logs/async_runner.error.log',
  multiple: true
)

And then did the below:

[root@ip-172-30-4-194 current]# ./script/startup/async_runner start
[root@ip-172-30-4-194 current]# cat ../support/pids/jobs_num0.pid
17433
[root@ip-172-30-4-194 current]# ps ax | grep jobs
17450 ?        Sl     0:00 jobs
17479 pts/2    S+     0:00 grep --color=auto jobs
[root@ip-172-30-4-194 current]# cat ../support/pids/jobs_num0.pid
cat: ../support/pids/jobs_num0.pid: No such file or directory

So the pid inside the file is not the same as the running process, and then the pid file disappears. Calling stop at this point does nothing.

[thuehlinger]

Which Ruby gem does AsyncRunner originate from? I do not see an import in your code. I guess that AsyncRunner is created new processes for the tasks which are then obviously not tracked by daemons. The main spawned process exits after having created the tasks and therefore its .pid file will be cleaned up. I guess its just not a good idea to combine AsyncRunner and daemons. What you actually want is having :multiple set to true and then call your ./script/startup/async_runner startas many times as the number of desired worker processes.