sshkit.ex icon indicating copy to clipboard operation
sshkit.ex copied to clipboard

Allow executing on hosts in parallel/groups

Open pmeinhardt opened this issue 8 years ago • 3 comments

As outlined in https://github.com/bitcrowd/sshkit.ex/issues/3, it would be nice to provide a way of specifying how to run

The SSHKit.run function should let us specify, whether we want to execute:

  • sequentially
  • in parallel
  • in groups of a given size

pmeinhardt avatar May 04 '17 05:05 pmeinhardt

I implemented SSHKit.run's "mode: :parallel" option for asynchronous run.

https://github.com/seungjin/sshkit.ex/commit/3806c94919943db33868d14444f1445f42077a6e

Planning to add test for it.

seungjin avatar Aug 27 '18 13:08 seungjin

Nice, thank you @seungjin 👍

Some quick, early feedback and thoughts:

  • If you could add tests, that'd be great 🙂
  • I think that if we provide a timeout option, that should be used in all versions (seq., parallel…) of the SSHKit.run function
  • The internal run = fn host -> … end function in both the sequential & parallel versions are the same, so maybe this could be consolidated and instead passed to an internal function (outline/some inspiration below)

Cheers 👋 😃


def run(context, command, mode \\ :sequential) do
  cmd = Context.build(context, command)

  op = fn host ->
    {:ok, conn} = SSH.connect(host.name, host.options)
    res = SSH.run(conn, cmd)
    :ok = SSH.close(conn)
    res
  end

  perform(context.hosts, op, mode)
end

defp perform(hosts, op, :sequential) do
  Enum.map(hosts, op)
end

defp perform(hosts, op, :parallel) do
  hosts
  |> Enum.map(fn host -> Task.async(fn -> op.(h) end) end)
  |> Enum.map(fn task -> Task.await(task) end)
end

pmeinhardt avatar Aug 29 '18 14:08 pmeinhardt

Ah, and if you open a PR and reference this issue, we could add feedback/discussions in the right places @seungjin ✌️

pmeinhardt avatar Aug 29 '18 14:08 pmeinhardt