erikfrey
Mac OS X Support
(I'm working on this one — if I find something I'll send a pull request. Consider this as much a development log as a feature request!)
Apple's version of nc doesn't support -q, though -w seems to do something similar. Even with the change, it results in the rather mystifying behavior of reading the input and producing no output, and I have no idea why.
(This is of course after installing pv from Homebrew; one could easily alias pv=cat if one was so inclined!)
A few discoveries:
First, the reason awk is so slow (I think) is that it the "distribution" isn't actually a distribution at all, at least as I understood it. At least on my machine, seed(...) only accepts an integer parameter, and AWK casts anything that's not a number to 0. This means that doing a distributed computation with non-integral input data will only hit a single node.
Second, I'm wondering if perhaps xargs -P might help simplify the internal architecture of this script a bit. Right now nc fails on me when I run nc with -q0, since that's not supported on OS X. It seems like this option is there to control termination — I'm away from my normal Linux box so I can't check on the function of that flag. Using xargs automatically manages sub-processes in a clean manner, and terminates once all have finished, which simplifies the logic significantly.