- implement hooks

[mk01]

• enable user to extend plotman's functionality / plotting flow, by providing external scripts, without need to touch plotmans internals

• many of current feature requests can be solved with lighweight scripts (bash/python/...), e.g. #712, #711, #677, #638, #582 ...

• process cpu and io niceness can also be manipulated, affinity to cpu or NUMA on big server hardware can be set differently based on stage or plotter

• all new code is in new library 'hooks' + two new shell scripts directly inside hooks.d directory (located inside plotman's own config directory

• supplied hook serves as reference implementation / example, providing functionality for #677

• modification of existing plotman's code is only to call entrypoint in hooks.py and pass current jobs list containing jobs objects

• currently manager.py maybe_start_new_plot() and plotman.py kill is injected with call into hooks.try_run() and hooks.run() respectively

• try_run consumes fully refreshed jobs[], compares phase to previous job phase and if it is changed, calls hooks.run()

• run() takes plotmans environment, extends it with particular job's metadata and calls all executable files from hooks.d directory having extension .sh or .py

• scripts are called synchronously, cmd exec is waiting until the script process returns. that means the implementation is not suitable for LONG running actions.

• anyhow, plotman CLI can be called without issues from within the hooks, recursion in job_refresh -> try_run -> hooks script -> plotman cmd is being checked for

• stdout/stderr of running the hook scripts is currently forced to /tmp/plotman-hooks.log

2021-07-03T20:45:29 [01-check-dstdir-free.sh(2fe2355c)] INFO: processing plot id 2fe2355c8e6e7faf21e1aec895f0167c7e72054a5e3d39ddfee854193c7cd00c (ph: 1:5)
2021-07-03T20:45:29 [01-check-dstdir-free.sh(2fe2355c)] INFO: plot not at phase 3:6/7, hook done
2021-07-03T20:45:50 [01-check-dstdir-free.sh(480fd668)] INFO: processing plot id 480fd6680080bf018a532940b442a27565ab0c9616e2f7900aad6423a486899e (ph: 3:7)
2021-07-03T20:45:50 [01-check-dstdir-free.sh(480fd668)] INFO: there is 1386222714880 available space at /chia/013/, which is > k32plot 108984795136. job can continue.

new: src/plotman/hooks.py src/plotman/resources/hooks.d

update: src/plotman/configuration.py src/plotman/manager.py setup.cfg