ktools icon indicating copy to clipboard operation
ktools copied to clipboard

Empty loss stream cause run_ktools to hang

Open johcarter opened this issue 4 years ago • 2 comments

Issue Description

Occasionally run_ktools.sh hangs when there is one or more empty pipes. The cause appears to be when summarycalc receives an empty loss stream. It is an intermittent problem, sometimes the script completes successfully even if some pipes loss streams are empty.

This is more likely to happen;

  • When there are more processes than events in the model run (e.g. a multi-core machine with a single event model)
  • When there are many events that do not cause a loss to the portfolio and they all coincide in a single pipe

Part of the solution is setting the number of parallel processes appropriately for the run. However, there will still be plausible cases where no losses are generated in a whole pipe (the portfolio is very small, or low risk, or there is a high loss threshold applied to losses). This bug needs to be fixed so that the script always completes, even though some pipes may not contribute any losses to outputs.

Steps to Reproduce (Bugs only)

  1. Simple script and data package for Paris Windstorm

Version / Environment information

Ubuntu 20 Observed both natively and inside model execution worker docker container Observed in latest ktools and 6 month old version

Example data / logs

johcarter avatar Jun 16 '21 09:06 johcarter

Example from @benhayes21 test_named_pipe_fail.tar.gz

sambles avatar Jun 29 '21 10:06 sambles

to test

johcarter avatar Nov 09 '22 10:11 johcarter

@johcarter - I tested this with 1.23.14 today with PiWind using 2 events and 8 processors and the issues doesn't seem to be there

The example script above still hangs - though I'm not sure what it's supposed to be doing?

benhayes21 avatar Dec 16 '22 10:12 benhayes21