Very high CPU usage when reload_interval=0 and running in-app on Linux

[urig]

I am using hupper to hot-reload a Dash application running in waitress. hupper is set to run with reload_interval=0 and it works fine on Windows but appears to behave badly by consuming 100% CPU in Linux.

Steps to reproduce:

1. Run a python docker container:

   docker run -it --cap-add SYS_PTRACE python:3.9.13-buster /bin/bash
   

2. In the container install py-spy: pip install py-spy
3. Install poetry: curl -sSL https://install.python-poetry.org | python3 -
4. Add poetry to path: export PATH="/root/.local/bin:$PATH"
5. Clone the app into the container: git clone https://github.com/entropy-lab/entropy.git
6. Checkout to a revision where hupper is enabled: git checkout tags/v0.11.0
7. cd entropy && poetry install
8. Run the app with poetry run entropy serve .

Actual results:

Run top and you will see that the app consumes ~100% CPU. Run py-spy --pid <process id> with the process id of the app as it is running and you will see that hupper's polling mechanism is the consumer:

  %Own   %Total  OwnTime  TotalTime  Function (filename)
 92.00%  92.00%   717.0s    717.0s   get_mtime (hupper/polling.py)

Expected results:

hupper should consume a reasonably small amount of CPU when running in the app.

Miscellaneous :

You can see how hupper is used by the app here: https://github.com/entropy-lab/entropy/blob/85536ef62f724290d35eea0eb7e7c47134ab05f5/entropylab/dashboard/init.py#L61

Thanks, urig

[mmerickel]

Probably not what you want to hear but I’d suggest setting the interval back to the default of 1. I can’t see a reason to set it lower and the high cpu usage makes perfect sense.

If you want a better mechanism than polling you can install watchman and it will use that way more efficiently.

I’m tempted to just error out when it’s less than 1 to protect against this. I have no idea why it works fine on windows…..

[urig]

@mmerickel many thanks for the prompt response. Setting the interval to 1 has indeed resolved things for me on Linux.

+1 for setting it to error on 0 so people don't end up unnecessarily stressing their CPUs.

[mmerickel]

fixed in 0efccba1cf9b4d6395b7b699750eb1adca32f760