java-dataloader icon indicating copy to clipboard operation
java-dataloader copied to clipboard

ScheduledDataLoaderRegistry can leak scheduler threads

Open DanielThomas opened this issue 2 years ago • 0 comments

Related to https://github.com/graphql-java/java-dataloader/pull/135, I noticed that the executor isn't shut down, so if it's used, would leak the single core thread.

Since JDK 9 the default keepalive timeout is 10ms, so allowCoreThreadTimeOut is a potential solution, if you want to avoid explicit shutdown() in close():

https://github.com/openjdk/jdk/blob/caf71810f85ea55083ce7d1c76307a0c42d9be0e/src/java.base/share/classes/java/util/concurrent/ScheduledThreadPoolExecutor.java#L429-L443

Incidentally, folks on JDK 21 using JSON thread dumps should watch for this, because it's tricky to troubleshoot:

  • On JDK 21 ThreadPoolExecutors register as ThreadContainers, a concept added to support thread dumps with virtual threads
  • We happen have a periodic thread dump script, which takes JSON thread dumps
  • TheadContainers#children doesn't scale well, on my machine with 30K containers (2m of runtime @ 250 rps) ThreadDumper takes a minute
  • On a real application instance, that caused ThreadDumper to run indefinitely, because the application outpaces the ThreadDumper's ability to walk thread containers
  • ThreadDumper also dutifully collects the ThreadContainers onto a list, which caused the weak references to the containers to live forever, causing a memory leak

DanielThomas avatar Nov 12 '23 23:11 DanielThomas