Concurrency issue on high core count systems

[Jotschi]

Request processing gets stuck on high core count systems.

Not yet sure what causes this issue but I was able to reproduce it with the demo image of 1.2.1 and 1.4.0 using:

ab -n 20000 -c 2000 http://server:8080/api/v1/demo/webroot/automobiles

21:48:26.336 [] ERROR [vert.x-eventloop-thread-6] [c.g.m.r.r.FailureHandler] - Error in request for path {GET /api/v1/demo/webroot/automobiles}
java.lang.RuntimeException: java.util.concurrent.TimeoutException: The source did not signal an event for 40 seconds and has been terminated.
        at io.reactivex.internal.util.ExceptionHelper.wrapOrThrow(ExceptionHelper.java:46)
        at io.reactivex.internal.observers.BlockingMultiObserver.blockingGet(BlockingMultiObserver.java:93)
        at io.reactivex.Single.blockingGet(Single.java:2737)
        at com.gentics.mesh.graphdb.spi.Database.lambda$null$3(Database.java:134)
        at io.vertx.core.impl.ContextImpl.lambda$executeBlocking$2(ContextImpl.java:316)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.util.concurrent.TimeoutException: The source did not signal an event for 40 seconds and has been terminated.
        at io.reactivex.internal.operators.single.SingleTimeout$TimeoutMainObserver.run(SingleTimeout.java:123)
        at io.reactivex.internal.schedulers.ScheduledDirectTask.call(ScheduledDirectTask.java:38)
        at io.reactivex.internal.schedulers.ScheduledDirectTask.call(ScheduledDirectTask.java:26)
        at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
        at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        ... 1 common frames omitted

• -c 16 - works
• -c 32 - works
• -c 40 - fails
• -c 64 - fails

CPU: Xeon E2699 v3 (18 Core - 32 HT) Env: Docker Image