Provide an option to wait for all currently-being-processed / currently-queued requests / events to complete before closing vertx

[azurvii]

Describe the feature

When call vertx.close(), provide an option to allow waiting for all currently-being-processed / currently-queued requests / events to complete before shutting down. Anything added after vertx.close() call should be fine to drop.

Use cases

This feature could help to achieve no down time in rolling upgrade deployments, e.g. in Kubernetes.

For example, in 4.5.7, vertx.close() interrupts the worker threads, and forcefully close all active HTTP connections. This would cause the client requests to fail. This could cause failures during the time when rolling upgrades are happening in k8s.

If these requests are allowed to complete normally before shutting down vertx, the rolling upgrades could be more likely seamless.

I made a reproducer to demonstrate the issue: https://github.com/azurvii/issue-vertx-not-waiting-http-requests.

In this reproducer, a server vertx instance is doing some lengthy work (Thread.sleep(5000)) when processing requests, and when a client (a separate vertx instance) is waiting for a request to return, a vertx.close() is called on the server vertx instance, which can possibly from a shutdown hook. The client would get an error and an empty response.

[vietj]

that is a feature expected for Vert.x 5