spring-projects
Add ability for user to configure whether to fail fast on startup
Application fails to connect to Pulsar; however, it swallows / logs the exception and does not restart
Is there anyway to add a configuration so that if a failure to connect to Pulsar happens on boot, it restarts? For context this is what happened: We applied a rolling restart to all of our applications Spring Boot Pulsar Client tried to connect to pulsar; however, pulsar was not finished spinning up. So, Pulsar just gave us back connection refused.
Error was like this:
java.util.concurrent.CompletionException: org.apache.pulsar.shade.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: \"},{\"attempt\":5,\"error\":\"java.util.concurrent.CompletionException:
org.apache.pulsar.client.api.PulsarClientException: java.util.concurrent.CompletionException: org.apache.pulsar.shade.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: \"},{\"attempt\":6,\"error\":\"java.util.concurrent.CompletionException:
org.apache.pulsar.client.api.PulsarClientException: java.util.concurrent.CompletionException: org.apache.pulsar.shade.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: \"}]}\n\tat org.apache.pulsar.client.api.PulsarClientException.unwrap(PulsarClientException.java:1074)\n\tat org.apache.pulsar.client.impl.ConsumerBuilderImpl.subscribe(ConsumerBuilderImpl.java:104)\n\tat
org.springframework.pulsar.core.DefaultPulsarConsumerFactory.createConsumer(DefaultPulsarConsumerFactory.java:89)\n\tat org.springframework.pulsar.listener.DefaultPulsarMessageListenerContainer$Listener.<init>(DefaultPulsarMessageListenerContainer.java:298)\n\tat org.springframework.pulsar.listener.DefaultPulsarMessageListenerContainer.doStart(DefaultPulsarMessageListenerContainer.java:126)\n\tat org.springframework.pulsar.listener.AbstractPulsarMessageListenerContainer.start(AbstractPulsarMessageListenerContainer.java:110)\n\tat
Final note, this is my first time submitting an issue on an open source project. If I goofed up just let me know please.
Hi @nerdBoi , First of all, your issue submission is just fine and thank you for taking the time to submit the issue. Much appreciated.
Currently the producer and consumers expect the successful connection to Pulsar up front. For the producer side (PulsarTemplate) it will throw the exception outward and the app will fail to start. However, the consumer (the @PulsarListener) containers have this buried down deeper and they let the app start eventhough there was an exception.
Connection retries
Longer term it would be nice if the client connect could be retried (more resilient). There is some work going on to enable https://github.com/spring-projects/spring-pulsar/issues/422 which may open the door for this ability (we will keep this in mind as we get further into that one).
FailFast Config Option
In the mean time, making the fail fast configurable and consistent would be helpful.
Also, out of curiosity, which version are you using?
Awesome! I appreciate the swift reply and acknowledgement.
version: 0.2.0
I will follow: https://github.com/spring-projects/spring-pulsar/issues/422 going forward!
Yea the FailFast config option would be awesome to configure. I think this is about it to the conversation?
Again, thanks for replying.
[!NOTE] This issue is about improving startup and it speaks of 2 features "Connection Retries" and "FailFast Config".
#816 has a similar theme but is only about "Connection Retries" (see original desc. As such, we will focus on "Connection Retries" in #816 and this ticket will solely be "Fail Fast Config".
cc: @eljefe6a
Hi @nerdBoi - the fix for fail fast config was just merged. The follow up is to add Spring Boot config props to drive this. Until then the default is to fail fast.
Closing via https://github.com/spring-projects/spring-pulsar/commit/a518e110893772a86239a1252b27126c28286188