nodejs-pubsub icon indicating copy to clipboard operation
nodejs-pubsub copied to clipboard

Retry INTERNAL errors on more request types ?

Open n-e opened this issue 2 years ago • 1 comments

Hi,

We're getting transient INTERNAL errors when calling createTopic or createSubscription:

Error: 13 INTERNAL: A service error has occurred. Please retry your request. If the error persists, please report it. [code=e8c0]
    at callErrorFromStatus (/usr/src/app/node_modules/@grpc/grpc-js/build/src/call.js:31:19)
    at Object.onReceiveStatus (/usr/src/app/node_modules/@grpc/grpc-js/build/src/client.js:192:76)
    at Object.onReceiveStatus (/usr/src/app/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:360:141)
    at Object.onReceiveStatus (/usr/src/app/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:323:181)
    at /usr/src/app/node_modules/@grpc/grpc-js/build/src/resolving-call.js:94:78
    at process.processTicksAndRejections (node:internal/process/task_queues:77:11)
for call at
    at ServiceClientImpl.makeUnaryRequest (/usr/src/app/node_modules/@grpc/grpc-js/build/src/client.js:160:34)
    at ServiceClientImpl.<anonymous> (/usr/src/app/node_modules/@grpc/grpc-js/build/src/make-client.js:105:19)
    at /usr/src/app/node_modules/@google-cloud/pubsub/build/src/v1/subscriber_client.js:227:29
    at /usr/src/app/node_modules/google-gax/build/src/normalCalls/timeout.js:44:16
    at repeat (/usr/src/app/node_modules/google-gax/build/src/normalCalls/retries.js:80:25)
    at /usr/src/app/node_modules/google-gax/build/src/normalCalls/retries.js:118:13
    at OngoingCall.call (/usr/src/app/node_modules/google-gax/build/src/call.js:67:27)
    at NormalApiCaller.call (/usr/src/app/node_modules/google-gax/build/src/normalCalls/normalApiCaller.js:34:19)
    at /usr/src/app/node_modules/google-gax/build/src/createApiCall.js:84:30

According to the docs, these errors can be retried.

However, the nodejs client doesn't retry them, see https://github.com/googleapis/nodejs-pubsub/blob/main/src/v1/subscriber_client_config.json#L46 for example.

Should this be changed, at least for idempotent requests?

n-e avatar Mar 07 '23 14:03 n-e

We were just talking about this last week. @kamalaboulhosn This might fit into that discussion.

feywind avatar Mar 07 '23 21:03 feywind

Ping @kamalaboulhosn, not sure what the state of the INTERNAL retries is.

feywind avatar Apr 02 '24 20:04 feywind

We're reassessing the errors for auto-retry now, will consider this as part of that.

feywind avatar May 02 '24 18:05 feywind