nodejs-pubsub icon indicating copy to clipboard operation
nodejs-pubsub copied to clipboard

Misleading error message

Open hspak opened this issue 2 years ago • 1 comments

Environment details

  • OS: Cloud Run Service
  • Node.js version: 18.16.0
  • npm version: 9.5.1
  • @google-cloud/pubsub version: 3.5.0

Steps to reproduce

  1. Publish messages with orderingKey set.
  2. Wait for a message to fail.
  3. All subsequent messages fail, but with a timeout error message (way before the timeout).

I suspect that we are running into this behavior, but the error message does not seem indicative of that: https://github.com/googleapis/nodejs-pubsub/blob/6c38e7bd29d1351830c80ade7162b6d5cf9cbe1e/src/topic.ts#L861-L887

I'm sorry I don't have a good, small reproduction case but here's effectively the code being run:

import { PubSub, Topic as PubSubTopic } from '@google-cloud/pubsub';

const pubsub = new PubSub(clientConfig);
const topic = pubsub.topic('TopicName', {
  messageOrdering: true,
  batching: {
    maxMessages: 100,
    maxMilliseconds: 250,
  },
};

await topic.publishMessage({
  data: somePayload,
  attributes: someAttrs,
  orderingKey: 'myOrderingKey'
});                                        

Here's a timestamp from Cloud Logging showing that the publish failed due to a timeout after 50 seconds only after less than a second of running: Screenshot 2023-08-07 at 2 27 57 PM

I believe the error message is misleading and incorrect here.

hspak avatar Aug 07 '23 21:08 hspak

@hspak Thanks for the issue. It does look like there's something odd going on there with the error message.

feywind avatar Aug 11 '23 16:08 feywind

I think if there's a bug here, it has to be something down in gax - it seems to know that its timeout is 50000, but it's firing immediately anyway.

However, I wonder if the problem is really that the timeout error is coming from a previous attempt to send. Like:

  • Attempt to send a message with an ordering key
  • Publish RPC is called, queue is cleared
  • Another message is added to the new queue
  • The RPC for the old queue times out

@hspak Is this something you're still having trouble with? This has been open a while.

feywind avatar May 23 '25 19:05 feywind

The service that had this issue was shut down a while back for us. It looks like no one else has seen this issue, so I'll go ahead and close this!

hspak avatar May 23 '25 19:05 hspak