python-task-queue icon indicating copy to clipboard operation
python-task-queue copied to clipboard

risk of getting empty messages

Open xiuliren opened this issue 7 years ago • 1 comments

sqs can return empty messages if the message number is low (<1000), maybe this explains that why the last few hundred tasks takes longer time.

this package set the wait time to be 0, and could potentially get empty messages. https://github.com/seung-lab/python-task-queue/blob/master/taskqueue/aws_queue_api.py#L60

here is the documentation of the long polling: https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-long-polling.html https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/working-with-messages.html#setting-up-long-polling

as suggested, simply setting the waittime to 20 seconds should fix this.

xiuliren avatar Jan 22 '19 19:01 xiuliren

Wait time has been set to 20 sec since https://github.com/seung-lab/python-task-queue/commit/a355d042b8f126947d458a4e82f69a0d3ebd79a8

william-silversmith avatar Jul 24 '20 05:07 william-silversmith