python-logging-rabbitmq icon indicating copy to clipboard operation
python-logging-rabbitmq copied to clipboard

self.queue.task_done() can be called when no message was got due to continue executing finally block anyway leading to ValueError exception

Open kmorwath opened this issue 3 years ago • 0 comments

The changes in version 2.2 for fix #25 in python_logging_rabbitmq/handlers_oneway.py may have introduced an issue. Before the Queue.Empty exception was never raised because record, routing_key = self.queue.get() had no timeout. Now when the exception is raised if no messages arrives within 10s, the exception handler will call "continue" but still the "finally" block is executed anyway - and queue.task_done() could be called more times than put() and it will lead to a ValueError exception.

queue.task_done() should be called in a inner "try..finally" block after a message has been dequeued actually, for example:

record, routing_key = self.queue.get(block=True, timeout=10) try: #Actually got a message ... try to send the message ... finally: queue.task_done()

Moreover when is_stopping is set the loop is exited before queue.task_done() is called, and messages still in the queue are not processed. If on the other side of the queue something attempts to call queue.join() it could never return.

kmorwath avatar May 10 '22 14:05 kmorwath