Manually marking Service Bus Queue Function as Failed

[garoplin]

Is your question related to a specific version? If so, please specify: Programming Model V1, Azure FUnction v4, python 3.10

What binding does your question apply to, if any? (e.g. Blob Trigger, Event Hub Binding, etc) Service Bus Queue

Question

Currently I have policy of retrying setup on function and Service Bus level. When the processing throws an error the message goes back to queue 3 times as expected (defined by property Max delivery count). However, for some types of exceptions I don't want to retry processing, because I know the retry won't help and it's pointless to waist computing power for that. So I can't catch the exception on the top level, check whether I should retry or no. The problem is that I still want to throw that exception after catching it, so the function execution will be marked as failed. Is there any way to mark the message as completed and then throw the exception? Or to silence the exception but add some kind of property to function execution context, so that it will be visible as failed?

[vijaystroup]

Facing this issue right now with local testing. Tried just raising an Exception but that causes the function to fire immediately again ignoring the message lock. ~One solution I found for now is marking a function as failed by doing sys.exit(0) which seems to have the desired outcome but this produces a weird start-up traceback but after will perform as expected. I don't feel comfortable doing this thought because I do not know what it will do when deployed.~ From the above the function is executing when the message should be unlocked but it does not run properly (does not actually execute the contents of the function). Work around for now is just to manually add back in a message to the queue.

There should be a functionality like in c# to be able to set autoComplete to False or a special exception in the azure.functions.ServiceBus class that a service bus trigger will listen to and mark it as a failed function without trying to retry the function again.

[garoplin]

Any update on this?

[JanuszNowak]

Any update on this ?

[andreyshv]

The only solution i've found for now is to move message to dead letter queue and then rethrow exception. This sample can help

[alecglen]

@andreyshv I'm running into this too and think you're right that dead-lettering is the correct action. However, I'm stuck on actually implementing the call to dead_letter_message() as shown in that example code because it expects an azure.servicebus.ServiceBusReceivedMessage object instead of the azure.functions.ServiceBusMessage type the function provides. Any chance you have a solution to that problem handy?

[andreyshv]

@alecglen sorry, but i work with .net not python. Below is my code in c#. It's differs from python of course but idea the same.

internal class CheckStatusFunction
    {
        private readonly IRunner _runner;

        public CheckStatusFunction(IRunner runner)
        {
            _runner = runner;
        }

        [Function(nameof(CheckStatusFunction))]
        public async Task RunAsync(
            [ServiceBusTrigger("%ServiceBusSender:QueueName%", AutoCompleteMessages = false)] ServiceBusReceivedMessage receivedMessage,
            ServiceBusMessageActions messageActions,
            CancellationToken cancellationToken)
        {
            var message = receivedMessage.Body.ToObjectFromJson<CheckOperationMessage>();

            var isSuccess = false;
            Exception exception = null;

            try
            {
                isSuccess = await _runner.RunAsync(message, cancellationToken);
            }
            catch (Exception ex)
            {
                exception = ex;
            }

            if (isSuccess)
            {
                await messageActions.CompleteMessageAsync(receivedMessage);
            }
            else
            {
                await messageActions.DeadLetterMessageAsync(receivedMessage);
                if (exception != null)
                    throw exception;
            }
        }
    }
}

[alecglen]

One would hope, but unfortunately the Python worker is not designed to support that flow currently.

Ultimately for our Python Function, we worked around this by flipping the script. The Function is configured without retries so errors are dead-lettered by default. Then for the specific cases that we do see value in retrying, we catch the errors and manually add a new copy of the message to queue before exiting.

Definitely some drawbacks and wouldn't work for every case, but it's working for us.

[trin94]

Regarding the code snippet from @andreyshv, I think ServiceBusReceivedMessage now is supported since this PR.

It's also on PyPi: https://pypi.org/project/azurefunctions-extensions-bindings-servicebus/

The only think left - at least from that sample - seems to be ServiceBusMessageActions. Really looking forward to these bindings as they will make the Service Bus integration much more mature and useful 👍

[hallvictoria]

Yes, binding to a ServiceBusReceivedMessage is now supported. You can learn more here: Azure Service Bus for Python

Message settlement (and the corresponding ServiceBusMessageActions object) is still pending, but this work is in progress. Tentative ETA is August - September.