Azure
[Bug] Blob trigger uses incorrect authentication method
Expected Behavior
When using identity-based connection settings it should use the managed identity auth method.
Actual Behavior
It tries to use the connection string method and fails because it tries to parse a non-existent setting.
Steps to Reproduce
- Create identity-based connections in the environment variables, in this case:
DATALAKE__queueServiceUri&DATALAKE__blobServiceUri - Create a blob trigger using
connection="DATALAKE" - Create a test blob in the path defined for the blob trigger
- It will result into erroring parsing the non-existent "DATALAKE" env variable.
If you use the AzureWebJobsStorage__accountname and set connection="AzureWebJobsStorage" without having the AzureWebJobsStorage setting it works, so it's only for custom connections which are documented here
Relevant code being tried
{
"IsEncrypted": false,
"Values": {
"FUNCTIONS_WORKER_RUNTIME": "python",
"AzureWebJobsFeatureFlags": "EnableWorkerIndexing",
"AzureWebJobsStorage": "UseDevelopmentStorage=true",
"DATALAKE__blobServiceUri": "https://REDACTED.blob.core.windows.net/",
"DATALAKE__serviceUri": "https://REDACTED.blob.core.windows.net/",
"DATALAKE__queueServiceUri": "https://REDACTED.queue.core.windows.net/"
}
}
import logging
import os
import azure.functions as func
from azure.identity import DefaultAzureCredential
from azure.storage.blob import BlobClient
import azurefunctions.extensions.bindings.blob as blob
bp = func.Blueprint()
@bp.function_name(name="process_blob_weather_data")
@bp.blob_trigger(
arg_name="sourceblob", path="bronze/weatherdata/{name}.json", connection="DATALAKE"
)
def process_blob_weather_data(sourceblob: blob.BlobClient):
logging.info('Starting to process blob')
logging.info(
f"Python blob trigger function processed blob! Perms \n"
f"Properties: {sourceblob.get_blob_properties()}\n"
f"Blob content head: {sourceblob.download_blob().read(size=1)}"
)
logging.info('Blob processed')
Relevant log output
ContextEnabledTask exception was never retrieved
future: <ContextEnabledTask finished name='Task-8' coro=<Dispatcher._dispatch_grpc_request() done, defined at C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.11\WINDOWS\X64\azure_functions_worker\dispatcher.py:263> exception=UnboundLocalError("cannot access local variable 'http_v2_enabled' where it is not associated with a value")>
Traceback (most recent call last):
File "C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.11\WINDOWS\X64\azure_functions_worker\dispatcher.py", line 565, in _handle__invocation_request
args[pb.name] = bindings.from_incoming_proto(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.11\WINDOWS\X64\azure_functions_worker\bindings\meta.py", line 183, in from_incoming_proto
return deferred_bindings_decode(binding=binding,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.11\WINDOWS\X64\azure_functions_worker\bindings\meta.py", line 301, in deferred_bindings_decode
deferred_binding_type = binding.decode(datum,
^^^^^^^^^^^^^^^^^^^^^
File "C:\Source\InSparkSolutions\smallestAzureDataplatform\functions\.venv\Lib\site-packages\azurefunctions\extensions\bindings\blob\blobClientConverter.py", line 43, in decode
return BlobClient(data=data).get_sdk_type()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Source\InSparkSolutions\smallestAzureDataplatform\functions\.venv\Lib\site-packages\azurefunctions\extensions\bindings\blob\blobClient.py", line 35, in get_sdk_type
return BlobClientSdk.from_connection_string(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Source\InSparkSolutions\smallestAzureDataplatform\functions\.venv\Lib\site-packages\azure\storage\blob\_blob_client.py", line 347, in from_connection_string
account_url, secondary, credential = parse_connection_str(conn_str, credential, 'blob')
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Source\InSparkSolutions\smallestAzureDataplatform\functions\.venv\Lib\site-packages\azure\storage\blob\_shared\base_client.py", line 368, in parse_connection_str
conn_str = conn_str.rstrip(";")
^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'rstrip'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.11\WINDOWS\X64\azure_functions_worker\dispatcher.py", line 274, in _dispatch_grpc_request
resp = await request_handler(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.11\WINDOWS\X64\azure_functions_worker\dispatcher.py", line 655, in _handle__invocation_request
if http_v2_enabled:
^^^^^^^^^^^^^^^
UnboundLocalError: cannot access local variable 'http_v2_enabled' where it is not associated with a value
requirements.txt file
azure-functions pandas requests azure-identity azure-storage-blob azurefunctions-extensions-bindings-blob
Where are you facing this problem?
Local - Core Tools
Function app name
No response
Additional Information
No response
Hi @donheerschap, thanks for reporting.
What version of azurefunctions-extensions-bindings-blob are you using? Support for managed identity was added in 1.0.0b2. If you're not using that version, can you update and try again?
Hi @hallvictoria I updated it, I was on version 1.0.0b1 locally, my CICD for the FA already installed 1.0.0b2 and shows another error in the logs, which now also show locally:
[2024-10-25T06:30:46.545Z] Executed 'Functions.process_blob_weather_data' (Failed, Id=0a4d9f70-4774-4994-8b3d-229ca51f7de2, Duration=82ms)
[2024-10-25T06:30:46.548Z] System.Private.CoreLib: Exception while executing function: Functions.process_blob_weather_data. System.Private.CoreLib: Result: Failure
Exception: ClientAuthenticationError: Operation returned an invalid status 'Server failed to authenticate the request. Please refer to the information in the www-authenticate header.'
ErrorCode:NoAuthenticationInformation
Stack: File "C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.11\WINDOWS\X64\azure_functions_worker\dispatcher.py", line 604, in _handle__invocation_request
call_result = await self._loop.run_in_executor(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\DonHeerschap\AppData\Local\Programs\Python\Python311\Lib\concurrent\futures\thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.11\WINDOWS\X64\azure_functions_worker\dispatcher.py", line 933, in _run_sync_func
return ExtensionManager.get_sync_invocation_wrapper(context,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.11\WINDOWS\X64\azure_functions_worker\extension.py", line 215, in _raw_invocation_wrapper
result = function(**args)
^^^^^^^^^^^^^^^^
File "C:\Source\InSparkSolutions\smallestAzureDataplatform\functions\blueprints\blob_process_weather_data.py", line 19, in process_blob_weather_data
f"Properties: {sourceblob.get_blob_properties()}\n"
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Source\InSparkSolutions\smallestAzureDataplatform\functions\.venv\Lib\site-packages\azure\core\tracing\decorator.py", line 94, in wrapper_use_tracer
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Source\InSparkSolutions\smallestAzureDataplatform\functions\.venv\Lib\site-packages\azure\storage\blob\_blob_client.py", line 1080, in get_blob_properties
process_storage_error(error)
File "C:\Source\InSparkSolutions\smallestAzureDataplatform\functions\.venv\Lib\site-packages\azure\storage\blob\_shared\response_handlers.py", line 186, in process_storage_error
exec("raise error from None") # pylint: disable=exec-used # nosec
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<string>", line 1, in <module>
File "C:\Source\InSparkSolutions\smallestAzureDataplatform\functions\.venv\Lib\site-packages\azure\storage\blob\_blob_client.py", line 1070, in get_blob_properties
blob_props = cast(BlobProperties, self._client.blob.get_properties(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Source\InSparkSolutions\smallestAzureDataplatform\functions\.venv\Lib\site-packages\azure\core\tracing\decorator.py", line 94, in wrapper_use_tracer
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Source\InSparkSolutions\smallestAzureDataplatform\functions\.venv\Lib\site-packages\azure\storage\blob\_generated\operations\_blob_operations.py", line 1900, in get_properties
map_error(status_code=response.status_code, response=response, error_map=error_map)
File "C:\Source\InSparkSolutions\smallestAzureDataplatform\functions\.venv\Lib\site-packages\azure\core\exceptions.py", line 161, in map_error
raise error
.
This is the RBAC of the targeted storage account (db-test being SP being set on my local machine with environment vars with AZURE_TENANT_ID, AZURE_CLIENT_ID, & AZURE_CLIENT_SECRET), which I think is not missing anything:
Looks like a credential is missing. The error is happening when trying to use the client, not when creating it. I'll look into adding a change that will allow users to input their own credentials, but for now, we can create a default one on the extension side and use that to authenticate correctly. Here is the BlobServiceClient documentation for reference.
The fix for this will be in azurefunctions-extensions-bindings-blob version 1.0.0b3. It might take some time to test, so ETA is sometime next week. I'll post an update here when it's finished!
Thanks for your patience!
Posting an update here -- the work has been tested and validated, but as we are currently halting releases for the holidays, the fix will be released sometime early January. I'll update here once it has been released. Thanks for your patience, and sorry for the delay!
v1.0.0b3 has been released! Sorry for the delay, and please let me know if you're still facing issues with this