Azurite icon indicating copy to clipboard operation
Azurite copied to clipboard

Blob tag queries do not support tag keys with special characters except for `_`

Open delreluca opened this issue 1 year ago • 1 comments

According to the docs both tag keys and values can contain the special characters +-.:=_/. However most of them need to be enclosed in double quotes for it to work in the query.

Azurite’s query parser always restricts tag keys to only allow _ as special character. This is correct when encountering an unquoted tag key I think, but it’s not correct when the tag key is quoted.

It is possible to set such a tag in Azurite and retrieve it back without any problems, so this is strictly a problem in the query parser it seems. I haven’t stepped through it because I don’t know TypeScript enough but these seems to be the relevant lines

https://github.com/Azure/Azurite/blob/1dd0784621a0e6069a7ccf4f4560d2bda6803a44/src/blob/persistence/QueryInterpreter/QueryParser.ts#L317-L348

https://github.com/Azure/Azurite/blob/1dd0784621a0e6069a7ccf4f4560d2bda6803a44/src/blob/persistence/QueryInterpreter/QueryParser.ts#L409-L441

There is a distinction between quoted and unquoted keys but both branches eventually call the same validation function that only accepts _ as special character.

Exemplary exception from the .NET SDK below. A query like this works in az CLI against actual Azure Blob Storage.

Azure.RequestFailedException: Error parsing query at or near character position 19: unexpected 'key.with.dots'
RequestId:6c6038aa-441e-49e5-9b53-3a2aedbeafa7
Time:2024-12-14T16:20:31.299Z
Status: 400 (Error parsing query at or near character position 19: unexpected 'key.with.dots')
ErrorCode: InvalidQueryParameterValue

Additional Information:
QueryParameterName: where
QueryParameterValue:     "key.with.dots" = 'value'

Content:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Error>
  <Code>InvalidQueryParameterValue</Code>
  <Message>Error parsing query at or near character position 19: unexpected 'key.with.dots'
RequestId:6c6038aa-441e-49e5-9b53-3a2aedbeafa7
Time:2024-12-14T16:20:31.299Z</Message>
  <QueryParameterName>where</QueryParameterName>
  <QueryParameterValue>    "key.with.dots" = 'value'</QueryParameterValue>
</Error>

Headers:
Server: Azurite-Blob/3.33.0
x-ms-error-code: InvalidQueryParameterValue
x-ms-request-id: 6c6038aa-441e-49e5-9b53-3a2aedbeafa7
Date: Sat, 14 Dec 2024 16:20:31 GMT
Connection: keep-alive
Keep-Alive: REDACTED
Transfer-Encoding: chunked
Content-Type: application/xml

   at Azure.Storage.Blobs.ContainerRestClient.FilterBlobsAsync(Nullable`1 timeout, String where, String marker, Nullable`1 maxresults, IEnumerable`1 include, CancellationToken cancellationToken)
   at Azure.Storage.Blobs.BlobContainerClient.FindBlobsByTagsInternal(String marker, String expression, Nullable`1 pageSizeHint, Boolean async, CancellationToken cancellationToken)
   at Azure.Storage.Blobs.Models.FilterBlobsAsyncCollection.GetNextPageAsync(String continuationToken, Nullable`1 pageSizeHint, Boolean async, CancellationToken cancellationToken)
   at Azure.Storage.StorageCollectionEnumerator`1.StorageAsyncPageable.GetAsyncEnumerator(CancellationToken cancellationToken)+MoveNext()
   at Azure.Storage.StorageCollectionEnumerator`1.StorageAsyncPageable.GetAsyncEnumerator(CancellationToken cancellationToken)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult()
   at MyApp.App.Worker.FindSomethingToWorkOn(CancellationToken cancellationToken) in /path/to/src/Worker.cs:line 32

delreluca avatar Dec 14 '24 16:12 delreluca

@EmmaZhu Would you please help to look at the blob tag issue?

blueww avatar Dec 16 '24 03:12 blueww