Custom search attributes having hyphen("-") in the name is returning error while filtering Workflows

[enakshipriya]

I have a self hosted Temporal server setup which uses auroradb and elasticsearch for visibility. I created a custom search attribute named "abc-test". While creation there was no error. I tried using this custom search attribute in one of my workflows which also got created without any error and I was able to see the newly created search attribute in the workflow history. However on trying to query the workflows through UI, I am getting error:

There is an error with filtering Workflows.
invalid query: unable to convert filter expression: unable to convert left side of "abc - test = 'xyz'": invalid expression: must be a column name but was *sqlparser.BinaryExpr

Expected Behavior

It should not return any error.

Actual Behavior

It seems that although there is no restriction on using hyphen it is somewhat acting as a delimeter and inserting spaces while filtering. If we are not using hyphen in the name then it works as expected.

Steps to Reproduce the Problem

1. On a self hosted server, create a custom search attribute "abc-test".
2. Use the newly created search attribute in any workflow execution
3. On UI, while querying, try to query the workflow using the search attribute as a filter.

Specifications

• Version: [1.22.6]
• Platform: All

Slack link: https://temporalio.slack.com/archives/CTRCR8RBP/p1710752031783399

[alexseedkou]

This issue comes from the sqlparser as hyphen - is reserved to SQL.

Currently, a workaround here is to use your custom filter directly in the search bar with the delimiter backtick ("`"). For example `abc-test`="something".

To fix the issue when using the filter option in the UI, a way is to add the backtick to the customer search attribute when sending the query request to the backend server from UI or frontend .

[diervo]

I run into the same issue without the hyphen:

invalid query: unable to convert filter expression: unable to convert left side of "OrgId = 'foo'": filter by 'OrgId' not supported for standard visibility

I just upgraded (self hosted) to the latest version (1.22.7 , ui 2.26.1) and latest schema versions so not sure if it could be a regression?

[alexseedkou]

I run into the same issue without the hyphen:

invalid query: unable to convert filter expression: unable to convert left side of "OrgId = 'foo'": filter by 'OrgId' not supported for standard visibility

I just upgraded (self hosted) to the latest version (1.22.7 , ui 2.26.1) and latest schema versions so not sure if it could be a regression?

Did you try the search directly to use the filter?

[rodrigozhou]

It looks like you're using standard visibility, which does not support custom search attributes. Please refer to v1.20.0 release notes for upgrade instructions.

[alexseedkou]

The issue should be fixed with the hyphen after the merge of this PR