datafold
Databricks: support column of type `Text()` as key
Is your feature request related to a problem? Please describe.
We have tables that contain key columns with the type Text(). We can not change these columns; we have to use the tables as is.
Describe the solution you'd like
I want to use columns with the type Text() as the key.
Describe alternatives you've considered
None. If this feature will/can not be supported, then I will see if can create views that convert the key columns to numeric columns.
Additional context
Running on Databricks
This issue has been marked as stale because it has been open for 60 days with no activity. If you would like the issue to remain open, please comment on the issue and it will be added to the triage queue. Otherwise, it will be closed in 7 days.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
Is your feature request related to a problem? Please describe.
We have tables that contain key columns with the type
Text(). We can not change these columns; we have to use the tables as is.Describe the solution you'd like
I want to use columns with the type
Text()as the key.Describe alternatives you've considered
None. If this feature will/can not be supported, then I will see if can create views that convert the key columns to numeric columns.
Additional context
Running on Databricks
Thanks for reporting this @JCZuurmond, we're not currently working on this but may in the future, in the meantime we'd happily accept a community contribution
A similar issue has cropped up related to Databricks Date PKs:
ERROR - Cannot use a column of type UnknownColType(text='DATE') as a key
This issue has been marked as stale because it has been open for 60 days with no activity. If you would like the issue to remain open, please comment on the issue and it will be added to the triage queue. Otherwise, it will be closed in 7 days.
Although we are closing this issue as stale, it's not gone forever. Issues can be reopened if there is renewed community interest. Just add a comment and it will be reopened for triage.
Yes, same issue here... All columns we use have surrogate keys which are strings.
Context on why arbitrary strings are not supported (without some work) in #539
https://github.com/datafold/data-diff/issues/539#issuecomment-1773354733
Hi all -
I'm sorry for the delay in following up on this. @JCZuurmond, thank you for taking the time to raise this issue!
We made a hard decision to sunset the data-diff package and won't provide further development or support.
If that's of interest, over the past few months, we have rewritten the diffing engine in Datafold Cloud and solved many issues that existed in this package.
-Gleb
