databricks-sql-python
databricks-sql-python copied to clipboard
SEA: normalise column names in metadata queries
What type of PR is this?
- [x] Feature
Description
The Thrift backend returns a different schema in response to metadata queries (getting catalogs, schemas, tables, columns) than the SEA backend. Here, we normalise the column names and types returned by SEA to align with those from Thrift.
Column values will be addressed in #662.
We store the metadata columns, column mappings, etc. as state in the ResultSet instead of applying the changes once on the stored data because we may not get the data all at once, i.e., in case of CloudFetch results, we may have to get the data as multiple tables that need not be fetched at once. Thus, we store the columns and mappings, and apply it on the resultant of each fetch call.
How is this tested?
- [x] Unit tests
- [ ] E2E Tests
- [ ] Manually
- [ ] N/A