kamu-cli
kamu-cli copied to clipboard
Feature/306 use arrow schema instead of parquet schema in gql and cli
Description
Closes: https://github.com/kamu-data/kamu-cli/issues/306
Done:
- Updated QueryService::get_schema() to return Arrow schema sourced from SetDataSchema event.
- Existing logic that returns schema of the last Parquet file in the dataset has been moved into get_schema_parquet() method.
- Updated kamu inspect schema command and renamed json format to parquet-json. Introduced arrow-json output format.
- Used serde tp convert arrow to json.
- Updated current_schema() GraphQL API Extend DataSchemaFormat with ArrowJson.
To do after review:
- Create a kamu-web-ui ticket to migrate schema displayed to ArrowJson.
- Add more tests?
Checklist before requesting a review
- [ ] CHANGELOG.md updated
- [ ] API changes are backwards-compatible
- [ ] Workspace layout changes include a migration
- [ ] Documentation update PR: <link or N/A>
- [ ] Dataset pipelines update scheduled if needed
- [ ] Unit-tests added