cube icon indicating copy to clipboard operation
cube copied to clipboard

fix(schema-compiler) Resolve rollup join pre-aggregation param allocator issue

Open pauldheinrichs opened this issue 1 year ago • 4 comments

Check List

  • [x] Tests has been run in packages where changes made if available
  • [x] Linter has been run for changed code
  • [ ] Tests for the changes have been added if not covered yet
  • [ ] Docs have been added / updated if required

Issue Reference this PR resolves Resolves: https://github.com/cube-js/cube/issues/8268

Description of Changes Made (if issue reference is not provided)

Resolves:

  • Pre aggregation rollup_join query invalidations for cross datasource utilizing different parameter symbols.
  • IE: Oracle, Postgres, Druid, MSSQL

Summary:

So this is a fun one, here's the situation. At first i thought this was entirely community druid problem but tracked it down to an underlying issue with rollup_joins.

If we rollup_join two datasources that contain a different paramAllocator symbol (IE: $1 and ?) the default paramAllocator would inject the wrong parameters into the pre-aggregation sql code invalidating the cache on the API node and causing weird behaviour. This would only present itself when actually querying the data and thus causing issues.


Smoke tested:

  • Refresh schedules refreshing pre-aggregations
  • Generated pre-aggregations

Thanks ❤️

pauldheinrichs avatar May 19 '24 03:05 pauldheinrichs

The latest updates on your projects. Learn more about Vercel for Git ↗︎

8 Ignored Deployments
Name Status Preview Comments Updated (UTC)
examples-angular-dashboard ⬜️ Ignored (Inspect) Visit Preview Jun 24, 2024 0:30am
examples-react-d3 ⬜️ Ignored (Inspect) Visit Preview Jun 24, 2024 0:30am
examples-react-dashboard ⬜️ Ignored (Inspect) Visit Preview Jun 24, 2024 0:30am
examples-react-data-table ⬜️ Ignored (Inspect) Visit Preview Jun 24, 2024 0:30am
examples-react-highcharts ⬜️ Ignored (Inspect) Visit Preview Jun 24, 2024 0:30am
examples-react-material-ui ⬜️ Ignored (Inspect) Visit Preview Jun 24, 2024 0:30am
examples-react-pivot-table ⬜️ Ignored (Inspect) Visit Preview Jun 24, 2024 0:30am
examples-vue-query-builder ⬜️ Ignored (Inspect) Visit Preview Jun 24, 2024 0:30am

vercel[bot] avatar May 19 '24 03:05 vercel[bot]

@pauldheinrichs Thanks for contributing! Could you please set the allocator to undefined in the branch without queryFactory as well? It should be consistent.

paveltiunov avatar May 23 '24 03:05 paveltiunov

@pauldheinrichs Thanks for contributing! Could you please set the allocator to undefined in the branch without queryFactory as well? It should be consistent.

@paveltiunov For sure, i've updated the newSubQuery method to behave the same!

pauldheinrichs avatar May 23 '24 12:05 pauldheinrichs

@paveltiunov sorry for the ping - curious if there's any other adjustments needed from my end here to push this forward!

pauldheinrichs avatar Jun 06 '24 19:06 pauldheinrichs

Closing as it was fixed in #8619

paveltiunov avatar Sep 13 '24 03:09 paveltiunov