phoenix icon indicating copy to clipboard operation
phoenix copied to clipboard

PHOENIX-7006: Configure maxLookbackAge at table level

Open sanjeet006py opened this issue 2 years ago • 5 comments

Design doc

  • Support for table level max lookback age in SYSCAT.
  • Support for table level max lookback age in CREATE/ALTER TABLE.
  • Block setting/altering max lookback age for views and indexes.
  • Retrieve max lookback age for view and index from data table via hierarchical lookup and store it only for data table.
  • Invalidate metadata cache on alter table for child views and associated indexes (view indexes and data table indexes) so that new PTable instance in loaded in cache with update max lookback age.
  • Use table level max lookback age in coproc hooks wherever cluster level max lookback age is being used. Cluster level max lookback age will be fallback if table level max lookback age is not set.
  • Use table level max lookback in Index verification in GlobalIndexRegionScanner. Cluster level max lookback age will be fallback if table level max lookback age is not set.
  • Use table level max lookback age in IndexScrutinyTool.
  • Test coverage for all the above items.

sanjeet006py avatar Dec 05 '23 21:12 sanjeet006py

Another thing we should consider is that maxlookback on index should inherit the value of the base table. The behavior should be exactly similar to TTL where both the data table and index have same values and you cannot directly set ttl on index. It always inherits the TTL value from base table.

tkhurana avatar Dec 07 '23 21:12 tkhurana

Another thing we should consider is that maxlookback on index should inherit the value of the base table. The behavior should be exactly similar to TTL where both the data table and index have same values and you cannot directly set ttl on index. It always inherits the TTL value from base table.

Yes, I will be doing that only. Current set of changes include SYSCAT schema changes and CREATE/ALTER Table support. In the subsequent changes I will be adding support to block max lookback age modifications for index.

sanjeet006py avatar Dec 08 '23 11:12 sanjeet006py

Jenkins test this please

sanjeet006py avatar Dec 19 '23 19:12 sanjeet006py

@gjacoby126 @haridsv please re-review the PR, I have addressed the earlier comments. Thanks

sanjeet006py avatar Jan 17 '24 03:01 sanjeet006py

@kadirozde I have addressed all the review comments. Please re-review. Thanks

sanjeet006py avatar Feb 25 '24 21:02 sanjeet006py

@sanjeet006py the changes overall looks good now on the latest revision. There are some merge conflicts though.

virajjasani avatar Mar 06 '24 00:03 virajjasani

Thanks a lot @virajjasani . I have resolved the conflicts. The conflicts were around the version changes for 5.3.0 creation. Please re-review. Thanks

sanjeet006py avatar Mar 06 '24 15:03 sanjeet006py

Once current PR validation is done. Shall I squash and rebase all my changes into one commit or merge procedure will take care of that?

sanjeet006py avatar Mar 06 '24 15:03 sanjeet006py

Triggered one more build. Once we get results, will merge the PR unless there are any objections.

virajjasani avatar Mar 09 '24 05:03 virajjasani

@virajjasani the latest build is complete. Are we good to merge the PR? Thanks

sanjeet006py avatar Mar 10 '24 23:03 sanjeet006py

Test failures on the build are not relevant.

virajjasani avatar Mar 11 '24 20:03 virajjasani