apache
KYLIN-5748 Skip build flat table for index build job if all new created indexes can be built from existed parent index
For an index build job, if all the new added indexes can be built from their existed parent index. No index was built from new created flat table. For this case, there won't be any data inconsistency across indexes. And there is no need to build a new flat table to serve the stage of index build by layer. So I think we can skip stage BUILD DICT and stage GENERATE FLAT TABLE for an index build job in condition that all new added indexes can be built from their existed parent index
Your proposal will be invalid in this use case: if you add a new bitmap type metric to the model, you can build the new index separately, and the task type is also INDEX_BUIDL. If you skip building the dictionary, an error will occur when generating encoding columns for the flat table later.
@zznlime Does this pr need to be merged into the kylin code? If necessary, please send pr to @liyang-kylin . I will resolve this issue.
