opaque-sql icon indicating copy to clipboard operation
opaque-sql copied to clipboard

the stage EncryptedAddDummyRowExec is meaningless

Open Oliver001 opened this issue 4 years ago • 3 comments

Instead of inserting a blank line into a data block, we can pass it through the parameters of the function NonObliviousSortMergeJoin.

#215

Oliver001 avatar Apr 29 '21 07:04 Oliver001

Meaningless how? Inserting it as a parameter to NonObliviousSortMergeJoin means modifying the Flatbuffers definition, the corresponding serialization of equi-joins, and involves adding additional functionality to the C++ code to actually write output according to a certain schema. Using a dummy row allows us to use an already existing functionality.

octaviansima avatar Apr 29 '21 17:04 octaviansima

we can still use this code to generate a block, and pass it through a parameter to NonObliviousSortMergeJoin.

val nullRowsBlock = Utils.encryptInternalRowsFlatbuffers(
            Seq(InternalRow.fromSeq(Seq.fill(buildPlan.output.length)(null))),
            buildPlan.output.map(_.dataType),
            useEnclave = true,
            isDummyRows = true
          ) // the code to generate a null row
// Block(enclave.NonObliviousSortMergeJoin(eid, joinExprSer, block.bytes)) // the origin line
Block(enclave.NonObliviousSortMergeJoin(eid, joinExprSer, block.bytes, nullRowsBlock.bytes)) // add a parameter

The Flatbuffers definition is no need to change. And no needed to adding additional functionality. Only should we modify the definition of non_oblivious_sort_merge_join and read the null row just like

non_oblivious_sort_merge_join(uint8_t *join_expr, size_t join_expr_length,
                                   uint8_t *input_rows, size_t input_rows_length,
                                   uint8_t *null_row, size_t null_row_length, // the line added
                                   uint8_t **output_rows, size_t *output_rows_length)
// add tow line to get the null row
RowReader null_row_r(BufferRefView<tuix::EncryptedBlocks>(null_row, null_row_length));
const tuix::Row *null_row = null_row_r.peek();

If we want to implement the shuffleHashJoin, this maybe a more gentle way.

Oliver001 avatar Apr 30 '21 03:04 Oliver001

@Oliver001 sounds good to me! If you want to open a PR I'm happy to review.

octaviansima avatar May 25 '21 18:05 octaviansima