beam icon indicating copy to clipboard operation
beam copied to clipboard
verified publisher icon apache

Pass user specified destination type to UpdateSchemaDestination

Open ahmedabu98 opened this issue 3 years ago • 3 comments

Fixes #22543

When using multiple partitions for a BigQuery FILE_LOADS write, the user's dynamic destination type isn't properly passed into UpdateSchemaDestination. This leads to a ClassCastException when attempting to get the table and schema using the dynamic destination object.

The user's dynamic destination object is configured with a particular parameter type (represented by DestinationT) for getTable() and getSchema (see here). The current implementation from #17365 doesn't respect this user type and instead passes in a fixed TableDestination argument.

ahmedabu98 avatar Aug 08 '22 20:08 ahmedabu98

Run PostCommit_Java_DataflowV2

ahmedabu98 avatar Aug 08 '22 20:08 ahmedabu98

R: @johnjcasey

ahmedabu98 avatar Aug 08 '22 20:08 ahmedabu98

Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control

github-actions[bot] avatar Aug 08 '22 20:08 github-actions[bot]

Run PostCommit_Java_DataflowV2

ahmedabu98 avatar Aug 11 '22 22:08 ahmedabu98

This LGTM. Have you retried the original failing custom dynamic destinations?

johnjcasey avatar Aug 12 '22 13:08 johnjcasey

Yup, left a comment in #22543

ahmedabu98 avatar Aug 12 '22 14:08 ahmedabu98