Added new templates for PubsubAvroToBigtable and PubsubProtoToBigtable

[Vasu7052]

Hi Team,

Added new v2 templates for streaming real-time binary data from Pub/Sub Subscription to a Bigtable table. I have added support for two Serialization formats which are Avro and Protobuf. Both of them have separate templates.

[google-cla[bot]]

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

[bvolpato]

@Vasu7052 Thanks for the contribution! Are you interested in getting this reviewed? The next step would be signing the Contributor License Agreement (CLA) as commented above.

[Vasu7052]

Hey @bvolpato. I've signed the CLA as you mentioned and the scan is reflecting the same. Please guide me through the next steps.

[Vasu7052]

R: @zhoufek

[Vasu7052]

@Vasu7052 Thanks for the contribution! It is looking solid -- I've added a few suggestions based on a quick review. Let me know if you have any questions.

(I will leave for others to comment more on the Bigtable specifics.)

cc @Polber

@bvolpato I've tried resolving your suggestions. do tell me if I miss anything or if any further changes is required.

[Vasu7052]

@bvolpato just a gentle reminder that is there any change required from my end or we can close this PR?

[bvolpato]

@bvolpato just a gentle reminder that is there any change required from my end or we can close this PR?

Hi @Vasu7052 - thanks again for contributing. It seems good from a quick glance, but I still need some time to look at it carefully (Tentative is still this week).

[Vasu7052]

@bvolpato apologies for bothering you again, but is there anything left for this PR on my side?

[Vasu7052]

@bvolpato can you please tell me the blocker in merging this PR?

[bvolpato]

Sorry, this wasn't forgotten - we were in the middle of some large changes for templates. The templates now require metadata annotations so we can release them properly when contributed externally: https://github.com/GoogleCloudPlatform/DataflowTemplates#metadata-annotations

Do you want to add such annotations on the templates? I can also help if you prefer. I believe https://github.com/GoogleCloudPlatform/DataflowTemplates/blob/main/v2/pubsub-binary-to-bigquery/src/main/java/com/google/cloud/teleport/v2/templates/PubsubAvroToBigQuery.java and https://github.com/GoogleCloudPlatform/DataflowTemplates/blob/main/v2/pubsub-binary-to-bigquery/src/main/java/com/google/cloud/teleport/v2/templates/PubsubProtoToBigQuery.java should still serve as examples.

[Vasu7052]

Sure @bvolpato , I'll do the needful changes.

[Vasu7052]

@bvolpato i've tried adding annotations to the required classes, can you please check and do let me know if anything required from my end.

[github-actions[bot]]

This pull request has been marked as stale due to 180 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time. Thank you for your contributions.

[github-actions[bot]]

This pull request has been closed due to lack of activity. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.