DataflowTemplates icon indicating copy to clipboard operation
DataflowTemplates copied to clipboard

Kafka To Elasticsearch Template

Open Cherepushko opened this issue 4 years ago • 9 comments

This is the initial PR for kafka-to-elasticsearch Dataflow template. It contains pom.xml and the empty directories that will be filled in the future pull requests. As a placeholder for the empty folders is used .gitkeep.

Cherepushko avatar Oct 26 '21 07:10 Cherepushko

@prathapreddy123 @suyograo

Cherepushko avatar Oct 26 '21 11:10 Cherepushko

@prathapreddy123 is it better to place this inside of google cloud to elasticsearch folder?

suyograo avatar Oct 28 '21 04:10 suyograo

@prathapreddy123 is it better to place this inside of google cloud to elasticsearch folder?

Since kafka is third party product this template should remain outside google cloud to elasticsearch. Separate directory (i.e kafka-to-elasticsearch) should be fine.

prathapreddy123 avatar Oct 29 '21 13:10 prathapreddy123

@prathapreddy123 can we get a review please?

suyograo avatar Nov 05 '21 05:11 suyograo

@prathapreddy123 can we get a review please?

@suyograo - will review by early next week.

prathapreddy123 avatar Nov 05 '21 13:11 prathapreddy123

As discussed earlier, this PR will be reworked to use original ElasticsearchIO from Apache Beam 2.34.

Cherepushko avatar Feb 11 '22 17:02 Cherepushko

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale[bot] avatar Apr 16 '22 10:04 stale[bot]

Comment just to remove "stale" label.

Cherepushko avatar Apr 26 '22 14:04 Cherepushko

Hi @Cherepushko -- thanks for your contribution here. Unfortunately we ran behind on reviews, and the structure of the project changed a bit. Is there still interest in pursuing this template? We can work together on a plan to resolve conflicts + review the code.

bvolpato avatar Apr 10 '23 13:04 bvolpato