dagger icon indicating copy to clipboard operation
dagger copied to clipboard
verified publisher icon raystack

Add support for Avro data

Open prakharmathur82 opened this issue 4 years ago • 1 comments

Currently, Dagger only supports SerDe for Protobuf encoded messages.

  • Avro SerDe is already supported in Flink. Add support for enabling it.
  • Apart from SerDe, postprocessors, preprocessors, some functions like ElementAt, HashTransformer also rely on Dynamic Message APIs. Need to ensure they work for Avro data as well.

prakharmathur82 avatar Aug 13 '21 07:08 prakharmathur82

Stencil also soon plans to add support for AVRO schemas. So, we would also need to consider how to integrate with schema registry.

ravisuhag avatar Aug 13 '21 07:08 ravisuhag