versatile-data-kit
versatile-data-kit copied to clipboard
Create a demo of reading from kafka in a data job
What is the feature request? What problem does it solve? If a user needs to process larger amounts of data with VDK they should be able to deploy multiple pod instances and use kafka consumers to scale out.
In this ticket we want an example of a single pod deployment reading from kafka. When the topic is drained we then want the pod to die.