Enhance the clickstream ksqlDB demo to create connectors using ksqlDB
You can now create connectors within ksqlDB. We're already using SOURCE connectors within the demo. We should update the demo to:
- Not run connect as a separate container, but run within ksqlDB
- Use the
CREATE SINK CONNECTORsyntax in ksqlDB to create the connectors to sink to ES, and removeksql-connect-es-grafana.sh.
The whole create source connectors, execute sql statements, create sink connectors, could be in a single SQL script....
Not run connect as a separate container, but run within ksqlDB
In 5.5.0-post, connect is not a separate container, it is already run within ksqlDB.
Use the CREATE SINK CONNECTOR syntax in ksqlDB to create the connectors to sink to ES, and remove ksql-connect-es-grafana.sh.
So that leaves the question about the second bullet point, whether the Elasticsearch sink connectors should also use the CLI CREATE SINK CONNECTOR. The demo intentionally does not do that in order to show users how to use the REST API to the connect cluster.
I've always viewed the clickstream demo as a ksqlDB demo, and so I'm be leaning towards using ksqlDB features wherever possible. Maybe you view it more as a CP demo, and in which case I can see why this would be useful.
If you feel it's more the latter, then feel free to close this :D