examples icon indicating copy to clipboard operation
examples copied to clipboard

Add automated testing for examples

Open big-andy-coates opened this issue 5 years ago • 4 comments

It would be great is we could automate the testing of these examples. The Kafka tutorials micro-site is a good example of how this can be done.

Such automation would:

  1. Ensure we know everything is working for the current / past releases
  2. Give us early warnings / failures if an upcoming release brakes anything, (e.g. if we have a branch building of ksqlDB's master branches latest images
  3. Speed up the task of updating the examples

big-andy-coates avatar Jun 11 '20 12:06 big-andy-coates

For example, testing and updating the clickstream demo is currently a manual process. Yes, there are lots of scripts to help, but essentially there are about 20 different things to execute to run through the example. When needing to make changes and retest, this is time consuming and error prone. Would be great if there was a single command to test the example!

big-andy-coates avatar Jun 11 '20 12:06 big-andy-coates

@big-andy-coates wrt to your second comment:

there are about 20 different things to execute to run through the example

Are you aware of https://github.com/confluentinc/examples/blob/5.5.0-post/clickstream/start.sh ? It's a single command ./start.sh that runs everything end-to-end, you don't have to run "20 different things".

(Even more so, every demo in this repo has a ./start.sh which has reduced the urgency to create a more automated test with formalized pass/fail validation steps.)

ybyzek avatar Jun 11 '20 12:06 ybyzek

I did not - but do now ;)

big-andy-coates avatar Jun 11 '20 12:06 big-andy-coates

@big-andy-coates I filed DEVX-1864 to call out the existence of start.sh in the Clickstream demo docs.

ybyzek avatar Jun 11 '20 13:06 ybyzek