Draft: chore: Add an end-to-end batching benchmark

[orf]

This is a bit of a weird PR, sorry. I have been playing a lot with the internals of Vector recently for a project of mine to integrate Vector as a Python library. Vector is a really nicely structured project, and after a few wrong turns I was able to integrate it in-proces with Python as a library.

To explore how everything works (especially the service abstractions) I built the following dummy source and sink, and I figured I'd try and contribute it as a end-to-end Criterion benchmark for a couple of reasons:

1. I found it quite hard to find completely minimal/bare-bones non-test sources and sinks, so perhaps this could form an example for contributors?
2. An end-to-end benchmark involving producing and consuming might be an useful thing to include?
   • I was curious about potential overhead incurred with having to repeatedly allocate Vec's to call SourceSender::send_batch and I had no idea where to start profiling this.

Feel free to close this, but I figured it might be useful to someone in the future and I didn't want it to go to waste.

[bits-bot]

All committers have signed the CLA.

[orf]

So after a bit of work on this I came to similar conclusions as the linked document - there’s a lot of noise, and while the timings are stable it’s not clear if the timings are fully accurate in measuring the things we want to measure.

There’s definitely value in an example like this I feel, so maybe it should just be an example rather than a benchmark?

[jszwedko]

So after a bit of work on this I came to similar conclusions as the linked document - there’s a lot of noise, and while the timings are stable it’s not clear if the timings are fully accurate in measuring the things we want to measure.

There’s definitely value in an example like this I feel, so maybe it should just be an example rather than a benchmark?

Agreed, I think there is some value in a heavily documented example that is in-tree so that it is compiled and kept up-to-date. We have some example sink code in this tutorial: https://github.com/vectordotdev/vector/tree/master/docs/tutorials/sinks but I think it suffers from the fact that it isn't in-tree and thus can become stale without notice.

Would you want to move the example implementations and drop the benchmarks. I think tests/examples/ could make a good place?