incubator-xtable Open Telemetry Integration

Instrument the project with open telemetry or something similar to get a better understanding of the performance characteristics of Onetable

Sep 21 '23 23:09 the-other-tim-brown

@the-other-tim-brown I would love to take a look at this. Can you give me some starting points?

Dec 12 '23 17:12 taher-cldcvr

@ashvina correct me if I'm wrong on any of this.

I think the general idea was to integrate with https://opentelemetry.io/ so we can export information about how long the conversion is taking to allow users to better understand the overhead. With OTel specifically, we can even introduce spans for sections of the code to allow finer grain views instead of a simple metric reporting total conversion time. This allows us to point to specific parts of the code taking more time than others.

Dec 12 '23 18:12 the-other-tim-brown

Hi @taher-cldcvr Good timing. This feature seems to be getting some traction. Our main goal is to generate valuable telemetry. As Tim pointed out, opentelemetry is a good option. Do you have experience with OT? It would be great to start a design proposal on this and discuss. Thoughts?

Dec 12 '23 19:12 ashvina

@ashvina I do not have experience with open telemetry, however, on exploration this seems more like a mix of sentry and metrics reporting, which I should be able to start a design discussion. I will put up a doc around this and get a community opinion on this.

Dec 14 '23 06:12 taher-cldcvr