openzipkin
Find a place to document dependencies UI troubleshooting
I've written the below brain dump, as folks trying to write new instrumentation often run into problems getting the dependency UI working. I'm not sure where to place this doc, but since 3 repositories include code around this topic, likely here is best.
One tricky part around this is that instrumentation need to follow some practice, or else they won't show up at all. This is a documentation concern because instrumentation are written in any language, and usually "look at this scala/java file" isn't always helpful.
The dependency graph view is special cased for zipkin "server" annotations. That's due to a number of reasons, including keeping the rules simple, the input data small and avoiding double-counting.
The following patterns will trigger nodes in the dependency tree.
- server span
- A span has a SERVER_RECV("sr") annotation, but no CLIENT_ADDR("ca")
- uninstrumented client+server span
- A span has a CLIENT_ADDR("ca") binary annotation and a SERVER_RECV("sr") annotation
- A span has both CLIENT_ADDR("ca") and SERVER_ADDR("sa") binary annotations
- client-only span
- A span has a SERVER_ADDR("sa") binary annotation, but no SERVER_RECV("sr")
If spans don't produce combinations like above, stop here, and try and figure out why or what's missing.
If spans look right per above, yet no dependency graph is showing up in either mem or mysql storage...
catch a trace via http://zipkinhost:9411/api/v1/trace/your_trace_id and ask about it in https://gitter.im/openzipkin/zipkin, preferably a pared down trace (vs one with a 100+ spans).
If you want details, there are a number of tests that account for behaviors, addressing specific instrumentation use cases. https://github.com/openzipkin/zipkin-java/blob/master/zipkin/src/test/java/zipkin/internal/DependencyLinkSpanTest.java https://github.com/openzipkin/zipkin-java/blob/master/zipkin/src/test/java/zipkin/internal/DependencyLinkerTest.java https://github.com/openzipkin/zipkin-java/blob/master/zipkin/src/test/java/zipkin/DependenciesTest.java
other note of extreme interest.
currently, only in-memory (dev) and jdbc (mysql) do on-demand dependency linking. If you are using cassandra, the only current way to create links is by running https://github.com/openzipkin/zipkin-dependencies-spark In other words, if you are running cassandra and see no links, it might just be because the job to make them wasn't run!
Thanks for capturing this; I think with a little editing this is almost ready to go straight into documentation.
