KyleBanks
Possible performance issue detected in downstream library using `depth`
Hi @KyleBanks,
An issue has been reported recently in a downstream tool, swaggo/swag, which uses depth to calculate required dependencies in order to build valid OpenAPI 2.0 output as a result of parsing code comments in a given folder structure.
Under https://github.com/swaggo/swag/issues/1032 the user @matrixik has exquisitely pointed out, there seems to be a performance issue in swag, which I have narrowed down, with the help of @sdghchj to be a problem with depth.
- Could you give a look at the flamegraphs provided under the issue linked and verify the assumption the problem lies in this tool is correct?
- Would you be able and willing to help fixing it?
We use swaggo throughout our platform at Compensate in order to build the documentation for all our services.
Thank you,
-Manuel
@magandrez indeed it does point to depth as the culprit.
Admittedly performance was never a major concern in the design as I hadn't anticipated it having any real-time applications, so there's likely some room for improvement in that area. I'll see if I can get some time over the holidays to dig into it a little deeper.
I suspect the recursive use of build.Import from the stdlib is going to be the culprit. It's pretty central to the design of depth, and I'm already calling Import only as many times as needed, so that's going to be tricky to solve for... Needs some benchmarking to confirm this assumption though.
@KyleBanks Yep, indeed it would require some benchmarking. I will be available during holidays for some hacking, so let me know if I can be of any help.
It would be good to have that fixed, as swaggo is one of the two main projects to build and serve OpenAPI documentation in Go from documentation strings (i.e.: it is pretty big deal).
I suspect the recursive use of build.Import from the stdlib is going to be the culprit. It's pretty central to the design of depth, and I'm already calling Import only as many times as needed, so that's going to be tricky to solve for... Needs some benchmarking to confirm this assumption though.
I use depth.Resolve() in my demo code, i found it like @KyleBanks said that build.Import spent much time.
build.Import single call pprof:
