parca icon indicating copy to clipboard operation
parca copied to clipboard

symbolizer: Wrong order of segments from pprof heap profile flamegraph

Open bwplotka opened this issue 3 years ago • 5 comments

Just for awareness--there might be bug in stack trace ordering in flame graph. For example, see the bytes.Split at the bottom. It comes before sum.Sum which is wrong:

image

It's the sum.Sum that calls bytes.Split. pprof profile obtained from HTTP looks fine. I downloaded using Parca download button and it shows correct order:

image

bwplotka avatar Aug 04 '22 21:08 bwplotka

Thank you very much for reporting! Are both of these collected from the HTTP endpoints or is the Parca one from the Parca Agent?

brancz avatar Aug 08 '22 12:08 brancz

Since this is not incorrect in the downloaded version, I suspect this is just a mistake in rendering inlined functions in the icicle graph in the wrong order.

brancz avatar Aug 09 '22 12:08 brancz

hmm ... it seems like that already does the right thing https://github.com/parca-dev/parca/blob/790294ac3518c6edacc68645b3e678b626a6caff/pkg/query/flamegraph.go#L287-L301

Which makes it seem like either this is about asynchronous symbolization or the profile that was written was already symbolized and had an unexpected order.

Could you clarify where this data came from?

brancz avatar Aug 09 '22 13:08 brancz

I might have reproduced this. I'll look into it in my first freecycle. https://pprof.me/f5c1a5d CleanShot 2023-03-24 at 13 32 13 CleanShot 2023-03-24 at 13 32 27

kakkoyun avatar Mar 24 '23 13:03 kakkoyun

Sorry, missed the discussion. It's either HTTP or benchmark --memprofile flag.

bwplotka avatar Mar 24 '23 13:03 bwplotka