Compiling a generic ToJSON with stack build --profile consumes much more memory

[acrylic-origami]

This may be more of a question than it is an issue per se, but I've noticed that compiling a genericToEncoding for a timid-enough datatype:

data A = A [Int] [(Int, ((String, (Int, Int)), [(Int, Int)]))] deriving Generic
instance ToJSON A where
  toEncoding = genericToEncoding defaultOptions

• stack build takes 288MB
• stack build --profile takes 2264MB, which seems like quite a lot.

Progressively simpler datatypes take less and less memory, with data A = A Int taking 342MB with profiling and 218MB without. This is with:

• Aeson 1.4.7.1
• Stack version 2.3.1

The project I benchmarked is available at this repo. See [no-]profile[-simple] files for the memory profiling.

Again, not sure if this is more to do with GHC's treatment of profiling, Stack's --profile option, aeson, or GHC generics. I'm also not sure how justified this expectation of lower memory consumption is. I'm personally interested in this because of a cost center profiling tool I'm making, but imagined others had run into these issues trying to profile some Aeson-dependent project as well.

[bergmark]

Profiling takes more memory, I'm sure there are things we could do in aeson that could alleviate that but at the end of the day I think it's up to GHC to do those improvements. Any optimizations we make for this are not guaranteed to hold when the next GHC ships (true for all optimizations, but profiling is probably much less of a priority for the GHC developers). Performance is important in aeson, but optimizing for profiling is bound to be a pretty low priority as we care more about the actual code that will run.

These things are tricky. If you find an improvement that can be done within aeson we are likely to accept a PR but we might be hesitant if it complicates the code base, or there isn't a reliable way to identify regressions.

Also please make sure you are using the latest version of aeson when filing an issue (though for this particular case I don't think it would make a difference). The GHC version you are using is also important, that might actually make a difference here.

[acrylic-origami]

@bergmark Yeah, that makes sense to me. Thanks for your comments.

To explain myself a bit, I'm mainly just concerned with practicality of compiling a project with profiling, and for all the dependencies to compile with reasonable resources. Ultimately, I was just curious if any identifiable patterns here were leading to high memory usage with generics + profiling, and what this could mean for the appeal of compiling with profiling. Without anything specific, it's definitely a question on GHC itself as you say.

P.S. Oops, forgot to write the GHC version was 8.8.4.