Provide access to different intermediate stages of mlir pipeline within the program

[paul0403]

Currently, to access the intermediate stages of the milr compilation pipeline, we need to do @qjit(keep_intermediate=True), and access the produced files with either python's open(), or some sort of subprocess.call("less/cat ...").

It would be very helpful to have then accessible in the QJIT object itself. Right now there only is QJIT.mlir, which is the mlir before it's sent to the pipeline (aka the mlir before any passes are run).

[josh146]

@paul0403 you can use this function: https://docs.pennylane.ai/projects/catalyst/en/latest/code/api/catalyst.debug.print_compilation_stage.html

[paul0403]

@paul0403 you can use this function: https://docs.pennylane.ai/projects/catalyst/en/latest/code/api/catalyst.debug.print_compilation_stage.html

This function still needs the keep_intermediate=True.

Although, if we are not keeping the intermediates, dragging all the stage mlirs as gigantic strings might not be ideal.

[josh146]

I think the reason to moving to a functional form was to avoid properties/attributes that depended on state? cc @dime10

[dime10]

I think we distilled the problem down to:

• in order to print intermediate compilation stages, you need to enable an option that dumps several files into the working directory

I mentioned to Paul that keep_intermediate option is necessary because during regular execution we don't want to store all the string representations of each pass output because it would slow down the normal use case. Making it opt-in means regular users don't pay the overhead cost.

So this issue would boil down to making intermediate compilation outputs available without dumping them to file / polluting the working directory, but still only on request to avoid the performance penalty.

[DSGuala]

Closing as this is no longer an issue after https://github.com/PennyLaneAI/catalyst/pull/981