cpytraceafl icon indicating copy to clipboard operation
cpytraceafl copied to clipboard

when and how 'rewriting_compile' function is called?

Open leveryd opened this issue 5 years ago • 3 comments

i read "rewriter.py" source code, "builtins.compile" confused me

@functools.wraps(original_compile)
    def rewriting_compile(*args, **kwargs):
        ...
builtins.compile = rewriting_compile

what i understand at first is "compile function has been hooked, when run 'python xxxx.py' and compile the code, builtins.compile will be called first, so rewriting_compile function will be called".

but after i run test code as below, i am confused.

import dis
import functools
import random
import os
import _frozen_importlib_external
import builtins
from sys import version_info
from ast import PyCF_ONLY_AST

original_compile = builtins.compile

@functools.wraps(original_compile)
def rewriting_compile(*args, **kwargs):
    flags = (len(args) >= 4 and args[3]) or kwargs.get("flags") or 0
    print("rewriting_compile")
    return False

builtins.compile = rewriting_compile

print("end")

"rewriting_compile" will not be printed.

so my question is " when and how 'rewriting_compile' function is called".

leveryd avatar Feb 18 '21 13:02 leveryd

I'm not sure why it would be called in your case - in python, the compilation step is generally only done when you import a new module (or e.g. use eval to run an arbitrary statement). This is why I mention that install_rewriter() needs to be called very early on in the python process, preferably before any interesting modules are imported.

risicle avatar Feb 18 '21 20:02 risicle

import functools
import builtins



original_compile = builtins.compile


@functools.wraps(original_compile)
def rewriting_compile(*args, **kwargs):
    flags = (len(args) >= 4 and args[3]) or kwargs.get("flags") or 0
    print("rewriting_compile")
    return False

builtins.compile = rewriting_compile
compile = rewriting_compile


import hashlib

eval("1==1")

# code = compile("", "name", "exec")

if "eval" or "import a new module" will call compile function,why above code does not print "rewriting_compile"?

leveryd avatar Feb 28 '21 10:02 leveryd

I'm not sure about the eval case (I've never really dug into the inner workings of eval) but for the import hashlib, this is a case that won't call builtins.compile because the bytecode for that module will have already been compiled & written to a .pyc file. Instead it will call _frozen_importlib_external._compile_bytecode when it tries to load that off disk. That's why I have to hook both of them in rewriter.py.

risicle avatar Feb 28 '21 13:02 risicle