vmprof 0.4 fails to compile with Python 3.6 under Windows 64 bit

[divenex]

vmprof 0.4 fails to compile with Python 3.6 under Windows 64 bit with the attached error log

vmprof_error.txt

[planrich]

Thanks, I'll have a look at that.

[planrich]

CPython 3.6 defines many int types on windows. They conflict with the third party library we use. This should work as of 4714450.

You can pull that with pip. Something along the lines: $ pip install git+https://github.com/vmprof/vmprof-python

should work.

[divenex]

Thanks for the prompt reply.

Now vmprof compiles and installs without errors. I also managed to successfully produce the time statistics on a minimal few-lines Python script.

However, as soon as I try to run "python -m vmprof script.py" on any of my real programs (which normally take a couple of seconds to run), the profiling never stops. I see a "Python" process in Task Manager which keeps growing in memory usage, until it fills up my RAM and I have to kill it.

[planrich]

Does your script use the C API or a library that does? This would explain the issue. It is the same that has been fixed for unix back then (locking issue with the GIL and acquiring the thread state).

[divenex]

My scripts use Numpy/Scipy. To be more specific with minimal examples illustrating the problem. The following command takes a fraction of a second to run

python -m vmprof --lines .\test.py

where test.py contains

def test():
    print("test")

if __name__ == '__main__':
    test()

The following very simple Numpy example still manages to run, but it is about the largest I can successfully run. It pushes the limit of my machine, it uses all four cores for 45s and produces 1400 lines of output (!), when using the same command as above

import numpy as np

def test():
    a = np.arange(10)
    print(a**2)

if __name__ == '__main__':
    test()

[planrich]

Thanks for your code snippets. I think I need some more infomation though. I tried to run both snippets and they both complete in less than 1 second (which is totally fine I guess). I run Windows 10 64 bit in a virtual machine, CPython 3.6.1rc1 and the latest head of vmprof.

Can you describe your system a bit more an how you installed and ran the code?

[divenex]

Thanks for testing under Windows. This is puzzling as it looks we have nearly the same configuration. My tests were also under Windows 10 (1607) 64 bit, with all updates applied. I used Python 3.6.1 (Miniconda)and I installed the latest vmprof from GitHub and Visual C++ 14 from here.

One question: is it normal for my Numpy example above to produce 1400 lines of output? I have not yet been able to see the output of a properly working vmprof...

The code was run with the commands I had give above.

[planrich]

Here how it looks for me:

It would be more interesting to see numpy code in a loop.

[divenex]

This is very different from what I obtain. Using the command python -m vmprof --lines .\test.py > output.txt I get this output.

[planrich]

I see, I have pushed 68350b3 which removes the <unknown code> entries. Short explanation: with --lines the stack traces size is doubled. The line number is included as a separate stack trace entry.<unknown code> corresponds to these entries which are not shown anymore.

[divenex]

The entries have indeed disappeared. However, when running exactly the same command as in your screenshot above, I get 340 lines of output (attached HERE) instead of the 22 you get. This seems to dramatically increase the time vmprof takes to run.

I have also tried to profiling test.py in Pycharm, which is supposed to support vmprof to nicely visualize line timing in the editor, but the profiling does not give any output: it takes forever until I have to kill it.

[planrich]

I have now also installed miniconda on my vm (cpython 3.6.0). Could not reproduce, I'm not sure what but I'm missing something. Would it be possible to get my hands on the binary profile win64 generated?

[planrich]

You could also upload the profile to vmprof.com (provide --web) and post the link.

[divenex]

HERE is the result of running python -m vmprof --web .\test.py

[mattip]

Closing, windows wheels are available and pass tests. Please open a new issue if the package is still failing to properly profile your app.