PyO3
DWARF debug symbol file is missing on Mac
Maturin fails to include DWARF debug symbol files in pip packages on Mac. This results in backtraces not displaying filenames and line numbers. On (at least some) Mac platforms, cargo will generate the following build artifacts for builds with debuginfo:
target
└── debug
├── build
├── deps
│ ├── libpython_cffi.dylib
│ ├── libpython_cffi.dylib.dSYM
│ │ └── Contents
│ │ └── Resources
│ │ └── DWARF
│ │ └── libpython_cffi.dylib
│ └── python_cffi.d
├── examples
├── incremental
│ ...
├── libpython_cffi.d
└── libpython_cffi.dylib
Crucially, the target/debug/libpython_cffi.dylib file (which is byte-for-byte identical to target/debug/deps/libpython_cffi.dylib) does not seem to contain symbols which are instead provided by the target/debug/deps/libpython_cffi.dylib.dSYM folder. This folder should be included in the pip wheel to obtain a site-packages structure as below:
/Users/clemens/anaconda3/envs/blobcache/lib/python3.8/site-packages/pyo3_backtrace_repro
├── __init__.py
├── __pycache__
│ └── __init__.cpython-38.pyc
├── libpyo3_backtrace_repro.dylib.dSYM
│ └── Contents
│ ├── Info.plist
│ └── Resources
│ └── DWARF
│ └── libpyo3_backtrace_repro.dylib
└── pyo3_backtrace_repro.cpython-38-darwin.so
Currently, the libpyo3_backtrace_repro.dylib.dSYM folder is missing which prevents backtraces from working properly. I have confirmed that adding this folder manually to site-packages fixes the issue, but still need help with making the necessary changes to maturin to properly include those files in the pip wheel. Some additional context in https://github.com/PyO3/pyo3/issues/766
Interesting, I wasn't aware that on mac the debug info is an extra directory! Could you copy the last few lines of cargo build --message-format json to see how cargo categorizes the debug symbol file?
Cargo doesn't seem to make any mention of it:
{"reason":"compiler-artifact","package_id":"python_cffi 0.1.0 (path+file:///Users/clemens/openai/python-rust-cffi)","target":{"kind":["cdylib"],"crate_types":["cdylib"],"name":"python_cffi","src_path":"/Users/clemens/openai/python-rust-cffi/src/lib.rs","edition":"2018","doctest":false},"profile":{"opt_level":"0","debuginfo":2,"debug_assertions":true,"overflow_checks":true,"test":false},"features":[],"filenames":["/Users/clemens/openai/python-rust-cffi/target/debug/libpython_cffi.dylib"],"executable":null,"fresh":false}
It shouldn't be hard to include the debug symbol file. Do you know any Python C/C++ extension that includes debug symbol files as a reference?
