Development Workflow/Build Issues (MacOS)

[rac0316]

Hello,

After some struggles I got master to build on Mac OS, and now I want to start integrating godot-python into a new Godot project. Trying to use the pong example as a template, however when I open it up in the editor, Godot complains about missing .py dependencies:

Is this expected? What I'm after is a little guide on how to make a Godot project from scratch, add the compiled godot-python library, and then build, but I don't see any documentation on how to accomplish this.

[touilleMan]

Hi @rac0316 !

I'm struggling a lot with Mac OS builds (I don't have a Mac so I can only rely on the CI which is far from ideal for debugging...) As a matter of fact I've disabled CI for Mac for the moment (you can see here the last failed build on the CI)

it up in the editor, Godot complains about missing .py dependencies: Is this expected?

Well it's kind of expected on MacOS ;-)

My guess is Godot-python failed to load on MacOS due to some specific requirements on this platform regarding dynamic libraries.

Here is what happen when Godot loads Godot-python:

• Godot load libpythonscript.so (this is the gdnative library considered as the entry point)
• libpythonscript.so loads libpython37.so (so the python interpeter)
• libpythonscript.so loads _godot.so (the _godot module exposed to python and written in cython)
• _godot.so loads libpython37.so (given it is a python module !) and also relies on global variables defined in libpythonscript.so

You can see there is a dependency loop libpythonscript.so -> _godot.so -> libpythonscript.so may prevent the loading (though it works fine on windows and linux...)

Can you run Godot from a terminal and have a look at the output to see if the Pythonscript module is successfully loaded ? (you should see something like Pythonscript 0.20.1+dev (CPython 3.7.1.final.0))

By the way I've recently updated the build system in order to improve the way the project is build, what is the commit you've tested ?

[rac0316]

I ended up getting it! The issue is related to the other bug #154 where you have to set LD_LIBRARY_PATH (Linux) or DYLD_LIBRARY_PATH (MacOS). This has to be done when launching editor as well, then the library loads just fine.

Latest commit I had was from 4/12: https://github.com/touilleMan/godot-python/commit/babebc5c1af78fc7db1e5d447146ad542b6c3ec2

Side note - HUGE build time improvement between this commit and the v0.20.1 release - awesome job!

I think it'd be good once the dust has settled on bugs to add a little documentation on what to do once you have a successful build (i.e. putting pythonscript.gdnlib and pythonscript folder in project folder). Due to the symbolic links in the examples, it's not self-explanatory until after the source has successfully built that these files need to be in the project folder.

[mikolak-net]

Adding to @rac0316 's comment (sorry for the mini-hijack BTW), it would be great for the Wiki to be enabled (or another mode for documentation implemented). Apart from the workflow, there are other topics that probably warrant attention (such as a more comprehensive explanation of how the Python API looks like), and are "contributable" for others apart from the maintainer - and the README is long enough as it is.

[touilleMan]

Side note - HUGE build time improvement between this commit and the v0.20.1 release - awesome job!

Not sure what's the reason... Are you sure you didn't compiled with bindings_generate_sample=true option (so that bindings.pyx only contains a subset of the api) ?

I ended up getting it! The issue is related to the other bug #154 where you have to set LD_LIBRARY_PATH (Linux) or DYLD_LIBRARY_PATH (MacOS). This has to be done when launching editor as well, then the library loads just fine.

That really interesting ! Now I wonder how we could configure the linker inform about the need to load another library before this one...

What is done on x11 (and osx so far but with no success...) is we set LINKFLAGS with f"-Wl,-rpath,'$$ORIGIN/{libpython_path}' to inform the library where it can find the library it needs to load:

https://github.com/touilleMan/godot-python/blob/382139bd52a9c842db2f81904c8b82d90728c757/site_scons/site_tools/cython.py#L82-L103

Are you aware of some difference in behavior on macOS ? I've seen that scons define LoadableModule and SharedLibrary which are two different thing on macOS, do you think you could investigate ? ;-)

Adding to @rac0316 's comment (sorry for the mini-hijack BTW), it would be great for the Wiki to be enabled (or another mode for documentation implemented)

Yup, documentation is always good ! I'd like to have things stable enough before investing in documentation though, ideally api should be pretty equivalent to Godot, but there is a lot of interesting features that should be documented like how to use pip or Cython for you game ;-)

[touilleMan]

@rac0316 about the huge improvement in compilation time... bfa9cd04a0fa2f2d35140edd0f5616b449fdfe28 :trollface:

[rac0316]

I'm not an expert with linking, but I can definitely try a few things if you have any hunches!

[rac0316]

Update on MacOS build - with the latest code (bindings_generate_sample=false), MacOS generates this error on build. Any ideas?

[touilleMan]

--strip-all option is passed to bindings.pyx compilation in order to strip symbols out of the final binary (bindings modules is huge so we save plenty of space by doing this, but this is done at the expense of readability during debug of course)

It's possible this option is not available on macOS's linker (from your output, it seems you use gcc as compiler but clang as linker)

So the simplest solution would be to disable this option on macOS, to do so you can comment here: https://github.com/touilleMan/godot-python/blob/ffba065b634c772418cd1eab69657e79c2406db5/pythonscript/godot/SConscript#L92-L94

[rac0316]

Got it, makes sense!

OK so, now it builds and I can get Python code to run, but if I try to keep the line from godot.globals import * in my Python files, I get No module named 'godot.globals' when running the project. Again, this is with bindings_generate_sample=false.

[touilleMan]

There is no more godot.globals module (you should only use from godot import xxx, for instance from godot import OK)

[mikolak-net]

@touilleMan : it's possible that @rac0316 was referring to the fact that the godot.globals import is still being emitted by the script template.

[touilleMan]

@mikolak-net good catch ! I've fixed this with 7fedd2fe85d098de511938f41ba2e49782bb3857

[matheus2740]

@rac0316 I just pushed a pull request with fixes for macOS compilation. I was only able to test in one computer, so if you could test it and provide feedback, that would be great!

[rac0316]

Reporting back - pulled latest and code builds on MacOS without any code changes! Also, I can load the editor without setting DYLD_LIBRARY_PATH!

Looks good.. Need me to test anything else?