Replace direct QtScript usage with an isolating interface.
This is documenting my attempts to isolate QtScript to interfaces that we control, to enable us to add or replace scripting engines in the future.
I've been looking this over a bit. Overall it looks good, but this is going to need documentation about what you're doing here, what's changing, and what the desired end state is.
So for instance:
- What are the high level changes being made?
- Why are we changing from QScriptValue to ScriptValuePointer?
- Why QSharedPointer and not std::shared_ptr? Nothing against Qt, but we're going to be working with non-Qt scripting engines anyway, so any integration benefits with Qt may be moot there.
- There's ScriptManager now. I take it manages scripts, what does that involve? Looking at it, I think there's stuff that doesn't seem to quite belong in it. That a ScriptManager loads scripts sounds reasonable, but why is resourcePath there? That seems like something that should be either an outside API (because it's not script management), or an interface implemented by each script engine (because perhaps different engines store stuff in different places).
- What will a second scripting engine look like?
- etc.
I think it's important to have a plan upfront here to be clear on what's changing, why, and to have a good idea of what's not done yet, what's intentional, and what's a bug.
Currently there are three interfaces being introduced here:
- ScriptEngine, to abstract access to QScriptEngine. It will only be creatable from a factory function.
- ScriptValue, to abstract access to QScriptValue. It will only be creatable from a ScriptEngine.
- ScriptContext, to abstract access to QScriptContext. It cannot be created but will be available through Scriptable or ScriptEngine.
These interfaces cannot be directly created and do not inherit from QObject (directly), so cannot have any slots or signals. All interfaces are pointers which is why ScriptValue is now a pointer (where QScriptValue is generally not a pointer). Using QSharedPointer for these objects also means that copying them doesn't involve calling into the scripting engine to make a copy of the actual value.
The choice of QSharedPointer over std::shared_ptr is mostly as our project is a Qt project. Choosing one over the other won't really affect much.
ScriptManager is a QObject containing most of the code that is common between the scripting engines. It will encapsulate the main Run() function as well as all the Vircadia objects that are shoved into globalObject().
I'm unsure what to do to make sure "tests" and "tests-manual" work, offhand they don't appear to be part of the project that gets compiled.
If it helps, here are one-line descriptions for at least the classes in script-engine: http://protocol.odys.se/script-engine/annotated.html
[ScriptInterface] are new abstract classes to create engine-independent interfaces [QtScript] are classes deemed qtscript-specific and can now only be accessed through a ScriptInterface object ScriptManager has taken over all the duties possible that ScriptEngine used to have while not being QtScript-specific (which is quite a bit)
everything else is as minimally touched while having scripting references redirected to engine-independent interfaces.
Very surprisingly... I'm not seeing any linker dependencies from script-engine to other libraries (except perhaps shared)
I haven't done any testing of this outside of windows interface, however there's enough green lights showing up with this that I think it's ready to get poked at by others (and hopefully all the checks will pass this time)
Android is expected not to build successfully.
The following links are available: build (macOS-latest, client)
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-Interface-PR1200-b1b0f5a-_4abbff21.dmg
build (ubuntu-18.04, full)
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-b1b0f5a-_4abbff21.sh
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-b1b0f5a-.tar_4abbff21.Z
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-b1b0f5a-.tar_4abbff21.gz
build (macOS-latest, full)
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-b1b0f5a-_4abbff21.dmg
build (windows-latest, full)
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-b1b0f5a-_4abbff21.exe
The following links are available: build (macOS-latest, client)
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-Interface-PR1200-bb11550-_ff46561a.dmg
build (ubuntu-18.04, full)
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-bb11550-.tar_ff46561a.gz
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-bb11550-.tar_ff46561a.Z
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-bb11550-_ff46561a.sh
build (macOS-latest, full)
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-bb11550-_ff46561a.dmg
build (windows-latest, full)
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-bb11550-_ff46561a.exe
The following links are available: build (macOS-latest, client)
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-Interface-PR1200-2336f63-_26c9810d.dmg
build (ubuntu-18.04, full)
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-2336f63-_26c9810d.sh
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-2336f63-.tar_26c9810d.gz
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-2336f63-.tar_26c9810d.Z
build (macOS-latest, full)
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-2336f63-_26c9810d.dmg
build (windows-latest, full)
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-2336f63-_26c9810d.exe
This latest commit has increased my concern about ScriptContext being primarily a shared_ptr. It can make sense in some circumstances but definitely not as a general fact. Need to review memory management surrounding this, it might just end up being a custom deleter somewhere.
Reviewed the code again and ScriptContext is a C++ wrapper around a script engine object that is always a shared_ptr, even if the engine object isn't.
The following links are available: build (macOS-latest, client)
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-Interface-PR1200-de691af-_4f03899f.dmg
build (macOS-latest, full)
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-de691af-_4f03899f.dmg
build (ubuntu-18.04, full)
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-de691af-.tar_4f03899f.Z
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-de691af-_4f03899f.sh
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-de691af-.tar_4f03899f.gz
build (windows-latest, full)
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-de691af-_4f03899f.exe
I've just created a rough diff of ScriptEngine and ScriptManager at https://github.com/odysseus654/vircadia/commit/4f631e5d9dd551962059ed55f827c7f29abea399, hoping this helps code review
I'm okay with merging master into this PR to clear up the conflicts if/when it wouldn't make any code reviews more difficult than they already are.
a few super small remaining things in the comments above being worked through but I think we should:
- merge master and resolve conflicts
- make sure everything builds
- start extensive testing...idk really know the best way to do this except "make sure important scripts still work" and watch for crashes
- merge!
The following links are available: build (macOS-10.15, client)
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-Interface-PR1200-97b2b96-_170a8f32.dmg
build (ubuntu-18.04, full)
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-97b2b96-.tar_170a8f32.Z
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-97b2b96-.tar_170a8f32.gz
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-97b2b96-_170a8f32.sh
build (windows-latest, full)
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-97b2b96-_170a8f32.exe
build (macOS-10.15, full)
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-97b2b96-_170a8f32.dmg
build (self-hosted_debian-11_aarch64, full)
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-97b2b96-_170a8f32.sh
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-97b2b96-.tar_170a8f32.Z
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-97b2b96-.tar_170a8f32.gz
Agreed with merging this into the next release (2022.1.1)
AppImage is here: https://appimage.moto9000.moe/experimental/Vircadia-x86_64_PR1200_e407507.AppImage I have encountered one crash while smoke testing. The build crashes when pressing a button in the Emote app. Relevant log entry:
[qml] [FlickableWebViewCore.qml] Web Entity JS message: file:///tmp/.mount_VircadsGDPJI/vircadia/interface/scripts/system/html/EmoteApp.html 126 Fall button click
The AppImage is hooked up to Sentry: https://sentry.vircadia.dev/organizations/vircadia/releases/PR1200_e407507/?project=2
The crash dump is unreadable, can you please register the symbols with Sentry and try again?
FYI I have successfully reproed this crash (with full stack trace) on windows
The following links are available: build (ubuntu-18.04, full)
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-15b6189-_e407507a.sh
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-15b6189-.tar_e407507a.Z
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-15b6189-.tar_e407507a.gz
build (macOS-10.15, client)
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-Interface-PR1200-15b6189-_e407507a.dmg
build (macOS-10.15, full)
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-15b6189-_e407507a.dmg
build (self-hosted_debian-11_aarch64, full)
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-15b6189-.tar_e407507a.Z
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-15b6189-.tar_e407507a.gz
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-15b6189-_e407507a.sh
Crash has been determined as a hole in this PR -- QtScript will "shortcut" property access if it detects it's not dealing with an object that inherits from QScriptable (and nothing inherits from QScriptable anymore) so I'm not able to get a proper "this".
Will look at this more "tomorrow", but it's starting to look like I'm going to have to write some code to intercept property access and properly record "this" so these kinds of actions can start working again.
The crash dump is unreadable, can you please register the symbols with Sentry and try again?
Yeah sorry, Sentry is pretty slow so even after 10 minutes after uploading the symbols it still couldn't use it for some reason. I assume you don't need another dump from me because you got a repro on your machine.
Just added a rather significant chunk of code taking control of translating calls between script and Qt away from QtScript, mostly after finding out the reason why JulianGro's reported crash was happening (QtScript was disabling some code because it detected we're no longer using QScriptable). This code was going to be necessary when dealing with V8 but I wasn't expecting to have to write it quite yet.
I'm not done with making changes here but this is big enough that I'd appreciate a code-review of this particular commit, also hoping this is in a place where we can start doing some more testing to uncover any issues going on (expecting most problems to show up as strange scripting errors at this point)
This will not be compiling under Windows until I do a bit more research trying to find out why MIDI broke. Looks like there's a conflict with main branch here too. I also have some changes in response to earlier code reviews that I have better answers for. I'm also wanting to delete a bunch of workaround code that is no longer needed and do a code-review of "adjacent" code to improve some odd things I saw while testing this.
The following links are available: build (ubuntu-18.04, full)
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-004dfa0-.tar_3b37bbc3.Z
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-004dfa0-_3b37bbc3.sh
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-004dfa0-.tar_3b37bbc3.gz
build (windows-latest, full)
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-004dfa0-_3b37bbc3.exe
build (self-hosted_debian-11_aarch64, full)
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-004dfa0-.tar_3b37bbc3.Z
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-004dfa0-_3b37bbc3.sh
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-004dfa0-.tar_3b37bbc3.gz
The following links are available: build (macOS-10.15, client)
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-Interface-PR1200-8c54ffa-_01446096.dmg
build (ubuntu-18.04, full)
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-8c54ffa-.tar_01446096.Z
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-8c54ffa-.tar_01446096.gz
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-8c54ffa-_01446096.sh
build (macOS-10.15, full)
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-8c54ffa-_01446096.dmg
build (windows-latest, full)
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-8c54ffa-_01446096.exe
build (self-hosted_debian-11_aarch64, full)
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-8c54ffa-.tar_01446096.Z
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-8c54ffa-_01446096.sh
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-8c54ffa-.tar_01446096.gz
The following links are available: build (macOS-10.15, client)
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-Interface-PR1200-ce854da-_01446096.dmg
build (macOS-10.15, full)
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-ce854da-_01446096.dmg
build (ubuntu-18.04, full)
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-ce854da-.tar_01446096.Z
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-ce854da-_01446096.sh
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-ce854da-.tar_01446096.gz
build (windows-latest, full)
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-ce854da-_01446096.exe
build (self-hosted_debian-11_aarch64, full)
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-ce854da-.tar_01446096.Z
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-ce854da-_01446096.sh
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-ce854da-.tar_01446096.gz
The following links are available: build (macOS-10.15, client)
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-Interface-PR1200-2f7f5b8-_7a491551.dmg
build (macOS-10.15, full)
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-2f7f5b8-_7a491551.dmg
build (ubuntu-18.04, full)
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-2f7f5b8-_7a491551.sh
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-2f7f5b8-.tar_7a491551.gz
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-2f7f5b8-.tar_7a491551.Z
build (windows-latest, full)
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-2f7f5b8-_7a491551.exe
build (self-hosted_debian-11_aarch64, full)
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-2f7f5b8-_7a491551.sh
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-2f7f5b8-.tar_7a491551.gz
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-2f7f5b8-.tar_7a491551.Z
The following links are available: build (macOS-10.15, client)
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-Interface-PR1200-66f303a-_bc8d57ac.dmg
build (macOS-10.15, full)
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-66f303a-_bc8d57ac.dmg
build (ubuntu-18.04, full)
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-66f303a-.tar_bc8d57ac.Z
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-66f303a-.tar_bc8d57ac.gz
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-66f303a-_bc8d57ac.sh
build (windows-latest, full)
- https://athena-public.s3.eu-west-3.amazonaws.com/builds/PR/vircadia_vircadia/PR1200/Vircadia-PR1200-66f303a-_bc8d57ac.exe
A couple of potential errors found while in the serverless tutorial... Serverless Tutorial Errors.txt