Initial support for VR headsets

[pellet]

This PR modifies the current rendering engine to allow support for using a head mounted display. Here is a github issue explaining the change: Support for Oculus/Meta Quest VR. PsychXR is utilized for seamlessly integrating HUD support into the existing codebase which depends on PsychoPy for presenting stimulus.

• The core.wait calls had to be removed so that the render loop could continually update the display in relation to the current position and angle of the head.
• The code changes are backwards compatible so that existing experiments can be ran and replicated without any change to the presented visual stimulus or eeg data.
• More VR headsets can be supported by adding OpenXR support to PsychXR or directly into eeg-notebooks.

The new feature functionality can be tested by connecting a supported headset(eg oculus/quest) and running the N170 experiment with the parameter use_vr=True Backwards compatibility can be tested by running any experiment without any changes needed.

[JohnGriffiths]

@pellet thanks for this. I am now back from pat leave and this is high up on my priority list. I first need to get my hands on a comp with a good enough graphics card for the quest screen sharing. Once I have that I will start testing this out and feed back here.

[pellet]

@JohnGriffiths sounds great :) Let me know if you run into any issues with the testing and I will be happy to help out. If the changes to the rendering engine somehow break some old experiments I can leave the existing rendering code in place and separate the VR rendering code to only run with use_vr=True. I've opened an issue with PsychXR regarding setAutoDraw(True) no working, and window.getActualFrameRate() crashing. The Quest 2 does up to 120hz which is nice for the SSVEP experiment 👍🏼

[tmorshed]

@pellet - getting the following error. Any thoughts?

In [1]: from psychopy import visual

In [2]: blah = visual.Rift(monoscopic=True,headlocked=True)
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
Cell In[2], line 1
----> 1 blah = visual.Rift(monoscopic=True,headlocked=True)

File ~\miniconda3\envs\expy-vr\lib\site-packages\psychopy\contrib\lazy_import.py:118, in ScopeReplacer.__call__(self, *args, **kwargs)
    116 def __call__(self, *args, **kwargs):
    117     obj = object.__getattribute__(self, '_resolve')()
--> 118     return obj(*args, **kwargs)

File ~\miniconda3\envs\expy-vr\lib\site-packages\psychopy\visual\rift.py:316, in Rift.__init__(self, fovType, trackingOriginType, texelsPerPixel, headLocked, highQuality, monoscopic, samples, mirrorMode, mirrorRes, warnAppFrameDropped, autoUpdateInput, legacyOpenGL, *args, **kwargs)
    312     raise RuntimeError("HMD service is not available or started, " +
    313                        "exiting.")
    315 if not libovr.isHmdConnected():
--> 316     raise RuntimeError("Cannot find any connected HMD, check " +
    317                        "connections and try again.")
    319 # create a VR session, do some initial configuration
    320 initResult = libovr.initialize()  # removed logging callback

RuntimeError: Cannot find any connected HMD, check connections and try again.

In [3]:

[tmorshed]

...tracked this down a bit more...

In [15]: import psychxr.drivers.libovr as libovr
In [16]: libovr.isHmdConnected()
Out[16]: False

[pellet]

Yep I get this error when my headset isn't linked up and active(wearing it)

[tmorshed]

The device now gets recognized if you do the in-VR setup. But that setup never seems to finish successfully. It gets stuck on a three-dot loading screen, a la

If you ignore the continuing screen, it seems to get detected by libovr in Python.

>>> import psychxr.drivers.libovr as libovr
>>> libovr.isHmdConnected()
True

Trying to run the experiment though, gets this error: wglDXOpenDeviceNV with minimal detail Which seems to be a DirectX/OpenGL-related error message. Could be a driver or communication issue.

Next steps:

• Will retry later with the other USB ports and cables to see if that solves anything.
• Maybe using AirLink will solve the issue
• Also, maybe using the official USB cable helps.

[JohnGriffiths]

I've reviewed the code and this is working nicely.

There is a build deployment failure currently which I don't understand as I don't think anything in the PR is touching CI type things.

In any case I am going to merge into `develop' and we can continue to solve any CI issues there.

[pellet]

Thanks @JohnGriffiths, I'll take a look at the build issues.