hand-tracking-controls component detaches children (+fix & ideas)

[coderofsalvation]

• A-Frame Version: 1.5.0
• Platform / Device: Firefox/Brave

Normal/Expected behaviour:

<a-entity laser-controls="hand:left">
  <!-- adding objects here will attach it to the controller just fine -->
</a-entity>

however...after adding hand-tracking-controls

<a-entity laser-controls="hand:left" hand-tracking-controls="hand:left">
  <!-- here the objects no longer show up..neither with controller nor hand-tracking :/ -->
</a-entity>

Fix

This patch will re-attach the children to the wrist.

AFRAME.components['hand-tracking-controls'].Component.prototype.onModelLoaded = function(onModelLoaded){
  return function(e){
    onModelLoaded.apply(this);
    // re-attach children 
    ([...this.el.children]).map( (c) => {
      if( c.object3D ){
        this.el.object3D.getObjectByName("wrist").add(c.object3D)
      }
    })
  }
}(AFRAME.components['hand-tracking-controls'].Component.prototype.onModelLoaded)

I'm not really sure what the right solution is, as not every hand model might contain a 'wrist' object.
I played with several ideas:

• add the code above directly to onModelLoaded of hand-tracking-controls
• introduce an attribute <a-entity hand-tracking-controls="children: 'wrist'"> which is wrist by default e.g.
• introduce an component hand-attach="wrist: #foo; index: #bar which allows for finer-grained parenting

One remaining question is how to deal with switching between hands/laser-controllers.

Hope this helps!

[dmarcos]

What is the scenario you need to switch from laser-controls to hand tracking and viceversa?

[coderofsalvation]

It's more accessibility/flexibility in XR in general:

• wear menu on your lasercontroller (if the user is holding controllers)
• wear menu on your wrist (if the user is using hands)

I think it's important to offer experiences which works both with lasercontrols and hand-tracking (not all XR/VR endusers have handtracking).

[herebefrogs]

For context, something similar happens with hand-controls in AFRAME 1.6.0 which prevents from dynamically attaching entities in the hand controller:

Several levels of declared directly in the DOM of the hand-controls entity will render find. For example, this will write 'foo bar' on the back of the hand model:

<a-entity hand-controls="hand: left; ...">
  <a-text value="foo" position="-0.028 0.05 0" rotation="185 90 0">
    <a-text value="bar" position="0.03 0 0"></a-text>
  </a-text>
</a-entity>

Where things go pear-shaped is when I attempt to do this programmatically, by detaching a group of entities from the scene and reattaching them to the hand-controls entity:

index.html:
<a-box id="foo" color="red" position="1 2 -3" width...>
  <a-box id="bar" color="yellow" position="-0.5 0 -1"></a-box>
</a-box>
<a-entity hand-controls="hand: left; ..."></a-entity>

some system or component:
foo = document.querySelector('#foo');
foo.removeFromParent();
foo.setAttribute('position', '0 0 0');     // #foo had a world position, resetting it so it's local to the hand controller's origin
document.querySelector('[hand-controls]').appendChild(foo);

This will put the #foo box in the hand model, but the #bar bar isn't rendered at all.

Happy to open a separate issue if you deem this not related.

[vincentfretin]

@herebefrogs reparenting an entity is not supported, see #2425 how to do it.