FaceMeshFaceGeometry icon indicating copy to clipboard operation
FaceMeshFaceGeometry copied to clipboard

Update to mediapipe facemesh

Open jthawme opened this issue 3 years ago • 1 comments

As it looks the tfjs facemesh model is deprecated, do you know of any idea of how to use this with MediaPipe's facemesh instead?

https://google.github.io/mediapipe/solutions/face_mesh.html

Any guidance would be appreciated, thanks so much

jthawme avatar Sep 12 '22 13:09 jthawme

I wouldn't feel confident enough for a PR on this, but to drop some further information, it looks very similar in a lot of ways, but the results from the facemesh library linked above come in like

function onResults(results) {
  const {multiFaceLandmarks} = results;
  
  // [
  //   ...468 items
  //   { x: 0.4, y: 0.6, z: -0.1 }
  // ]
}

They're all values from 0-1, except z I believe

Atm i am exploring using it by doing

geo.update({
  scaledMesh: multiFaceLandmarks[0].map(vec => Object.values(vec))
})

But its not 100% correct yet, but works without erroring and does create a mesh

jthawme avatar Sep 13 '22 13:09 jthawme