MediaPipeUnityPlugin icon indicating copy to clipboard operation
MediaPipeUnityPlugin copied to clipboard

Basic subscriber / event listener example

Open Wayne-nulitics opened this issue 4 years ago • 1 comments

@homuler Thank you for explaining the purpose of this port and use of mediapipe::CalculatorGraph, that we are looking for 'n a sample code snippet in C# that subscribes to the detection/graph calculation, we've been able to "track it down" to:

public class HandTrackingGraph : DemoGraph { private OutputStreamPoller<List> handLandmarksStreamPoller; ... var handLandmarks = isHandLandmarksPresent ? FetchNextHandLandmarks() : new List(); ... private void RenderAnnotation(WebCamScreenController screenController, HandTrackingValue value) { // NOTE: input image is flipped GetComponent().Draw( screenController.transform, value.HandLandmarkLists, value.Handednesses, value.PalmDetections, value.PalmRects, true); }

But how, from another scripts, can we subscribe the receive the value.HandLandmarkLists and value.Handednesses as they are being updated.

ie. we can't figure out how to subscribe to the var handLandmarks or something that provides that with handedness.

I've mapped out the basic flow through the demo as follow, just the key points, assume we can add something similar to the RenderAnnotation function? image

We are well versed in using it in Python, and the hand_landmark detection from here https://google.github.io/mediapipe/solutions/hands.html

Any help would be appreciated.

Wayne-nulitics avatar Apr 01 '21 11:04 Wayne-nulitics

#296 adds a working example. https://github.com/homuler/MediaPipeUnityPlugin/blob/7bb877de4887ad4bf23c72b39649c41bb3650d54/Assets/Mediapipe/Samples/Scenes/Hand%20Tracking/HandTrackingSolution.cs#L79

I will add documentation on this later.

homuler avatar Sep 26 '21 08:09 homuler