tfjs icon indicating copy to clipboard operation
tfjs copied to clipboard

Compare pose-detection prediction data with another GPU backend

Open gyagp opened this issue 3 years ago • 4 comments

We couldn't get CPU prediction data for pose-detection. So to compare the result, we use another GPU backend for reference.

To see the logs from the Cloud Build CI, please join either our discussion or announcement mailing list.


This change is Reviewable

gyagp avatar Jul 25 '22 08:07 gyagp

@haoyunfeix PTAL

gyagp avatar Jul 25 '22 08:07 gyagp

@Linchenn If pose-detection will be removed very soon, we don't need this patch. Otherwise, we hope to bring this in temporarily so that our daily test can pass.

gyagp avatar Jul 25 '22 09:07 gyagp

Thank you Yang! I have reached a consensus over the new model list with Na, Ping and Jiajia (I have pinged you on the spreadsheet), so I will update the model.json soon.

Even though the failure will disappear when the model is removed, I am curious about if WebGL/WebGPU's results are expected or CPU/WASM's results and hope we could fix the issue.

I took a look over the correctness check for pose-detection and the following codes returns an empty array:

<script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs"></script>
<script src="https://cdn.jsdelivr.net/npm/@tensorflow-models/toxicity"></script>
<script src="https://cdn.jsdelivr.net/npm/@tensorflow-models/pose-detection"></script>
<script>

async function loadImage(imagePath) {
    const image = new Image();
    const promise = new Promise((resolve, reject) => {
      image.crossOrigin = '';
      image.onload = () => {
        resolve(image);
      };
    });
  
    image.src = imagePath;
    return promise;
}

async function run() {
    const image = await loadImage('https://storage.googleapis.com/tfjs-models/assets/posenet/tennis_standing.jpg');
    const runtime = 'tfjs';
    const enableSmoothing = false;
    const modelType = 'lite';
    config = {runtime, enableSmoothing, modelType};
    model = await poseDetection.createDetector('BlazePose', config);
    tf.setBackend('webgl');
    let res = await model.detectPose(image);
    console.log(res);
}

run();
</script>

@ahmedsabie Could you help take a look at the above codes? Is empty array expected? Or did I miss something? (@lina128 is OOO this week)

By the way, I found the issue exists when adding pose-detection, https://github.com/tensorflow/tfjs/pull/5147

Linchenn avatar Jul 26 '22 00:07 Linchenn

Talked with Jiajia about new benchmarks, and she will leave some more comments in the spreadsheet. I will close this PR once the new benchmarks are merged.

gyagp avatar Jul 26 '22 03:07 gyagp