openpi icon indicating copy to clipboard operation
openpi copied to clipboard

[FEATURE] Inconsistent Video Playback Timing in DROID pi0 inference script

Open Everloom-129 opened this issue 9 months ago • 0 comments

Hi! Here is Tony from GRASP Lab@UPenn. When we are evaluating the pi0, we notice current implementation of examples/droid/main.py for pi0 only records frames at policy inference intervals. This results in inconsistent video playback speeds. The delays (~750ms–1s) inherent to inference and environment steps lead to misleading visualizations that roughly appear as real-time but do not precisely reflect true action timing.

**Expected Feature: ** Videos should play back consistently at real-time speed (e.g., 10 FPS), accurately representing agent behavior.

  • Implement timestamp-based frame recording
  • Alternatively, continuously record frames independently at a fixed frame rate (e.g., client-side recording).

Hope this could help future benchmark & evaluation!

Everloom-129 avatar Apr 13 '25 00:04 Everloom-129