openpi
openpi copied to clipboard
[FEATURE] Inconsistent Video Playback Timing in DROID pi0 inference script
Hi! Here is Tony from GRASP Lab@UPenn. When we are evaluating the pi0, we notice current implementation of examples/droid/main.py for pi0 only records frames at policy inference intervals. This results in inconsistent video playback speeds. The delays (~750ms–1s) inherent to inference and environment steps lead to misleading visualizations that roughly appear as real-time but do not precisely reflect true action timing.
**Expected Feature: ** Videos should play back consistently at real-time speed (e.g., 10 FPS), accurately representing agent behavior.
- Implement timestamp-based frame recording
- Alternatively, continuously record frames independently at a fixed frame rate (e.g., client-side recording).
Hope this could help future benchmark & evaluation!