VideoRectangles from predictions only appear if they all exist in the frame

[aounleonardo]

Describe the bug I am trying to label some VideoRectangles predictions but the tool is not displaying them until they all have a keyframe together. When trying to display many different tracks, it's then enough for one of them to not be displayed for all the others to disappear too. I believe this is a bug, because this doesn't reproduce if I create and display the VideoRectangles manually.

To Reproduce Steps to reproduce the behaviour:

1. Create Video tasks with the template pasted below
2. Upload json task containing video rectangle sequences (in my case task_501 (10).json)
3. Open task in frontend
4. Click on a frame where not all videoRectangles exist. See that none of the others appear.
5. Click on a frame where all of them exist. See that now, they exist.
6. You can also click on a frame where one of the VideoRectangles doesn't appear yet, and delete it. This would make the others appear..

Expected behavior The VideoRectangles should all be displayed, no matter what others are displayed.

Screenshots I have attached a couple of recordings, one showing the successful manual case, and the other showing the failure case, which is the one importing the predictions.

https://github.com/HumanSignal/label-studio/assets/11409469/e6df253a-7921-440d-b9ba-d4eb66a3e7a9

https://github.com/HumanSignal/label-studio/assets/11409469/d2f5d927-db59-4d7b-8472-83e050db0b70

Environment (please complete the following information):

• OS: MacOS but in a docker container, latest image
• Label Studio Version 1.11.0

Additional context Task template:

<View>
   <Labels name="videoLabels" toName="video" allowEmpty="true">
     <Label value="Player1" background="#1c3bb5"/>
     <Label value="Player2" background="#ff0000"/>
     <Label value="Player3" background="#04d600"/>
     <Label value="Player4" background="#d4ff00"/>
     <Label value="Player5" background="#fc38ff"/>
     <Label value="Player6" background="#ff9500"/>
     <Label value="Player7" background="#ff9500"/>
     <Label value="Player8" background="#ff9500"/>
     <Label value="Player9" background="#ff9500"/>
     <Label value="Player10" background="#ff9500"/>
     <Label value="Player11" background="#ff9500"/>
     <Label value="Player12" background="#ff9500"/>
  </Labels>
   
   <!-- Please specify FPS carefully, it will be used for all project videos -->
   <Video name="video" value="$video" framerate="29.97002997002997" muted="true"/>
   <VideoRectangle name="box" toName="video"/>
</View><!--{
 "video": "/static/samples/opossum_snow.mp4"
}-->

[Jerry2990]

hi, i am really interested in this work. However, i have some questions. Could you share about how to upload json task containing video rectangle sequences? Thanks a lot!

[Jerry2990]

Same problems.