label-studio icon indicating copy to clipboard operation
label-studio copied to clipboard

Exporting to COCO does not include extra bbox labels

Open ntakouris opened this issue 3 years ago • 0 comments

Describe the bug JSON after exporting to COCO does not contain extra bbox label info (see below steps)

To Reproduce Steps to reproduce the behavior:

  1. Create a new project
  2. Add the following code xml (modified image bboxes label playground template)

<View>  
    <Image name="image" value="$image" zoom="true" zoomControl="true"/>

        <!-- Add classes to select per bbox -->
      <RectangleLabels name="label" toName="image" showInline="true">
        <Label value="c" background="#0400fa"/>
        <Label value="d" background="#00ff2a"/>
      </RectangleLabels>

        <!-- Select a bbox and assign an extra label choice (for each head) -->
      <View visibleWhen="region-selected">
      <Header value="Extra Head: milciv"/>

        <Choices name="my-secondary-category" toName="image"
                 perRegion="true">
          <Choice value="a"/>
          <Choice value="b"/>
        </Choices>
      </View>

      <View style="width: 100%; display: block">
        <Header value="Select bbox after creation to add meta-labels"/>
      </View>
  
</View>

  1. Click Export -> COCO
  2. See output
{
  "images": [
    {
      "width": 1000,
      "height": 664,
      "id": 0,
      "file_name": "images/1/blabla1.jpg"
    },
  ],
  "categories": [
    {
      "id": 0,
      "name": "aircraft"
    },
    {
      "id": 1,
      "name": "a"
    },
    {
      "id": 2,
      "name": "b"
    },
    {
      "id": 3,
      "name": "c"
    },
    {
      "id": 4,
      "name": "d"
    },
  ],
  "annotations": [
    {
      "id": 0,
      "image_id": 0,
      "category_id": 5,
      "segmentation": [],
      "bbox": [
        153,
        332,
        1151,
        296
      ],
      "ignore": 0,
      "iscrowd": 0,
      "area": 340696
    },
  ],
  "info": {
    "year": 2022,
    "version": "1.0",
    "description": "",
    "contributor": "Label Studio",
    "url": "",
    "date_created": "2022-06-02 08:20:21.747930"
  }
}

Expected behavior There's no defined API to add extra annotations on bounding boxes for the COCO API, but:

  • I would expect the extra labels per bbox (labels "a" and "b") to be contained in this JSON somehow.
  • Most (if not all) COCO loader implementations would ignore extra fields in this json. For example, the annotation could look like:
    {
      "id": 0,
      "image_id": 0,
      "category_id": 5,
      "choices": {"my-secondary-category": { "category_id": <> }},
      "segmentation": [],
      "bbox": [
        153,
        332,
        1151,
        296
      ],
      "ignore": 0,
      "iscrowd": 0,
      "area": 340696
    }

Also, it doesn't make sense to bundle the categories created in the bbox choices (secondary ones), inside the main categories.

Environment:

  • Label Studio Version : 1.4.1.post1

ntakouris avatar Jun 02 '22 11:06 ntakouris