Compositor: Bottom-Up Clustering and Compositing for Robust Part and Object Segmentation

This repo contains the data and code for our paper Compositor: Bottom-Up Clustering and Compositing for Robust Part and Object Segmentation

Compositor is a simple yet effective architecture for jointly segmenting parts and objects in a bottom-up process, which clusters pixel embedding to part embedding and further composites part embedding to obtain object embedding, thus achieving more consistent and robust segmentation results. We provide Compositor implemented based on two mask transformer architectures: MaskFormer and kMaX-DeepLab in this repo.

Installation

See Installation Instructions.

Dataset Sources

• PartImageNet
• Pascal-Part

Getting Started

See Preparing Datasets for Compositor.

See Getting Started with Compositor.

Citing Compositor

If you use Compositor in your research, please use the following BibTeX entry.

@InProceedings{He_2023_CVPR,
    author    = {He, Ju and Chen, Jieneng and Lin, Ming-Xian and Yu, Qihang and Yuille, Alan L.},
    title     = {Compositor: Bottom-Up Clustering and Compositing for Robust Part and Object Segmentation},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2023},
    pages     = {11259-11268}
}

Acknowledgement

Mask2Former

kMaX-DeepLab