Does this method used for a two-stage object detector?

[kinredon]

Great works! But I wonder whether this method can be used for a two-stage object detector, e.g., Faster RCNN or not?

[akshaychawla]

Hi @kinredon ! Thank you for your interest in our work. Apologies for the long delay in my response.

While our code is built only for a specific one-stage detector: Yolo-v3, the method itself can be extended to other methods.

For generating images with a Faster RCNN, you will have to modify this script https://github.com/NVlabs/DIODE/blob/yolo/deepinversion_yolo.py. The model will have to be changed to a Faster-RCNN and the task_loss will change to the detection loss for a faster-rcnn model. All other losses such as image prior losses and deep feature loss will remain the same.

For distillation, here is a paper that performs knowledge distillation for Faster RCNN models: https://arxiv.org/abs/1906.03609 . I'm sure there is a repository somewhere that does distillation for 2 stage models.

[bshakya77]

@akshaychawla has DIODE been experimented with transformer-based object detection models like DINO or DETR? How effective/challenging would it be to use DIODE in transformer-based object detection models?