zhyever
Conceptual Questions about BinsFormer
Thanks for a great repo! I have a question about BinsFormer architecture especially about 'auxiliary classification.' From your code, I found that there are variables called class_num and n_bins. In my initial understanding from AdaBins is correct, I think AdaBins classifies the network output by n_bins, but it seems that BinsFormer's classification is done by class_num, where class here represents specific type of indoor scene (like bedroom, bathroom) in nyu. But if so, why do we need such class in BinsFormer and how class contributes to 'auxiliary classification'? If you need more clarification, I appreciate if you let me know.
I'm now revising this paper. Thanks for your questions. I may improve my description of the 'auxiliary classification' in this version.
Let me try to answer you more directly. Considering the bins queries b = b1, b2... bn, and a classification query c. We have n_bins(Adabins) bins queries and only one classification query c. Each bin query will be projected to the bin length and associated with its rank in b. The classification query c will be projected to the classification logit via MLP, which is used to classify the environment like bedroom, bathroom, ..., and so on.
The classification query is included in the self-attention of queries so that the environment information can be passed into bins queries. Hence, we claim it can further improve the model performance, which is proven in our ablation studies. Also, this supervision is more effective compared to the time-consuming chamfer loss used in Adabins.
