VAN-Classification icon indicating copy to clipboard operation
VAN-Classification copied to clipboard
verified publisher icon Visual-Attention-Network

Attention vs Add in LKA

Open iumyx2612 opened this issue 3 years ago • 1 comments

In table 3, changing attention (mul) to add reduces VAN performance from 75.4 to 74.6. I think this is really huge. However, in the ablation study, you stated that "Besides, replacing attention with adding operation is also not achieving a lower accuracy". Is it okay to say it like that since the performance drop is 0.8

Can't treat add as a type of attention function? In Attention Mechanisms in Computer Vision: A Survey, we have the formula: image I can treat function f here is an addition operation can't I?

iumyx2612 avatar Nov 30 '22 02:11 iumyx2612

@MenghaoGuo Hello can you explain this?

iumyx2612 avatar Jan 13 '23 05:01 iumyx2612