Slight difference between implementation and paper.

[AI-Guru]

Namaste!

I think this is my final issue :D

In the implementation of the classifier, you start with a single-head-attention, then you do the spatial transformation followed by multi-head-attention. In the paper, on the other hand, you start with the transformation.

I have the feeling that I have missed something. What is your opinion?

[AI-Guru]

Another difference. Correct me if I am wrong: The implementation of attn_feature does not 100% match. Especially "edge_feature = input_feature_tiled - neighbors" seems to be missing on the picture.

[AI-Guru]

Yes. The paper mentions that a single-head attention happens as part of the spatial transform step.