GreenMIM
GreenMIM copied to clipboard
LayneH
[NeurIPS2022] Official implementation of the paper 'Green Hierarchical Vision Transformer for Masked Image Modeling'.
Thanks for the great work! I would like to know that where the `Optimal Grouping with Dynamic Programming` process being conducted in your code? Hope for your reply.
Hello! Thanks for your interesting work! I have some doubt about the mask in relative bias table: if we already have attn+mask, why still need to multiply mask and res_pos:...
For the GPU memory restriction, I could use swin-tiny as my backbone. Did you try pretraining swin-tiny with GreenMIM? I tried it, but it seems get bad performance, even worse...
