A question about the design of decoder sampler.
Dear author,
Thank you for your excellent work! I found the solution elegant and concise in general after reading your paper. However, I have a question for the decoder sampler that bothers me for a long while.
It seems that you distribute the value of each position only to its four neighbor points. Is this true? If so, I wonder whether there will be "holes" in the upsampled feature maps in theory, especially when the spatial distortion is large. I've gone through the code but cannot really figure out the answer. Furthermore, if there are some holes, it may also be reasonable since that the subsequent layers might be responsible and able to fill them up.
It would be really nice if you can give me some comment on this. Thank you again!
Yours sincerely, Yuyao

Which "stars" each "circle" would average from?