Attention visuals tensor has exactly same values

[praggnanandhaa]

Hello sir, I appreciate your hard work. Now, straight to the issue.

The problem is that when I try to run the visualization.py file only the attention scores associated with the first hierarchy level are different from 0. I checked the tensors of the other hierarchy levels and each one of them has exactly the same values, that's why the attention scores are equal to zero. This problem persists with different models, trained with different parameters and with different data. Did I miss something?