Peacer68

Why is trained embedding orthogonal?

1

I load the model ema_0.9999_050000.pt shared by you (thanks for sharing), and find the **word_embedding.weight** is orthogonal, which is wierd! This means trained embedding failing to learn semantic relevance between...