KL divergency of t-sne in pretraining epochs

[xiqxin1]

Hi, Thank you so much for sharing the excellent work! I'm now also trying to see the effectiveness of SimCLR on HAR data, and want to discuss it with you.

Since we can output KL divergence of t-sne result to see the network feature space, I try to output the t-sne results at each pretraining epoch (the SimCLR head) using your plot function. However, the feature space doesn't change a lot.

If I understood correctly, the KL divergence is supposed to be smaller after many epochs, because the SimCLR is supposed to learn good features for different activities. However, as the SimCLR loss got smaller, the latent feature didn't get better. Did you have some idea about this problem?

Thank you very much for your help! Best Cassie