vid-koci

Hi, Thanks for your comments. For now, I will leave the code as it is, for reproducibility and consistency with the trained models. I will try to revisit and re-train...

That's correct. And the second bug is one redundant ".mean()" after loss_2 at the end of the line.

It should be `- loss_2` without .mean() because `.mean()` averages over the batch - which shouldn't happen at this stage.

@mhillebrand I believe that it should be `masked_lm_labels>-1` rather than `masked_lm_labels==-1` as -1 are exactly the ones that we are not interested in and are ignored by the CrossEntropy loss.

Hi, this error seems to be connected to CUDA/PyTorch and Nvidia driver error. Unfortunately, the code is not supported for more recent versions of PyTorch/CUDA, making it incompatible with newer...

Hi, the "easiest way" is to simply re-phrase your data format into a format that this code can already read (see examples in the data folder, I suggest files data/gap-test...

Indeed, most of the datasets came with a list of candidates. There were two that did not, WNLI and GAP (candidates were for evaluation only). In these cases, external systems...

Node2vec algorithm is quite memory consumptive. As you mentioned, the learning of embeddings is the slow part of the algorithm. The running time can be reduced (at the cost of...

Could you please send the exact parameters and data you ran the program with? The size of the graph should not cause the embeddings to be -nans.

The node2vec in my github should be the same. On my github there also exists a project called Veles, a heuristical approach to node2vec that uses much less memory. Unfortunately,...