Yueqian Wang

a bug (?) when distilling TinyBERT on regression tasks with task_distill.py

1

in TinyBERT/task_distill.py line 973: ``` python elif output_mode == "regression": loss_mse = MSELoss() cls_loss = loss_mse(student_logits.view(-1), label_ids.view(-1)) ``` so TinyBERT is actually learning from the label, maybe we should use...

请问可以提供UNIMO的预训练代码吗？

1

如题，对预训练过程中单模态数据的使用细节很感兴趣，谢谢

euler有没有支持动态网络embedding的算法？

1

例如我有一个有向同构图，图本身很大（有上亿条边）但更新的频率比较慢（例如每天增加/删除的边只占所有边数量的不到1%，因此用完整的图重新训练一次开销太大了），请问euler中有没有内置的模型支持每天输入一个增量图，只对增量图中涉及的节点的embedding进行更新的模型呀？ 一个问题是每天都会有一些已经之前存在的边被删除，因此不太了解怎么用euler那个json格式表示昨天还存在但今天被删除的边

though this is less important, I would like to ask if you can split the 30G checkpoint into several smaller files, like most models (e.g., llama, alpaca) do? This will...

which n-gram BLEU metric do you report in your paper?

1

 This is Table 1 in Result and Analysis section in your paper, which BLEU do you use here, BLEU-1, BLEU-2, BLEU-3 or BLEU-4？

Do you have plans to release all the captions of InternVid?

2

As described in the paper, there are 234M clips in the InternVid dataset in all, but the size of the largest subset which is publicly available is only 18M. Do...