xinhaoH

请问如何使用模型？

2

请问如何使用sgns.target.word-ngram.1-2.dynwin5.thr10.neg5.dim300.iter5这种文件呀？ 这种iter5文件是什么东西？ 自己看了一些回答还是一头雾水，可以提供如何使用模型的示例代码吗？谢谢！

no module named 'torchvision.layers'

1

When I run your code, there is an error 'no module named 'torchvision.layers'. and I can not use the link pip install git+git://github.com/pytorch/vision.git@24577864e92b72f7066e1ed16e978e873e19d13d. The solution of sudo apt-get install git-core...

The relationship between operator running time and number of runs

2

I used the same data to run the same function five times, and the five running times were: 1282.27764 ms, 0.35153 ms, 0.15597 ms, 0.1487 ms, 0.14346 ms. The difference...

What's the exact meaning of 8.2x enhancement in memory efficiency, prompting latency, and generation latency? Can you provide the evaluation code?

2

Thanks for your great work. Q1: I found that we should execute key_states = repeat_kv(key_states, self.num_key_value_groups) value_states = repeat_kv(value_states, self.num_key_value_groups) and then do past_key_value.update (kv_pruned / key|value_states). Since the pruned...

I noticed that in the prefill stage, although we prune the token number to max capacity prompt (e.g, 2k), we still use full attention to compute attention. For example, we...