GeneZC
GeneZC
I'm wondering whether it would be better if we extend the input to more layers to include more information on the board such as 'live 3' and 'sleep 4' ......
https://github.com/KaiyuYue/torchshard/blob/89e21def180bf6063ceb2e312a61631173abc7e7/projects/minGPT/main.py#L150 I have noticed that the `group_size` is set to `world_size` in examples, but in fact the `group_size` can be set to other numbers according to my understanding. https://github.com/KaiyuYue/torchshard/blob/main/torchshard/distributed/core.py#L18 I...
CoFi is a great work which may benefit the research in related areas. However, I have found the numbers of the task performance on other sparsities are not available. Could...
how did you get towe annotations of mams, manually or from other sources? btw, could you please provide mams-arts testset as well?
I understand that the `grad_k` should be the reduced sum among local ranks since Q is only a sub-sequence in each rank. However, I do no quite understand why a...
[Phoenix](https://github.com/FreedomIntelligence/LLMZoo/tree/main) is a multilingual instruction-following language model which aims to democratize ChatGPT across languages. It has achieved competitive performance with ChatGLM and Wenxin on Chinese following the evaluation protocol of...
How about the performance difference between token-gate and sentence gate? And how about the value of alpha for load balance loss?
Using pyserini to evaluate on NQ or TriviaQA requires an additional index of Wikipedia which seems to be redundant to me. It seems to be easier to save the doc...