LongAlign icon indicating copy to clipboard operation
LongAlign copied to clipboard

代码是这么写的?预先定义好句号的token id是什么意思,只能测试GLM呗?????

Open kkwhale7 opened this issue 1 year ago • 0 comments

https://github.com/THUDM/LongAlign/blob/9ae0b597737c6658f4350ef7a42d5d01980d142c/Needle_test/prompt.py#L214

搞了半天测试,原来needle一直在context开头,无语了 image 搞不懂,既然你们只想要30930作为句号的tokenizer,你们开源repo的意义是什么????????????????????????????????????????????????????

kkwhale7 avatar Nov 04 '24 17:11 kkwhale7