shelton-xiaoshuaichen

at the end of inf_sampling self.nw[_w][topic] += 1 self.nd[m][topic] += 1 self.nwsum[topic] += 1 self.ndsum[m] += 1 should be following? self.newnw[_w][topic] += 1 self.newnd[m][topic] += 1 self.newnwsum[topic] += 1 self.newndsum[m]...

### Is there an existing issue for this? - [X] I have searched the existing issues ### Current Behavior ChatGLM V1和V2的tokenization有差异，导致Finetune代码不支持v2 model V1的tokenization情况： def build_inputs_with_special_tokens( self, token_ids_0: List[int], token_ids_1: Optional[List[int]]...