Siqi Liang
Siqi Liang
We will check your code. Thank you very much!
A modified version can be ```python def __get_ppl(self, input_texts: List[str], mask_length=None): if self.call_api: return api_get_ppl(self.api_name, input_texts) self.tokenizer.padding_side = "right" inputs = self.tokenizer(input_texts, padding=True, return_tensors='pt', truncation=True) inputs = {k: v.to(self.model.device) for...
maybe you can check whether there are local partitioned data files based on path `os.path.join(self.path, type, "data{}.pkl".format(cid))`