evaluation accuracy remains 0

[2proveit]

Thanks for your great work in creating this dataset, I have questions while evaluating llama2-7b-chat on this dataset.

• The accuracy of the llama2-7b-chat output remains 0 when the training goes, here is my code:

def acc(eval_preds:EvalPrediction):
        logits, labels = eval_preds
        preds = tokenizer.batch_decode(logits, skip_special_tokens=True)
        labels = tokenizer.batch_decode(labels, skip_special_tokens=True)
        save_results(preds, labels) # save results to json file
        preds = [last_boxed_only_string(s) for s in preds]
        correct = 0
        total = 0
        for pred, label in zip(preds, labels):
            if is_equiv(pred, label):
                correct += 1
            total += 1
        return {"accuracy": correct / total}
    return acc

• whether the preprocess is required or recommended to use?