picard icon indicating copy to clipboard operation
picard copied to clipboard

model performance

Open qianmuuq opened this issue 2 years ago • 0 comments

When I use cxmefzzi's checkpoint, the input format is as follows:

[question] | [db_id] | [table]: [column], [column] | [table]:... |

But the test results are as follows

                 easy                 medium               hard                 extra                all                 

count 248 446 174 166 1034
===================== EXECUTION ACCURACY ===================== execution 0.754 0.691 0.489 0.386 0.623

====================== EXACT MATCHING ACCURACY ===================== exact match 0.835 0.709 0.448 0.319 0.632

There is a significant difference in performance compared to what was written in the paper. How should I improve it?

qianmuuq avatar Sep 08 '23 14:09 qianmuuq