gector icon indicating copy to clipboard operation
gector copied to clipboard

preprocessing data question

Open liwenju0 opened this issue 3 years ago • 0 comments

when i read the source code of preprocess_data.py file。 i am confused with below code: in function perfect_align :

image

when call apply_transformation, the cdoe ' '.join(T[j:k]) will insert 3 spaces between tokens.

but, look at the source code of apply_transformation: image

it will call check_equal, check_casetype, check_verb, check_plural sequencely, but i think inserted space will affect these check funciton.

liwenju0 avatar Jan 06 '23 03:01 liwenju0