gector
gector copied to clipboard
preprocessing data question
when i read the source code of preprocess_data.py file。 i am confused with below code: in function perfect_align :

when call apply_transformation, the cdoe ' '.join(T[j:k]) will insert 3 spaces between tokens.
but, look at the source code of apply_transformation:

it will call check_equal, check_casetype, check_verb, check_plural sequencely, but i think inserted space will affect these check funciton.