text-preprocessing icon indicating copy to clipboard operation
text-preprocessing copied to clipboard

A problem with dash character with check spelling

Open EAAcs opened this issue 3 years ago • 0 comments

when I use this method, it generates an error in check spelling because the remove_special_character doesn't remove the dash '-'. I also need to use check spelling def pre_process(input_text): input_text=remove_pattern(input_text,"@[\w]") input_text=remove_pattern(input_text,"#[\w]") preprocess_functions = [to_lower, remove_email, remove_url, remove_punctuation, remove_special_character, normalize_unicode, remove_number, remove_whitespace, remove_stopword, lemmatize_word, stem_word, check_spelling]
preprocessed_text = preprocess_text(input_text, preprocess_functions) return preprocessed_text

print(pre_process("The method is internal-based."))

Please, Could you suggest a solution?

EAAcs avatar Aug 31 '22 15:08 EAAcs