text-preprocessing
text-preprocessing copied to clipboard
A problem with dash character with check spelling
when I use this method, it generates an error in check spelling because the remove_special_character doesn't remove the dash '-'.
I also need to use check spelling
def pre_process(input_text):
input_text=remove_pattern(input_text,"@[\w]")
input_text=remove_pattern(input_text,"#[\w]")
preprocess_functions = [to_lower, remove_email, remove_url, remove_punctuation, remove_special_character, normalize_unicode, remove_number, remove_whitespace, remove_stopword, lemmatize_word, stem_word, check_spelling]
preprocessed_text = preprocess_text(input_text, preprocess_functions)
return preprocessed_text
print(pre_process("The method is internal-based."))
Please, Could you suggest a solution?