prose icon indicating copy to clipboard operation
prose copied to clipboard
verified publisher icon jdkato

The IterTokenizer does not support unicode emoji

Open jwmach1 opened this issue 6 years ago • 0 comments

I noticed that when a text has emoji (😀etc) that the IterTokenizer does not completely parse the text. I'm prepping a pull request, if this seems worthwhile

jwmach1 avatar Mar 01 '19 00:03 jwmach1