prose
prose copied to clipboard
Cannot reproduce tokenization example
Installed prose with go get github.com/jdkato/prose/v2 and copied the code from the tokenization example in the readme.
The output I get is
@jdkato NN
, ,
go VB
to TO
http://example.com NN
thanks NNS
: :
) )
. .
Instead of the expected one reported in the example:
// @jdkato NN
// , ,
// go VB
// to TO
// http://example.com NN
// thanks NNS
// :) SYM
// . .
The difference is that the smile ":)" symbol is not recognized.
$ go version go version go1.16.5 linux/amd64