fsimonjetz

add sentence delimiter option to preserve original sentencization

2

Hi there, it would be nice to have an option to preserve the original sentence boundaries by overriding the internal sentence splitter. The readme seems to imply that newlines will...

Separate documents?

3

The readme says "All data have been processed automatically so that it is not possible to reconstruct the original source texts." I'm considering to use German-Korean data for my PhD...

Installation via pip on Python 3.9 and newer fails with fatal error: xtensor/xarray.hpp: No such file or directory

4

Hi there, first of, this is an excellent library. I am utilizing it for a research project and had no issues installing and running it on Kaggle. Recently, however, Kaggle...