wego icon indicating copy to clipboard operation
wego copied to clipboard
verified publisher icon ynqa

Scalability / Datasets larger than RAM

Open gnewton opened this issue 6 years ago • 2 comments

Hello,

I have not tried this out yet (I am excited to) but was wondering how this dealt with very large datasets, particularly those larger than RAM. Can wego handle this use case?

Thanks, Glen

gnewton avatar Feb 11 '19 19:02 gnewton

@gnewton Thanks for your indication, and which is quite right. Currently, wego is not able to train models from huge datasets that are unacceptable for memory. Now I'm just tackling this issue. Please wait for resolved :)

ynqa avatar Feb 12 '19 08:02 ynqa

Any guidance on RAM required per words?

estebarb avatar Apr 22 '21 19:04 estebarb