cnglen
cnglen
Make ensure compatibility with TensorFlow, see https://github.com/tensorflow/io#tensorflow-version-compatibility
Yes, this is tedious!!!!!!!!!!!!
``` python text_vectorizer._self_tracked_trackables.pop() text_vectorizer._lookup_layer._self_tracked_trackables.pop() ``` It's caused by the large list member of: - text_vectorizer._self_tracked_trackables - text_vectorizer._lookup_layer._self_tracked_trackables It's seems it's not necessary to add the huge list to ._self_tracked_tradckables(which is...
You can change 'vsize = [1000, 10000, 100000, 1000000, 25000000]' to 'vsize = [1000, 10000, 100000, 1000000]' It takes too much time to save the huge preprocessor with 2500e4 vocab...
same error. Any suggestions?
## benchmark **bench mark result:** @zesterer - [windancer](https://github.com/cnglen/windancer) my implemenmation using chumsky: **1.7 second** - [orgize](https://github.com/PoiScript/orgize/) using nom: **2.4 ms** > Running benches/doc_parse.rs (target/release/deps/doc_parse-44ef6fb8a1ceb328) > Benchmarking org-doc-parse/windancer/test: Warming up for...
Thanks. The most of time spent is in the state.rewind() and state.save(). After remove unnecessary states, the time is not too much slow.