COPILOT-WDP

Results 4 comments of COPILOT-WDP

[Issue]: Large Dataset (200k input files, 100M input tokens): out of memory issue (xml.etree.ElementTree.ParseError)

Any ideas or suggestions on how to scale?

[Issue]: Global Search takes a long time to respond for large datasets

Can you share some more info about your dataset and the time it took to ingest? I am struggling a bit with larget datasets too.

Share my config to change to your local LLM and embedding

What dataset have you indexed? Would be curious to run the process using GPT-4o and compare to NotebookLM (running on Gemini 1.5 Pro I believe).

Consider adding an entity reconciliation step that merges nodes that seem to be duplicate

Which model have you used? GPT-4o? The documentation mentions a "destructive entity resolution" step which is not enabled by default. I believe it is not implemented at all in the...