Scrapegraph-ai
Scrapegraph-ai copied to clipboard
FetchNode - Optional timeout to control execution time
https://github.com/ScrapeGraphAI/Scrapegraph-ai/blob/739b05ac79abe3e48090e2ea24482fb19e868cef/scrapegraphai/nodes/fetch_node.py#L263
In the mentioned line the request is run without a timeout
The same principle apply to the PDF parsing that has no timeout.
The feature I request is to improve the handling of timeout by configuration.
Simply make it possible to add a timeout to the FetchNode and use it when performing long blocking requests.