Time out in HTMLFetcher

Open neerajbhatt opened this issue 10 years ago • 3 comments

In HTMLDocument fetch(final URL url) there is no timeout. Ideally after creating final URLConnection conn = url.openConnection(); time out should be given. Please assign issue to me and I will send a pull request

Oct 19 '15 12:10 neerajbhatt

+1, I was very dissapointed when it stuck at night.

Oct 19 '15 13:10 luckyace

try to set the proxy overriding the method

Feb 10 '16 12:02 deepcode-debug

Fetch HTML separately, and feed it in via the BoilerpipeSaxInput - you can have your own timeouts and use a pipeline.

May 19 '17 16:05 danizen