WebToEpub icon indicating copy to clipboard operation
WebToEpub copied to clipboard

Automatically handle 404 errors when try to get image from different host

Open bookimp opened this issue 1 year ago • 6 comments

Describe the bug A clear and concise description of what the bug is.

To Reproduce Steps to reproduce the behavior:

  1. Go to https://libread.com/libread/the-rebirth-of-an-ill-fated-consort-65376
  2. Download epub with defaul settings other than 3 second delay
  3. Get 403 error about logging in (site has no accounts) or solving CAPTCHA
  4. You also get a billion failed to dl image errors but I'm less concerned about that.

Expected behavior Epub to download without this popping up a dozen times

Screenshots Attached

Desktop (please complete the following information):

  • OS: Windows 11 Home 64bit
  • Browser: Opera LVL 5 (core: 109.0.5097.130)

WebToEpub Version 0.0.0.159

Additional context It did allow me to just hit ok on the alert, but I had to be on the download or it stalled.

Screenshot 2024-06-29 000104 The Rebirth of an Ill-Fated Consort Chapter 1-207.zip Just in case you want to see the file without having to download the whole thing.

bookimp avatar Jun 29 '24 07:06 bookimp

@bookimp try to update to version 0.0.0.160 it could be like this #1306 issue.

gamebeaker avatar Jun 29 '24 12:06 gamebeaker

My estimate was wrong. The problem is that the Autor embeded pictures witch get blocked with 403 error code. Example: https://freewebnovel.comenovel.com/the-rebirth-of-an-ill-fated-consort/chapter-106 image on this site with error: https://lh6.googleusercontent.com/xuxiYqUG6e6F7FurcfYQJWCAqmNYLQpD1HINXhnGhjGC6AXkZekhcTSn0Kzx-TFTuTDU3XtWGEPGh5dIyLFlaPk3d1aw0t79jlYc1iFZ I don't know what a solution would be like. Maybe something like a checkbox in the warning dialog "ignore future 403 warning for this session"

gamebeaker avatar Jun 29 '24 14:06 gamebeaker

more context: for each image that has this problem the 403 Warning pops up 2 times and the retry or skip dialog 1 time.

grafik grafik grafik

gamebeaker avatar Jun 29 '24 15:06 gamebeaker

@bookimp try to update to version 0.0.0.160 it could be like this #1306 issue.

Well, I've done this at least thanks. If it's a picture issue next time I start getting these I'll try clicking the button to not download them. I can add a cover myself.

bookimp avatar Jun 29 '24 16:06 bookimp

This is somewhat of a global issue I've seen on other sites as well...

I've looked into this before and played with a solution, but it takes quite a few changes to the underlying code. Keep in mind, when downloading the page we are already getting information about the image that we can work with.

  1. Check for failed or otherwise undisplayed image load in ImageCollector.fetchImages (failed load image has imageInfo.height == null)
  2. Don't bother trying to download it.

OR

  1. Add parameter to fetchImage to denote such failure.
  2. Disable errors or warning on "low hope" image downloads. Code or logic change in HttpClient.FetchErrorHandler.OnResponseError - Probably a wrapOption to skip the popup.

While fundamentally it's not a difficult change to make, 15 minutes or so after these investigations, I'm not sure either route is ideal.

Kiradien avatar Jun 30 '24 02:06 Kiradien

Reopening. this is "something I should really look at doing".

dteviot avatar Aug 09 '24 07:08 dteviot

@Kiradien @dteviot Not sure if the change to 403 addresses this. I am setting it as Complete if you disagree remove the Label.

Test versions for Firefox and Chrome have been uploaded to https://github.com/dteviot/WebToEpub/releases/tag/developer-build. Pick the one suitable for you, follow the "How to install from Source (for people who are not developers)" instructions at https://github.com/dteviot/WebToEpub/tree/ExperimentalTabMode#user-content-how-to-install-from-source-for-people-who-are-not-developers and let me know how it goes. Tested with:

  • https://www.scribblehub.com/series/945582/the-heavenly-martial-empress-returns-an-op-xianxia-returnee-litrpg/

gamebeaker avatar Sep 20 '24 15:09 gamebeaker

@bookimp

You should now be able to tell WebToEpub to ignore site (e.g. Patreon) that hosts problematic images.

Updated version (1.0.0.0) has been submitted to Firefox and Chrome stores. Firefox version is available now. Chrome might be available in a few hours to 21 days.

dteviot avatar Sep 26 '24 08:09 dteviot