bilix icon indicating copy to clipboard operation
bilix copied to clipboard
verified publisher icon HFrost0

下载时，HEAD 404异常

Open showthesunli opened this issue 3 years ago • 10 comments

bilix 版本

image

下载命令

bilix s https://www.bilibili.com/video/BV1kt411k7A3 --dir E:\tmp --max-con 20

问题描述

重复执行上述命令总是报下方相同的异常，无法定位是某一p的地址404或者是某个分片404。也无法知道是不是我本地的网络导致的问题。

异常信息

Traceback (most recent call last):
  File "c:\python38\lib\runpy.py", line 193, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "c:\python38\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "c:\users\showt\.local\bin\bilix.exe\__main__.py", line 7, in <module>
  File "C:\Users\showt\.local\pipx\venvs\bilix\lib\site-packages\click\core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "C:\Users\showt\.local\pipx\venvs\bilix\lib\site-packages\click\core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "C:\Users\showt\.local\pipx\venvs\bilix\lib\site-packages\click\core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Users\showt\.local\pipx\venvs\bilix\lib\site-packages\click\core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "C:\Users\showt\.local\pipx\venvs\bilix\lib\site-packages\bilix\_cli.py", line 268, in main
    loop.run_until_complete(task)
  File "c:\python38\lib\asyncio\base_events.py", line 616, in run_until_complete
    return future.result()
  File "C:\Users\showt\.local\pipx\venvs\bilix\lib\site-packages\bilix\download.py", line 371, in get_series
    await asyncio.gather(*cors)
  File "C:\Users\showt\.local\pipx\venvs\bilix\lib\site-packages\bilix\download.py", line 441, in get_video
    await asyncio.gather(*cors)
  File "C:\Users\showt\.local\pipx\venvs\bilix\lib\site-packages\bilix\download.py", line 577, in _get_media
    res = await self._req(media_urls[0], method='HEAD')
  File "C:\Users\showt\.local\pipx\venvs\bilix\lib\site-packages\bilix\download.py", line 545, in _req
    res.raise_for_status()
  File "C:\Users\showt\.local\pipx\venvs\bilix\lib\site-packages\httpx\_models.py", line 1510, in raise_for_status
    raise HTTPStatusError(message, request=request, response=self)
httpx.HTTPStatusError: Client error '404 Not Found' for url 'https://upos-sz-mirrorcos.bilivideo.com/upgcxcode/16/18/76821816/76821816-1-30080.m4s?e=ig8euxZM2rNcNbdlhoNvNC8BqJIzNbfqXBvEqxTEto8BTrNvN0GvT90W5JZMkX_YN0MvXg8gNEV4NC8xNEV4N03eN0B5tZlqNxTEto8BTrNvNeZVuJ10Kj_g2UB02J0mN0B5tZlqNCNEto8BTrNvNC7MTX502C8f2jmMQJ6mqF2fka1mqx6gqj0eN0B599M=&uipk=5&nbs=1&deadline=1653679938&gen=playurlv2&os=cosbv&oi=1885228412&trid=fd2cf300307c48e6aa28300fe24ec4d6u&platform=pc&upsig=ec230dd69a0462df746fde23b8ef006e&uparams=e,uipk,nbs,deadline,gen,os,oi,trid,platform&mid=0&bvc=vod&nettype=0&orderid=0,3&agrr=1&bw=139942&logo=80000000'
For more information check: https://httpstatuses.com/404

showthesunli avatar May 27 '22 17:05 showthesunli

在我本机未能复现。可能是网络原因导致的，这个错误的意思是向某一p的画面或者音频链接发起HEAD请求时服务器返回404状态，这个很奇怪🤔。至于如果再次访问这个报404的链接返回的是403错误，这个是可预料的，因为后端链接过一段时间就会失效。

关于定位，可以使用--max-con 1来一个个下载看谁报错。

HFrost0 avatar May 28 '22 00:05 HFrost0

隔了一天再下载又没问题了😅

showthesunli avatar May 28 '22 09:05 showthesunli

我今天也遇到这个问题，在向服务器发起HEAD请求后报404，但是使用GET则返回正常，可能是b站服务端的问题

HFrost0 avatar Jun 23 '22 12:06 HFrost0

版本0.6.11对该问题进行了处理，但问题仍然存在

HFrost0 avatar Jun 27 '22 03:06 HFrost0

我目前提交了一个新的版本对此问题进行了处理，但是还未经过测试（暂时没遇上），所以暂时未在pypi上发布

HFrost0 avatar Jul 14 '22 08:07 HFrost0

我目前提交了一个新的版本对此问题进行了处理，但是还未经过测试（暂时没遇上），所以暂时未在pypi上发布

@HFrost0 ，我pull了你的代码进行尝试，下载了两部视频，其中：这一部完全没有任何异常。

下载另外一部时 404HEAD异常出错。出错日志：

(Lighting-bilibili-download) liy@LEE-PC:~/Lighting-bilibili-download$ bilix s https://www.bilibili.com/video/BV16S4y1p7Dg --dir /mnt/e/tmp --max-con 10 --subtitle
警告：状态码异常404 HEAD https://upos-sz-estgoss.bilivideo.com/upgcxcode/47/66/760196647/760196647-1-30080.m4s?e=ig8euxZM2rNcNbdlhoNvNC8BqJIzNbfqXBvEqxTEto8BTrNvN0GvT90W5JZMkX_YN0MvXg8gNEV4NC8xNEV4N03e
N0B5tZlqNxTEto8BTrNvNeZVuJ10Kj_g2UB02J0mN0B5tZlqNCNEto8BTrNvNC7MTX502C8f2jmMQJ6mqF2fka1mqx6gqj0eN0B599M=&uipk=5&nbs=1&deadline=1657796530&gen=playurlv2&os=upos&oi=1885228211&trid=49989deb523e4e8db37502
5a261ed1ccu&mid=0&platform=pc&upsig=3ba5c8c88b0d4ac24ff1a8692855647b&uparams=e,uipk,nbs,deadline,gen,os,oi,trid,mid,platform&bvc=vod&nettype=0&orderid=0,1&agrr=1&bw=30547&logo=80000000
警告：状态码异常404 HEAD https://upos-sz-estgoss.bilivideo.com/upgcxcode/47/66/760196647/760196647-1-30080.m4s?e=ig8euxZM2rNcNbdlhoNvNC8BqJIzNbfqXBvEqxTEto8BTrNvN0GvT90W5JZMkX_YN0MvXg8gNEV4NC8xNEV4N03e
N0B5tZlqNxTEto8BTrNvNeZVuJ10Kj_g2UB02J0mN0B5tZlqNCNEto8BTrNvNC7MTX502C8f2jmMQJ6mqF2fka1mqx6gqj0eN0B599M=&uipk=5&nbs=1&deadline=1657796530&gen=playurlv2&os=upos&oi=1885228211&trid=49989deb523e4e8db37502
5a261ed1ccu&mid=0&platform=pc&upsig=3ba5c8c88b0d4ac24ff1a8692855647b&uparams=e,uipk,nbs,deadline,gen,os,oi,trid,mid,platform&bvc=vod&nettype=0&orderid=0,1&agrr=1&bw=30547&logo=80000000
【Udemy Tailwind CSS 热门课程】Tailwind CSS From Scratch  Learn By Building Projects-P9-003 Container & Spacing-中文（简体）.srt 完成
【Udemy Tailwind CSS 热门课程】Tailwind CSS From Scratch  Learn By Building Projects-P9-003 Container & Spacing-中文（自动翻译）.srt 完成
【Udemy Tailwind CSS 热门课程】Tailwind CSS From Scratch  Learn By Building Projects-P1-001 Introduction【01 - Introduction】-中文（简体）.srt 完成
【Udemy Tailwind CSS 热门课程】Tailwind CSS From Scratch  Learn By Building Projects-P3-004 What Is Tailwind CSS-中文（简体）.srt 完成
警告：状态码异常404 HEAD https://upos-sz-mirrorcos.bilivideo.com/upgcxcode/71/85/760198571/760198571-1-30216.m4s?e=ig8euxZM2rNcNbdlhoNvNC8BqJIzNbfqXBvEqxTEto8BTrNvN0GvT90W5JZMkX_YN0MvXg8gNEV4NC8xNEV4N0
3eN0B5tZlqNxTEto8BTrNvNeZVuJ10Kj_g2UB02J0mN0B5tZlqNCNEto8BTrNvNC7MTX502C8f2jmMQJ6mqF2fka1mqx6gqj0eN0B599M=&uipk=5&nbs=1&deadline=1657796530&gen=playurlv2&os=cosbv&oi=1885228211&trid=83160320a4064890b79
17724d134c287u&mid=0&platform=pc&upsig=fc53638988da7169f5b3a390cddbad11&uparams=e,uipk,nbs,deadline,gen,os,oi,trid,mid,platform&bvc=vod&nettype=0&orderid=0,3&agrr=1&bw=8055&logo=80000000
【Udemy Tailwind CSS 热门课程】Tailwind CSS From Scratch  Learn By Building Projects-P3-004 What Is Tailwind CSS-中文（自动翻译）.srt 完成



.........................省略........................





Traceback (most recent call last):
  File "/home/liy/.local/share/virtualenvs/Lighting-bilibili-download-C-ivWGZB/bin/bilix", line 33, in <module>
    sys.exit(load_entry_point('bilix', 'console_scripts', 'bilix')())
  File "/home/liy/.local/share/virtualenvs/Lighting-bilibili-download-C-ivWGZB/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/liy/.local/share/virtualenvs/Lighting-bilibili-download-C-ivWGZB/lib/python3.9/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/liy/.local/share/virtualenvs/Lighting-bilibili-download-C-ivWGZB/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/liy/.local/share/virtualenvs/Lighting-bilibili-download-C-ivWGZB/lib/python3.9/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/liy/Lighting-bilibili-download/bilix/_cli.py", line 268, in main
    loop.run_until_complete(task)
  File "/usr/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
    return future.result()
  File "/home/liy/Lighting-bilibili-download/bilix/download.py", line 381, in get_series
    await asyncio.gather(*cors)
  File "/home/liy/Lighting-bilibili-download/bilix/download.py", line 452, in get_video
    await asyncio.gather(*cors)
  File "/home/liy/Lighting-bilibili-download/bilix/download.py", line 609, in _get_media
    await asyncio.gather(*cors)
  File "/home/liy/Lighting-bilibili-download/bilix/download.py", line 630, in _get_media_part
    start, end = map(int, part_name.split('-')[-2:])
ValueError: invalid literal for int() with base 10: ''

showthesunli avatar Jul 14 '22 09:07 showthesunli

感谢，我复现了这个错误，现在正在debug

HFrost0 avatar Jul 14 '22 09:07 HFrost0

问题的原因是改用的GET请求返回的数据中total所在的位置发生了变化，目前版本已经处理，如果没问题可以发布新版本了。总结一下，对于这个问题的解决，主要通过

  1. 错误发生时尝试向备选服务器发起请求
  2. 2次失败后（通常由于没有备选服务器导致）改为GET请求

HFrost0 avatar Jul 14 '22 09:07 HFrost0

还是有问题，如图： image

HEAD 404 异常之后，换成 GET，结果GET也404了。 这部视频好奇怪啊，下载其他的都没碰到这个问题。

showthesunli avatar Jul 14 '22 14:07 showthesunli

我觉得要不然这问题还是放着吧，好像也不是大面积出现。 没必要针对啊B服务器的非标准化抽风一直做适配。。。。😅

showthesunli avatar Jul 14 '22 14:07 showthesunli

又优化了一次重试逻辑，现在应该很难发生这个问题了

HFrost0 avatar Aug 17 '22 00:08 HFrost0