D2wnloader icon indicating copy to clipboard operation
D2wnloader copied to clipboard

多线程下载多个文件请教

Open HenryBeWell opened this issue 4 years ago • 0 comments

您好,我尝试用线程池并发请求多个下载链接,然后在单个链接请求中调用您写的D2wnloader,但是出现了问题,想请教下,如有回复,不胜感激~

# obj包含了url等信息
class FileDownload:
    def download_all_reqs():
        with ThreadPoolExecutor(max_workers=8) as pool:
            for obj in objs:
                uuid = obj.get('uuid')
                logging.info(f'{uuid} is running')
                future_result = pool.submit(self.fetch_file, obj)
                results.append(future_result)
            wait(results, return_when=ALL_COMPLETED)
            logging.info('files ok')
    def fetch_file(self, obj: dict):
        uuid = obj.get('uuid')
        seq = obj.get('chunkSeq')
        url = obj.get('url')
        prefix = uuid + f'.{seq}'
        # response = requests.get(url, stream=True)
        dl = D2wnloader(url, prefix)
        dl.start()

问题打印:

----- D2wnloader [v2.0.b8] -----
[size] 15.7 MB
[info] 284.3 % | 13.5 MB/s | 8+73 4s    

HenryBeWell avatar Jul 15 '21 02:07 HenryBeWell