训练停止并未完成，控制台没有报错信息

[mopring]

你好，我遇到了以下问题，训练停止并未完成，控制台没有报错信息，在web端中显示error无法继续，错误信息如下 (py38) C:\Users\shd14\Documents\code\facechain-main\facechain-main>python app.py 2023-11-29 22:48:57,932 - modelscope - INFO - PyTorch version 2.0.1+cu117 Found. 2023-11-29 22:48:57,934 - modelscope - INFO - Loading ast index from C:\Users\shd14.cache\modelscope\ast_indexer 2023-11-29 22:48:57,984 - modelscope - INFO - Loading done! Current index file version is 1.9.5, with md5 0c667a039a9b29271347d22e770d05ed and a total number of 945 components indexed [['C:\Users\shd14\Documents\code\facechain-main\facechain-main/resources/inpaint_template\1.jpg'], ['C:\Users\shd14\Documents\code\facechain-main\facechain-main/resources/inpaint_template\2.jpg'], ['C:\Users\shd14\Documents\code\facechain-main\facechain-main/resources/inpaint_template\3.jpg'], ['C:\Users\shd14\Documents\code\facechain-main\facechain-main/resources/inpaint_template\4.jpg'], ['C:\Users\shd14\Documents\code\facechain-main\facechain-main/resources/inpaint_template\5.jpg']] [['resources/tryon_garment\garment1.png'], ['resources/tryon_garment\garment2.png'], ['resources/tryon_garment\garment3.png'], ['resources/tryon_garment\garment4.png']] Running on local URL: http://127.0.0.1:7860

Could not create share link. Missing file: C:\Users\shd14.conda\envs\py38\lib\site-packages\gradio\frpc_windows_amd64_v0.2.

Please check your internet connection. This can happen if your antivirus software blocks the download of this file. You can install manually by following these steps:

1. Download this file: https://cdn-media.huggingface.co/frpc-gradio-0.2/frpc_windows_amd64.exe
2. Rename the downloaded file to: frpc_windows_amd64_v0.2
3. Move the file to this location: C:\Users\shd14.conda\envs\py38\lib\site-packages\gradio 显存足够 ** Setting base model to SDXL ** --------uuid: qw ----------work_dir: C:\Users\shd14\Documents\code\facechain-main\facechain-main\worker_data\qw\AI-ModelScope/stable-diffusion-xl-base-1.0\person1 2023-11-29 22:50:03,867 - modelscope - INFO - PyTorch version 2.0.1+cu117 Found. 2023-11-29 22:50:03,869 - modelscope - INFO - Loading ast index from C:\Users\shd14.cache\modelscope\ast_indexer 2023-11-29 22:50:03,923 - modelscope - INFO - Loading done! Current index file version is 1.9.5, with md5 0c667a039a9b29271347d22e770d05ed and a total number of 945 components indexed [['C:\Users\shd14\Documents\code\facechain-main\facechain-main/resources/inpaint_template\1.jpg'], ['C:\Users\shd14\Documents\code\facechain-main\facechain-main/resources/inpaint_template\2.jpg'], ['C:\Users\shd14\Documents\code\facechain-main\facechain-main/resources/inpaint_template\3.jpg'], ['C:\Users\shd14\Documents\code\facechain-main\facechain-main/resources/inpaint_template\4.jpg'], ['C:\Users\shd14\Documents\code\facechain-main\facechain-main/resources/inpaint_template\5.jpg']] [['resources/tryon_garment\garment1.png'], ['resources/tryon_garment\garment2.png'], ['resources/tryon_garment\garment3.png'], ['resources/tryon_garment\garment4.png']] 2023-11-29 22:50:13.9588041 [W:onnxruntime:, graph.cc:3543 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2649'. It is not used by any node and should be removed from the model. 2023-11-29 22:50:13.9618147 [W:onnxruntime:, graph.cc:3543 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2644'. It is not used by any node and should be removed from the model. 2023-11-29 22:50:13.9648713 [W:onnxruntime:, graph.cc:3543 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2647'. It is not used by any node and should be removed from the model. 2023-11-29 22:50:13.9675510 [W:onnxruntime:, graph.cc:3543 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2658'. It is not used by any node and should be removed from the model. 2023-11-29 22:50:13.9700824 [W:onnxruntime:, graph.cc:3543 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2648'. It is not used by any node and should be removed from the model. 2023-11-29 22:50:13.9734617 [W:onnxruntime:, graph.cc:3543 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2657'. It is not used by any node and should be removed from the model. 2023-11-29 22:50:13.9761604 [W:onnxruntime:, graph.cc:3543 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2653'. It is not used by any node and should be removed from the model. 2023-11-29 22:50:13.9791752 [W:onnxruntime:, graph.cc:3543 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2652'. It is not used by any node and should be removed from the model. 2023-11-29 22:50:13.9819490 [W:onnxruntime:, graph.cc:3543 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2645'. It is not used by any node and should be removed from the model. 2023-11-29 22:50:13.9845896 [W:onnxruntime:, graph.cc:3543 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2643'. It is not used by any node and should be removed from the model. 2023-11-29 22:50:13.9878951 [W:onnxruntime:, graph.cc:3543 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2641'. It is not used by any node and should be removed from the model. 2023-11-29 22:50:13.9905723 [W:onnxruntime:, graph.cc:3543 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2633'. It is not used by any node and should be removed from the model. 2023-11-29 22:50:13.9935002 [W:onnxruntime:, graph.cc:3543 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2632'. It is not used by any node and should be removed from the model. 2023-11-29 22:50:13.9961173 [W:onnxruntime:, graph.cc:3543 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2624'. It is not used by any node and should be removed from the model. 2023-11-29 22:50:13.9986391 [W:onnxruntime:, graph.cc:3543 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2614'. It is not used by any node and should be removed from the model. 2023-11-29 22:50:14.0011645 [W:onnxruntime:, graph.cc:3543 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2613'. It is not used by any node and should be removed from the model. 2023-11-29 22:50:14.0036085 [W:onnxruntime:, graph.cc:3543 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2606'. It is not used by any node and should be removed from the model. 2023-11-29 22:50:14.0064339 [W:onnxruntime:, graph.cc:3543 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2598'. It is not used by any node and should be removed from the model. 2023-11-29 22:50:14.0091208 [W:onnxruntime:, graph.cc:3543 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2596'. It is not used by any node and should be removed from the model. 2023-11-29 22:50:14.0116885 [W:onnxruntime:, graph.cc:3543 onnxruntime::Graph::CleanUnusedInitializersAndNodeArgs] Removing initializer 'const_fold_opt__2594'. It is not used by any node and should be removed from the model. load checkpoint from local path: C:\Users\shd14.cache\modelscope\hub\damo\cv_ddsar_face-detection_iclr23-damofd\pytorch_model.pt load checkpoint from local path: C:\Users\shd14.cache\modelscope\hub\damo\cv_ddsar_face-detection_iclr23-damofd\pytorch_model.pt load checkpoint from local path: C:\Users\shd14.cache\modelscope\hub\damo\cv_ddsar_face-detection_iclr23-damofd\pytorch_model.pt 000.jpg 0.9586985558271408 black_eyes, glasses, looking_at_viewer, mole, solo, transparent_background 001.jpg 0.9590332955121994 bald, full_body, realistic, simple_background, solo, transparent_background, white_background 002.jpg 0.9537132531404495 closed_eyes, glasses, realistic, simple_background, solo, transparent_background, white_background 003.jpg 0.9648549258708954 1boy, bald, closed_eyes, full_body, male_focus, realistic, smile, solo, transparent_background 004.jpg 0.955781988799572 bald, closed_eyes, full_body, old, realistic, solo, transparent_background 005.jpg 0.9634375646710396 bald, closed_eyes, facial_hair, full_body, realistic, smile, solo, transparent_background 006.jpg 0.9590725600719452 bald, beard, closed_eyes, facial_hair, full_body, mustache, old, old_man, realistic, solo, transparent_background, what, wrinkled_skin 007.jpg 0.9532539397478104 bald, black_eyes, full_body, mole, solo, transparent_background 008.jpg 0.9544237852096558 bald, full_body, transparent_background 009.jpg 0.9599556475877762 bald, closed_eyes, facial_hair, full_body, old, old_man, realistic, smile, solo, transparent_background, what, wrinkled_skin Detecting multiple faces, do not use image 010.jpg. [['black_eyes', 'glasses', 'looking_at_viewer', 'mole', 'solo', 'transparent_background'], ['bald', 'full_body', 'realistic', 'simple_background', 'solo', 'transparent_background', 'white_background'], ['closed_eyes', 'glasses', 'realistic', 'simple_background', 'solo', 'transparent_background', 'white_background'], ['1boy', 'bald', 'closed_eyes', 'full_body', 'male_focus', 'realistic', 'smile', 'solo', 'transparent_background'], ['bald', 'closed_eyes', 'full_body', 'old', 'realistic', 'solo', 'transparent_background'], ['bald', 'closed_eyes', 'facial_hair', 'full_body', 'realistic', 'smile', 'solo', 'transparent_background'], ['bald', 'beard', 'closed_eyes', 'facial_hair', 'full_body', 'mustache', 'old', 'old_man', 'realistic', 'solo', 'transparent_background', 'what', 'wrinkled_skin'], ['bald', 'black_eyes', 'full_body', 'mole', 'solo', 'transparent_background'], ['bald', 'full_body', 'transparent_background'], ['bald', 'closed_eyes', 'facial_hair', 'full_body', 'old', 'old_man', 'realistic', 'smile', 'solo', 'transparent_background', 'what', 'wrinkled_skin']] 0.png a handsome man 1.png a handsome man 2.png a handsome man 3.png a handsome man 4.png a handsome man 5.png a handsome man 6.png a handsome man 7.png a handsome man 8.png a handsome man 9.png a handsome man instance_data_dir C:\Users\shd14\Documents\code\facechain-main\facechain-main\worker_data\qw\training_data\AI-ModelScope/stable-diffusion-xl-base-1.0\person1