tickstep
问题：上传文件遇到内存错误
[7819] 2022-09-05 11:42:22 已经禁用秒传检测，直接上传 [7685] ↑ 6.73GB/6.95GB 5.48MB/s(73.83MB/s) in 16m39.4s ............[7821] 2022-09-05 11:42:22 文件上传结果： 耗时 1秒 panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0xabe5a3]
goroutine 541374 [running]: github.com/tickstep/aliyunpan/internal/functions/panupload.(*UploadTaskUnit).Run(0xc000f9ce70, 0xc0298e4480) /Users/tickstep/Documents/Workspace/go/projects/aliyunpan/internal/functions/panupload/upload_task_unit.go:354 +0xa83 github.com/tickstep/aliyunpan/internal/taskframework.(*TaskExecutor).Execute.func1(0xc0015644b0, 0xc000286630, 0xc0009f4ac8) /Users/tickstep/Documents/Workspace/go/projects/aliyunpan/internal/taskframework/executor.go:111 +0x62 created by github.com/tickstep/aliyunpan/internal/taskframework.(*TaskExecutor).Execute /Users/tickstep/Documents/Workspace/go/projects/aliyunpan/internal/taskframework/executor.go:108 `+0x7c
在上传文件夹占用空间大概在1T以上时，必然会遇到这个报错，我是用的命令是 aliyunpan u ./ok /U2 --norapid --ow -pp 报错后再次使用该命令尝试继续上传会报错： [0] 当前文件上传最大并发量为: 15, 上传分片大小为: 10.00MB 打开上传未完成数据库错误: panupload.UploadingDatabase.UploadingList: []*panupload.Uploading: panupload.Uploading.LocalFileMeta: Path: localfile.SymlinkFile.LogicPath: readStringSlowPath: unexpected end of input, error found in #10 byte of ...|icPath":"/|..., bigger context ...|12},"upload_done":true}]}},{"path":{"logicPath":"/|...
帮忙收集一些debug日志。 你单个文件的最大大小是多少，我有点怀疑过大文件不允许这样传。
有日志文件可以方便查看吗？单个文件最大不会超过100G
debug日志你看readme文档，有教程怎么开。
我使用aliyunpan u ./ok /U2/ --ow 2>&1 > ./debuglog.log命令没能成功将debug日志保存到文件中。我主要有几个文件上传失败。文件中最后一段log是 [6426] 2022-09-07 06:57:22 准备上传: /root/Downloads/ok/HUNTER X HUNTER 2011 BLURAY COFFRET FRANCAIS 2xBLURAYBOX ^~^DragonMax^~^/HUNTER_X_HUNTER_DISC_17-DragonMax/BDMV/STREAM/00043.m2ts => /U2/ok/HUNTER X HUNTER 2011 BLURAY COFFRET FRANCAIS 2xBLURAYBOX ^~^DragonMax^~^/HUNTER_X_HUNTER_DISC_17-DragonMax/BDMV/STREAM/00043.m2ts [6426] 2022-09-07 06:57:22 检测秒传中, 请稍候... [6426] 2022-09-07 06:57:22 秒传失败，开始正常上传文件 [6412] 2022-09-07 06:57:22 文件上传结果： 失败！上传文件失败,The resource upload_id cannot be found. upload is not exist or has been completed 耗时 0秒 [6478] 2022-09-07 06:57:22 准备上传: /root/Downloads/ok/HUNTER X HUNTER 2011 BLURAY COFFRET FRANCAIS 2xBLURAYBOX ^~^DragonMax^~^/HUNTER_X_HUNTER_DISC_18-DragonMax/BDMV/BACKUP/index.bdmv => /U2/ok/HUNTER X HUNTER 2011 BLURAY COFFRET FRANCAIS 2xBLURAYBOX ^~^DragonMax^~^/HUNTER_X_HUNTER_DISC_18-DragonMax/BDMV/BACKUP/index.bdmv [6478] 2022-09-07 06:57:22 检测秒传中, 请稍候... [6478] 2022-09-07 06:57:22 秒传失败，开始正常上传文件 [6183] 2022-09-07 06:57:22 文件上传结果： 失败！上传文件失败,The resource upload_id cannot be found. upload is not exist or has been completed 耗时 0秒 [6531] 2022-09-07 06:57:22 准备上传: /root/Downloads/ok/HUNTER X HUNTER 2011 BLURAY COFFRET FRANCAIS 2xBLURAYBOX ^~^DragonMax^~^/HUNTER_X_HUNTER_DISC_18-DragonMax/BDMV/STREAM/00004.m2ts => /U2/ok/HUNTER X HUNTER 2011 BLURAY COFFRET FRANCAIS 2xBLURAYBOX ^~^DragonMax^~^/HUNTER_X_HUNTER_DISC_18-DragonMax/BDMV/STREAM/00004.m2ts [6531] 2022-09-07 06:57:22 检测秒传中, 请稍候... [6531] 2022-09-07 06:57:22 秒传失败，开始正常上传文件 [6296] 2022-09-07 06:57:22 文件上传结果： 失败！上传文件失败,The resource upload_id cannot be found. upload is not exist or has been completed 耗时 0秒 [6414] 2022-09-07 06:57:22 文件上传结果： 失败！上传文件失败,The resource upload_id cannot be found. upload is not exist or has been completed 耗时 0秒 [9198] ↑ 1.32GB/7.08GB 6.07MB/s(29.26MB/s) in 3m26s ............[6422] 2022-09-07 06:57:22 文件上传结果： 失败！上传文件失败,The resource upload_id cannot be found. upload is not exist or has been completed 耗时 0秒 [6419] ↑ 4.94GB/5.09GB 4.63MB/s(28.59MB/s) in 1m34s ............[6426] 2022-09-07 06:57:22 文件上传结果： 失败！上传文件失败,The resource upload_id cannot be found. upload is not exist or has been completed 耗时 0秒 [6478] 2022-09-07 06:57:22 文件上传结果： 失败！上传文件失败,The resource upload_id cannot be found. upload is not exist or has been completed 耗时 0秒 [6531] 2022-09-07 06:57:22 文件上传结果： 失败！上传文件失败,The resource upload_id cannot be found. upload is not exist or has been completed 耗时 0秒 [9198] ↑ 1.47GB/7.08GB 4.38MB/s(38.55MB/s) in 3m51s ............... [6419] 2022-09-07 06:57:22 上传文件成功, 保存到网盘路径: /U2/ok/HUNTER X HUNTER 2011 BLURAY COFFRET FRANCAIS 2xBLURAYBOX ^~^DragonMax^~^/HUNTER_X_HUNTER_DISC_17-DragonMax/BDMV/STREAM/00032.m2ts [9051] ↑ 4.64GB/7.09GB 8.19MB/s(15.53MB/s) in 11m16s ............[6419] 2022-09-07 06:57:22 文件上传结果： 成功！ 耗时 1分59秒 [9051] ↑ 4.93GB/7.09GB 7.87MB/s(39.96MB/s) in 11m59s ............[9286] 2022-09-07 06:58:22 文件上传结果： 失败！创建上传任务失败：context deadline exceeded (Client.Timeout or context cancellation while reading body),context deadline exceeded (Client.Timeout or context cancellation while reading body) 耗时 3分29秒 [6077] ↑ 0B/1.20MB 0B/s(28.57MB/s) in 6m54s ............[8463] 2022-09-07 07:02:22 文件上传结果： 失败！上传文件失败,context canceled 耗时 6分42秒 [8463] 上传文件失败, context canceled, 重试 2/3 [9198] ↑ 3.86GB/7.08GB 6.37MB/s(0B/s) in 9m47s ............[9150] 2022-09-07 07:03:22 文件上传结果： 失败！上传文件失败,context canceled 耗时 13分46秒 [9150] 上传文件失败, context canceled, 重试 1/3 [9199] ↑ 3.87GB/7.07GB 6.00MB/s(16.65MB/s) in 9m49s ............. [9051] 2022-09-07 07:03:22 上传文件成功, 保存到网盘路径: /U2/ok/めぞん一刻/MAISON_IKKOKU_13/BDMV/STREAM/00000.m2ts [9051] ↑ 7.09GB/7.09GB 2.90MB/s(68.83MB/s) in 17m18s ............[9051] 2022-09-07 07:03:22 文件上传结果： 成功！ 耗时 19分20秒 [5713] ↑ 0B/1.15MB 0B/s(1.88MB/s) in 10m0s ............[9199] 2022-09-07 07:05:22 文件上传结果： 失败！上传文件失败,context canceled 耗时 13分23秒 [9199] 上传文件失败, context canceled, 重试 1/3 [5713] ↑ 0B/1.15MB 0B/s(0B/s) in 16m10s ............................ [9198] 2022-09-07 07:11:22 上传文件成功, 保存到网盘路径: /U2/ok/めぞん一刻/MAISON_IKKOKU_16/BDMV/STREAM/00000.m2ts [9198] 2022-09-07 07:11:22 文件上传结果： 成功！ 耗时 19分35秒 [5713] ↑ 0B/1.15MB 0B/s(0B/s) in 3h4m19s ............
DEBUG日志一直在循环打印这一段log,已经持续好几个小时 [2022-9-7 15:56:54.085] do request url: https://api.aliyundrive.com/v2/file/get_upload_url [2022-9-7 15:56:54.181] DEBUG: UPLOADER INFO: multiUpload worker upload file done [2022-9-7 15:56:54.181] DEBUG: UPLOADER WARN: upload err: UrlExpired, id: 0 [2022-9-7 15:56:54.181] DEBUG: UPLOADER INFO: begin to upload part: 0 [2022-9-7 15:56:54.181] do request url: https://api.aliyundrive.com/v2/file/get_upload_url [2022-9-7 15:56:54.378] DEBUG: UPLOADER INFO: multiUpload worker upload file done [2022-9-7 15:56:54.378] DEBUG: UPLOADER WARN: upload err: UrlExpired, id: 0 [2022-9-7 15:56:54.378] DEBUG: UPLOADER INFO: begin to upload part: 0 [2022-9-7 15:56:54.378] do request url: https://api.aliyundrive.com/v2/file/get_upload_url [2022-9-7 15:56:54.469] DEBUG: UPLOADER INFO: multiUpload worker upload file done [2022-9-7 15:56:54.469] DEBUG: UPLOADER WARN: upload err: UrlExpired, id: 0 [2022-9-7 15:56:54.469] DEBUG: UPLOADER INFO: begin to upload part: 0
我使用aliyunpan u ./ok /U2/ --ow 2>&1 > ./debuglog.log命令没能成功将debug日志保存到文件中。我主要有几个文件上传失败。文件中最后一段log是 [6426] 2022-09-07 06:57:22 准备上传: /root/Downloads/ok/HUNTER X HUNTER 2011 BLURAY COFFRET FRANCAIS 2xBLURAYBOX ^~^DragonMax^~^/HUNTER_X_HUNTER_DISC_17-DragonMax/BDMV/STREAM/00043.m2ts => /U2/ok/HUNTER X HUNTER 2011 BLURAY COFFRET FRANCAIS 2xBLURAYBOX ^~^DragonMax^~^/HUNTER_X_HUNTER_DISC_17-DragonMax/BDMV/STREAM/00043.m2ts [6426] 2022-09-07 06:57:22 检测秒传中, 请稍候... [6426] 2022-09-07 06:57:22 秒传失败，开始正常上传文件 [6412] 2022-09-07 06:57:22 文件上传结果： 失败！上传文件失败,The resource upload_id cannot be found. upload is not exist or has been completed 耗时 0秒 [6478] 2022-09-07 06:57:22 准备上传: /root/Downloads/ok/HUNTER X HUNTER 2011 BLURAY COFFRET FRANCAIS 2xBLURAYBOX ^~^DragonMax^~^/HUNTER_X_HUNTER_DISC_18-DragonMax/BDMV/BACKUP/index.bdmv => /U2/ok/HUNTER X HUNTER 2011 BLURAY COFFRET FRANCAIS 2xBLURAYBOX ^~^DragonMax^~^/HUNTER_X_HUNTER_DISC_18-DragonMax/BDMV/BACKUP/index.bdmv [6478] 2022-09-07 06:57:22 检测秒传中, 请稍候... [6478] 2022-09-07 06:57:22 秒传失败，开始正常上传文件 [6183] 2022-09-07 06:57:22 文件上传结果： 失败！上传文件失败,The resource upload_id cannot be found. upload is not exist or has been completed 耗时 0秒 [6531] 2022-09-07 06:57:22 准备上传: /root/Downloads/ok/HUNTER X HUNTER 2011 BLURAY COFFRET FRANCAIS 2xBLURAYBOX ^~^DragonMax^~^/HUNTER_X_HUNTER_DISC_18-DragonMax/BDMV/STREAM/00004.m2ts => /U2/ok/HUNTER X HUNTER 2011 BLURAY COFFRET FRANCAIS 2xBLURAYBOX ^~^DragonMax^~^/HUNTER_X_HUNTER_DISC_18-DragonMax/BDMV/STREAM/00004.m2ts [6531] 2022-09-07 06:57:22 检测秒传中, 请稍候... [6531] 2022-09-07 06:57:22 秒传失败，开始正常上传文件 [6296] 2022-09-07 06:57:22 文件上传结果： 失败！上传文件失败,The resource upload_id cannot be found. upload is not exist or has been completed 耗时 0秒 [6414] 2022-09-07 06:57:22 文件上传结果： 失败！上传文件失败,The resource upload_id cannot be found. upload is not exist or has been completed 耗时 0秒 [9198] ↑ 1.32GB/7.08GB 6.07MB/s(29.26MB/s) in 3m26s ............[6422] 2022-09-07 06:57:22 文件上传结果： 失败！上传文件失败,The resource upload_id cannot be found. upload is not exist or has been completed 耗时 0秒 [6419] ↑ 4.94GB/5.09GB 4.63MB/s(28.59MB/s) in 1m34s ............[6426] 2022-09-07 06:57:22 文件上传结果： 失败！上传文件失败,The resource upload_id cannot be found. upload is not exist or has been completed 耗时 0秒 [6478] 2022-09-07 06:57:22 文件上传结果： 失败！上传文件失败,The resource upload_id cannot be found. upload is not exist or has been completed 耗时 0秒 [6531] 2022-09-07 06:57:22 文件上传结果： 失败！上传文件失败,The resource upload_id cannot be found. upload is not exist or has been completed 耗时 0秒 [9198] ↑ 1.47GB/7.08GB 4.38MB/s(38.55MB/s) in 3m51s ............... [6419] 2022-09-07 06:57:22 上传文件成功, 保存到网盘路径: /U2/ok/HUNTER X HUNTER 2011 BLURAY COFFRET FRANCAIS 2xBLURAYBOX ^~^DragonMax^~^/HUNTER_X_HUNTER_DISC_17-DragonMax/BDMV/STREAM/00032.m2ts [9051] ↑ 4.64GB/7.09GB 8.19MB/s(15.53MB/s) in 11m16s ............[6419] 2022-09-07 06:57:22 文件上传结果： 成功！ 耗时 1分59秒 [9051] ↑ 4.93GB/7.09GB 7.87MB/s(39.96MB/s) in 11m59s ............[9286] 2022-09-07 06:58:22 文件上传结果： 失败！创建上传任务失败：context deadline exceeded (Client.Timeout or context cancellation while reading body),context deadline exceeded (Client.Timeout or context cancellation while reading body) 耗时 3分29秒 [6077] ↑ 0B/1.20MB 0B/s(28.57MB/s) in 6m54s ............[8463] 2022-09-07 07:02:22 文件上传结果： 失败！上传文件失败,context canceled 耗时 6分42秒 [8463] 上传文件失败, context canceled, 重试 2/3 [9198] ↑ 3.86GB/7.08GB 6.37MB/s(0B/s) in 9m47s ............[9150] 2022-09-07 07:03:22 文件上传结果： 失败！上传文件失败,context canceled 耗时 13分46秒 [9150] 上传文件失败, context canceled, 重试 1/3 [9199] ↑ 3.87GB/7.07GB 6.00MB/s(16.65MB/s) in 9m49s ............. [9051] 2022-09-07 07:03:22 上传文件成功, 保存到网盘路径: /U2/ok/めぞん一刻/MAISON_IKKOKU_13/BDMV/STREAM/00000.m2ts [9051] ↑ 7.09GB/7.09GB 2.90MB/s(68.83MB/s) in 17m18s ............[9051] 2022-09-07 07:03:22 文件上传结果： 成功！ 耗时 19分20秒 [5713] ↑ 0B/1.15MB 0B/s(1.88MB/s) in 10m0s ............[9199] 2022-09-07 07:05:22 文件上传结果： 失败！上传文件失败,context canceled 耗时 13分23秒 [9199] 上传文件失败, context canceled, 重试 1/3 [5713] ↑ 0B/1.15MB 0B/s(0B/s) in 16m10s ............................ [9198] 2022-09-07 07:11:22 上传文件成功, 保存到网盘路径: /U2/ok/めぞん一刻/MAISON_IKKOKU_16/BDMV/STREAM/00000.m2ts [9198] 2022-09-07 07:11:22 文件上传结果： 成功！ 耗时 19分35秒 [5713] ↑ 0B/1.15MB 0B/s(0B/s) in 3h4m19s ............
DEBUG日志一直在循环打印这一段log,已经持续好几个小时 [2022-9-7 15:56:54.085] do request url: https://api.aliyundrive.com/v2/file/get_upload_url [2022-9-7 15:56:54.181] DEBUG: UPLOADER INFO: multiUpload worker upload file done [2022-9-7 15:56:54.181] DEBUG: UPLOADER WARN: upload err: UrlExpired, id: 0 [2022-9-7 15:56:54.181] DEBUG: UPLOADER INFO: begin to upload part: 0 [2022-9-7 15:56:54.181] do request url: https://api.aliyundrive.com/v2/file/get_upload_url [2022-9-7 15:56:54.378] DEBUG: UPLOADER INFO: multiUpload worker upload file done [2022-9-7 15:56:54.378] DEBUG: UPLOADER WARN: upload err: UrlExpired, id: 0 [2022-9-7 15:56:54.378] DEBUG: UPLOADER INFO: begin to upload part: 0 [2022-9-7 15:56:54.378] do request url: https://api.aliyundrive.com/v2/file/get_upload_url [2022-9-7 15:56:54.469] DEBUG: UPLOADER INFO: multiUpload worker upload file done [2022-9-7 15:56:54.469] DEBUG: UPLOADER WARN: upload err: UrlExpired, id: 0 [2022-9-7 15:56:54.469] DEBUG: UPLOADER INFO: begin to upload part: 0
看着是上传链接过期了导致不断重试。 你把aliyunpan_uploading.json的文件删除，再上传应该就能上传成功了。我看看怎么改善下这个问题