aws
support continuing from aborted multipart upload
Currently the s3handler seems to always create a new upload context.
It would be nice to have the option to resume an earlier upload in the event that it was interrupted.
Marking as feature request.
Because the CLI currently will abort any in progress multipart uploads if the user Ctrl-Cs the process or in the event of an unknown failure, we'd also have to add an option you'll have to specify during the initial transfer that tells the CLI not to abort pending multipart uploads so they can be later resumed.
Hmm, in that case see #1428, because something didn't clean those up after failure.
Definitely see a great use for resuming an earlier upload context.
We use awscli sync to regularly sync a folder's contents to S3. Those folders often contain very large video files - dozens of GBs - that take many hours to upload. On occasion, every hour or so, internet connectivity can be lost, resulting in (32, 'Broken pipe') errors. When the cron job tries again, however, the s3handler creates a new upload context. That means that sync starts all over uploading that gigantic file. Then, because the pipe breaks every hour, the upload fails yet again. The giant file is now stuck, unable to fully upload due to the occasional broken pipe.
If we picked up where we left off in the multipart upload, the file would still make it up to S3 despite the failures. Ideally, aws sync would never exit - it would simply handle the Broken pipe error and gracefully resume.
Hi tyleha,
i was uploading big file to a s3 bucket but connection dropped my file. I can list the bucket and find the upload-id and retrieve the Etag of the last uploaded part. Is it possible to resume the upload from the last uploaded part?
Good Morning!
We're closing this issue here on GitHub, as part of our migration to UserVoice for feature requests involving the AWS CLI.
This will let us get the most important features to you, by making it easier to search for and show support for the features you care the most about, without diluting the conversation with bug reports.
As a quick UserVoice primer (if not already familiar): after an idea is posted, people can vote on the ideas, and the product team will be responding directly to the most popular suggestions.
We’ve imported existing feature requests from GitHub - Search for this issue there!
And don't worry, this issue will still exist on GitHub for posterity's sake. As it’s a text-only import of the original post into UserVoice, we’ll still be keeping in mind the comments and discussion that already exist here on the GitHub issue.
GitHub will remain the channel for reporting bugs.
Once again, this issue can now be found by searching for the title on: https://aws.uservoice.com/forums/598381-aws-command-line-interface
-The AWS SDKs & Tools Team
Based on community feedback, we have decided to return feature requests to GitHub issues.
Still would love to see this as a feature. The ability to specify an upload-id and have it continue the upload from where it left off would be very helpful.
Resume uploads is really really important when dealing with cellular network in iot applications. Would love to see this added in boto3 since it was available in boto
