s5cmd icon indicating copy to clipboard operation
s5cmd copied to clipboard

Add extra virtual host style endpoinds support for s5cmd

Open LinPr opened this issue 11 months ago • 2 comments

Question

Currently, s5cmd will use path style endpoint by default when sending request to remote object storage, only when the remote endpoints match the following two endpoints will send request by virtual host style. and they are hard coded in program. check this page https://github.com/peak/s5cmd/blob/master/storage/s3.go

	// Amazon Accelerated Transfer endpoint
	transferAccelEndpoint = "s3-accelerate.amazonaws.com"

	// Google Cloud Storage endpoint
	gcsEndpoint = "storage.googleapis.com"

But in some situation, we can’t send request to the remote endpoint with path style request. For example , the alibaba object storage service.. It just banned the path style request by default, but only support virtual host style request by default , or you could ask for the official technical support , adding a white list for your path style endpoints. that not convenient when you have a bunch of buckets.

issue reference

#518

Suggestion

So, I recommend to add a extra command line flag, and also works by passing value by environment variable , which support the users to add their custom endpoints. when sending request to those endpoints, those requests will automatically be transformed and sent by virtual host style Also, this fucntion won't have any side affect on the original s5cmd command line funcions when not set.

And I will make a PR for this Issue, I have implement this and well tested, hope this could be merged, and help people who face the same issue like me.

LinPr avatar Feb 18 '25 03:02 LinPr

+1 I'm using custom cloud that supports only virtual-host style.

artemnikitin2023 avatar May 05 '25 16:05 artemnikitin2023

+1 I'm using custom cloud that supports only virtual-host style.

unfortunately,this project seems not being updated and merged new PR anymore, I have created this PR long time ago 😅

LinPr avatar May 06 '25 16:05 LinPr