googleCloudStorageR icon indicating copy to clipboard operation
googleCloudStorageR copied to clipboard

Upload object with metadata in folder fails due to urlencode

Open mzusev opened this issue 4 years ago • 3 comments

Example:

meta <- gcs_metadata_object("test_folder/mtcars.csv",
                            metadata = list(id = '4D67djkdJL8'))

gcs_upload(mtcars,
           name = "test_folder/mtcar.csv",
           object_metadata = meta,
           predefinedAcl = "bucketLevel")

Fails with error:

Error: API returned: Value 'test_folder%2Fmtcars.csv' in content does not agree with value 'test_folder/mtcars.csv'. This can happen when a value set through a parameter is inconsistent with a value set in the request.

Forward slash seems to be due to the URLencode:

object_name <- if(!is.null(object_name)) URLencode(object_name)

Workaround is to update object name directly before calling gcs_upload:

meta$name <- "test_folder/mtcars.csv"

But this may interfere with other reserved characters for url encoding.

mzusev avatar Jun 09 '21 16:06 mzusev

Thanks for the report. I wonder if URL encoding is not necessary anymore in the gcs_upload()

MarkEdmondson1234 avatar Jun 10 '21 06:06 MarkEdmondson1234

Hi! I encountered a similar problem today while trying to add caching details via gcs_metadata_object().

gcs_upload() and gcs_metadata_object() use different encodings for slashes. While gcs_upload uses "/", gcs_metadata_object() uses "%2F".

This is my work around, but a more stable fix would be great:

metaData <- gcs_metadata_object(object_name = paste0("folder/images/", filename_pure), cacheControl = "max-age=0, no-cache, no-store")

metaData[["name"]] <- gsub("%2F","/",metaData[["name"]])

ghost avatar Aug 20 '21 12:08 ghost

Thanks! Looks like an inconsistency between the functions, will look to standardise

MarkEdmondson1234 avatar Aug 21 '21 07:08 MarkEdmondson1234