cloudyr
Cannot read/write csv files on s3
Please specify whether your issue is about:
- [x] a possible bug
- [x] a question about package functionality
- [x] a suggested code or documentation change, improvement to the code, or feature request
If you are reporting (1) a bug or (2) a question about code, please supply:
- a fully reproducible example using a publicly available dataset (or provide your data)
- if an error is occurring, include the output of
traceback()run immediately after the error occurs - the output of
sessionInfo()
Put your code here:
## load package
library("aws.s3")
#For Reading
Scoretest <- s3read_using(FUN = read.csv, object = "s3://xx-xxx-xxx-xx-databricks-x-xx/xxxx_member_xxx_mask.csv")
Error in parse_aws_s3_response(r, Sig, verbose = verbose) :
Forbidden (HTTP 403).
#For Writing
#Method 1
x <- rawConnection(raw(0), "w")
write.csv(Score, x)
put_object(rawConnectionValue(x), object= "s3://xx-xxx-xxx-xx-databricks-x-xx/Suspicious_list1.csv" )
Error in parse_aws_s3_response(r, Sig, verbose = verbose) :
Forbidden (HTTP 403).
#Method 2
tmp <- tempfile()
on.exit(unlink(tmp))
write.csv(mtcars, file = tmp)
put_object(tmp, object = "mtcars.csv", bucket = "s3://xx-xxx-xxx-xx-databricks-x-xx")
Error in parse_aws_s3_response(r, Sig, verbose = verbose) :
Bad Request (HTTP 400).
#Method 3
s3write_using(mtcars, FUN = write.csv, object = "s3://xx-xxx-xxx-xx-databricks-x-xx/cars.csv")
Error in parse_aws_s3_response(r, Sig, verbose = verbose) :
Forbidden (HTTP 403).
## I am using platform Databricks but that should not be an issue.
## As on the same platform I am able to read via PANDAS in PYTHON
sessionInfo()
R version 3.4.4 (2018-03-15)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 16.04.4 LTS
Matrix products: default
BLAS: /usr/lib/openblas-base/libblas.so.3
LAPACK: /usr/lib/libopenblasp-r0.2.18.so
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
Also I am able to read via - below code
obj <-get_object("s3://xx-xxx-xxx-xx-databricks-x-xx/xxxx_member_xxx_mask.csv", check_region = FALSE)
csvcharobj <- rawToChar(obj)
con <- textConnection(csvcharobj)
data <- read.csv(file = con)
Please Help me solve this i have spent two days going through blogs/issues on git and stack still unable to resolve
@leeper help needed !!
I had a similar problem when I started using the package, but I got around it by passing the bucket name into functions (e.g. s3read_using and s3write_using) without putting s3:// in front of it. Would something like that work in your use case?
