Azure
HTTPError: 403 Client Error In merge task foreach %dopar%
I have been using Azure Batch for a while and have come across a new error in the merge task even on the sample code. this appears to happen with any size of data while running the foreach %dopar%. has anyone else noticed this?
HTTPError: 403 Client Error: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature. for url: https://_batchaccount._blob.core.windows.net/job12345?sv=2016-05-31&sr=c&st=2020-07-18T21%3A09%3A30Z&se=2020-07-21T21%3A09%3A30Z&sp=rwcl&sig=signature%3D&include=metadata&restype=container&comp=list&maxresults=1000
Reproduce Example generateCredentialsConfig("credentials.json")
setCredentials("credentials.json") generateClusterConfig("cluster.json")
cluster <- makeCluster("cluster.json")
Register your parallel backend registerDoAzureParallel(cluster)
setVerbose(FALSE) setAutoDeleteJob(FALSE)
mean_change = 1.001 volatility = 0.01 opening_price = 100
getClosingPrice <- function() { days <- 1825 # ~ 5 years movement <- rnorm(days, mean=mean_change, sd=volatility) path <- cumprod(c(opening_price, movement)) closingPrice <- path[days] return(closingPrice) }
Optimize runtime. Chunking allows running multiple iterations on a single R instance. opt <- list(chunkSize = 10) start_p <- Sys.time() closingPrices_p <- foreach(i = 1:1000, .combine='c', .options.azure = opt) %dopar% { replicate(100000, getClosingPrice()) } end_p <- Sys.time()
hist(closingPrices_s)
