Azure
Cluster fails to provision if nodes are rebooted and the storage SAS has expired
The error is:
BlobSource: https://******.blob.core.windows.net/spark-node-scripts/node-scripts.zip?sr=b&sp=r&se=2018-01-19T02%3A22%3A57Z&sv=2015-07-08&sig=mSVuZVmWRZ9EKij8DUtQSraFRSE1zALfLoXt7tnvfhY%3D
FilePath: /mnt/batch/tasks/startup/wd/node-scripts.zip
Can we get around using blob SAS's if we have access to the shared keys?
Alternatively so that we can't see the shared keys in clear text, but we still have them on node, can we simply re-generate a SAS every time?
I like the idea of regenerating it as needed. I'm not sure that will be necessary once we start encrypting the keys on the node, though.
We will need to update some logic here. Currently this is downloaded via the start task's resource file list. It will need to be downloaded as part of the body of the start task instead.
A temporary fix will be to simply extend the expiry date for the SAS token (maybe to 1 month?). This is just pushing the boundary though, not a real fix.