Connell Donaghy

fault tolerance

3

hi all, a few questions. 1 -- does restarting the process on a server cause spark tasks to fail, or will they wait and retry once server is up and...

fault tolerance of restarting server

7

I'm a little confused, shouldn't a server using local storage that restarts be able to accept/handle downloads if the service can restart in 15 seconds or so, saving its state?...

write amplification

2

i'm noticing running some spark apps that produce 11TB of shuffle data on external shuffle service, that they produce closer to 18TB of shuffle data on remote shuffle service. is...

spark 3.1/3.2?

10

hi all, I saw there is a spark30 branch for spark 3.0.x supported in the readme. there also seems to be a spark31 branch but wondering is there any plans...