Firestorm
Firestorm copied to clipboard
Tencent
Firestorm is a Remote Shuffle Service, and provides the capability for Apache Spark and Apache Hadoop MapReduce applications to store shuffle data on remote servers
The current blockId is designed as following: ``` // BlockId is long and composed by partitionId, executorId and AtomicInteger // AtomicInteger is first 19 bit, max value is 2^19 -...
hi all, a few questions. 1 -- does restarting the process on a server cause spark tasks to fail, or will they wait and retry once server is up and...
It would be good to have a web UI in coordinated server to show the detailed information about shuffle server, job shuffle metrics and others. Simply like Spark web UI...
