spark-memory icon indicating copy to clipboard operation
spark-memory copied to clipboard

A tool to get better debug info on spark's memory usage

Results 4 spark-memory issues
Sort by recently updated
recently updated
newest added

Updates and Fixes to Make spark-memory work on opensource spark ``` [INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ spark-memory-core_2.11 --- [INFO] Building jar: /Users/loandrew/Workspaces/Spark-memory/src/Spark-memory/core/target/spark-memory-core_2.11-0.1.0-SNAPSHOT.jar [INFO] [INFO] --- maven-assembly-plugin:2.2-beta-5:single (make-assembly) @ spark-memory-core_2.11 ---...

``` [hadoop@ip-10-0-61-104 ~]$ spark-submit --class com.databricks.spark.sql.perf.runExperiment --jars s3://loandrew-emr-dev/tools/spark-memory/spark-memory-core_2.11-0.1.0-SNAPSHOT.jar --driver-java-options "-XX:MaxMetaspaceSize=200M -XX:+PrintGCDetails -Dmemory.monitor.enabled=true -Dmemory.monitor.freq=100" --conf spark.executor.extraJavaOptions="-XX:MaxMetaspaceSize=200M -XX:+PrintGCDetails -Dmemory.monitor.enabled=true -Dmemory.monitor.freq=100" --conf spark.executor.plugins="com.cloudera.spark.MemoryMonitorExecutorExtension" /home/hadoop/spark-sql-perf.jar --benchmark TPCDS_2_4 --databaseName tpcds_loandrew --jobName q95 --queryNames q95 --runWarmup...

What license is this under? It's a bit ambiguous atm.