spark-examples

spark-examples copied to clipboard

Submitting code to AWS EMR

2

I tried to run your code to AWS EMR and I get the following error - > "FailureDetails": { "Reason": "Main Class not found.", "LogFile": "s3://some_locatoin/clusterId/steps/stepId/stderr.gz", "Message": "java.lang.ClassNotFoundException: org.sparkexample.WordCountTask" }...

tomron

For "./bin/spark-submit --class org.sparkexample.WordCount --master local[2] //target/spark-examples-1.0-SNAPSHOT.jar / /" The class name should be WordCountTask now. And also, I don't see any output result in output directory, only console.

autofei

Hi, I had to change new WordCountTask().run(args[1]); to new WordCountTask().run(args[0]); to get the job to run in Spark Anthony

28formica77

Create Dockerfile

1

prasadlolugu