SparkUnitTestingExamples
SparkUnitTestingExamples copied to clipboard
tmalaska
This project is a collection of Spark Unit Tests Examples to help new Spark users have good examples on how to unit start their code for Spark Core, Spark SQL, and Spark Streaming
Spark Unit Test Examples
In this github you will find examples for Spark Core, Spark SQL, and Spark Streaming unit test.
This is by no means the only way to unit test Spark, it is just to be used as a guide for training
Running
Simply load this project into your IDE and execute the test classes.
Back sure to add the following JVM parameters -Xmx1536m -XX:MaxPermSize=512m -XX:ReservedCodeCacheSize=512m
Or just use mvn test
tmalaska