frameless icon indicating copy to clipboard operation
frameless copied to clipboard
verified publisher icon typelevel

Add CI tests over a remote Spark cluster

Open imarios opened this issue 8 years ago • 2 comments

  • Bring a very basic cluster up and then deploy a toy Frameless project packaged using sbt-assembly
  • Read data back/forth from Paraquet
  • Read data back/forth from CSV and JSON

imarios avatar Oct 21 '17 21:10 imarios

maybe as a first step it should be investigated if there is a way to run the tests in cluster mode on travis.

tscholak avatar Oct 21 '17 21:10 tscholak

here is something useful: https://github.com/holdenk/spark-testing-base/blob/master/.travis.yml

tscholak avatar Oct 21 '17 21:10 tscholak