spark-csv icon indicating copy to clipboard operation
spark-csv copied to clipboard

Configurable null values

Open petro-rudenko opened this issue 10 years ago • 8 comments

There's datasets where each column has it's own marker for missing values. spark-csv assumes only empty string for missing values. To avoid additional data transformation and saving on user's side would be great to specify a set of null markers and replace them to empty string on a library side.

petro-rudenko avatar Jun 08 '15 01:06 petro-rudenko

I ran into the same problem. Here is how I fixed it. Hopefully it would help others

  1. remove .ivy2/cashe rm -rf ~/.ivy2/cache

  2. redo git clone

  3. now go into project root and run build/sbt assembly

you should have a build success (at least for me). Good luck!

spark-water avatar Feb 05 '20 22:02 spark-water