Spark 3.1 on Windows build test issue

Open clee704 opened this issue 4 years ago • 0 comments

Here's the logs: https://dev.azure.com/ossspark/78bda1b1-2d15-4e6d-8327-a916f6cb1f9f/_apis/build/builds/2223/logs/48

Newly added tests are failing only for Windows / Spark 3.1.

The exception is: java.lang.UnsatisfiedLinkError: org.apache.hadoop.io.nativeio.NativeIO$POSIX.stat(Ljava/lang/String;)Lorg/apache/hadoop/io/nativeio/NativeIO$POSIX$Stat;

I suspect this is because Spark 3.1 depends on Hadoop 3.2.0 but we use 2.8.1 for testing.

Jul 28 '21 09:07 clee704