Praneet Sharma comments

Results 6 comments of


                                            Praneet Sharma

Messages older than 14 days are not being deleted.

@rhld13 let me know if you want to work on this issue.

Is .avro extension necessary for reading avro files?

Hi @devender-yadav where are you setting the property? In the runtimeConfig of SparkSession?

ClassNotFoundException: com.databricks.spark.avro

Please keep spark-avro*.jar in Spark driver's and executor's classpath.

write as bytes decimal logical type

Hey @eliviu spark-avro currently doesn't support Decimal LogicalType. And hence decimal type is converted to String. You can look at the code for conversion in the function createConverterToAvro in AvroOutputWriter.scala...

java.lang.NoSuchMethodError: org.apache.spark.sql.DataFrameReader.load

Hi @Leela1229 Please tell us what version of Spark and Spark-avro are you using?

Save Compressed avro to Hdfs using PySpark

Hi @YonoloX Spark 1.6.x uses version 2.0.1 of spark-avro library. As per spark-avro 2.0.1 doc, the following compression types are supported when writing avro to disk - **uncompressed**, **snappy**, **deflate**....