databricks
Fix Scala ArrayBuffer cast to Spark ArrayData for Scala 2.13
Closes #688
Fix problem with cast scala.collection.mutable.ArrayBuffer to org.apache.spark.sql.catalyst.util.ArrayData
Hm, I don't think it's quite that simple unfortunately - it causes some test to fail.
I'm not totally sure why. It seems like building an Array this way doesn't quite work, ends up with 1 element not 3 for some reason, when 3 should be the result of the array
It may be that you have to try returning an array later somewhere, I'm not sure.
We'd also have to make a change in spark to match this, as the code really lives there now, but that's for later.
Hm, I don't think it's quite that simple unfortunately - it causes some test to fail. I'm not totally sure why. It seems like building an Array this way doesn't quite work, ends up with 1 element not 3 for some reason, when 3 should be the result of the array xs_any column.
It may be that you have to try returning an array later somewhere, I'm not sure.
We'd also have to make a change in spark to match this, as the code really lives there now, but that's for later.
Okay) I really bad in Scala. After that errors I know my target on future :) I think problem in immutability Array type. I think that problem need harder solution, but we can try start tests and dream about success)