Edsko de Vries
Edsko de Vries
Per @mboes request, here's a summary of my attempt to add a binding to [`mapPartitions` for dataframes](https://spark.apache.org/docs/2.0.0/api/java/org/apache/spark/sql/Dataset.html#mapPartitions(org.apache.spark.api.java.function.MapPartitionsFunction,%20org.apache.spark.sql.Encoder)). The Java function has signature ``` java Dataset mapPartitions(MapPartitionsFunction f, Encoder encoder) ```...
Note that once we move to the dataset API we'll have a potentially more challenging Implicit to deal with: `Encoding`.
I have to admit I often use `generics-sop` simply for it's awesome products :)
Yes, that's certainly possible. The original branch from which I was working does in fact do that. For my current purposes however, it turns out this representation is still not...
Perhaps @kosmikus did some on the original compact representation branch?
Well actually for `Map` and co I typically import qualified all the time ``` import qualified Data.Map as Map ``` and then use `Map.map`.
Yeah, I know that they weren't intended to be used by other types, but sometimes that does arise naturally. But perhaps this is indeed a bit of a quagmire (did...
(Actually, it's `HTraversable` that we ended up using in the end.)
Yeah, we have ```haskell -- | N-ary traversable functors -- -- TODO: Don't provide Elem explicitly (just instantiate @c@)? -- TODO: Introduce HTraverse into SOP? class NTraversable (f :: (k...
Oh! Yes, perhaps so :flushed: