apache
[SYSTEMDS-3421] Adds missing read/writeExternal to ColumnEncoderUDF.
The read/writeExternal functions for ColumnEncoderUDF where missing. This also adds the necessary switch case in the EncoderFactory and a test that currently does nothing.
I encountered this bug when running the FTBench T2 in federated mode with two actual workers. The main issue is that the ColumnEncoderUDF objects on the workers end up with null as _fName because the function name is not sent.
@mboehm7
There are two versions of T2 (KDD dataset) -- one with UDF-based scaling and another with scaling outside of transformencode. The UDF task implementation is still in the experimental phase. Internally it uses our eval mechanism. I am not very sure how would that fit in the federated setup. But the other implementation should work out of the box in federated.
That is a good point @phaniarnab - the UDF column encoder would not be valid for federated row-partitions we currently work with for the other benchmark use cases. However, for column-partitioned federated data it should work.
