Yunhui YZ Zhang
Yunhui YZ Zhang
[SimpleSparkSerializer](https://github.com/combust/mleap/blob/087e8f04f79d08bad5e37600c4668dfb0328013f/python/mleap/pyspark/spark_support.py#L36) use the default `json` format. But for model like Word2Vec, size can be huge and json decode with current design will cause heap OOM. To ease this problem, protobuf...
MleapReflection use `Thread.currentThread().getContextClassLoader` as classLoader for `mirror`. In some case we need to specified classLoader without changing the current thread's classLoader. Providing a set method here will make this hack...
Existing ChiSqSelectorModel assume filterIndiecs is sorted But since spark 3.0.0, filterIndiecs is not sorted any more. This PR simply sort the filterIndiecs and use it later.
Related to https://github.com/combust/mleap/pull/760. We currently convert Sparse Tensor to Dense Tensor before calling tensorflow. This behavior will cause memory issue and is ineffective. Since Tensorflow also support Sparse Tensor, we...
When a pool has multiple instance e.g: cpu and gpu mix pool. We would like to specify instance type
current abstract class SparkParityBase has a variable `val spark = SparkEnv.spark` Even user override it in child class, it will still create a spark session. change it to `def spark...
Call Stacks ``` java.lang.IllegalArgumentException: size of dimensions must equals size of values at ml.combust.mleap.tensor.Tensor$.normalizeDimensions(Tensor.scala:63) at ml.combust.mleap.tensor.Tensor$.create(Tensor.scala:33) at ml.combust.bundle.tensor.TensorSerializer$.fromProto(TensorSerializer.scala:74) at ml.combust.bundle.dsl.Value.getTensor(Value.scala:323) ``` Possible cause In `ml.combust.bundle.tensor.TensorSerializer$.toProto`, we save `rawValue`, But in...
Current implementation always will convert Tensor to Vector Bug is hidden in `tt.dimensions.size` where `tt.dimensions` is `Option[Seq[Int]]`, so calling `size` on Some will have size of 1 and calling `size`...
sinopia@882d14f30a1b:/opt/sinopia$ npm -v sinopia version 2.14.4 fatal --- uncaught exception, please report this Error: not implemented at Readable._read (/opt/sinopia/node_modules/sinopia/node_modules/readable-stream/lib/_stream_readable.js:496:22) at Readable.read (/opt/sinopia/node_modules/sinopia/node_modules/readable-stream/lib/_stream_readable.js:373:10) at Readable.resume (/opt/sinopia/node_modules/sinopia/node_modules/readable-stream/lib/_stream_readable.js:742:12) at Readable.on (/opt/sinopia/node_modules/sinopia/node_modules/readable-stream/lib/_stream_readable.js:708:10) at Storage.get_url...
Hi, Your multi-threading implementation give me a lot of inspirations for how to use the estimator API. After some dig-in, I found there is way to use only a simple...