miceforest

miceforest copied to clipboard

Imputed values differ after a kernel is loaded

7

Hi @AnotherSamWilson, The related issue is the fact that for large dataset (my case at least) the imputed values for the same dataset differ before (trained kernel) and after (that...

KaikeWesleyReis

See if tree compilation will speed up impute_new_data

1

https://github.com/siboehm/lleaves

AnotherSamWilson

Right now random state management is all over the place, and inelegant, especially inside the default mean matching functions. See if most random processes can be switched over to use...

samFarrellDay

lightgbm can handle the following. No reason we can't add: H2O DataTable's Frame scipy.sparse Look into lightgbm.Sequence... might be no point.

samFarrellDay

Actually storing the latest imputation values can take up a lot of memory. We have all the information we need to generate imputation values when complete_data() is called, why not...

samFarrellDay

Process finished with exit code -1073741571 (0xC00000FD)

3

Getting this reliably. Chased it down to scipy.Spatial.KDtree. Changing leafsize doesn't help. candidate_preds is float64, shape (294695, 1).

samFarrellDay

For correlations, could be the % of matching imputed categories. For distributions, could be a bar/boxplot (depending on datasets?) of the histograms values.

samFarrellDay

Have had good experience with Bayesian optimization in the past. Lightweight implementation: https://github.com/fmfn/BayesianOptimization

samFarrellDay

plot_imputed_distributions() fails with LinAlgError: Singular Matrix

3

Hello, Hope you are doing well!! I was working with MiceForest on a toy dataset to understand how it works. And, during that came across "LinAlgError: singular matrix" while generating...

raj-shr-git

AnotherSamWilson