NearestNeighborModels.jl icon indicating copy to clipboard operation
NearestNeighborModels.jl copied to clipboard

[testing] check results are coherent vs sklearn when using sample weights

Open tlienart opened this issue 6 years ago • 0 comments

We've recently added (https://github.com/alan-turing-institute/MLJModels.jl/pull/125) the possibility to add weights to samples in KNNC, KNNR. It seems fine but it would still be good to check this a bit more and ideally against an external benchmark like Sklearn which I believe supports sample weights as well.

Steps:

  • ~~be on the dev branch of MLJModels~~ edit This now lives at NearestNeighborModels (current repo)
  • generate some dummy data with dummy weights (see also examples in tests for NearestNeighbors though it'd be better to use less dumb data where classes overlap a bit)
  • save the data and do the same analysis in sklearn
  • check that the results look roughly similar (like accuracy within +- 5%)

tlienart avatar Nov 11 '19 08:11 tlienart