Statistics.jl icon indicating copy to clipboard operation
Statistics.jl copied to clipboard

Inconsistencies in cor calculation

Open bkamins opened this issue 5 years ago • 2 comments

Here are some cor corner cases that would be good to fix by 2.0 release of Julia:

julia> cor(Union{Float64, Missing}[missing]) # this is the expected result
1.0

julia> cor([missing])
missing

julia> cor(Any[missing])
ERROR: MethodError: no method matching zero(::Type{Any})

julia> cor(Any[1])
ERROR: MethodError: no method matching zero(::Type{Any})

EDIT: this is based on the assumption that if someone wants cor and has missing this missing represents a number (and not e.g. a string)

bkamins avatar Jan 03 '21 18:01 bkamins

Here is another edge case (with Statistics1,10.0 and Julia 1.10)

julia> cor(fill(missing,2,2))
ERROR: MethodError: no method matching copy(::Missing)

julia> cor(fill(missing,3,3))
ERROR: MethodError: no method matching copy(::Missing)

julia> cor(fill(missing,4,4)) #this is the expected result
4×4 Matrix{Missing}:
 missing  missing  missing  missing
 missing  missing  missing  missing
 missing  missing  missing  missing
 missing  missing  missing  missing

PGS62 avatar Feb 17 '24 15:02 PGS62

julia> cor(fill(missing,4,4)) #this is the expected result
4×4 Matrix{Missing}:
 missing  missing  missing  missing
 missing  missing  missing  missing
 missing  missing  missing  missing
 missing  missing  missing  missing

Is that the expected result? Or should it be akin to the NaN:

julia> cor(fill(NaN,4,4))
4×4 Matrix{Float64}:
   1.0  NaN    NaN    NaN
 NaN      1.0  NaN    NaN
 NaN    NaN      1.0  NaN
 NaN    NaN    NaN      1.0

mbauman avatar Aug 27 '24 21:08 mbauman