Unexpected table schema

[baumgold]

It appears that Arrow.jl somehow incorrectly handles columns with nullable types (Union with Nothing) as the schema of the resulting table containing such a column doesn't match what's reported by Tables.schema.

For example, below is an example with a column of type Union{Int64,Nothing} that gets somehow interpreted by Arrow.jl as Union{Missing, Nothing, Int64} (should not include Missing).

julia> using Arrow

julia> tbl = Arrow.Table(Arrow.tobuffer((nothingints=Vector{Union{Int64,Missing}}([1,2,3,missing]),)))
Arrow.Table with 4 rows, 1 columns, and schema:
 :nothingints  Union{Missing, Int64}

julia> tbl = Arrow.Table(Arrow.tobuffer((nothingints=Vector{Union{Int64,Nothing}}([1,2,3,nothing]),)))
Arrow.Table with 4 rows, 1 columns, and schema:
 :nothingints  Union{Missing, Nothing, Int64}

julia> using Tables

julia> Tables.schema((nothingints=Vector{Union{Int64,Missing}}([1,2,3,missing]),))
Tables.Schema:
 :nothingints  Union{Missing, Int64}

julia> Tables.schema((nothingints=Vector{Union{Int64,Nothing}}([1,2,3,nothing]),))
Tables.Schema:
 :nothingints  Union{Nothing, Int64}