Boolean column with all nulls causes `npm run sources` sources to fail

[archiewood]

Steps To Reproduce

Create a source query with a boolean column containing only nulls

select null::bool as country, 100 as sales 
union all 
select null::bool as country, 200 as sales
union all 
select null::bool as country, 300 as sales

Environment

• Database: postgres, duckdb (assume also others but not tested)

Expected Behavior

A column with nulls should be returned

Actual Behaviour

Invalid argument error: Need at least 1 bytes for bitmap in buffers[0] in array of type Boolean, but got 0

Workarounds

Don't select this column with nulls as it is contains no information anyway

[mcrascal]

Only occurs on bool columns

[csjh]

https://github.com/apache/arrow-rs/tree/master/arrow-data/src/data.rs#L778

Comes from arrow-rs, which is a dependency of parquet-wasm, so it's something in multipart parquet creation

[csjh]

Awaiting apache/arrow#39776

[mcrascal]

@csjh in the meantime can we provide something more informative in the error message?

[csjh]

Yeah, but something that just occurred to me is that we're still pinned on version 11.0.0, so even if it is fixed upstream we'll still have the faulty version (at least until duckdb-wasm bumps their versions?). Not sure what the best message to put in the error/warning would be considering that.

[csjh]

How bad would it be to just convert boolean columns to true/false? Or maybe having a special case when the column is fully null?

[archiewood]

Special case when column is fully null feels not too bad.

I'd probably advocate for one of the following

1. Error Message along the lines of:

• Error: Column "column_name" is of type Boolean and contains only null values, which is not allowed. Consider removing the column from the query or casting it to another type

2. Cast it to a string by default, and print a warning:

• Warning: Column "column_name" (type Boolean) contains only null values so has been cast to varchar

3. Fill with false by default, and print a warning:

• Warning: Column "column_name" (type Boolean) contains only null values so has been infilled with false values.

[archiewood]

@csjh is this done?

[csjh]

Fixed by #1521