Enable Writer to ingest array values
Currently all array values in DataFrame will be converted into string, but it's not ideal.
The most challenging part of this topic is in InsertIntoWriter since it requires carefully escaping quotes of array elements.
To make the situation simpler, it might be okay to support array column only in BulkImportWriter and SparkWriter first of all, because these writers directly load DataFrame / CSV file and automatically cares the type matters within the data format. This makes the behavior of BulkImportWriter / SparkWriter vs. InsertIntoWriter inconsistent though.
cc: @chezou
Or, we could have a way to update schema after uploading the dataframe.
Start implementation for BulkImportWriter.
Concerns for other writers are:
- InsertIntoWriter requires tons of SQL escape
- Introducing list handling in SparkWriter requires handling
ArrayTypewhich can be confusing for spark unfamiliar users.