Bugfix import avro with logicalType and default values
If the avro being imported contains default or logicalType, the datacontract definition skips it on both sides. This bug fix will enable importing avro definitions that support logicalType and/or default values for fields.
Hi @enriquecatala, we'll need to postpone this PR a bit, until https://github.com/datacontract/datacontract-specification/issues/63 is decided, how we handle server-specific attributes, such as avroLogialTypes. Feel free to contribute your requirements and thoughts there.
We can continue, as https://github.com/datacontract/datacontract-specification/issues/63 is decided:
The Data Contract now should look like this:
dataContractSpecification: 0.9.3
id: my-data-contract-id
info:
title: My Data Contract
version: 0.0.1
models:
Test:
config:
avroNamespace: mynamespace.com
fields:
test_id:
type: string
required: true
description: id documentation test
device_id:
type: int
required: true
test_value:
type: double
required: true
num_items:
type: int
required: true
processed_timestamp:
type: timestamp_ntz
required: true
description: 'The date the event was processed: for more info https://avro.apache.org/docs/current/spec.html#Local+timestamp+%28microsecond+precision%29'
config:
avroType: long
avroLogicalType: local-timestamp-micros
description:
type: string
required: true
is_processed:
type: boolean
required: true
config:
avroDefault: false
i´ll close the PR since changes are totally different. I´ll try to implement this feature with new specification