datacontract-cli icon indicating copy to clipboard operation
datacontract-cli copied to clipboard

Bugfix import avro with logicalType and default values

Open enriquecatala opened this issue 1 year ago • 2 comments

If the avro being imported contains default or logicalType, the datacontract definition skips it on both sides. This bug fix will enable importing avro definitions that support logicalType and/or default values for fields.

enriquecatala avatar May 08 '24 15:05 enriquecatala

Hi @enriquecatala, we'll need to postpone this PR a bit, until https://github.com/datacontract/datacontract-specification/issues/63 is decided, how we handle server-specific attributes, such as avroLogialTypes. Feel free to contribute your requirements and thoughts there.

jochenchrist avatar May 08 '24 16:05 jochenchrist

We can continue, as https://github.com/datacontract/datacontract-specification/issues/63 is decided:

The Data Contract now should look like this:

dataContractSpecification: 0.9.3
id: my-data-contract-id
info:
  title: My Data Contract
  version: 0.0.1
models:
  Test:
    config:
      avroNamespace: mynamespace.com
    fields:
      test_id:
        type: string
        required: true
        description: id documentation test
      device_id:
        type: int
        required: true
      test_value:
        type: double
        required: true
      num_items:
        type: int
        required: true
      processed_timestamp:
        type: timestamp_ntz
        required: true
        description: 'The date the event was processed: for more info https://avro.apache.org/docs/current/spec.html#Local+timestamp+%28microsecond+precision%29'
        config:
          avroType: long
          avroLogicalType: local-timestamp-micros
      description:
        type: string
        required: true
      is_processed:
        type: boolean
        required: true
        config:
          avroDefault: false 

jochenchrist avatar May 16 '24 15:05 jochenchrist

i´ll close the PR since changes are totally different. I´ll try to implement this feature with new specification

enriquecatala avatar May 22 '24 21:05 enriquecatala