data-prepper icon indicating copy to clipboard operation
data-prepper copied to clipboard

Consolidate processor configurations for actions when there're key conflicts

Open oeyh opened this issue 2 years ago • 0 comments

Is your feature request related to a problem? Please describe. These processors have similar overwrite_if_key_exists options:

  • add_entries
  • copy_values
  • rename_keys
  • parse_json
  • parse_ion
  • key_value

grok processor has a keys_to_overwrite option so that you can determine to overwrite or append (default) for each grok generated field.

There are 3 basic options when adding fields but the key already exists: overwrite, append or skip. We should consider making the options available and consistent throughout processors.

Describe the solution you'd like A clear and concise description of what you want to happen.

Describe alternatives you've considered (Optional) A clear and concise description of any alternative solutions or features you've considered.

Additional context See comments from https://github.com/opensearch-project/data-prepper/pull/4143

oeyh avatar Feb 22 '24 21:02 oeyh