filter_modify: Add data type condition checker to `Modify` filter.
Summary
It would be good if Fluent Bit supports strong data type condition check when applied in structured log stream. I implemented a data type condition checker for Modify filter.
Check data type using lua filter isn't clear enough for end-user and costs more performance overhead.
Usage
Modify filter supports two new conditions. See modify_filter_example.conf below.
| Condition | Parameter 1 | Parameter 2 | Description |
|---|---|---|---|
| Key_Value_Type_Matches | STRING:KEY | STRING:VALUE | Is true if KEY exists and its value data type is VALUE. |
| Key_Value_Type_Does_Not_Match | STRING:KEY | STRING:VALUE | Is true if KEY exists and its value data type is not VALUE. |
Parameter 2 data type string literal could be:
-
null -
bool -
number -
int -
float -
str -
array -
map
[INPUT]
Name dummy
Tag test
dummy {"nullType": null, "boolType": true, "intType": 123, "floatType": 3.14159, "strType": "awesome!", "arrayType": [1, 2, 3], "mapType": {"k": "v"}}
[OUTPUT]
Name stdout
Match *
[FILTER]
Name modify
Match *
Condition Key_Value_Type_Matches nullType null
Condition Key_Value_Type_Matches boolType bool
Condition Key_Value_Type_Matches intType number
Condition Key_Value_Type_Matches intType int
Condition Key_Value_Type_Matches floatType float
Condition Key_Value_Type_Matches strType str
Condition Key_Value_Type_Matches arrayType array
Condition Key_Value_Type_Matches mapType map
Condition Key_Value_Type_Does_Not_Match mapType array
Rename nullType nullTypeMatches
Rename mapType mapTypeDoesNotMatch
modify_filter_example.conf
Unit Tests
Here comes unit testing results.
13/31 Test #13: flb-rt-filter_kubernetes ......... Passed 193.00 sec
Start 14: flb-rt-filter_parser
14/31 Test #14: flb-rt-filter_parser ............. Passed 20.00 sec
Start 15: flb-rt-filter_modify
15/31 Test #15: flb-rt-filter_modify ............. Passed 55.00 sec
Start 16: flb-rt-core_engine
16/31 Test #16: flb-rt-core_engine ............... Passed 20.00 sec
Addresses
#2530
Enter [N/A] in the box, if an item is not applicable to your change.
Testing Before we can approve your change; please submit the following in a comment:
- [x] Example configuration file for the change
- [x] Debug log output from testing the change
- [x] Attached Valgrind output that shows no leaks or memory corruption was found
Documentation
- [x] Documentation required for this feature
Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.
thanks for this PR.
- is it
nila configuration value ?, if so, can it benull? , that's more representative for end-users - does
numbercoversintandfloat?
@edsiper Fine.
- Change
niltonull. -
numbercovers bothintandfloat, corresponding to JSON number data type.
Is there any chance this might eventually be merged?