fluent-plugin-anonymizer icon indicating copy to clipboard operation
fluent-plugin-anonymizer copied to clipboard

Fix config sample in README

Open pavdmyt opened this issue 5 years ago • 0 comments

I was not able to mask IPv4 or IPv6 address using ipv4_mask_subnet and ipv6_mask_subnet parameters from sample config. Here is the output:

$ docker run -ti --rm -v $PWD:/fluentd/etc fluentd-with-anonymizer:1 -c /fluentd/etc/fluent.conf             
fluentd -c /fluentd/etc/fluent.conf
2020-06-16 14:57:58 +0000 [info]: parsing config file is succeeded path="/fluentd/etc/fluent.conf"
2020-06-16 14:57:58 +0000 [info]: gem 'fluent-mixin-rewrite-tag-name' version '0.1.0'
2020-06-16 14:57:58 +0000 [info]: gem 'fluent-plugin-anonymizer' version '1.0.0'
2020-06-16 14:57:58 +0000 [info]: gem 'fluentd' version '1.11.0'
2020-06-16 14:57:58 +0000 [warn]: both of Plugin @id and path for <storage> are not specified. Using on-memory store.
2020-06-16 14:57:58 +0000 [warn]: both of Plugin @id and path for <storage> are not specified. Using on-memory store.
2020-06-16 14:57:58 +0000 [info]: using configuration file: <ROOT>
  <source>
    @type dummy
    tag "raw.dummy"
    dummy [{"host":"10.102.3.80","member_id":"12345","mail":"[email protected]"},{"host":"2001:db8:0:8d3:0:8a2e::","member_id":"61f6c1b5f19e0a7f73dd52a23534085bf01f2c67","mail":"eeb890d74b8c1c4cd1e35a3ea62166e0b770f4f4"}]
  </source>
  <filter raw.**>
    @type anonymizer
    <mask sha1>
      keys user_id, member_id, mail
      salt "mysaltstring"
    </mask>
    <mask network>
      keys host
      ipv4_mask_subnet 24
      ipv6_mask_subnet 104
    </mask>
  </filter>
  <match raw.**>
    @type stdout
  </match>
</ROOT>
2020-06-16 14:57:58 +0000 [info]: starting fluentd-1.11.0 pid=7 ruby="2.5.8"
2020-06-16 14:57:58 +0000 [info]: spawn command to main:  cmdline=["/usr/bin/ruby", "-Eascii-8bit:ascii-8bit", "/usr/bin/fluentd", "-c", "/fluentd/etc/fluent.conf", "-p", "/fluentd/plugins", "--under-supervisor"]
2020-06-16 14:57:59 +0000 [info]: adding filter pattern="raw.**" type="anonymizer"
2020-06-16 14:57:59 +0000 [info]: adding match pattern="raw.**" type="stdout"
2020-06-16 14:57:59 +0000 [info]: adding source type="dummy"
2020-06-16 14:57:59 +0000 [warn]: #0 both of Plugin @id and path for <storage> are not specified. Using on-memory store.
2020-06-16 14:57:59 +0000 [warn]: #0 both of Plugin @id and path for <storage> are not specified. Using on-memory store.
2020-06-16 14:57:59 +0000 [warn]: parameter 'ipv4_mask_subnet' in <mask network>
  keys host
  ipv4_mask_subnet 24
  ipv6_mask_subnet 104
</mask> is not used.
2020-06-16 14:57:59 +0000 [warn]: parameter 'ipv6_mask_subnet' in <mask network>
  keys host
  ipv4_mask_subnet 24
  ipv6_mask_subnet 104
</mask> is not used.
2020-06-16 14:57:59 +0000 [info]: #0 starting fluentd worker pid=21 ppid=7 worker=0
2020-06-16 14:57:59 +0000 [info]: #0 fluentd worker is now running worker=0
2020-06-16 14:58:00.044048270 +0000 raw.dummy: {"host":"10.102.3.80","member_id":"5ab2cebb0537866c4a0cd2e2f3502c0976b788da","mail":"7e9d6dbefa72d56056c8c740b34b5c0bbfec8d87"}
2020-06-16 14:58:01.061253723 +0000 raw.dummy: {"host":"2001:db8:0:8d3:0:8a2e::","member_id":"445514dfcd82b2a8b94ec6763afa6e349e78c5f8","mail":"54608576c8d815a4ffd595a3c1fe72751ed04424"}
2020-06-16 14:58:02.087428669 +0000 raw.dummy: {"host":"10.102.3.80","member_id":"b14a8f98019ec84c6fe329d5af62c46bb45348f8","mail":"723da8084da3438d9287b44e5a714b70e10a9755"}
2020-06-16 14:58:03.003302007 +0000 raw.dummy: {"host":"2001:db8:0:8d3:0:8a2e::","member_id":"d38ebb9b96c0cbffd4136935c7f6fe9dd05980cd","mail":"b6f9d777831cbecfd2ea806f5f62f79a275bbb82"}
^C2020-06-16 14:58:03 +0000 [info]: Received graceful stop

fluentd-with-anonymizer is a fluent/fluentd:v1.11-1 with fluent-plugin-anonymizer pre-installed.

pavdmyt avatar Jun 16 '20 15:06 pavdmyt