td-agent throwing error - 2023-05-22 14:08:44 +0000 [error]: config error file="/etc/td-agent/td-agent.conf" error_class=Fluent::ConfigError error="'columnify' utility must be in PATH for -h compression"

Open DivyaYash opened this issue 2 years ago • 3 comments

2023-05-22 14:08:44 +0000 [error]: config error file="/etc/td-agent/td-agent.conf" error_class=Fluent::ConfigError error="'columnify' utility must be in PATH for -h compression"

which columnify

/root/go/bin/columnify

columnify -h

Usage of columnify: columnify [-flags] [input files] -output string path to output file; default: stdout -parquetCompressionCodec string parquet compression codec, default: SNAPPY (default "SNAPPY") -parquetPageSize int parquet file page size, default: 8kB (default 8192) -parquetRowGroupSize int parquet file row group size, default: 128MB (default 134217728) -recordType string record data format type, [avro|csv|jsonl|ltsv|msgpack|tsv] (default "jsonl") -schemaFile string path to schema file -schemaType string schema type, [avro|bigquery]

May 22 '23 14:05 DivyaYash

Install Columnify on the PATH where the Fluentd process can be found. Alternatively, add the path where you installed Columnify to your Fluentd PATH.

Sep 25 '23 01:09 okkez

@DivyaYash @okkez how did you resolved this issue ? How I can add package path to fluentd path

Jan 13 '24 07:01 shubhdeep22

@shubhdeep22 Read the documents carefully, please.

https://docs.treasuredata.com/display/public/PD/td-agent+Reference
https://docs.fluentd.org/

Jan 19 '24 07:01 okkez