Ingestion Task Shows SUCCESS status, but Pushed 0 segments

[mounikanakkala]

Affected Version

0.22.1.61

Description

Hi Everyone,

We have observed a scenario where the Ingestion Task Shows SUCCESS status but the logs showed that it pushed 0 segments.

Background We have the following inputFormat

"inputFormat":{
            "type":"tsv",
            "findColumnsFromHeader":true,
            "delimiter": "|"
         }

And our input files are of the extension .csv.gz usually. This extension works fine as expected (meaning ingestion succeeds and segments are created)

Problem We were accidentally creating files without the .csv.gz extension. That's when the ingestion status showed SUCCESS but the logs showed that it pushed 0 segments and 0 bytes. Is that an expected behavior?

Can Druid fail or provide a different Status instead of SUCCESS when it did not push any segments because current behavior is misleading.

[a2l007]

It's possible that Druid ignored the unparseable rows. Please check the task reports corresponding to the ingestion task to see if there were unparseable records encountered. If you require tasks to fail on unparseable rows, an option is to set reportParseExceptions to true in your ingestion tuningConfig.

[github-actions[bot]]

This issue has been marked as stale due to 280 days of inactivity. It will be closed in 4 weeks if no further activity occurs. If this issue is still relevant, please simply write any comment. Even if closed, you can still revive the issue at any time or discuss it on the [email protected] list. Thank you for your contributions.

[github-actions[bot]]

This issue has been closed due to lack of activity. If you think that is incorrect, or the issue requires additional review, you can revive the issue at any time.