spring-batch icon indicating copy to clipboard operation
spring-batch copied to clipboard

Proper default value for skipLimit

Open samuelstein opened this issue 1 year ago • 3 comments

Recently I stumbled over the default value of the skipLimit in the class FaultTolerantStepBuilder. It is set by default to 0. If you don't use a SkipPolicy and configure it instead with skip(Exception.class) it is very easy to forget setting the additional property skipLimit. Could you provide a more suitable default value? My suggestion would be Integer.MAX_VALUE. What do you think?

samuelstein avatar Sep 12 '24 13:09 samuelstein

That's a valid point, thank you for raising it!

My suggestion would be Integer.MAX_VALUE

This means that by default we would theoretically allow the entire dataset to be skipped, which in turn means there is something fundamentally wrong with the input (bad data format, unexpected input, etc). As a user, I would prefer my job to fail fast in the first dozen or hundred skips rather than waiting until the end just to discover that my job was skipping the entire data set. Do you see my point?

I guess 10 or 100 is a reasonable default. Wdyt?

I will plan the change in the next minor release.

fmbenhassine avatar Sep 16 '24 06:09 fmbenhassine

Hi @fmbenhassine, this makes abolutely sense to me. I would prefer at least 10. Then you have a good balance between fail-fast and misconfiguration.

samuelstein avatar Sep 17 '24 07:09 samuelstein

I have opened a pull request that addresses this issue: #4668

Ian3110 avatar Sep 22 '24 07:09 Ian3110