aws S3 streaming upload random naming strategy always overwrite the initial file (without suffix)

[jingwang]

When choosing namingStrategy(AWSS3NamingStrategyEnum.random) and restartingPolicy(AWSS3RestartingPolicyEnum.lastPart), the very first file without any UUID suffix is always overwritten when the application restarts. One would expect in this case it would start with a file with a new UUID suffix.

version:

        <camel-quarkus.version>2.11.0</camel-quarkus.version>
        <quarkus-plugin.version>2.11.1.Final</quarkus-plugin.version>
        <quarkus.platform.artifact-id>quarkus-bom</quarkus.platform.artifact-id>
        <quarkus.platform.group-id>io.quarkus</quarkus.platform.group-id>
        <quarkus.platform.version>2.11.1.Final</quarkus.platform.version>

[zhfeng]

@jingwang Thanks for reporting! - can you provide a reproducer or unit test for this issue? I will take a look.

[jingwang]

Thanks @zhfeng . Here is the test with docker-compose using localstack:

docker-compose.yaml

version: '3.8'
services:
  localstack:
    image: localstack/localstack:latest
    environment:
      - AWS_DEFAULT_REGION=us-west-2
      - EDGE_PORT=4566
      - SERVICES=s3
    ports:
      - '4566-4583:4566-4583'

Aws2S3Route.java

import org.apache.camel.builder.endpoint.EndpointRouteBuilder;
import org.apache.camel.builder.endpoint.dsl.AWS2S3EndpointBuilderFactory;
import org.apache.camel.component.aws2.s3.stream.AWSS3NamingStrategyEnum;
import org.apache.camel.component.aws2.s3.stream.AWSS3RestartingPolicyEnum;
import javax.enterprise.context.ApplicationScoped;

@ApplicationScoped
public class Aws2S3Route extends EndpointRouteBuilder {
    @Override
    public void configure() throws Exception {

            AWS2S3EndpointBuilderFactory.AWS2S3EndpointProducerBuilder builder = aws2S3("bucket1")
                    .region("us-west-2")
                    .autoCreateBucket(true)
                    .useDefaultCredentialsProvider(true)
                    .streamingUploadMode(true)
                    .overrideEndpoint(true)
                    .uriEndpointOverride("http://localhost:4566")
                    .batchMessageNumber(2)
                    .streamingUploadTimeout(5000)
                    .namingStrategy(AWSS3NamingStrategyEnum.random)
                    .restartingPolicy(AWSS3RestartingPolicyEnum.lastPart)
                    .keyName("file.txt");

            from(cron("test").schedule("0/5 * * * * ?"))
                    .setBody(constant("test message"))
                    .to(builder);
    }
}

pom.xml

...
        <camel-quarkus.version>2.11.0</camel-quarkus.version>
        <quarkus-plugin.version>2.11.1.Final</quarkus-plugin.version>
        <quarkus.platform.artifact-id>quarkus-bom</quarkus.platform.artifact-id>
        <quarkus.platform.group-id>io.quarkus</quarkus.platform.group-id>
        <quarkus.platform.version>2.11.1.Final</quarkus.platform.version>
...
        <dependency>
            <groupId>org.apache.camel.quarkus</groupId>
            <artifactId>camel-quarkus-aws2-s3</artifactId>
        </dependency>

        <dependency>
            <groupId>org.apache.camel.quarkus</groupId>
            <artifactId>camel-quarkus-cron</artifactId>
        </dependency>

Steps to reproduce:

• Start docker-compose

> docker-compose up

• On a separate tab, start application

> export AWS_ACCESS_KEY_ID=accesskey
> export AWS_SECRET_ACCESS_KEY=secretkey
> mvn clean quarkus:dev

• On a separate tab, monitor the s3 bucket and check the timestamp of file.txt

aws --endpoint-url=http://localhost:4566 s3 ls s3://bucket1 --recursive

Expected: When the application is restarted, the timestamp of file.txt should not change (i.e., it is not overwritten) Actual: Every time when the application is restarted, the timestamp of file.txt changes.