Add support for spring.config.import with S3 backend

[celcius112]

Type: Feature

First of all, as a disclaimer, I am not sure whether this feature request should be in this repo or in https://github.com/spring-cloud/spring-cloud-config, as there is already a S3 implementation for the legacy bootstrap.

Similar to the parameter store and secrets manager configuration, it would be great if AWS S3 could be used as external configuration backend, such as:

spring:
  config:
    import: aws-s3:my-bucket

At the moment, spring-cloud-config does not seem to propose Spring Boot's spring.config.import property for S3. Since using spring.config.import with parameter-store-config and secrets-manager-config simultaneously with the legacy bootstrap with s3 does not seem to work out of the box (AwsSecretsManagerPropertySourceLocator and AwsSecretsManagerConfigDataLocationResolver are both created simultaneously), it is at the moment a blocking point for us for migrating to the new spring.config.import and correcting https://github.com/awspring/spring-cloud-aws/issues/159.

[MatejNedic]

Hey @celcius112 , S3 module currently doesn't support spring.config.import . Could you please explain in greater detail what would you expect from this feature to support?

[celcius112]

Hello @MatejNedic,

At the moment, spring-cloud-config-server proposes a S3 module that retrieves .properties, .yml or .json files from a S3 bucket. Using the legacy bootstrap the properties are imported into the application environment.

I have implemented what spring-cloud-config-server proposes, but using Spring Boot's spring.config.import:

spring:
  config:
    import:
      - aws-s3:bucket-name-1;file1.yml
      - aws-s3:bucket-name-1;file2.yml

I don't think this module should have much more functionality than that, except maybe having a way to override the S3Client using the BootstrapContext.

I guess the usage would be similar to spring-cloud-aws's parameter-store and secrets-manager starters.

[kunalvarpe]

@maciejwalkowiak @MatejNedic, I would like to contribute here, I have done some changes in my local. Before pushing that I have some queries/doubts.

• As per above comment the expectation is spring confing import should load properties from file.yml and file2.yml from the given buckets.
• What if file has profile mentioned with it, in such situation should we respet the profile as well. e.g.

spring:
  config:
    import:
     - aws-s3:bucket-name-1;file1-test.yml
     - aws-s3:bucket-name-1;file2.yml
     - aws-s3:bucket-name-2;file3-integration.yml

• What would be other usecases here?

[maciejwalkowiak]

@kunalvarpe good question. It should be consistent with other property sources, for example - if you use spring.config.import with a filesystem location, does it respect profiles? It should work the same with S3 location.

[kunalvarpe]

Hi @maciejwalkowiak I have push the initial commit to the PR #849. Could you please let me know if I am doing the changes in correct direction, also I would be needed some more guidence on respecting the profiles support.

[vishal94136]

Hello @MatejNedic,

At the moment, spring-cloud-config-server proposes a S3 module that retrieves .properties, .yml or .json files from a S3 bucket. Using the legacy bootstrap the properties are imported into the application environment.

I have implemented what spring-cloud-config-server proposes, but using Spring Boot's spring.config.import:

spring:
  config:
    import:
      - aws-s3:bucket-name-1;file1.yml
      - aws-s3:bucket-name-1;file2.yml

I don't think this module should have much more functionality than that, except maybe having a way to override the S3Client using the BootstrapContext.

I guess the usage would be similar to spring-cloud-aws's parameter-store and secrets-manager starters.

is it possible to have aws s3 config server without legacy bootstrap?

[maciejwalkowiak]

@vishal94136 we have S3 spring.config.import in progress here: https://github.com/awspring/spring-cloud-aws/pull/849. It will be included in 3.2.0. Spring Boot's "bootstrap" phase is not legacy - this is the new way of handling such things. What's legacy is using Spring Cloud Bootstrap 🙃

[vishal94136]

@maciejwalkowiak thanks a lot for replying and sorry I sentenced it wrong I meant the spring cloud bootstrap.