go icon indicating copy to clipboard operation
go copied to clipboard
verified publisher icon stellar

services/galexie: add AWS S3 datastore plugin

Open hawkaa opened this issue 8 months ago • 3 comments

PR Checklist

PR Structure

  • [x] This PR has reasonably narrow scope (if not, break it down into smaller PRs).
  • [x] This PR avoids mixing refactoring changes with feature changes (split into two PRs otherwise).
  • [x] This PR's title starts with name of package that is most changed in the PR, ex. services/friendbot, or all or doc if the changes are broad or impact many packages.

Thoroughness

  • [ ] This PR adds tests for the most critical parts of the new functionality or fixes.
  • [x] I've updated any docs (developer docs, .md files, etc... affected by this change). Take a look in the docs folder for a given service, like this one.

Release planning

  • [x] I've reviewed the changes in this PR and if I consider them worthwhile for being mentioned on release notes then I have updated the relevant CHANGELOG.md within the component folder structure. For example, if I changed horizon, then I updated (services/horizon/CHANGELOG.md. I add a new line item describing the change and reference to this PR. If I don't update a CHANGELOG, I acknowledge this PR's change may not be mentioned in future release notes.
  • [ ] I've decided if this PR requires a new major/minor version according to semver, or if it's mainly a patch change. The PR is targeted at the next release branch if it's not a patch change.

What

Add AWS S3 pluggable DataStore backend for Galexie, alongside existing GCS support.
Update developer documentation and example config to reflect selection and configuration of both backends.

Why

Adds broader cloud support for Galexie users and clarifies how to configure and extend storage backends with concise, improved documentation.

Known limitations

  • S3 feature set covers main workflow, but advanced options (e.g. custom endpoints, advanced KMS) aren’t comprehensively tested.
  • Authored and contributed by a Go newcomer—please review for typical Go best practices and plugin idioms.

hawkaa avatar May 22 '25 10:05 hawkaa

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security		 Vulnerability Quality Maintenance License
Addedgithub.com/​aws/​aws-sdk-go-v2@​v1.36.371100100100100
Addedgithub.com/​aws/​smithy-go@​v1.22.372100100100100
Addedgithub.com/​aws/​aws-sdk-go-v2/​config@​v1.29.1487100100100100
Addedgithub.com/​aws/​aws-sdk-go-v2/​service/​s3@​v1.79.394100100100100
Addedgithub.com/​aws/​aws-sdk-go-v2/​feature/​s3/​manager@​v1.17.7596100100100100

View full report

socket-security[bot] avatar May 22 '25 10:05 socket-security[bot]

We have tests for the GCS datastore which use a GCS emulator to test the end to end behavior of the datastore operations. I think we can take a similar approach and use minio as an s3 servicefor end to end testing.

tamirms avatar May 22 '25 21:05 tamirms

Hi everyone, and thank you very much for the detailed review! I haven't forgotten about this PR, I've just prioritized differently. Will find some time to address the comments in the PR and adjust the code accordingly :)

hawkaa avatar May 27 '25 06:05 hawkaa

I was planning to address the comments in this PR, but I realized that s3 support was already implemented by someone else. Thanks for your time anyway!

hawkaa avatar Jul 29 '25 08:07 hawkaa

Hi @hawkaa, I saw you closed this PR yourself, and I really appreciate your understanding.

I still want to sincerely apologize. I was in a rush for this feature and completely missed the great work you had already done here. I feel bad that your effort was duplicated.

Your approach was very insightful. Thanks again for everything! ♥️

overcat avatar Jul 29 '25 10:07 overcat

Hi, absolutely nothing to apologize for!! I discarded my change in my fork and started using what you did and it worked splendidly! Perfect outcome.

On Tue, Jul 29, 2025 at 12:06 PM Jun Luo @.***> wrote:

overcat left a comment (stellar/go#5708) https://github.com/stellar/go/pull/5708#issuecomment-3131700213

Hi @hawkaa https://github.com/hawkaa, I saw you closed this PR yourself, and I really appreciate your understanding.

I still want to sincerely apologize. I was in a rush for this feature and completely missed the great work you had already done here. I feel bad that your effort was duplicated.

Your approach was very insightful. Thanks again for everything! ♥️

— Reply to this email directly, view it on GitHub https://github.com/stellar/go/pull/5708#issuecomment-3131700213, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA4INSE2LYBFOBVM2OM5O3L3K5BQXAVCNFSM6AAAAAB5VUHOJ6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTCMZRG4YDAMRRGM . You are receiving this because you were mentioned.Message ID: @.***>

hawkaa avatar Jul 29 '25 10:07 hawkaa