iceberg-rust icon indicating copy to clipboard operation
iceberg-rust copied to clipboard

REST catalog: support AWS sigV4

Open xxchan opened this issue 9 months ago • 8 comments

Is your feature request related to a problem or challenge?

For AWS glue/s3 tables

rest.sigv4-enabled, rest.signing-name, rest.signing-region

https://github.com/apache/iceberg-python/issues/1405

rest_catalog = load_catalog(
  catalog_name,
  **{
    "type": "rest",    
    "warehouse":"arn:aws:s3tables:<Region>:<accountID>:bucket/<bucketname>",
    "uri": "https://s3tables.<Region>.amazonaws.com/iceberg",
    "rest.sigv4-enabled": "true",
    "rest.signing-name": "s3tables",
    "rest.signing-region": "<Region>"
  }
)                

Describe the solution you'd like

No response

Willingness to contribute

None

xxchan avatar Apr 23 '25 08:04 xxchan

Found a problem in reqsign:

  1. Reqsign got canonical request like: canonical request: GET /iceberg/v1/arn%3Aaws%3As3tables%3Aap-southeast-1..., but should have been sth like /iceberg/v1/arn%253Aaws%253As3tables%253Aap-southeast-1%..., where reqsign wrongly decoded %3.
  2. x-amz-content-sha256 seems to be required for this endpoint, which is not natively supported by reqsign.

xxchan avatar Apr 23 '25 10:04 xxchan

@xxchan I'd like to work on this.

ananthaksr avatar Apr 23 '25 20:04 ananthaksr

@ananthaksr Hi, thanks for offering help, but I've already finished a working draft. #1241

xxchan avatar Apr 24 '25 02:04 xxchan

I've also implemented this in https://github.com/apache/iceberg-rust/pull/917

phillipleblanc avatar May 01 '25 14:05 phillipleblanc

@xxchan How is your change coming along ?

gsoundar avatar May 07 '25 22:05 gsoundar

This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.

github-actions[bot] avatar Nov 06 '25 00:11 github-actions[bot]

Missing feature.

liurenjie1024 avatar Nov 06 '25 07:11 liurenjie1024

It seems reqsign has large changes, and becomes opendal-reqsign https://github.com/apache/opendal-reqsign. Maybe time to give another try.

On Thu, Nov 6, 2025 at 3:16 PM Renjie Liu @.***> wrote:

liurenjie1024 left a comment (apache/iceberg-rust#1236) https://github.com/apache/iceberg-rust/issues/1236#issuecomment-3495442676

Missing feature.

— Reply to this email directly, view it on GitHub https://github.com/apache/iceberg-rust/issues/1236#issuecomment-3495442676, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJBQZNKWO37WOTPCAOAKK7333LYUHAVCNFSM6AAAAAB3VVEL3WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTIOJVGQ2DENRXGY . You are receiving this because you were mentioned.Message ID: @.***>

xxchan avatar Nov 06 '25 07:11 xxchan