All calls to S3 API fail with error: 405 (MethodNotAllowed)

[nickbrennan1]

Issue report

What version of MicroCeph are you using ?

Name Version Rev Tracking Publisher Notes microceph 0+git.287ee68 807 quincy/edge canonical✓ held

What are the steps to reproduce this issue ?

1. Install MicroCeph on a vanilla Ubuntu 22.04 VM
2. Enable the dashboard and create a user/ bucket
3. Enable RGW

What happens (observed behaviour) ?

At this point any calls to the RGW endpoint (default port 80) appear to fail, ie to set a new policy on the bucket:

# s3cmd setpolicy ./ceph-pol s3://ceph/
ERROR: S3 error: 405 (MethodNotAllowed)

And from the RGW logs:

2024-04-23T19:16:29.281+0000 7f0bec688640  1 ====== starting new request req=0x7f0ca83c4710 =====
2024-04-23T19:16:29.281+0000 7f0bec688640  1 req 15239953914733753145 0.000000000s handler->ERRORHANDLER: err_no=-2003 new_err_no=-2003
2024-04-23T19:16:29.281+0000 7f0bec688640  1 ====== req done req=0x7f0ca83c4710 op status=0 http_status=405 latency=0.000000000s ======
2024-04-23T19:16:29.281+0000 7f0bec688640  1 beast: 0x7f0ca83c4710: ::1 - - [23/Apr/2024:19:16:29.281 +0000] "PUT /?policy HTTP/1.1" 405 189 - - - latency=0.000000000s

All calls will fail except an INFO on the bucket, ie:

root@ceph:~# s3cmd info s3://ceph/
s3://ceph/ (bucket):
   Location:  us-east-1
   Payer:     none
   Expiration Rule: none
   Policy:    <?xml version="1.0" encoding="UTF-8"?><ListAllMyBucketsResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Owner><ID>test</ID><DisplayName>test</DisplayName></Owner><Buckets><Bucket><Name>ceph</Name><CreationDate>2024-04-23T10:47:41.982Z</CreationDate></Bucket></Buckets></ListAllMyBucketsResult>

   CORS:      <?xml version="1.0" encoding="UTF-8"?><ListAllMyBucketsResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Owner><ID>test</ID><DisplayName>test</DisplayName></Owner><Buckets><Bucket><Name>ceph</Name><CreationDate>2024-04-23T10:47:41.982Z</CreationDate></Bucket></Buckets></ListAllMyBucketsResult>

Gist with detail here

What were you expecting to happen ?

Expectation is that standard API calls will work

Additional comments.

Possibly the problems are occurring because no privileges are set on hte bucket yet, but without the ability to apply a policy there appears to be no way round this

[UtkarshBhatthere]

Hmm, this could be an interesting, can you please describe the enable dashboard and create user/bucket step. ? Also, Can you try achieving the same (user/bucket) creation using the radosgw-admin cli ?

We don't really test for setting up policies for buckets, it would be awesome if you could contribute a CI test for it. Public Matrix chat room

[nickbrennan1]

Thanks for the suggestion, I'll retry with the CLI. Have added the notes on enabling dashboard etc in a gist above

[nickbrennan1]

I've been able to create a user and add capabilities with radosgw-admin. However, there appears to be no option to create bucket, either in the radosgw-admin help page or the main guide:

  bucket list                list buckets (specify --allow-unordered for faster, unsorted listing)
  bucket limit check         show bucket sharding stats
  bucket link                link bucket to specified user
  bucket unlink              unlink bucket from specified user
  bucket stats               returns bucket statistics
  bucket rm                  remove bucket
  bucket check               check bucket index
  bucket chown               link bucket to specified user and update its object ACLs
  bucket reshard             reshard bucket
  bucket rewrite             rewrite all objects in the specified bucket
  bucket sync checkpoint     poll a bucket's sync status until it catches up to its remote
  bucket sync disable        disable bucket sync
  bucket sync enable         enable bucket sync
  bucket radoslist           list rados objects backing bucket's objects

....and if I use s3cmd I get the same behaviour as the problem statement:

root@ceph:~/ceph# s3cmd mb s3://clitest
ERROR: S3 error: 405 (MethodNotAllowed)

radosgw-admin will list available buckets and their content. I can also use s3cmd PUT to copy up objects, so some of the interface works and the creds in .s3cfg are correct.