Daniel
Daniel
I just tested again using the info from my initial post and I can confirm, even with distStorageType set to AWS, connectivity to S3 does not work, results in the...
Hi there, another update from my previous testing not defining host causes the above wait-for-s3 error, but it's not clear what to define it as. s3.amazonaws.com does not work.
Some more examples: This does not work, it complains that it's missing a secret named dremio-externals3 ```yaml externalS3: accessKeyID: -- accessKeySecret: -- bucket: my bucket # this value is required...
``` Caused by: com.dremio.plugins.util.ContainerNotFoundException: Error while looking up bucket scratch at com.dremio.plugins.s3.store.S3FileSystem.getUnknownContainer(S3FileSystem.java:373) at com.dremio.plugins.util.ContainerFileSystem.getFileSystemForPath(ContainerFileSystem.java:321) at com.dremio.plugins.util.ContainerFileSystem.mkdirs(ContainerFileSystem.java:533) at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:2496) at com.dremio.exec.hadoop.HadoopFileSystem.mkdirs(HadoopFileSystem.java:338) at com.dremio.io.file.FilterFileSystem.mkdirs(FilterFileSystem.java:106) at com.dremio.exec.store.dfs.LoggedFileSystem.mkdirs(LoggedFileSystem.java:133) at com.dremio.exec.store.dfs.FileSystemPlugin.createIfNecessary(FileSystemPlugin.java:951) ... 9 common frames...
```yaml externalS3: bucket: real-bucket path: /dremio host: realbucket.s3.amazonaws.com existingSecret: dremio-secrets existingSecretAccessKeyIDKey: DREMIO_AWS_ACCESS_KEY_ID existingSecretKeySecretKey: DREMIO_AWS_SECRET_ACCESS_KEY region: us-east-2 ``` In this example, dremio fails when it attempts to write to a bucket...
#34784 is caused by this error, can Bitnami assist before I have to tear up the chart?
@carrodher any thoughts here? Chart is busted when trying to use s3 in any fashion. I do not believe you can use S3 for either the distributed store or internally...
I wanted to also point out that if I run JUST the docker image, this and #34784 work just fine. So the issue lies somewhere in the chart. cc @juan131
I understand, I'm notifying Juan because the issues are related.
Hi, I've put some more time into debugging this and found that using anything but instance auth in core-site.xml causes the connection to attempt to reach out to minio rather...