Add scylladb

[mkorolyov]

ScyllaDB is an open-source distributed NoSQL wide-column data store. It gains wide popularity as a drop-in replacement for Cassandra but a significantly faster one.

This PR adds the ScyllaDB module.

[guy9]

Thanks @mkorolyov , interested in this as well

[mykaul]

I wish there was a way NOT to provide your own scylla.yaml, but merge your defaults with the default scylla.yaml - the end result is that you might be missing some new defaults if you bring your own scylla.yaml.

[mkorolyov]

I wish there was a way NOT to provide your own scylla.yaml, but merge your defaults with the default scylla.yaml - the end result is that you might be missing some new defaults if you bring your own scylla.yaml.

This looks like a separate feature for merging config files. Usually what I used to see in a majority of the applications is that you are using the default config file and overriding it with ENVs or flags or providing your own config file. Providing own config file which will still be merged with some default looks like implicit behavior which in my opinion will confuse users. WDYT?

[guy9]

Any updates on merging this?

[mykaul]

I wish there was a way NOT to provide your own scylla.yaml, but merge your defaults with the default scylla.yaml - the end result is that you might be missing some new defaults if you bring your own scylla.yaml.

This looks like a separate feature for merging config files. Usually what I used to see in a majority of the applications is that you are using the default config file and overriding it with ENVs or flags or providing your own config file. Providing own config file which will still be merged with some default looks like implicit behavior which in my opinion will confuse users. WDYT?

Yes, my ask is exactly this - let's use the default Scylla YAML that comes with Scylla, and allow overriding values in it.

[vbazhmin]

Hello folks. Any updates on this one ? ;)

[eddumelendez]

Hi, I'll review the PR once the other comments have been addressed. It is great seeing Scylla team reviewing the PR.

[fruch]

Hi, I'll review the PR once the other comments have been addressed. It is great seeing Scylla team reviewing the PR.

hopefully it won't be dragged like it's counterpart: https://github.com/testcontainers/testcontainers-python/pull/167

[eddumelendez]

@fruch LMK once this is ready to review and will do it as soon as possible. I still see some comments that hasn't been resolved. So, that's the reason why I haven't reviewed yet.

[eddumelendez]

Hi, I know this is still in progress because I see some comments unresolved. But, JFTR it would be nice to add an integration test enabling ssl and add it to the docs.

[eddumelendez]

I'm updating the PR based on https://github.com/testcontainers/testcontainers-go/pull/2919/

[mkorolyov]

Hi @mykaul,

I've looked into implementing your suggestion, and it seems there are a few challenges with achieving this functionality:

1. ScyllaDB Configuration Dependency:
   The ScyllaDB service requires a single configuration file, typically located at /etc/scylla/scylla.yaml in the official Docker image. It's straightforward to replace this file in the container using a volume mount, e.g., -v ~/master_scylla.yaml:/etc/scylla/scylla.yaml.

2. Merging Custom Configuration:
   To merge a custom partial configuration with the default one, we would need to process and generate the final configuration file before the container starts and then mount it to the container.

3. Extracting Default Configuration:
   One potential approach is to extract the default configuration file directly from the Docker image without starting the container. This can be done using Docker commands, such as:

   docker create --name temp-container scylladb/scylla  
   docker cp temp-container:/etc/scylla/scylla.yaml ./scylla.yaml  
   docker rm temp-container  
   

While this method works, it feels like a bit of a hack to me, and I’m not entirely comfortable with introducing such a workaround into the workflow.

4. Maintaining Up-to-Date Configurations: Alternatively, the latest default ScyllaDB configuration file can be found in their repository here. However, storing a snapshot of this file in the testcontainers-java repository introduces a maintenance issue: it would become outdated as soon as updates are made upstream.

5. Dynamic Fetching Concerns: Dynamically fetching the latest configuration file from the ScyllaDB repository during container setup (e.g., via HTTP calls) is also problematic and may not align with project standards or be acceptable to the maintainers.

Given these constraints, do you have other suggestions for achieving this functionality? Specifically, how can we ensure the availability of an up-to-date configuration file while maintaining simplicity and compatibility with the existing workflow?

Looking forward to your thoughts!

[mkorolyov]

@eddumelendez Hey, thanks for stepping in and polishing PR. Just asking, why you removed custom config file related changes?

[mykaul]

@mkorolyov - thanks for looking into this. I think your point no. 1 makes a convincing argument - it's trivial to just pass a yaml file, it's quite standard, so anyone using the Scylla testcontainer can do it. Let's just document it.

[mkorolyov]

@mykaul Hey, added possibility to easily replace config file for test container and documented it.

@eddumelendez Hey, added ssl integration test and documented it.

Could you pls take a look so we could proceed with it?

Appreciate your help and time!

[mkorolyov]

@eddumelendez Hey, could you please take a look? I've updated PR according to your suggestions.

[eddumelendez]

Thanks for your contribution, @mkorolyov !