[Enhancement]: Create volumes for testcontainers

[pjroth]

Module

Core

Proposal

I recently needed to create two test containers that shared data storage. One container would generate data, and the second would read this data. My first thought was to create a docker volume and share the volume in the two containers. This avoids bind mounts which complicates things greatly. Bind mounts are harder to make work cross platform and there are restrictions on bind mounts when I run in my CI tool, bitbucket pipelines.

However, I cannot find any way to do this with testcontainers. There is a way to use withVolumesFrom but there is no way to create the volume as best I can tell. Here is an example of what I was trying to do with test containers but shown with docker commands.

Create a first container with a annonymous volume: docker run -it --rm --name source --volume=/share ubuntu bash

Create a second container that uses the same volume from the first: docker run -it --rm --volumes-from source --name dest ubuntu bash

Then in the source container I write a file into the /share directory and I can cat it out in the dest container. I would like to set this up using test containers.

Thanks for this great tool!

[fokion]

will try to have a look

[fokion]

 File tempDirectory = Files.createTempDirectory("test").toFile();
        GenericContainer<?> container = new GenericContainer<>(TestImages.TINY_IMAGE)
            .withCommand("sh", "-c", "echo \"test\">/workdir/test.txt;");

        container.addFileSystemBind(tempDirectory.getAbsolutePath(), "/workdir", BindMode.READ_WRITE);

        container.start();
        container.stop();

        assertTrue(tempDirectory.exists() && tempDirectory.isDirectory());
        assertEquals(1,tempDirectory.listFiles().length);
        File tempFile = tempDirectory.listFiles()[0];
        assertTrue(new File(tempDirectory,"test.txt").exists());
        assertEquals("test", Files.readAllLines(tempFile.toPath(), java.nio.charset.StandardCharsets.UTF_8).stream().filter(s -> !s.isEmpty()).findFirst().orElse(""));

        GenericContainer<?> container2 = new GenericContainer<>(TestImages.TINY_IMAGE)
            .withCommand("sh", "-c", "cat /workdir/test.txt");

        container2.addFileSystemBind(tempDirectory.getAbsolutePath(), "/workdir", BindMode.READ_ONLY);
        container2.start();

        assertEquals("test", container2.getLogs().replaceAll("\n",""));
        container2.stop();

[pjroth]

Just FYI (since I see the example code above uses the file system and bind mounts) my use case is to NOT require file system bind mounts. These make cross platform builds more complicated. For example during CI i’m actually doing builds with docker in docker. Getting temp directory access is confusing and complicated in this scenario and then requires builds to understand the runtime environment and behave differently in CI/macOS/linux/windows.

Thanks for all the hard work!

[pjroth]

Perhaps I misunderstood the proposed API addition. I see there are two new proposed methods, the SELF withVolume(String containerPath) that creates a volume as in my docker command examples I believe? If this will allow me to create containers as in my docker CLI example that works for me! :)

[fokion]

There is one existing method that creates a bind to the filesystem and what I did is to have the withVolume that will generate a temporary volume that you can consume from the other container. One limitation is that you cannot stop the container that has the volume as that will destroy the volume itself. So both containers must be up and running.

[fokion]

What is your opinion @eddumelendez ? Is that proposed solution acceptable or do we want to do any modifications?

[halotukozak]

ping, ping