Add support for Docker Swarm

[mattdy]

What would you like to be added?

As per a discussion on Discord (https://discord.com/channels/921520481163673640/1353780808439042118) - it seems that the Docker container auto-discovery added in #2957 does not function correctly when utilizing Docker Swarm.

While I'm not that familiar with the Docker APIs, other pieces of software implementing container auto-discovery seem to integrate two separate methods of discovery:

For example, Traefik: https://github.com/traefik/traefik/blob/master/pkg/provider/docker/pdocker.go vs https://github.com/traefik/traefik/blob/master/pkg/provider/docker/pswarm.go

Would it be possible for CrowdSec to also support Docker Swarm for auto-discovery?

Why is this needed?

Extends support for container auto-discovery, in a common use-case that currently doesn't function as intended

[github-actions[bot]]

@mattdy: Thanks for opening an issue, it is currently awaiting triage.

In the meantime, you can:

1. Check Crowdsec Documentation to see if your issue can be self resolved.
2. You can also join our Discord.
3. Check Releases to make sure your agent is on the latest version.

Details

I am a bot created to help the crowdsecurity developers manage community feedback and contributions. You can check out my manifest file to understand my behavior and what I can do. If you want to use this for your project, you can check out the BirthdayResearch/oss-governance-bot repository.

[github-actions[bot]]

@mattdy: There are no 'kind' label on this issue. You need a 'kind' label to start the triage process.

• /kind feature
• /kind enhancement
• /kind refactoring
• /kind bug
• /kind packaging

Details

I am a bot created to help the crowdsecurity developers manage community feedback and contributions. You can check out my manifest file to understand my behavior and what I can do. If you want to use this for your project, you can check out the BirthdayResearch/oss-governance-bot repository.

[mattdy]

/kind enhancement

[LaurenceJJones]

Tagging milestone as 1.7.0 as the feature can be useful and my brief late night read last night it seems this should be achievable, however, containers endpoint we currently use is only for local containers to the node itself. We have to use the nodes endpoint instead / or in conjunction if we can detect if node is apart of a swarm and has manager rights.

Note a sub issue should also be considered that we currently dont do any validation on the logging driver, we should also detect if the logging driver is supportable since they may have journald configured and we cannot get the logs via docker api.

[johnweland]

Sorry if this sounds dumb, but would this be the reason running this in swarm mode if I run docker service ps crowdsec is shows up as running but then if I docker ps it doesn't show as a container (ergo I cannot docker exec <containername> -it /bin/sh in order to run cli commands other get my bouncer connected?

I am new to using docker in swarm mode.

[LaurenceJJones]

Hey @mattdy,

I have a working PR linked above, and after digging into it, it looks like Swarm handles logs quite differently compared to standard containers.

In short, all logs in Swarm are grouped by service and accessed via the service name or ID. Based on that, I added a use_service_labels option to enable the same type of “discovery,” but for Swarm services.

One important note: If you enable both use_service_labels and use_container_labels, you may end up with duplicate log ingestion since both will collect the same logs through different paths. Most users likely won’t deploy workloads on the manager node, but it’s still worth mentioning.

[mattdy]

Amazing, looking forward to trying it out!

From a quick look, I agree that use_service_labels and use_container_labels should be mutually exclusive. I do deploy workloads on manager nodes, but they will still be picked up through the service labels.

Thanks again for your work on this!