Fails to connect to podman / could not find a container

[puppetjoy]

With Bolt 4.0.0 and Podman 5.1.1, running against a podman target results in a failure:

> bolt command run hostname -t 'podman://gitlab-runner'
Started on podman://gitlab-runner...
Failed on podman://gitlab-runner:
  Failed to connect to podman://gitlab-runner: Could not find a container with name or ID matching 'gitlab-runner'
Failed on 1 target: podman://gitlab-runner
Ran on 1 target in 0.17 sec

I traced this through the code to find that extract_json in lib/bolt/transport/podman/connection.rb is returning nil. The code comments say podman ps returns pretty json and it does some weird check for bracket and newline positions, which is failing.

My version of Podman returns plain JSON like:

> podman ps --format '{{json .}}'
{"AutoRemove":false,"Command":["run","--user=gitlab-runner","--working-directory=/var/lib/gitlab-runner"],"Created":"2024-09-23T16:59:14.480429711-04:00","CreatedAt":"","CIDFile":"","Exited":false,"ExitedAt":1733704249,"ExitCode":0,"ExposedPorts":null,"Id":"66802ad43d1594ce73605f0a0a136214855e6ebb303c117b818d98e39bce44e6","Image":"docker.io/alpinelinux/gitlab-runner:latest","ImageID":"07690a5c49398fbb755585ceda4c5f6922fc9d80c9ba6544f83e2af5cab0de0a","IsInfra":false,"Labels":{"nest.podman.version":"5.1.1"},"Mounts":["/etc/gitlab-runner","/var/run/docker.sock"],"Names":["gitlab-runner"],"Namespaces":{},"Networks":["podman"],"Pid":4430,"Pod":"","PodName":"","Ports":null,"Restarts":0,"Size":null,"StartedAt":1733799305,"State":"running","Status":""}
{"AutoRemove":false,"Command":["foreground"],"Created":"2024-09-23T16:59:43.958958621-04:00","CreatedAt":"","CIDFile":"","Exited":false,"ExitedAt":1733818042,"ExitCode":0,"ExposedPorts":null,"Id":"1102aea2aa69e720f22de2e78c8b76eb737d848f7f1474a90540e9097dabe3db","Image":"registry.gitlab.james.tl/nest/forks/container-puppetserver:8.2.3","ImageID":"2d8b94a230bd76ab7340f77047bd1852e531b19ba7b7ded433213ebda94d5a12","IsInfra":false,"Labels":{"io.buildah.version":"1.30.0","nest.podman.version":"5.1.1","org.label-schema.build-date":"","org.label-schema.dockerfile":"/Dockerfile","org.label-schema.license":"Apache-2.0","org.label-schema.maintainer":"Voxpupuli Team <[email protected]>","org.label-schema.name":"Puppet Server (release)","org.label-schema.schema-version":"1.0","org.label-schema.url":"https://github.com/voxpupuli/container-puppetserver","org.label-schema.vcs-ref":"","org.label-schema.vcs-url":"https://github.com/voxpupuli/container-puppetserver","org.label-schema.vendor":"Voxpupuli","org.label-schema.version":"8.2.3","org.opencontainers.image.ref.name":"ubuntu","org.opencontainers.image.version":"22.04"},"Mounts":["/etc/eyaml","/docker-custom-entrypoint.d","/etc/puppetlabs/code","/etc/puppetlabs/puppet","/etc/puppetlabs/puppetserver/ca","/var/cache/r10k"],"Names":["puppetserver"],"Namespaces":{},"Networks":[],"Pid":509007,"Pod":"8d74e3414da50347f24c45192c622f545ea69fe8304706dd1791eb16d496a27c","PodName":"","Ports":[{"host_ip":"","container_port":8080,"host_port":8080,"range":2,"protocol":"tcp"},{"host_ip":"","container_port":8140,"host_port":8140,"range":1,"protocol":"tcp"},{"host_ip":"","container_port":80,"host_port":8180,"range":1,"protocol":"tcp"}],"Restarts":0,"Size":null,"StartedAt":1733818046,"State":"running","Status":"healthy"}

As a workaround, I removed the function so that it inherits format_json from lib/bolt/transport/docker/connection.rb and it works:

> bolt command run hostname -t 'podman://gitlab-runner'
Started on podman://gitlab-runner...
Finished on podman://gitlab-runner:
  66802ad43d15
Successful on 1 target: podman://gitlab-runner
Ran on 1 target in 0.24 sec

I imagine a proper solution involves maintaining backwards compatibility with however podman used to produce JSON.

(btw, it is very cool applying puppet code to a container with bolt!)