containers
Replicate environment variable from local environment fails with default `.env` -file
Describe the bug
If environment variable value is defined in default .env -file and environment variable is replicated in compose.yml file, environment variable is not passed to container
To Reproduce
.env:
TEST=test
compose.yml:
services:
test:
image: busybox
environment:
- TEST
command: printenv
Expected behavior
Environment variable TEST should be included in output
Actual behavior
Environment variable TEST is not included in output
Output
$ podman compose run --rm test
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
container=podman
TERM=xterm
HOME=/root
HOSTNAME=aa7b6faaf810
My environment:
$ podman compose run --rm test
container=podman
TERM=xterm
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOME=/root
HOSTNAME=6da914b12a2e
$ podman compose version
podman-compose version 1.2.0
podman version 5.1.2
$ cat /etc/os-release
NAME="Arch Linux"
PRETTY_NAME="Arch Linux"
ID=arch
BUILD_ID=rolling
ANSI_COLOR="38;2;23;147;209"
HOME_URL="https://archlinux.org/"
DOCUMENTATION_URL="https://wiki.archlinux.org/"
SUPPORT_URL="https://bbs.archlinux.org/"
BUG_REPORT_URL="https://gitlab.archlinux.org/groups/archlinux/-/issues"
PRIVACY_POLICY_URL="https://terms.archlinux.org/docs/privacy-policy/"
LOGO=archlinux-logo
$ pacman -Qi podman podman-compose
Name : podman
Version : 5.1.2-1
Description : Tool and library for running OCI-based containers in pods
Architecture : x86_64
URL : https://github.com/containers/podman
Licenses : Apache-2.0
Groups : None
Provides : None
Depends On : catatonit conmon containers-common oci-runtime gcc-libs glibc iptables device-mapper libdevmapper.so gpgme libgpgme.so=11-64 libseccomp
libseccomp.so=2-64 passt
Optional Deps : apparmor: for AppArmor support
btrfs-progs: support btrfs backend devices
cni-plugins: for an alternative container-network-stack implementation
fuse-overlayfs: for storage driver in rootless environment
slirp4netns: for alternative rootless network support
podman-compose: for docker-compose compatibility [installed]
podman-docker: for Docker-compatible CLI [installed]
Required By : podman-compose podman-docker
Optional For : None
Conflicts With : None
Replaces : None
Installed Size : 77.29 MiB
Packager : David Runge <[email protected]>
Build Date : Fri 12 Jul 2024 12:26:12 AM EEST
Install Date : Sat 13 Jul 2024 07:33:05 PM EEST
Install Reason : Explicitly installed
Install Script : No
Validated By : Signature
Name : podman-compose
Version : 1.2.0-1
Description : A script to run docker-compose.yml using podman
Architecture : any
URL : https://github.com/containers/podman-compose
Licenses : GPL-2.0-only
Groups : None
Provides : None
Depends On : podman python python-dotenv python-yaml
Optional Deps : aardvark-dns: resolve hostnames of linked containers [installed]
Required By : None
Optional For : podman
Conflicts With : None
Replaces : None
Installed Size : 486.06 KiB
Packager : David Runge <[email protected]>
Build Date : Fri 05 Jul 2024 11:00:08 AM EEST
Install Date : Thu 11 Jul 2024 09:50:35 AM EEST
Install Reason : Explicitly installed
Install Script : No
Validated By : Signature
It seems that if I interpolate value in compose.yml it picks the value correctly from .env:
$ cat .env
TEST=test
$ cat compose.yml
services:
test:
image: busybox
environment:
- TEST=$TEST
command: printenv
$ podman compose run --rm test
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
container=podman
TERM=xterm
TEST=test
HOME=/root
HOSTNAME=cb5574ddcaef
On the other hand if I set environment variable as local environment variable then it is replicated correctly:
$ cat .env
TEST=test
$ cat compose.yml
services:
test:
image: busybox
environment:
- TEST
command: printenv
$ TEST=local podman compose run --rm test
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
container=podman
TERM=xterm
TEST=local
HOME=/root
HOSTNAME=481b5794f566
Might be the same issue as:
- https://github.com/containers/podman-compose/issues/891
Related:
- https://github.com/containers/podman-compose/issues/970
- https://github.com/containers/podman-compose/issues/813
- https://github.com/containers/podman-compose/issues/999
not sure if i'm doing this correctly, but this issue has bothered me and i've implemented a fix that seems to work great in my (admittedly short) tests:
https://github.com/containers/podman-compose/compare/main...terminatorbs:podman-compose:patch-1
sorry i'm not knowledgeable to add tests for this. also no worries if the change doesn't make it into a PR, i just wanted a fix for myself because for example Jitsi has like hundreds of env vars passed like this and i thought i'd rather fix it here than write hacky scripts that modify compose files to add interpolation to every environment: entry.
edit: the code change is in the commit, but here it is with 2 comments added that kinda explain what it does:
env = norm_as_list(cnt.get("environment", {}))
for e in env:
if '=' in e:
# value is already assigned, directly add to podman_args
podman_args.extend(["-e", e])
else:
# if we find a key in our env, pass it and it's value to the container
if e in compose.environ:
podman_args.extend(["-e", f"{e}={compose.environ[e]}"])
this is (as of right now) from line 1100 in podman-compose.py, in the container_to_args() function.
edit 2: if the change seems reasonable and someone thinks it might be good to merge, i am willing to invest the time to make a test or two for it and open a proper PR.
@hmontone Thank you for opening this issue!
Please update your podman-compose to the newest main branch, it should now be working correctly (fixed by #1248).
If you still see any (other) problems, please open a new bug report.
@terminatorbs you were right about the solution for this issue.
Next time feel free to open a pull request with your changes and tests. Maintainers will comment on how to improve the change (if improvements are needed) and your code will land in podman-compose project soon enough.
Thank you for your contribution!