command is truncated from docker-compose file

[disaster123]

Hi,

i've the following compose file:

    command: bash -c 'for i in `seq 1 30`; do node main.js && s=$$? && break || s=$$?; echo "Tried $$i times. Waiting 5 secs..."; sleep 5; done; (exit $$s)'

But the container start fails with a syntax error and podman ps -a shows:

podman ps -a --no-trunc
9d9e22f463ffaf597608a02d76e8830d219ef24b69b6a0548936fdc90d4007c6  docker.io/library/mongo:latest                           bash -c "for i in `seq 1 30`; do mongo mongo/rocketchat --eval \"rs.initiate({ _id: 'rs0', members: [ { _id: 0, host: 'localhost:27017' } ]})\" && s=$ && break || s=$ echo \"Tried $ times. Waiting 5 secs...\"; sleep 5; done; (exit $  7 minutes ago  Exited (1) 2 minutes ago  0.0.0.0:3000->3000/tcp  rocket_mongo-init-replica_1

so the comand got truncated and last chars are missing: $s)'

Greets, Stefan

[disaster123]

print(json.dumps(compose, indent = 2))

this already shows that while parsing $$? or $$i seems to be interpreted. So the command isn't truncted like i thought but those vars seem to be replaced by empty strings.

[disaster123]

Here is written that you can use $$a to escape the $ sign to get the original var. But that does not work in podman-compose.

https://docs.docker.com/compose/compose-file/#variable-substitution

[disaster123]

OK bug is in rec_subs - this function breaks the command.

[disaster123]

I'M not sure whether this really fixes this bug but using a negative look behind fixes my situation:

 139 var_re = re.compile(r'(?<!\$)\$(\{(?:[^\s\$:\-\}]+)\}|(?:[^\s\$\{\}]+))')
 140 var_def_re = re.compile(r'(?<!\$)\$\{([^\s\$:\-\}]+)(:)?-([^\}]+)\}')
 141 var_err_re = re.compile(r'(?<!\$)\$\{([^\s\$:\-\}]+)(:)?\?([^\}]+)\}')

[disaster123]

While this fixes the problem that a "wrong" command is shown in ps -a it still does not fix the issue that podman cannot parse it.

Now ps -a looks right but podman still logs:

i: -c: line 0: unexpected EOF while looking for matching `''
i: -c: line 1: syntax error: unexpected end of file

But manually executing the command works fine.

[disaster123]

OK but this again is also an issue in podman-compose.

the printed podman command from podman-compose is:

podman run --name=rocket_rocketchat_1 -it --pod=rocket -l io.podman.compose.config-hash=123 -l io.podman.compose.project=rocket -l io.podman.compose.version=0.0.1 -l com.docker.compose.container-number=1 -l com.docker.compose.service=rocketchat -e PORT=3000 -e ROOT_URL=http://netflow.vms.phoffice -e MONGO_URL=mongodb://mongo:27017/rocketchat -e MONGO_OPLOG_URL=mongodb://mongo:27017/local -e Accounts_UseDNSDomainCheck=False --mount type=bind,source=/home/podman/rocket/uploads,destination=/app/uploads --add-host rocketchat:127.0.0.1 --add-host rocket_rocketchat_1:127.0.0.1 --add-host mongo:127.0.0.1 --add-host rocket_mongo_1:127.0.0.1 --add-host mongo-init-replica:127.0.0.1 --add-host rocket_mongo-init-replica_1:127.0.0.1 rocket.chat:latest bash -c 'for i in `seq 1 30`; do node main.js && s=$? && break || s=$?; echo "Tried $i times. Waiting 5 secs..."; sleep 5; done; (exit $s)'

while this fails if podman-compose executes it it works fine if manually executed

[disaster123]

OK the next problem is that podman-compose just splits the command here: podman_args.extend(command.split())

but this won't work you cannot simply split this like you want.

[disaster123]

We can simply fix this by using shlex. podman_args.extend(shlex.split(command))

will work fine

[disaster123]

Complete patch to fix both bugs:

--- podman_compose.py   2020-02-11 10:05:08.138391256 +0100
+++ podman_compose.py.patched   2020-02-11 10:06:54.905890890 +0100
@@ -17,6 +17,7 @@ import time
 import re
 import hashlib
 import random
+import shlex
 
 from threading import Thread
 
@@ -136,9 +137,9 @@ def fix_mount_dict(mount_dict, proj_name
 # ${VARIABLE?err} raise error if not set
 # $$ means $
 
-var_re = re.compile(r'\$(\{(?:[^\s\$:\-\}]+)\}|(?:[^\s\$\{\}]+))')
-var_def_re = re.compile(r'\$\{([^\s\$:\-\}]+)(:)?-([^\}]+)\}')
-var_err_re = re.compile(r'\$\{([^\s\$:\-\}]+)(:)?\?([^\}]+)\}')
+var_re = re.compile(r'(?<!\$)\$(\{(?:[^\s\$:\-\}]+)\}|(?:[^\s\$\{\}]+))')
+var_def_re = re.compile(r'(?<!\$)\$\{([^\s\$:\-\}]+)(:)?-([^\}]+)\}')
+var_err_re = re.compile(r'(?<!\$)\$\{([^\s\$:\-\}]+)(:)?\?([^\}]+)\}')
 
 def dicts_get(dicts, key, fallback='', fallback_empty=False):
     """
@@ -535,7 +536,7 @@ def container_to_args(compose, cnt, deta
     command = cnt.get('command')
     if command is not None:
         if is_str(command):
-            podman_args.extend(command.split())
+            podman_args.extend(shlex.split(command))
         else:
             podman_args.extend(command)
     return podman_args

[muayyad-alsadi]

@disaster123 Please send me pull request

[profihost]

I currently found out that she command.split bug is already fixed in devel and that the $$ bug is fixed here: https://github.com/containers/podman-compose/pull/97

So a merge of https://github.com/containers/podman-compose/pull/97 is missing and a new release ;-)

[metal3d]

seems related to #129 and #115

[rugk]

So a merge of #97 is missing and a new release ;-)

The PR has been merged. So can this issue be closed?