amazon-linux-2023 icon indicating copy to clipboard operation
amazon-linux-2023 copied to clipboard

[Feature Request] - consider removing LimitNOFILE from docker.service configuration

Open pluralistix opened this issue 1 year ago • 0 comments

Is your feature request related to a problem? Please describe. Currently we are using a CI/CD on ECS where the instances run with an al2023-ami-ecs-hvm-* ami. That (still) brings LimitNOFILE=infinity with it.

systemctl cat docker.service | sed -rn '1p;/^LimitNOFILE/p'
# /usr/lib/systemd/system/docker.service
LimitNOFILE=infinity
systemctl show docker.service | grep LimitNOFILE
LimitNOFILE=infinity
LimitNOFILESoft=infinity

The foo comes in, when "people" do build very old software...

time DOCKER_BUILDKIT=0 docker build --no-cache -t foo -f - >/dev/null 2>&1 . <<EOF
FROM python:3.7-stretch
RUN echo "deb http://archive.debian.org/debian stretch stretch-security main contrib" > /etc/apt/sources.list; apt-get update; apt-get install --no-install-recommends --no-install-suggests -y supervisor
EOF
echo $?
real    0m6.768s
user    0m0.096s
sys     0m0.175s
0

time docker build --ulimit nofile=1024:4096 --no-cache -t foo -f - >/dev/null 2>&1 . <<EOF
FROM python:3.7-stretch
RUN echo "deb http://archive.debian.org/debian stretch stretch-security main contrib" > /etc/apt/sources.list; apt-get update; apt-get install --no-install-recommends --no-install-suggests -y supervisor
EOF
echo $?
real    0m7.415s
user    0m0.110s
sys     0m0.139s
0

But

time docker build --no-cache -t foo -f - >/dev/null 2>&1 . <<EOF
FROM python:3.7-stretch
RUN echo "deb http://archive.debian.org/debian stretch stretch-security main contrib" > /etc/apt/sources.list; apt-get update; apt-get install --no-install-recommends --no-install-suggests -y supervisor
EOF
echo $?
real    16m41.333s
user    0m0.537s
sys     0m0.897s
0

spending all the time on /usr/bin/python /usr/bin/pycompile -p python-meld3, /usr/bin/python /usr/bin/pycompile -p python-pkg-resources etc.

Describe the solution you'd like Consider shipping it as it was changed in here

Describe alternatives you've considered building with DOCKER_BUILDKIT=0 or adjusting each docker build command and passing "reasonable" e.g. --ulimit nofile=1024:4096 config.

We already use

/etc/sysconfig/docker
OPTIONS="--default-ulimit nofile=32768:65536"

But that does not help, as buildkit inherits this from docker.service. See

docker build --no-cache --progress=plain -t foo -f - . <<EOF
FROM python:3.7-stretch
RUN getconf OPEN_MAX
EOF
#5 [2/2] RUN getconf OPEN_MAX
#5 0.501 1073741816

but

DOCKER_BUILDKIT=0 docker build --no-cache -t foo -f - 2>/dev/null . <<EOF
FROM python:3.7-stretch
RUN getconf OPEN_MAX
EOF
Step 2/2 : RUN getconf OPEN_MAX
 ---> Running in 7befb30c2fe4
32768

pluralistix avatar Jul 23 '24 10:07 pluralistix