Failure to Start RKT Pod when SELinux is set to Enforcing.

[joshw123]

Issue Report

Bug

We have recently enabled SELinux on CoreOS and since we are seeing problems starting RKT Pods, When attempting to start we get the following.

$ sudo rkt run coreos.com/etcd:v2.0.10
Error: Unable to determine interpreter for "/etcd"
$ sudo rkt run docker://alpine --insecure-options=image
Error: Unable to determine interpreter for "/lib/ld-musl-x86_64.so.1"

Container Linux Version

$ cat /etc/os-release
NAME="Container Linux by CoreOS"
ID=coreos
VERSION=1688.5.3
VERSION_ID=1688.5.3
BUILD_ID=2018-04-03-0547
PRETTY_NAME="Container Linux by CoreOS 1688.5.3 (Rhyolite)"
ANSI_COLOR="38;5;75"
HOME_URL="https://coreos.com/"
BUG_REPORT_URL="https://issues.coreos.com"
COREOS_BOARD="amd64-usr"
BUG_REPORT_URL="https://issues.coreos.com"

Environment

On Premise.

Expected Behavior

Pod should start as normal

Actual Behavior

Pod Fails to Start

Reproduction Steps

Start CoreOS Instance and attempt to run a RKT Container like above.

Other Information

[iaguis]

Ref: https://github.com/coreos/bugs/issues/2231

[glevand]

@joshw123 I recommend you run auditctl -D to clear the audit filter rules, start systemctl -f to monitor the logs, then run your rkt command. Do you see anything relevant appear in the log?

[joshw123]

Thank You @glevand. The results are below.

auditctl -D No rules

audit[1113]: AVC avc:  denied  { write } for  pid=1113 comm="systemd" name="core_pattern" dev="proc" ino=25511 scontext=system_u:system_r:svirt_lxc_net_t:s0:c332,c680 tcontext=system_u:object_r:sysctl_kernel_t:s0 tclass=file permissive=0

audit[1113]: AVC avc:  denied  { create } for  pid=1113 comm="systemd" name="blk" scontext=system_u:system_r:svirt_lxc_net_t:s0:c332,c680 tcontext=system_u:object_r:svirt_lxc_file_t:s0:c332,c680 tclass=blk_file permissive=0

audit[1113]: AVC avc:  denied  { remount } for  pid=1113 comm="systemd" scontext=system_u:system_r:svirt_lxc_net_t:s0:c332,c680 tcontext=system_u:object_r:svirt_lxc_file_t:s0:c332,c680 tclass=filesystem permissive=0

audit[1113]: AVC avc:  denied  { search } for  pid=1113 comm="systemd" name="unix" dev="proc" ino=25528 scontext=system_u:system_r:svirt_lxc_net_t:s0:c332,c680 tcontext=system_u:object_r:sysctl_net_unix_t:s0 tclass=dir permissive=0

audit[1122]: AVC avc:  denied  { mounton } for  pid=1122 comm="(nginx)" path="/opt/stage2/proxy/rootfs/proc/bus" dev="proc" ino=4026531854 scontext=system_u:system_r:svirt_lxc_net_t:s0:c332,c680 tcontext=system_u:object_r:proc_t:s0 tclass=dir permissive=0

audit[1113]: AVC avc:  denied  { sendto } for  pid=1113 comm="systemd" path="/systemd/nspawn/notify" scontext=system_u:system_r:svirt_lxc_net_t:s0:c332,c680 tcontext=system_u:system_r:kernel_t:s0 tclass=unix_dgram_socket permissive=0

audit[1187]: AVC avc:  denied  { write } for  pid=1187 comm="systemd" name="core_pattern" dev="proc" ino=28813 scontext=system_u:system_r:svirt_lxc_net_t:s0:c211,c880 tcontext=system_u:object_r:sysctl_kernel_t:s0 tclass=file permissive=0

audit[1187]: AVC avc:  denied  { create } for  pid=1187 comm="systemd" name="blk" scontext=system_u:system_r:svirt_lxc_net_t:s0:c211,c880 tcontext=system_u:object_r:svirt_lxc_file_t:s0:c211,c880 tclass=blk_file permissive=0

audit[1187]: AVC avc:  denied  { remount } for  pid=1187 comm="systemd" scontext=system_u:system_r:svirt_lxc_net_t:s0:c211,c880 tcontext=system_u:object_r:svirt_lxc_file_t:s0:c211,c880 tclass=filesystem permissive=0

audit[1187]: AVC avc:  denied  { search } for  pid=1187 comm="systemd" name="unix" dev="proc" ino=28830 scontext=system_u:system_r:svirt_lxc_net_t:s0:c211,c880 tcontext=system_u:object_r:sysctl_net_unix_t:s0 tclass=dir permissive=0

audit[1196]: AVC avc:  denied  { mounton } for  pid=1196 comm="(nginx)" path="/opt/stage2/proxy/rootfs/proc/bus" dev="proc" ino=4026531854 scontext=system_u:system_r:svirt_lxc_net_t:s0:c211,c880 tcontext=system_u:object_r:proc_t:s0 tclass=dir permissive=0

audit[1187]: AVC avc:  denied  { sendto } for  pid=1187 comm="systemd" path="/systemd/nspawn/notify" scontext=system_u:system_r:svirt_lxc_net_t:s0:c211,c880 tcontext=system_u:system_r:kernel_t:s0 tclass=unix_dgram_socket permissive=0

systemd[1]: Stopped Container rkt-ad69bb98-0a5a-41e2-a5c3-8686d83dbd26.
systemd-machined[919]: Machine rkt-ad69bb98-0a5a-41e2-a5c3-8686d83dbd26 terminated.

This appears to reveal something, Ill look a little further.

[obikay200]

#Bug

We also have the same problem with enabled SELinux on CoreOS and seeing problems starting RKT Pods, When attempting to start we get the following.

$ sudo rkt run docker://nginx:alpine --insecure-options=image Error: Unable to determine interpreter for "/lib/ld-musl-x86_64.so.1"

Container Linux Version

NAME="Container Linux by CoreOS"
ID=coreos
VERSION=1911.3.0
VERSION_ID=1911.3.0
BUILD_ID=2018-11-05-1815
PRETTY_NAME="Container Linux by CoreOS 1911.3.0 (Rhyolite)"
ANSI_COLOR="38;5;75"
HOME_URL="https://coreos.com/"
BUG_REPORT_URL="https://issues.coreos.com"
COREOS_BOARD="amd64-usr"

journalctl output:

journalctl -u run-rcee2e52305d84bfe858178f5237af78b.service
-- Logs begin at Wed 2018-11-07 20:08:10 UTC, end at Thu 2018-11-08 21:19:44 UTC. --
Nov 08 21:19:33 localhost systemd[1]: Started /bin/rkt run --insecure-options=image --port=80-tcp:80 docker://nginx.
Nov 08 21:19:34 localhost rkt[1193]: Error: Unable to determine interpreter for "/lib64/ld-linux-x86-64.so.2"
Nov 08 21:19:34 localhost systemd[1]: run-rcee2e52305d84bfe858178f5237af78b.service: Main process exited, code=exited, status=203/EXEC
Nov 08 21:19:34 localhost systemd[1]: run-rcee2e52305d84bfe858178f5237af78b.service: Failed with result 'exit-code'.