systemd-bootchart icon indicating copy to clipboard operation
systemd-bootchart copied to clipboard

zero-byte svg file generated when systemd-bootchart invoked using "init" kernel parameter

Open jamuir opened this issue 1 year ago • 4 comments

According to man systemd-bootchart, the recommended way of running systemd-bootchart is using the init kernel parameter (which you can set from the grub menu):

init=/usr/lib/systemd/systemd-bootchart

Unfortunately, this seems to have stopped working.

On Fedora 41, with systemd v256, the generated svg file is zero bytes:

Fedora Linux 41 (Server Edition)
Kernel 6.11.4-301.fc41.aarch64 on an aarch64 (tty1)
vbox login: jmuir
Password:
Last login: Tue Jan 14 14:16:28 on tty1
jmuir@vbox:~$ ls -1 /run/log
total 0
rw-rw-rw-.  1 root root             0 Jan 14 14:22 bootchart-20250114-1422.svg
drwxr-sr-x+ 2 root systemd-journal 40 Jan 14 14:22 journal

Here is my kernel command-line:

jmuir@vbox:~$ xargs -n1 < /proc/cmdline
BOOT_IMAGE=(hdB,gpt2)/vmlinuz-6.11.4-301.fc41.aarch64
root=/dev/mapper/fedora_vbox-root
ro
rd.lum.lu=fedora_vbox/root
rhgb
init=/usr/1ib/systemd/systemd-bootchart

systemd-bootchart should run for about 3 mins (1800 samples at 10 samples/sec):

jmuir@vbox:~$ cat /etc/systemd/bootchart.conf
# This file is part of systemd.
#
# systemd is free software; you can redistribute it and/or modify it
# under the terms of the GNU Lesser General Public License as published by
# the Free Software Foundation; either version 2.1 of the License, or
# (at your option) any later version.
#
# Entries in this file show the compile time defaults.
# You can change settings by editing this file.
# Defaults can be restored by simply deleting this file.
# See bootchart.conf (5) for details.

[Bootchart]
Samples=1800
Frequency=10
#Relative=no
#Filter=yes
#Output=<folder name, defaults to /run/log>
#Init=/path/to/init-binary
#PlotMemoryUsage=no
#PlotEntropyGraph=no
#ScaleX=100
#ScaleY=20
#ControlGroup=no
#PerCPU=no

Version information:

jmuir@vbox:~$ systemctl --version
systemd 256 (256.7-1.fc41)
+PAM *AUDIT +SELINUX -APPARMOR *IMA SMACK *SECCOMP -GCRYPT *GNUTS OPENSSL *ACL *BLKID +CURL *ELFUTILS +FIDO2 +IDN -IDN -IPTC +KMOD *LIBCRYPTSETUP *LIBCRYPTS
ETUP_PLUGINS +LIBFDISK +PCREZ +PWQUALITY +P11KIT +QRENCODE +TPM +BZIP2 +LZA +XZ *ZLIB *ZSTD +BPF_FRAMEWORK *XKBCOMMON UTMP *SYSUINIT *LIBARCHIVE
jmuir@vbox:~$ strings /usr/lib/systemd/systemd-bootchart | grep systemd-bootchart
{"type" : "rpm", "name": "systend-bootchart"
systemd-bootchart wrote %s
systemd-bootchart: sample time overrun %i times
systemd-bootchart-235-1.fc41.aarch64 .debug

jamuir avatar Jan 14 '25 20:01 jamuir

I was able to capture an strace log using the following debug init program:

// init-jm.c
#include <stdio.h>
#include <unistd.h>
#include <fcntl.h>

#define BOOTCHART "/usr/lib/systemd/systemd-bootchart"
#define SYSTEMD   "/usr/lib/systemd/systemd"
#define STRACE    "/usr/bin/strace"

int main() {
  pid_t fpid;
  pid_t mypid;
  int fd = -1;
  fd = open("/dev/kmsg", O_WRONLY);

  mypid = getpid();

  dprintf(fd, "begin init-jm, mypid=%d\n", mypid);

  fpid = fork();
  if (fpid < 0) {
    perror("call to fork() failed");
  } else if (fpid == 0) {
    mypid = getpid();
    dprintf(fd, "this is the child, mypid=%d\n", mypid);
    execl(STRACE, "@strace", "-o", "/run/log/strace.log", BOOTCHART, NULL);
  } else {
    mypid = getpid();
    dprintf(fd, "this is the parent, mypid=%d\n", mypid);
    execl(SYSTEMD, SYSTEMD, NULL);
  }

  dprintf(fd, "end init-jm, mypid=%d\n", mypid);

  return 0;
}

Here is the kernel command-line I used (note that I am running in permissive mode) and the resulting contents of /run/log:

jmuir@vbox:~$ xargs -n1 < /proc/cmdline 
BOOT_IMAGE=(hd0,gpt2)/vmlinuz-6.11.4-301.fc41.aarch64
root=/dev/mapper/fedora_vbox-root
ro
rd.lvm.lv=fedora_vbox/root
rhgb
enforcing=0
init=/usr/lib/systemd/init-jm
jmuir@vbox:~$ ls -l /run/log
total 96
-rw-rw-rw-. 1 root root                0 Jan 14 16:00 bootchart-20250114-1600.svg
drwxr-sr-x+ 2 root systemd-journal    40 Jan 14 16:00 journal
-rw-rw-rw-. 1 root root            97423 Jan 14 16:00 strace.log

The strace log shows that systemd-bootchart is sent SIGHUP just after it starts executing.

...
pread64(166, "systemd-bootcha (512, #threads: "..., 4095, 0) = 4095
getdents64(5, 0xaaaad4d34640 /* 0 entries */, 32768) = 0
close(7)                                = 0
clock_nanosleep(CLOCK_REALTIME, 0, {tv_sec=0, tv_nsec=83836333}, NULL) = ? ERESTART_RESTARTBLOCK (Interrupted by signal)
--- SIGHUP {si_signo=SIGHUP, si_code=SI_USER, si_pid=1, si_uid=0} ---
--- SIGCONT {si_signo=SIGCONT, si_code=SI_USER, si_pid=1, si_uid=0} ---
rt_sigreturn({mask=[]})                 = -1 EINTR (Interrupted system call)
close(9)                                = 0
close(8)                                = 0
close(11)                               = 0
close(10)                               = 0
close(13)                               = 0
...

in src/bootchart.c you can see that systemd-bootchart catches SIGHUP and then sets a global variable to exit its main loop nicely.

Full strace log is attached. strace-init-jm.log

jamuir avatar Jan 14 '25 21:01 jamuir

@bluca , the root cause here is that PID 1 is sending SIGHUP to systemd-bootchart shortly after it starts up.

could this be a systemd defect?

maybe the issue should be moved/linked there.

jamuir avatar Jan 16 '25 14:01 jamuir

Maybe not a bug, although if pid == 1 we can certainly ignore SIGHUP and a few other signals...

sofar avatar Feb 09 '25 21:02 sofar

Yep I've noticed something similar myself. bootchart v235. systemd v257.4 Report isn't completely zero but seems cut early in boot at 3.3 sec or sth.

ootchart for srv0 - Fri, 27 Jun 2025 23:38:51 +0200 System: Linux 6.16.0-rc3 #26 SMP PREEMPT_DYNAMIC Mon Jun 23 01:34:14 CEST 2025 x86_64 CPU: Intel(R) Core(TM) i7-14700K Boot options: root=UUID=a569985f-a17f-4791-8aa8-b6d7e0d05604 rw rootflags=subvol=@rootfs transparent_hugepage=madvise init_on_alloc=0 init=/lib/systemd/systemd-bootchart initcall_debug consoleblank=180 quiet intel_iommu=on systemd.machine_id=4d478806645249708407d893e8506db2 Build: Debian GNU/Linux 12 (bookworm) Log start time: 3.363s Idle time: Not detected Graph data: 25.000 samples/sec, recorded 8000 total, dropped 0 samples, 356 processes, 283 filtere

I've enabled systemd service (post boot) it starts at 3.8 sec and look "normalish"

System: Linux 6.16.0-rc3 #26 SMP PREEMPT_DYNAMIC Mon Jun 23 01:34:14 CEST 2025 x86_64 CPU: Intel(R) Core(TM) i7-14700K Boot options: root=UUID=a569985f-a17f-4791-8aa8-b6d7e0d05604 rw rootflags=subvol=@rootfs transparent_hugepage=madvise init_on_alloc=0 init=/lib/systemd/systemd-bootchart initcall_debug consoleblank=180 quiet intel_iommu=on systemd.machine_id=4d478806645249708407d893e8506db2 Build: Debian GNU/Linux 12 (bookworm) Log start time: 3.815s Idle time: 19.143s Graph data: 25.000 samples/sec, recorded 8000 total, dropped 1 samples, 844991 processes, 267 filtered

idk

?

elmystico avatar Jun 30 '25 11:06 elmystico