scaleway
apt-get --reinstall install linux-headers-`uname -r` fails
sudo apt-get --reinstall install linux-headers-uname -r fails, also "linux-image-extra-$(uname -r)" fails.
Tried to reinstall kernel according to your docs, first it needed apt-get install gcc and apt-get install libssl-dev , otherwise the process would fail. Then it failed on 'make menuconfig KERNEL=3.18-std' with error No rule to make target `menuconfig'.
Scaleway is turning such a headache for simply installing flynn.io which is supported on all other kvm vps.
Hi @jojobo,
At boot, we are loading our own kernel before loading your disk, see https://www.scaleway.com/docs/bootscript-and-how-to-use-it/
So basically, if you apt-get install linux-headers, aptitude will download the package and write it on your disk, but it won't be taken into account because our bootscript is always loaded before your disk content
We are working on a boot on local kernel bootscript, but for now you still need to use our bootscripts
If you need a new module or special kernel configuration, do not hesitate to open an issue here, we are always happy to enhance our provided kernels
About the make menuconfig KERNEL=3.18-std, this Makefile is used to build new kernels (this is how we are building the Scaleway kernels), but it won't help you to update it on your server, the only way is to change your bootscript (see https://www.scaleway.com/docs/bootscript-and-how-to-use-it/)
Cheers
@moul thanks for thorough answer.
I'm trying to install flynn (flynn.io) and here's the issue https://github.com/flynn/flynn/issues/2696 .
And here's a suggestion from flynn member https://github.com/flynn/flynn/issues/2709#issuecomment-213594407 .
How can I solve this now? I already tried all available bootscripts.
Ok nice, can you just try to use the Docker bootscript then ? it is shipped with AUFS support
Else, can you give us instructions to reproduce your flynn installation error, so we can try to deeply investigate the error ?
I ran the installation manually without apt-get install -y "linux-headers-$(uname -r)" or "linux-image-extra-$(uname -r)", cause they return Unable to locate package linux-headers-4.5.1-docker-1 .
But installation still fails with:
Setting up linux-headers-4.4.0-21-lowlatency (4.4.0-21.37~14.04.1) ...
Examining /etc/kernel/header_postinst.d.
run-parts: executing /etc/kernel/header_postinst.d/dkms 4.4.0-21-lowlatency /boot/vmlinuz-4.4.0-21-lowlatency
configure: error:
*** Please make sure the kmod spl devel <kernel> package for your
*** distribution is installed then try again. If that fails you
*** can specify the location of the spl objects with the
*** '--with-spl-obj=PATH' option.
Error! Bad return status for module build on kernel: 4.4.0-21-lowlatency (x86_64)
Consult /var/lib/dkms/zfs/0.6.5.6/build/make.log for more information.
And then it exits with modprobe: FATAL: Module zfs not found.
Tried to install zfs manually in a couple of way, but modprobe zfs still returns module not found.
Ok, so I need to get zfs into the kernel
Can you give me instructions about how you manually tried to install flynn so I can reproduce your environment ? it will be a lot easier for me to try my fixes
To install manually, you need to run https://github.com/flynn/flynn/blob/v20160412.0/script/install-flynn.tmpl on server, WITHOUT apt-get install -y "linux-headers-$(uname -r)" and "linux-image-extra-$(uname -r)", and run apt-key adv --keyserver keyserver.ubuntu.com --recv E871F18B51E0147C77796AC81196BA81F6B0FC61.
That's it.
@moul is there a reason a non-vanilla kernel needs to be used? I imagine something in your netboot/remote disk infrastructure needs to be compiled in rather than loaded from initrd? I'm not familar with Scaleway so apologies if this something already discussed elsewhere.
If you were able to use vanilla kernels you could then just use the headers available in the distro packages to build DKMS modules like ZFS.
Failing that you should probably just also package your kernels for the distros so that people are able to install them to the local disk even if they won't be used so that kernel modules not required to boot the server can be built against them and stored on the network disk and modprobed after system startup.
Ofcourse being able to load the kernel from the disk image would be preferable but I would imagine this is somewhat more difficult without reasonably sophisticated boot infrastructure to first boot an initrd to mount the rootfs, parse the local bootloader configuration, get the guest kernel, push this to somewhere that you can get on the next reboot and cycle the machine. Possible but fairly complicated.
@josephglanville
is there a reason a non-vanilla kernel needs to be used? I imagine something in your netboot/remote disk infrastructure needs to be compiled in rather than loaded from initrd?
I suggest you to look at this schema: https://github.com/scaleway/initrd/tree/master/Linux#schema-boot-timeline
In short: we are attaching your volumes (disks) using NBD which is not handled by the boot loaders (u-boot for C1, ipxe for VPS and C2*), so we can't get your kernel from your /boot folder and that's why we created bootscript to give you ability to use different kernels.
We are working on solutions to bypass this limitation so you can just boot using your local kernel
If you were able to use vanilla kernels you could then just use the headers available in the distro packages to build DKMS modules like ZFS.
That's exactly what I understood yesterday, I will try to add a script that sync everything needed to build new modules easily using DKMS, so I can add ZFS to fix this issue, but also improve the ease of customisation for every other usages
thanks for confirming the idea
Failing that you should probably just also package your kernels for the distros so that people are able to install them to the local disk even if they won't be used so that kernel modules not required to boot the server can be built against them and stored on the network disk and modprobed after system startup.
I tried to do it using the builtin make deb-pkg, but sadly it will only work for Debian/Ubuntu, so I'm not 100% satisfied and I'm looking for another solution that may work everywhere, if you have any idea
Of course being able to load the kernel from the disk image would be preferable but I would imagine this is somewhat more difficult without reasonably sophisticated boot infrastructure to first boot an initrd to mount the rootfs, parse the local bootloader configuration, get the guest kernel, push this to somewhere that you can get on the next reboot and cycle the machine. Possible but fairly complicated.
I have a proof of concept that first load our kernel, attach your disk using nbd, and then use kmod to switch to the disk kernel, but it's not ideal
We definitely need to add the "local boot" support
@jojobo, no sorry, I still have issues releasing a kernel in a way dkms can work out of the box, as soon as I fix this, it would be a lot easier to build new modules and release zfs for this issue
@moul currently using DigitalOcean because I can deploy with Flynn there, will be waiting for this to be supported and to work out of the box so I can leverage Scaleway.
Thank you for taking this on :)
@bbraga, ok :)
actually, we have successfuly built zfs module using dkms, but I didn't found a solution to package it with the kernel so basically you can have flynn running on Scaleway right now, but it may take about 1 hour to prepare the host before being able to install zfs :/
Here is the $> history output of an host where I successfully built zfs: https://gist.github.com/moul/18e8ec495728e9db3c1e253719bd0d49
I added a note about how to build modules using DKMS on the README.md,
I also created a new issue (#275) to try to do it from our kernel builder, so you can have the module directly without waiting for about 10 minutes
@moul I went through your gist but it fails.
root@scw-1d76da:/usr/src/zfs-0.6.5.6/scripts# cat dkms.con
cat: dkms.con: No such file or directory
root@scw-1d76da:/usr/src/zfs-0.6.5.6/scripts# dkms
root@scw-1d76da:/usr/src/zfs-0.6.5.6/scripts# ^C
root@scw-1d76da:/usr/src/zfs-0.6.5.6/scripts#
root@scw-1d76da:/usr/src/zfs-0.6.5.6/scripts# apt-get install zfs-utils
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package zfs-utils
root@scw-1d76da:/usr/src/zfs-0.6.5.6/scripts# apt-get install zfsutils
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package zfsutils
root@scw-1d76da:/usr/src/zfs-0.6.5.6/scripts# apt-get install zfsutils-linux
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package zfsutils-linux
Hi @jojobo, can you try this instead: https://github.com/scaleway/kernel-tools#how-to-build-a-custom-kernel-module ?
@moul . I tried your suggestion, but install zfsutils-linux fails again.
HOSTCC scripts/conmakehash
HOSTCC scripts/sortextable
HOSTCC scripts/asn1_compiler
HOSTCC scripts/extract-cert
make: Leaving directory `/usr/src/linux-4.5.1'
root@scw-413b14:~# apt-get install zfsutils-linux
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package zfsutils-linux
I tried on TYPE VC1S instance, if that makes a difference.
How ok, apparently your error is not related with the kernel stuff, but with aptitude
Can you ensure your local cache is update with apt-get update then run the apt-get install zfs-utils-linux ?
If it still doesn't work, can you give me the result of
-
apt-cache search zfs -
cat /etc/scw-release -
lsb_release -a
Thanks
Still not working @moul .
E: Unable to locate package zfsutils-linux
root@scw-451fff:~# apt-cache search zfs
parted - disk partition manipulator
bzflag-server - bzfs - BZFlag game server
zfs-fuse - ZFS on FUSE
root@scw-451fff:~# cat /etc/scw-release
IMAGE_FLAVORS="common docker-based feature-motd upstart"
IMAGE_TOOLS_CHECKOUT="764a58e59811d5f6217a3f71a4e009d6ec3e5138"
IMAGE_ID="Ubuntu Trusty"
IMAGE_RELEASE=2016-04-06
IMAGE_CODENAME=ubuntu
IMAGE_DESCRIPTION="Ubuntu Trusty (14.04 LTS)"
IMAGE_HELP_URL="https://community.scaleway.com"
IMAGE_SOURCE_URL="https://github.com/scaleway/image-ubuntu"
IMAGE_DOC_URL="https://scaleway.com/docs"
root@scw-451fff:~# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 14.04.4 LTS
Release: 14.04
Codename: trusty
root@scw-451fff:~#
@moul, @jgillich , the whole point was to install flynn, and flynn recommended ubuntu 14.04. I don't know if they support ubuntu 16 yet.
do you have a contact from the flynn team ? maybe they can help us to fix this the best way
I think you already seen I raised this issue on flynn/flynn https://github.com/flynn/flynn/issues/2696 https://github.com/flynn/flynn/issues/2709 . The other participants are from flynn team.
zfsutils-linux is the package name on 16.04, the flynn install script adds a PPA that has ubuntu-zfs (looks like both the userland and the kernel module, so install will probably fail).
@jgillich the ubuntu-zfs package builds the kernel modules using DKMS, even if it ships some pre-compiled modules those would be intended for use via DKMS caching mechanism.
Hailing from flynn/flynn here. Unfortunately I don't use Scaleway so it's harder for me understand why DKMS doesn't "just work".
@moul so, do you think it's impossible to install flynn on scaleway for now?