| Summary: | dracut: initramfs and installer are missing most kernel modules since glibc 2.44 | ||
|---|---|---|---|
| Product: | IPFire | Reporter: | Michael Tremer <michael.tremer> |
| Component: | --- | Assignee: | Assigned to nobody - feel free to grab it and work on it <nobody> |
| Status: | NEW --- | QA Contact: | |
| Severity: | Crash | ||
| Priority: | Will affect an average number of users | CC: | adolf.belka, arne.fitzenreiter |
| Version: | 2 | ||
| Hardware: | unspecified | ||
| OS: | Unspecified | ||
| Bug Depends on: | |||
| Bug Blocks: | 14055 | ||
Update: Adolf has sent an update to dracut 112 (patchwork 10498 and 10497), which already contains the upstream fix (40e7af014b47 is part of 112). So we do not need to carry our own backport.
On my "next" branch (ms/next, not pushed there yet):
- My backport and "core206: Ship dracut-ng" have been reverted.
- Adolf's two patches have been applied ("dracut: Update to version 112 and move from dracut-ng to dracut" and "core206: Ship dracut").
- Core Update 206 additionally removes /usr/lib/dracut/dracut-init.sh and /usr/lib/dracut/dracut-version.sh, which dracut 112 no longer ships.
A full clean build is running to verify that the system initramfs and the installer contain all drivers again. I will post the results here.
These changes have now been pushed to ms/next (up to e0aeaa308). Correction: the history on ms/next has been cleaned up. My backport and its reverts are gone, and the branch now simply has Adolf's dracut 112 update (a7e9225a9), "core206: Ship dracut" (e0410f46e) and the removal of the leftover dracut-ng files (6335bfa98). The commit hashes in my earlier comments no longer exist. A full clean build of next with dracut 112 has finished successfully (x86_64). Results: - System initramfs: 588 kernel modules (was 69). It contains e.g. megaraid_sas, mpt3sas, hv_storvsc, vmw_pvscsi, raid1, xen-blkfront and the common network drivers again. It is now 54 MB instead of 44 MB, which is expected. - Installer: contains all network and storage drivers again (e1000e, igb, r8169, megaraid_sas, mpt3sas, hv_storvsc, vmw_pvscsi, raid1, ...). Side effect: because dracut finds all modules again, the installer's dependency on the "drm" dracut module added all GPU drivers and their firmware (nvidia, amdgpu, i915, ...), which made the installer 192 MB instead of 56 MB. The installer only needs a text console, which the kernel provides with built-in simpledrm/efifb/vesafb, so "installer: Don't add any graphics drivers" removes that dependency. The installer is now 55 MB with all drivers. Not tested yet: booting the ISO and an installed system on real hardware or VMs with one of the affected controllers. Correction to the description: existing systems would have been affected, too. Core Update 205 ships glibc 2.44 (config/rootfiles/core/205/filelists/<arch>/glibc), and its update.sh regenerates all initramfs images with "dracut --regenerate-all". With dracut-ng 108, every system that installed Core Update 205 would have received an initramfs without most storage drivers, and systems whose root file system needs one of them (MegaRAID, MPT3SAS, HPE Smart Array, Adaptec, Microsemi, Hyper-V, VMware PVSCSI, Xen, software RAID, ...) might not have booted afterwards. Core Update 205 has not been released yet, so no systems have been affected. The fix for master has been pushed to ms/master: - dracut: Update to version 112 and move from dracut-ng to dracut - installer: Don't add any graphics drivers (otherwise the installer grows to about 192 MB because of GPU firmware) - core205: Ship dracut - core205: Remove files from dracut-ng that dracut no longer ships plus a few commits that make the installer smaller. Core Update 205 now ships dracut 112 itself, so Core Update 206 no longer needs to ship it again. The branch for master has not been built yet. The fix has been pushed to origin/master (Core Update 205 now ships dracut 112, up to e46c6d544) and origin/next (up to c85bad4d0). The master branch has not been built separately; next has been built and the initramfs and the installer contain all drivers again. |
Since glibc 2.44, dracut no longer installs any kernel modules that are selected by their directory (e.g. "instmods =drivers/net/ethernet" or "=drivers/md"). It finds the directories, but silently skips every file in them. Cause: glibc 2.44 switched to the gnulib fts implementation, which honours FTS_NOSTAT. dracut-install walks module directories with FTS_NOSTAT, so regular files are now reported as FTS_NSOK instead of FTS_F, and dracut-install ignores them. Only modules that are named explicitly end up in the initramfs. Upstream has fixed this in dracut-ng commit 40e7af014b47 ("fix(dracut-install): remove FTS_NOSTAT in install_modules() fts traversal"). dracut-ng 108, which we ship, does not have the fix. Impact (x86_64, built from next with glibc 2.44): - The system initramfs only contains 69 modules. Missing are, among others: MegaRAID (megaraid_sas), LSI/Broadcom SAS (mpt3sas), HPE Smart Array (hpsa), Adaptec (aacraid), Microsemi (smartpqi), Hyper-V storage (hv_storvsc), VMware PVSCSI, Xen block devices, software RAID (raid1, raid10, raid456) and SD/MMC. These are all modules in our kernel. A system whose root file system needs one of them may not boot after its initramfs has been regenerated. - The installer only contains six network drivers (the ones the qemu-net module names explicitly: e1000, e100, pcnet32, ne2k-pci, 8139cp, virtio_net). Common NICs like e1000e, igb, ixgbe, i40e, ice, igc and r8169 are missing, so installing over the network does not work on most hardware. - Storage that is built into the kernel (AHCI, SCSI disks, USB storage, NVMe and virtio via explicit names) is not affected. Affected: Anything built with glibc 2.44, which was merged into master on 2026-07-27. Please check whether the Core Update 205 ISO and images were built with it. Existing systems that were installed earlier and are updated through Pakfire still have an older glibc and should not be affected until glibc 2.44 is shipped to them. Reproducer (inside the build chroot): mkdir /tmp/x /usr/lib/dracut/dracut-install -D /tmp/x --kerneldir /lib/modules/$(KVER) -m =drivers/net/ethernet find /tmp/x -name "*.ko*" | wc -l returns 0 without the fix and 191 with it. Status: A fix has been prepared and pushed to my "next" branch (ms/next), but it is not in origin/next yet: - d37759686 dracut-ng: Find kernel modules in directories again (backports the upstream fix) - 9059d4893 core206: Ship dracut-ng (Core Update 206 regenerates all initramfs images anyway, which will then contain all drivers again) A full build to verify the resulting images is currently running.