Created attachment 979 [details] screenshot during reboot using ipfire since years - at the moment build 159 is running fine. After trying to update to build 162 ipfire did not boot any more - waited for build 163 - but problem still exisits. Even if trying to install fresh from ISO it states "no disks found". Virtualization-environment is Citrix XenServer 7.6.
Created attachment 980 [details] screenshot after 30secs during reboot
Created attachment 981 [details] last lines in rdsosreport.txt
Is there any progress to be expected here ?
Hello, (In reply to NV Norddeutsche Vermoegen from comment #3) > Is there any progress to be expected here ? as far as I know nobody on the team has access to such a device in order to reproduce this problem. So this rather hard. I would recommend reaching out for commercial support from Lightning Wire Labs if this is time-critical for you.
Hi Michael, thank you for the fast answer. It's not timecritical, because older version still works flawless. I'll be patient and check the next builds than....if none works we'll have to look for a new proxy-solution at some time...
This is caused by incorrect regeneration of the initrd image in core 163. The commit https://git.ipfire.org/?p=ipfire-2.x.git;a=commit;h=e9084d9b4fb32373663f462bec97537ef3dd05fb added the missing kernel module for Xen, but the regeneration of the initrd image is done incorrectly I think, also as discussed here https://community.ipfire.org/t/update-161-162-ipfire-doesnt-boot/6895/17 The point is that the dracut command used by the commmit in core 163 https://git.ipfire.org/?p=ipfire-2.x.git;a=blobdiff;f=config/rootfiles/core/163/update.sh;h=f0bc7fb55bed64811c12c52f06b9435cea49a714;hp=04ca3fb614a69bcea2ff9a623d323202c7fa6e2e;hb=e9084d9b4fb32373663f462bec97537ef3dd05fb;hpb=21bac1ba86b17038d34f9fbf9489aff82280ed72 is just doing " +# rebuild initrd +dracut --force --early-microcode --strip --verbose --xz + " This will only regenerate the initrd image for the current running version, which is 5.10.x, not the 5.15.x that was updated to in core 162 and 163. So the initrd for the 5.15.x will never get the xen kernel module added. Looking at how dracut was invoked in other older core updates, I think you should be using this script https://git.ipfire.org/?p=ipfire-2.x.git;a=blob;f=src/scripts/rebuild-initrd;h=3dd08dc215f1d68eafb9969018d1dd96105ddd3c;hb=e9084d9b4fb32373663f462bec97537ef3dd05fb when a need for dracut update is present in a new core version.
So I think the fix for this, is to include a dracut regeneration for the 5.15.x or whatever is the latest version that gets deployed with the core165 update. When one then updates from core 161 or earlier, to core165, then the system should boot correctly afterwards
(In reply to Alf Høgemark from comment #6) > This is caused by incorrect regeneration of the initrd image in core 163. > > The commit > https://git.ipfire.org/?p=ipfire-2.x.git;a=commit; > h=e9084d9b4fb32373663f462bec97537ef3dd05fb > added the missing kernel module for Xen, but the regeneration of the initrd > image is done incorrectly I think, also as discussed here > https://community.ipfire.org/t/update-161-162-ipfire-doesnt-boot/6895/17 > > The point is that the dracut command used by the commmit in core 163 > https://git.ipfire.org/?p=ipfire-2.x.git;a=blobdiff;f=config/rootfiles/core/ > 163/update.sh;h=f0bc7fb55bed64811c12c52f06b9435cea49a714; > hp=04ca3fb614a69bcea2ff9a623d323202c7fa6e2e; > hb=e9084d9b4fb32373663f462bec97537ef3dd05fb; > hpb=21bac1ba86b17038d34f9fbf9489aff82280ed72 > > is just doing > " > +# rebuild initrd > +dracut --force --early-microcode --strip --verbose --xz > + > " > > > This will only regenerate the initrd image for the current running version, > which is 5.10.x, not the 5.15.x that was updated to in core 162 and 163. > So the initrd for the 5.15.x will never get the xen kernel module added. > > > Looking at how dracut was invoked in other older core updates, I think you > should be using this script > > https://git.ipfire.org/?p=ipfire-2.x.git;a=blob;f=src/scripts/rebuild-initrd; > h=3dd08dc215f1d68eafb9969018d1dd96105ddd3c; > hb=e9084d9b4fb32373663f462bec97537ef3dd05fb > > when a need for dracut update is present in a new core version. Well, it seems like the rebuild-initrd will also just regenerate for the current running version, which will not help. Maybe the script should have an option to invoke the dracut with the "regenerate all" option.
Hope this could help some people to upgrade to the latest version. To upgrade an old system and circumvent the bug make a backup! run the upgrade .... BUT before reboot run dracut --regenerate-all --force --early-microcode --strip --verbose --xz then just reboot. You may check before boot with lsinitramfs. As there is no lsinitramfs on ipfire i copied the kernel to another host and run lsinitramfs /tmp/initramfs-5.15.6-ipfire.img | grep xen You should see at least the "xen_blkfront" driver to be able to detect the xen virtual disk. There is already a link to my comment on howto solve this update Bug. https://community.ipfire.org/t/update-161-162-ipfire-doesnt-boot/6895/15?u=cmisch by the way KVER=`uname -r` dracut --force --early-microcode --xz /boot/initramfs-$KVER.img $KVER will not help as current running kernel is the old one! The the update.sh should always use the --regenerate-all option. dracut --regenerate-all --force --early-microcode --strip --verbose --xz
I can now confirm that it works upgrading from core160 to core164 when running virtualized on Xen. I think this is caused by core164 shipping a newer kernel version than core163, and that the initramfs shipped with core164 contains the xen_blkfront module. Since the timestamp of the initramfs is from 9/3-2022, and I upgraded on 11/3, I must conclude that the initramfs is shipped with the upgrade if there is a new kernel version, and not generated by dracut during upgrade.
Bumping this to MODIFIED since upcoming Core Update 167 contains the dracut changes.
https://blog.ipfire.org/post/ipfire-2-27-core-update-167-is-available-for-testing
Updated my System to Core Update 167 Development Build: master/24c8e6a6 No Problems to boot my Xen based ipfire. This is the first time i see 2 kernels at /boot vmlinuz-5.15.23-ipfire and vmlinuz-5.15.32-ipfire As i remember there was only the latest kernel before. Hopefully the old one will be removed before/during next update as there is not enough space for another kernel in /boot and the initrd grows too. same "old" initrd before and after installing the update. -rw-r--r-- 1 root root 19627296 Mar 9 21:59 initramfs-5.15.23-ipfire.img -rw-r--r-- 1 root root 26558400 Apr 12 09:40 initramfs-5.15.23-ipfire.img You are using now dracut --regenerate-all --force why you stripped the other options? --early-microcode --strip --verbose --xz Will not help much. So i assume that there are much more modules in initrd now. -rw------- 1 root root 26552906 Apr 12 09:40 initramfs-5.15.32-ipfire.img -rw------- 1 root root 25217248 Apr 12 09:58 /tmp/initramfs-5.15.32-ipfire.img
(In reply to Christof Misch from comment #13) > You are using now > dracut --regenerate-all --force > why you stripped the other options? > --early-microcode --strip --verbose --xz They have moved to a configuration file.
Thanks for info I was not aware of the conf file /usr/lib/dracut/dracut.conf.d/ipfire.conf I have dracut only on ipfire and not using it elsewhere. So you switched to # Compress using Zstandard compress="zstd" Which did not compress like "xz". But anyway the boot works!
Compression ratio is only a few kilobytes away from XZ, but performs a lot faster and decompresses somewhat faster.
Thanks for clarification
https://blog.ipfire.org/post/ipfire-2-27-core-update-167-released