Hello all, I’m having some network problems on my server (Ubuntu Lunar Lobster Server Edition), namely my plex port is no longer reachable from outside the network despite it being port forwarded and allowed through ufw. A few days ago I used airvpn’s eddie gui program, and it completely wrecked my network. Caused it to be completely cut off from internet when disabled, even with the killswitch function off.
I rolled back to a snapshot I’d made earlier using snapper and grub-btrfs and thought the problem was solved. But while I regained the ability to access the internet, it still seems to have messed something up. I can’t seem to get my kernel upgraded. I have 6.2.0-25 installed, but I can only boot into 6.2.0-24, even when I apt removed the older kernel. I believe this is what is causing my networking issues, though even if it’s not I’d need a solution so as to be able to keep my kernel updated.
I’m at a bit of a loss at this point and am wondering if I’ve dug myself in too deep with grub-btrfs. I’m contemplating getting a fresh image on there and sticking to manual use of snapper.
Any help would be much appreciated, thanks.
Did you setup this server with a boot partition?
Rolling back the root subvolume with an older kernel could create a mismatch to the boot partitions kernel. Not sure if that’s contributing to your problem, but might be worth looking into.
Yes, I have a separate boot partition. How would I fix a mismatch? The only posible solution I’ve found is to directly edit the grub entries, but that’s a bit beyond what I’ve done with grub before. Thanks for the response!
Only thing I can think of is the subvolumes left over might be causing an issue. I don’t know how snapper performs a rollback and I’ve seen a couple ways to do it. Sometimes its modifying the default subvolume to the snapshot you want to rollback to. This is fine IF your kernel params are NOT specifying the subvol. That would look like this
rootflags=subvol=subvolume_name.Usually I just rename the subvols and make sure the snapshot I’m renaming to replace the current root subvol is not set to read only.
So rename current root to root.broken Then rename snapshot to root Then set readonly prop to false.
Well, I tried rolling back to another snapshot and checked my kernel params and its not in there. My port issue seems to be fixed but i still do not boot into the new kernel after updating and rebooting. Edit: I think I need to clean up my /boot directory somehow since it’s partition is not btrfs. But I’m unsure how to do it, or where to read up on it.
Well as long as you take a snapshot of your data and move that snapshot to another drive, you’re free to tinker without real worry. Arch Linux wiki should have lots of examples of what you’re trying to fix or modify. There’s always a fresh install as well, which isn’t ideal most of the time but as long as the data is safe, then its an option. Best of luck, hope you’re able to resolve the issue.
Thanks, it’s looking like I’ll be doing a reset. It looks like grub-btrfs creates its own entries and points towards the boot directory within the /.snapshot/ directory, completely ignoring my boot partitions current files.
Ubuntu was never my first choice but was necessary for using my arc380 with plex transcoding. Might as well take this opportunity to move back to debian with bookworm. Only trouble is I would prefer a 6.2 over 6.1 kernel distribution, and while I enjoy arch on my laptops, I’m not sure I’d want to update my server so frequently.
I think you’ve hit the nail on the head. The way grub-btrfs works is by changing the default root snapshot subvolume. I’m still not sure what I did wrong to get the rollback stuck to the particular kernel. But I’ll give what you’ve written here a try. Thanks again.
Have you tried a normal “grub-update” command to make sure it gets the right kernel to boot up for you? Not 100% familiar with Ubuntu, but perhaps that’s all you’d need to run to get it booting to the right version.
Yes, I’ve done that to no avail. And when I run “dpkg -l | grep linux-image” it shows I have five different kernels on my system, despite me only having the latest one installed through apt. Edit: Scratch that, I think I was misreading the input but the fact remains I can’t seem to get rid of 6.2.0-24
Edit: autocorrect being annoying
Again, no Ubuntu expert here, so do this at your own risk (someone jump in and say this is a bad idea if it is). Not irreversible as you can just revert what I’m telling you via live CD, but just wanted to leave the warning.
If
uname -rshows the old kernel *-24, and grub update is choosing that instead of *-25, perhaps Ubuntu relies on the last change/created date or something weird. What you can check is if *-24 AND *-25 show up in /usr/src. If both are there, do (as root/sudo):mv /usr/src/*6.2.0-24* /root. Now rungrub-updateagain. I think it should tell you which version it chose for the boot menu.Next try:
grep 6.2.0 /boot/grub/grub.cfgto verify you see the new version.- If you see it, reboot and see if it starts with it. If you can’t boot, go through the liveCD and move that directories back to /usr/src, chroot to the drive (see https://www.turnkeylinux.org/docs/chroot-to-repair-system for how to do this properly), and after chrooting, run
grub-updateagain. - If you don’t see it or get nothing back from grep, move the directory back and re-run
grub-updateto avoid not being able to boot back in. This wasn’t the fix we were hoping for.
^ this is all assuming Ubuntu puts its kernels in /usr/src and uses that location as reference to what’s available/installed. If you see nothing there, then something else will need to be attempted
Edit 2: another thing to check is if the compiled kernels are in /boot. Basically follow the same instructions above but use /boot/*-24 instead. This might actually be what needs to be done now that I think about it, not necessarily /usr/src.
- If you see it, reboot and see if it starts with it. If you can’t boot, go through the liveCD and move that directories back to /usr/src, chroot to the drive (see https://www.turnkeylinux.org/docs/chroot-to-repair-system for how to do this properly), and after chrooting, run