Greetings Forum,
Running PVE 9.1.9 on a Dell T630 & a Dell T620. After a recent upgrade, no network connection. When booting into proxmox-kernel-6.17.13-6-pve from GRUB, network functions normally.
When booting proxmox-kernel-7.0.0-3-pve, 'ip a' shows 'nic0' DOWN. 'ip link set nic0 up' brings it UP, but no connection and is no longer UP after a reboot.
Has anybody experienced this with the upgrade to kernel-7?
Hard to say without any logs/journals/config, but maybe 7.0.x names the interface(s) in a different way.
If for example 6.17.x names an interface eth0 and that eth0 name is used in the configurations, it won't work if 7.0.x names it nic0.
If so, either change the config so it uses nic0 instead of the old interface name, or use interface-pinning to keep the old name.
See: https://pve.proxmox.com/pve-docs/chapter-sysadmin.html#network_override_device_names
Last edited:
as the previous poster already said, logs of the boot as well as an overview of the network configuration would be interesting:
Code:
journalctl -b > syslog.txt
cat /etc/network/interfaces
cat /etc/network/interfaces.d/sdnas the previous poster already said, logs of the boot as well as an overview of the network configuration would be interesting:
Code:
journalctl -b > syslog.txt cat /etc/network/interfaces cat /etc/network/interfaces.d/sdn
Stefan & daanw, thank you for responding. syslog.txt attached from a boot into kernel-6.17.13-6-pve. (I cannot ssh in to capture output when booting into kernel-7*, but can capture terminal screenshots via iDRAC if needed.)
Attached is an image of 'ip a' and /etc/network/interfaces from kernel-7.0.0-3-pve (on the left) and kernel-6.17.13-6-pve (on the right) for comparison. Both identify nic0 & nic1, but the boot into kernel-7 has no vmbr interface entry.
Code:
root@t620:~# stat /etc/network/interfaces.d/sdn
stat: cannot statx '/etc/network/interfaces.d/sdn': No such file or directoryServer's NIC is set to 10.0.0.41/16, 10.0.0.2 gateway
I cannot ssh in to capture output when booting into kernel-7*, but can capture terminal screenshots via iDRAC if needed.
You can get the bootlog from a previous 7.* boot with journalctl -b -1 (or -2, -3, etc.). Please post that.
Last edited:
Hello, I seem to have the same issue. Updated Proxmox from 8 to 9 and one of my two servers ended up not having a functioning network connection afterwards.
does list the onboard nic which was used before as eno1 but vmbr0 does not appear. In addition eno1 is marked down.
I checked
but it did not look suspicious to me. It lists eno1 as bridge-ports for vmbr0.
My machine is running kernel 6.8.12-25-pve on pve 9.2.2 - unfortunately I do not know which kernel was running before the update while I was still on 8.x.
Code:
/etc/network/interfaces.d/is empty.
I am currently fighting some issues to get a usb thumb drive working to get text versions of the logs. If screenshots would help, let me know.
Kind regards
Max
Attached is an image of 'ip a' and /etc/network/interfaces from kernel-7.0.0-3-pve (on the left) and kernel-6.17.13-6-pve (on the right) for comparison. Both identify nic0 & nic1, but the boot into kernel-7 has no vmbr interface entry.
Hello, I seem to have the same issue. Updated Proxmox from 8 to 9 and one of my two servers ended up not having a functioning network connection afterwards.
Is there any error output when running ifreload -avd ?
This creates quite a bit of output, so it might be best to boot the failing kernel, run the command and save it into a text file, reboot into the working kernel and then download the resulting logs / reports.
Hi Stefan,
when I try to run ifreload -avd I get permission denied (logged in as root). This seems strange to me.
By the way here are the outputs of ip a
Code:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host noprefixroute
valid_lft forever preferred_lft forever
2: eno1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether c8:**:**:**:**:** brd ff:ff:ff:ff:ff:ff
altname enp9s0
altname enxc87f5400c838and cat /etc/network/interfaces
Code:
# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!
auto lo
iface lo inet loopback
iface eno1 inet manual
auto vmbr0
iface vmbr0 inet static
address 10.241.1.113/24
gateway 10.0.10.8
bridge-ports eno1
bridge-stp off
bridge-fd 0
# source /etc/network/interfaces.d/*I compared them to the other server that is running fine and besides the naming of the interface I could not spot a difference.
Kind regards
Max
Last edited:
when I try to run ifreload -avd I get permission denied (logged in as root). This seems strange to me.
This is very odd...
can you post the output of
Code:
which ifreload
stat /usr/sbin/ifreload
apt-cache policy ifupdown2This is very odd...
can you post the output of
Code:
which ifreload stat /usr/sbin/ifreload apt-cache policy ifupdown2
which ifreload does not return anything
here is the output of stat /usr/sbin/ifreload
Code:
File: /usr/sbin/ifreload -> ../share/ifupdown2/ifupdown2
Size: 28 Blocks: 0 IO Block: 4096 symbolic link
Device: 252,1 Inode: 148057 Links: 1
Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2026-05-28 15:40:44.333912993 +0200
Modify: 2026-01-22 22:11:27.000000000 +0100
Change: 2026-05-22 17:38:12.675072859 +0200
Birth: 2026-05-22 17:38:12.668072834 +0200and apt-cache policy ifupdown2
Code:
ifupdown2:
Installed: 3.3.0-1+pmx12
Candidate: 3.3.0-1+pmx12
Version table:
*** 3.3.0-1+pmx12 500
500 http://download.proxmox.com/debian/pve trixie/pve-no-subscription amd64 Packages
100 /var/lib/dpkg/status
3.3.0-1+pmx11 500
500 http://download.proxmox.com/debian/pve trixie/pve-no-subscription amd64 Packages
3.3.0-1+pmx10 500
500 http://download.proxmox.com/debian/pve trixie/pve-no-subscription amd64 Packages
3.3.0-1+pmx9 500
500 http://download.proxmox.com/debian/pve trixie/pve-no-subscription amd64 Packages
3.3.0-1+pmx8 500
500 http://download.proxmox.com/debian/pve trixie/pve-no-subscription amd64 Packages
3.3.0-1+pmx7 500
500 http://download.proxmox.com/debian/pve trixie/pve-no-subscription amd64 Packages
3.0.0-1.3 500
500 http://ftp.de.debian.org/debian trixie/main amd64 Packages
I attached the output of journalctl -b > syslog.txt, too.
When I monitored the boot of the boot of the machine I saw apparmor failing to start which I could verify via systemctl status apparmor.service.
Might all of this point into the direction of a failed upgrade?
Thanks
Max
Seems like systemd-udev-settle.service is failing, which in turn causes ifupdown2-pre to fail. This is usually caused by some piece of hardware / driver failing to initialize.
Code:
May 28 10:20:49 t620 systemd[1]: ifupdown2-pre.service: Main process exited, code=exited, status=1/FAILURE
May 28 10:20:49 t620 systemd[1]: ifupdown2-pre.service: Failed with result 'exit-code'.
May 28 10:20:49 t620 systemd[1]: Failed to start ifupdown2-pre.service - Helper to synchronize boot up for ifupdown.
May 28 10:20:49 t620 systemd[1]: Dependency failed for networking.service - Network initialization.
May 28 10:20:49 t620 systemd[1]: networking.service: Job networking.service/start failed with result 'dependency'.
May 28 10:20:49 t620 systemd[1]: systemd-udev-settle.service: Main process exited, code=exited, status=1/FAILURE
May 28 10:20:49 t620 systemd[1]: systemd-udev-settle.service: Failed with result 'exit-code'.
May 28 10:20:49 t620 systemd[1]: Failed to start systemd-udev-settle.service - Wait for udev To Complete Device Initialization.This seems to be caused by an issue with initializing a drive which I suppose is a Fusion-io ioDrive (? - never heard before):
Code:
May 28 10:18:55 t620 kernel: <6>fioinf Fusion-io 1.65TB ioScale2 0000:05:00.0: probed fct0
May 28 10:18:55 t620 kernel: <6>fioinf Fusion-io 1.65TB ioScale2 0000:05:00.0: sector_size=512
May 28 10:18:55 t620 kernel: <6>fioinf Fusion-io 1.65TB ioScale2 0000:05:00.0: setting channel range data to [2 .. 4095]
May 28 10:18:55 t620 kernel: <6>fioinf Fusion-io 1.65TB ioScale2 0000:05:00.0: ***************************************************
May 28 10:18:55 t620 kernel: <6>fioinf Fusion-io 1.65TB ioScale2 0000:05:00.0: *** unclean shutdown detected, re-scanning log. ***
May 28 10:18:55 t620 kernel: <6>fioinf Fusion-io 1.65TB ioScale2 0000:05:00.0: *** this may take several minutes. ***
May 28 10:18:55 t620 kernel: <6>fioinf Fusion-io 1.65TB ioScale2 0000:05:00.0: ***************************************************Code:
May 28 10:18:50 t620 kernel: ------------[ cut here ]------------
May 28 10:18:50 t620 kernel: Unpatched return thunk in use. This should not happen!
May 28 10:18:50 t620 kernel: WARNING: arch/x86/kernel/cpu/bugs.c:3736 at __warn_thunk+0x10/0x20, CPU#6: (udev-worker)/910
May 28 10:18:50 t620 kernel: Modules linked in: ipmi_ssif(+) iomemory_vsl(OE+) mgag200(+) dell_smbios ghash_clmulni_intel dell_wmi_descriptor platform_profile acpi_power_meter aesni_intel ipmi_si dcdbas mei_me acpi_ipmi joydev input_leds rapl ipmi_devintf pcspkr mei intel_cstate mac_hid ipmi_msghandler zfs(PO) spl(O)
msr vhost_net vhost vhost_iotlb tap efi_pstore nfnetlink dmi_sysfs ip_tables x_tables autofs4 btrfs libblake2b xor raid6_pq dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio hid_generic usbkbd usbmouse usbhid uas hid usb_storage mpt3sas igb ahci ehci_pci libahci ioatdma raid_class lpc_ich i2c_algo_bit ehci_hcd
scsi_transport_sas dca wmi
May 28 10:18:50 t620 kernel: CPU: 6 UID: 0 PID: 910 Comm: (udev-worker) Tainted: P OE 7.0.0-3-pve #1 PREEMPT(lazy)
May 28 10:18:50 t620 kernel: Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE
May 28 10:18:50 t620 kernel: Hardware name: Dell Inc. PowerEdge T620/0F5XM3, BIOS 2.9.0 12/06/2019
May 28 10:18:50 t620 kernel: RIP: 0010:__warn_thunk+0x10/0x20
May 28 10:18:50 t620 kernel: Code: ff c3 cc cc cc cc 0f 1f 40 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 0f 1f 44 00 00 55 48 89 e5 48 8d 3d 10 97 66 02 <67> 48 0f b9 3a 5d 31 ff c3 cc cc cc cc 0f 1f 00 90 90 90 90 90 90
May 28 10:18:50 t620 kernel: RSP: 0018:ffffcc0ccfe3f918 EFLAGS: 00010246
May 28 10:18:50 t620 kernel: RAX: 0000000000000000 RBX: ffffffffc072b010 RCX: 0000000000000000
May 28 10:18:50 t620 kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffffffa55faa30
May 28 10:18:50 t620 kernel: RBP: ffffcc0ccfe3f918 R08: 0000000000000000 R09: 0000000000000000
May 28 10:18:50 t620 kernel: R10: ffff8886c7be7780 R11: 0000000000000000 R12: 0000000000000000
May 28 10:18:50 t620 kernel: R13: 0000000000000000 R14: ffffcc0ccfe3f998 R15: ffffffffc12f55c0
May 28 10:18:50 t620 kernel: FS: 00007020d284d9c0(0000) GS:ffff889e39a8f000(0000) knlGS:0000000000000000
May 28 10:18:50 t620 kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
May 28 10:18:50 t620 kernel: CR2: 00007020d13484c0 CR3: 000000012ee54004 CR4: 00000000001706f0
May 28 10:18:50 t620 kernel: Call Trace:
May 28 10:18:50 t620 kernel: <TASK>
May 28 10:18:50 t620 kernel: warn_thunk_thunk+0x16/0x30
May 28 10:18:50 t620 kernel: init_fio_driver+0x52/0x110 [iomemory_vsl]
May 28 10:18:50 t620 kernel: ? ____versions+0x33f2dfa46d60/0x33f2dfa46d60 [iomemory_vsl]
May 28 10:18:50 t620 kernel: do_one_initcall+0x5f/0x350
May 28 10:18:50 t620 kernel: do_init_module+0x97/0x290
May 28 10:18:50 t620 kernel: load_module+0x1eab/0x2150
May 28 10:18:50 t620 kernel: init_module_from_file+0xfd/0x160
May 28 10:18:50 t620 kernel: ? init_module_from_file+0xfd/0x160
May 28 10:18:50 t620 kernel: idempotent_init_module+0x110/0x300
May 28 10:18:50 t620 kernel: __x64_sys_finit_module+0x73/0xf0
May 28 10:18:50 t620 kernel: x64_sys_call+0xe2e/0x2390
May 28 10:18:50 t620 kernel: do_syscall_64+0x11c/0x14e0
May 28 10:18:50 t620 kernel: ? switch_fpu_return+0x62/0x100
May 28 10:18:50 t620 kernel: ? do_syscall_64+0x311/0x14e0
May 28 10:18:50 t620 kernel: ? count_memcg_events+0xeb/0x190
May 28 10:18:50 t620 kernel: ? handle_mm_fault+0x254/0x370
May 28 10:18:50 t620 kernel: ? do_user_addr_fault+0x2f8/0x820
May 28 10:18:50 t620 kernel: ? irqentry_exit+0xb2/0x600
May 28 10:18:50 t620 kernel: ? exc_page_fault+0x92/0x1c0
May 28 10:18:50 t620 kernel: entry_SYSCALL_64_after_hwframe+0x76/0x7e
May 28 10:18:50 t620 kernel: RIP: 0033:0x7020d2b1a7b9
May 28 10:18:50 t620 kernel: Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 27 66 0d 00 f7 d8 64 89 01 48
May 28 10:18:50 t620 kernel: RSP: 002b:00007fff19481a68 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
May 28 10:18:50 t620 kernel: RAX: ffffffffffffffda RBX: 00005b3f2ac41dd0 RCX: 00007020d2b1a7b9
May 28 10:18:50 t620 kernel: RDX: 0000000000000000 RSI: 00007020d211244d RDI: 000000000000003c
May 28 10:18:50 t620 kernel: RBP: 0000000000000000 R08: 0000000000000000 R09: 00005b3f2ab55c30
May 28 10:18:50 t620 kernel: R10: 0000000000000000 R11: 0000000000000246 R12: 00007020d211244d
May 28 10:18:50 t620 kernel: R13: 0000000000020000 R14: 00005b3f2ac41ec0 R15: 0000000000000000
May 28 10:18:50 t620 kernel: </TASK>
May 28 10:18:50 t620 kernel: ---[ end trace 0000000000000000 ]---Code:
May 28 10:18:50 t620 kernel: iomemory_vsl: module verification failed: signature and/or required key missing - tainting kernelSeems somebody ran into this already before:
I've been running this card using the above setup on my Proxmox 8.4.9 box for about 6 months with seemingly no issues, but had to take the box offline to do some upgrades over the weekend. I don't think anything I changed (added RAM, cloned the boot SSD although I've tried with the original in place and the same issue occurs) has caused this issue and it would've been the same if I'd simply rebooted.
When bringing the box back up, I noticed it wasn't bringing the ifupdown2-pre.service up, and doing a non-quiet boot this seemed related to the Fusion IO drive advising that it detected an...
I checked my bootlog and it looks like ifupdown2 starts just fine.
systemd-udev-settle starts fine, too - except for some warning about zfs import because it deprecated.
The apparmor stuff seems upset
Code:
May 28 15:51:19 pfvm-testmaster apparmor.systemd[1358]: Failed to load policy-features from '/usr/share/apparmor-features/features': No such file or directory
May 28 15:51:19 pfvm-testmaster apparmor.systemd[1339]: Error: At least one profile failed to load
May 28 15:51:19 pfvm-testmaster systemd[1]: apparmor.service: Main process exited, code=exited, status=1/FAILURE
May 28 15:51:19 pfvm-testmaster systemd[1]: apparmor.service: Failed with result 'exit-code'.
May 28 15:51:19 pfvm-testmaster systemd[1]: Failed to start apparmor.service - Load AppArmor profiles.
...
May 28 15:51:19 pfvm-testmaster lxc-apparmor-load[1545]: Failed to load policy-features from '/usr/share/apparmor-features/features': No such file or directory
...
May 28 15:51:19 pfvm-testmaster lxc-apparmor-load[1548]: Failed to load policy-features from '/usr/share/apparmor-features/features': No such file or directory
...
May 28 15:51:19 pfvm-testmaster lxc-apparmor-load[1551]: Failed to load policy-features from '/usr/share/apparmor-features/features': No such file or directoryI filtered the log for 'failed' and besides the ones mentioned here I could not find a lot of interesting stuff. What else could I look for?
Max
which ifreload does not return anything
here is the output of stat /usr/sbin/ifreload
Code:
File: /usr/sbin/ifreload -> ../share/ifupdown2/ifupdown2 Size: 28 Blocks: 0 IO Block: 4096 symbolic link Device: 252,1 Inode: 148057 Links: 1 Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2026-05-28 15:40:44.333912993 +0200 Modify: 2026-01-22 22:11:27.000000000 +0100 Change: 2026-05-22 17:38:12.675072859 +0200 Birth: 2026-05-22 17:38:12.668072834 +0200and apt-cache policy ifupdown2
Code:
ifupdown2: Installed: 3.3.0-1+pmx12 Candidate: 3.3.0-1+pmx12 Version table: *** 3.3.0-1+pmx12 500 500 http://download.proxmox.com/debian/pve trixie/pve-no-subscription amd64 Packages 100 /var/lib/dpkg/status 3.3.0-1+pmx11 500 500 http://download.proxmox.com/debian/pve trixie/pve-no-subscription amd64 Packages 3.3.0-1+pmx10 500 500 http://download.proxmox.com/debian/pve trixie/pve-no-subscription amd64 Packages 3.3.0-1+pmx9 500 500 http://download.proxmox.com/debian/pve trixie/pve-no-subscription amd64 Packages 3.3.0-1+pmx8 500 500 http://download.proxmox.com/debian/pve trixie/pve-no-subscription amd64 Packages 3.3.0-1+pmx7 500 500 http://download.proxmox.com/debian/pve trixie/pve-no-subscription amd64 Packages 3.0.0-1.3 500 500 http://ftp.de.debian.org/debian trixie/main amd64 PackagesI attached the output of journalctl -b > syslog.txt, too.
When I monitored the boot of the boot of the machine I saw apparmor failing to start which I could verify via systemctl status apparmor.service.Might all of this point into the direction of a failed upgrade?
Thanks
Max
Max - by chance, are you running a fusion-io drive?
Might all of this point into the direction of a failed upgrade?
This bootlog shows Linux kernel 6.8.12 belonging to Debian 12/bookworm and PVE 8?
I do see zfs 2.4.2-pve1 though, indicating PVE 9, which might also explain a number of zfs errors in the bootlog.
Please check with cat /etc/debian_version and pveversion.
If you are running Trixie 13.5 and Proxmox 9.2.2, please check what kernels are installed with dpkg -l | grep proxmox-kernel and check which are configured for booting using: proxmox-boot-tool kernel list. PVE9 should have 6.14.x, 6.17.x or 7.0.x.
If needed, install the latest with apt install proxmox-kernel-7.0 or apt install proxmox-kernel-6.17.
Something else, this system has corosync-qdevice while also being a node in a cluster?
As this is unrelated to the issue Psilospiral experiences, best open a new topic if you need further help.
Last edited:
Code:
May 28 10:18:50 t620 kernel: iomemory_vsl: module verification failed: signature and/or required key missing - tainting kernelSeems somebody ran into this already before:
I've been running this card using the above setup on my Proxmox 8.4.9 box for about 6 months with seemingly no issues, but had to take the box offline to do some upgrades over the weekend. I don't think anything I changed (added RAM, cloned the boot SSD although I've tried with the original in place and the same issue occurs) has caused this issue and it would've been the same if I'd simply rebooted.
When bringing the box back up, I noticed it wasn't bringing the ifupdown2-pre.service up, and doing a non-quiet boot this seemed related to the Fusion IO drive advising that it detected an...
daanw - you are correct. It seems an upgrade to the newest kernel and my fusion-io drivers did not agree. Thanks to Vladimir for helping with DKMS driver removal and reinstallation. I had to:
Get the versions installed with DKMS by requesting DKMS status
Then remove them all for the previous kernel
Code:
dkms remove iomemory-vsl/6.17.4-1-7e96ed7 --allThen remove the sources and reboot
Code:
rm -rf /usr/src/iomemory-vsl-6.17.4-1-7e96ed7
rebootMake sure I have headers for current kernel
Code:
uname -r
apt install proxmox-headers-6.17.13-6-pveClone the fusion-io driver
Code:
git clone https://github.com/RemixVSL/iomemory-vslMove into dir and compile
CSS:
cd iomemory-vsl
make dkmsVerified no kernels were pinned
Code:
proxmox-boot-tool kernel list
proxmox-boot-tool kernel unpinThen reboot.
Max - by chance, are you running a fusion-io drive?
Actually not. Just two nvme ssds and a sata one.
This bootlog shows Linux kernel 6.8.12 belonging to Debian 12/bookworm and PVE 8?
I do see zfs 2.4.2-pve1 though, indicating PVE 9, which might also explain a number of zfs errors in the bootlog.
Please check withcat /etc/debian_versionandpveversion.If you are running Trixie 13.5 and Proxmox 9.2.2, please check what kernels are installed with
dpkg -l | grep proxmox-kerneland check which are configured for booting using:proxmox-boot-tool kernel list. PVE9 should have 6.14.x, 6.17.x or 7.0.x.
If needed, install the latest withapt install proxmox-kernel-7.0orapt install proxmox-kernel-6.17.Something else, this system has corosync-qdevice while also being a node in a cluster?
As this is unrelated to the issue Psilospiral experiences, best open a new topic if you need further help.
Hi daanw,
thanks for the tips. I think I was able to move forward a bit but could not solve my issue, yet.
Here is the link to the new thread.
Max
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。