e1000 driver hang
invalid@exam
·
2026-06-24
·
via Proxmox Support Forum
In the past week we are seeing random e1000e 0000:00:1f.6 eno1: Detected Hardware Unit Hang failuresacross all our nodes, even different hardware hosts. Must do a reset of the host.
There are lots of references to this issue going back 5+ years. Was there a driver change with the latest updates? We've run years with this hardware without issue. Now just this week its popping up all over.
Kernel Version Linux 5.0.21-2-pve #1 SMP PVE 5.0.21-3 (Thu, 05 Sep 2019 13:56:01 +0200)
Sep 22 20:03:08 vmhost03 kernel: [154458.471981] e1000e 0000:00:1f.6 eno1: Detected Hardware Unit Hang:
Sep 16 14:36:41 vmhost03 kernel: [67010.834277] NETDEV WATCHDOG: eno1 (e1000e): transmit queue 0 timed out
I'm seeing this same issue on both my Intel NUC8i5BEH units (Intel® Ethernet Connection I219-V) - Both run PVE6 and produce the same set of "e1000e 0000:00:1f.6 eno1: Detected Hardware Unit Hang" error messages as above in /var/log/syslog, sometimes in excess of 150 times a day...
I assume this is a driver issue (given it happens on both hardware units)?
Hey,
I got the same error, see my forum post bug tracker on kernel.org also on newer kernel 5.0+DKMS or intel source )...lets hope is solves the hangs... will give feedback
Still got hangs and also a call trace this time:
Code:
[50316.713031] e1000e 0000:00:1f.6 eno1: Detected Hardware Unit Hang:
TDH <71>
TDT <9a>
next_to_use <9a>
next_to_clean <70>
buffer_info[next_to_clean]:
time_stamp <100becc36>
next_to_watch <71>
jiffies <100becd68>
next_to_watch.status <0>
MAC Status <40080083>
PHY Status <796d>
PHY 1000BASE-T Status <3800>
PHY Extended Status <3000>
PCI Status <10>
[50318.729038] e1000e 0000:00:1f.6 eno1: Detected Hardware Unit Hang:
TDH <71>
TDT <9a>
next_to_use <9a>
next_to_clean <70>
buffer_info[next_to_clean]:
time_stamp <100becc36>
next_to_watch <71>
jiffies <100becf60>
next_to_watch.status <0>
MAC Status <40080083>
PHY Status <796d>
PHY 1000BASE-T Status <3800>
PHY Extended Status <3000>
PCI Status <10>
[50320.749008] e1000e 0000:00:1f.6 eno1: Detected Hardware Unit Hang:
TDH <71>
TDT <9a>
next_to_use <9a>
next_to_clean <70>
buffer_info[next_to_clean]:
time_stamp <100becc36>
next_to_watch <71>
jiffies <100bed159>
next_to_watch.status <0>
MAC Status <40080083>
PHY Status <796d>
PHY 1000BASE-T Status <3800>
PHY Extended Status <3000>
PCI Status <10>
[50322.760939] e1000e 0000:00:1f.6 eno1: Detected Hardware Unit Hang:
TDH <71>
TDT <9a>
next_to_use <9a>
next_to_clean <70>
buffer_info[next_to_clean]:
time_stamp <100becc36>
next_to_watch <71>
jiffies <100bed350>
next_to_watch.status <0>
MAC Status <40080083>
PHY Status <796d>
PHY 1000BASE-T Status <3800>
PHY Extended Status <3000>
PCI Status <10>
[50322.952657] ------------[ cut here ]------------
[50322.952659] NETDEV WATCHDOG: eno1 (e1000e): transmit queue 0 timed out
[50322.952672] WARNING: CPU: 2 PID: 0 at net/sched/sch_generic.c:466 dev_watchdog+0x221/0x230
[50322.952673] Modules linked in: tcp_diag inet_diag binfmt_misc veth ebtable_filter ebtables ip_set ip6table_filter ip6_tables iptable_filter bpfilter softdog nfnetlink_log nfnetlink snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic ledtrig_audio intel_rapl x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel crct10dif_pclmul crc32_pclmul ghash_clmulni_intel snd_soc_skl snd_soc_hdac_hda snd_hda_ext_core snd_soc_skl_ipc snd_soc_sst_ipc i915 snd_soc_sst_dsp kvmgt vfio_mdev mdev vfio_iommu_type1 snd_soc_acpi_intel_match vfio snd_soc_acpi kvm snd_soc_core aesni_intel irqbypass snd_compress ac97_bus aes_x86_64 snd_pcm_dmaengine crypto_simd wmi_bmof snd_hda_intel snd_hda_codec snd_hda_core snd_hwdep drm_kms_helper cryptd glue_helper intel_cstate intel_rapl_perf drm pcspkr i2c_algo_bit mei_me intel_wmi_thunderbolt fb_sys_fops snd_pcm syscopyarea snd_timer sysfillrect snd sysimgblt soundcore mei intel_pch_thermal acpi_pad mac_hid acpi_tad vhost_net vhost tap ib_iser rdma_cm iw_cm
[50322.952693] ib_cm ib_core iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi sunrpc ip_tables x_tables autofs4 zfs(PO) zunicode(PO) zlua(PO) zcommon(PO) znvpair(PO) zavl(PO) icp(PO) spl(O) btrfs xor zstd_compress raid6_pq libcrc32c uas usb_storage ahci e1000e i2c_i801 libahci wmi pinctrl_cannonlake video pinctrl_intel
[50322.952703] CPU: 2 PID: 0 Comm: swapper/2 Tainted: P O 5.0.21-4-pve #1
[50322.952704] Hardware name: Intel(R) Client Systems NUC8i3BEH/NUC8BEB, BIOS BECFL357.86A.0071.2019.0510.1505 05/10/2019
[50322.952705] RIP: 0010:dev_watchdog+0x221/0x230
[50322.952706] Code: 00 49 63 4e e0 eb 92 4c 89 ef c6 05 7d 40 ee 00 01 e8 03 22 fc ff 89 d9 4c 89 ee 48 c7 c7 e0 aa 1d a8 48 89 c2 e8 71 4f 77 ff <0f> 0b eb c0 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48
[50322.952707] RSP: 0018:ffff9ae8adb03e68 EFLAGS: 00010286
[50322.952707] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000006
[50322.952708] RDX: 0000000000000007 RSI: 0000000000000096 RDI: ffff9ae8adb16440
[50322.952708] RBP: ffff9ae8adb03e98 R08: 0000000000000001 R09: 00000000000003cb
[50322.952709] R10: 0000000000000004 R11: 0000000000000000 R12: 0000000000000001
[50322.952709] R13: ffff9ae8a192c000 R14: ffff9ae8a192c4c0 R15: ffff9ae8a1c75280
[50322.952710] FS: 0000000000000000(0000) GS:ffff9ae8adb00000(0000) knlGS:0000000000000000
[50322.952711] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[50322.952711] CR2: 0000000001880000 CR3: 000000017560e003 CR4: 00000000003626e0
[50322.952711] Call Trace:
[50322.952713] <IRQ>
[50322.952715] ? pfifo_fast_enqueue+0x120/0x120
[50322.952717] call_timer_fn+0x30/0x130
[50322.952718] run_timer_softirq+0x3e4/0x420
[50322.952720] ? ktime_get+0x40/0xa0
[50322.952721] ? lapic_next_deadline+0x26/0x30
[50322.952723] ? clockevents_program_event+0x93/0xf0
[50322.952724] __do_softirq+0xdc/0x2f3
[50322.952726] irq_exit+0xc0/0xd0
[50322.952727] smp_apic_timer_interrupt+0x79/0x140
[50322.952728] apic_timer_interrupt+0xf/0x20
[50322.952728] </IRQ>
[50322.952730] RIP: 0010:cpuidle_enter_state+0xbd/0x450
[50322.952731] Code: ff e8 17 9d 85 ff 80 7d c7 00 74 17 9c 58 0f 1f 44 00 00 f6 c4 02 0f 85 63 03 00 00 31 ff e8 2a d2 8b ff fb 66 0f 1f 44 00 00 <45> 85 ed 0f 89 cf 01 00 00 41 c7 44 24 08 00 00 00 00 48 83 c4 18
[50322.952731] RSP: 0018:ffffbddc41973e60 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff13
[50322.952732] RAX: ffff9ae8adb221c0 RBX: ffffffffa8553e60 RCX: 000000000000001f
[50322.952732] RDX: 00002dc4b9b788ec RSI: 000000002aaaaaaa RDI: 0000000000000000
[50322.952733] RBP: ffffbddc41973ea0 R08: 0000000000000000 R09: 0000000000021a80
[50322.952733] R10: 0000895ded97c40b R11: ffff9ae8adb21044 R12: ffff9ae8adb2cd00
[50322.952734] R13: 0000000000000004 R14: ffffffffa8553ff8 R15: ffffffffa8553fe0
[50322.952735] cpuidle_enter+0x17/0x20
[50322.952737] call_cpuidle+0x23/0x40
[50322.952737] do_idle+0x22c/0x270
[50322.952738] cpu_startup_entry+0x1d/0x20
[50322.952740] start_secondary+0x1ab/0x200
[50322.952741] secondary_startup_64+0xa4/0xb0
[50322.952742] ---[ end trace 68ab007781b80a74 ]---
[50322.952754] e1000e 0000:00:1f.6 eno1: Reset adapter unexpectedly
[50322.952881] vmbr0: port 1(eno1) entered disabled state
[50326.795286] e1000e: eno1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
[50326.795367] vmbr0: port 1(eno1) entered blocking state
[50326.795370] vmbr0: port 1(eno1) entered forwarding state
[ICODE][/QUOTE]
Still got hangs and also a call trace this time:
Maybe should try to disable tcp segmentation offload and generic segmentation offload
Code:
ethtook -K <interface> tso off gso off
There is known problem with Intel i218/i219 NIC and e1000e driver - buffer overrun while the I219 is processing DMA transactions. Problem was fixed in kernel 4.15 https://github.com/torvalds/linux/commit/b10effb92e272051dd1ec0d7be56bf9ca85ab927
But I'm on a 5.X-kernel and the problem's the same.
But I'm on a 5.X-kernel and the problem's the same.
I'm on a 5.X-kernel too, udp buffer overrun was fixed in e1000 driver in 4.15 kernel while caused segmentaion offload limitations, so you should turn off it if you want full speed.
Did you try solution before writing answer? For me disabling in driver tcp segmentation offload (tso) and generic segmentation offload (gso) solved problem.
Last edited: Nov 25, 2019
this fixed the problem for me also, but degraded the transfer speed..
I also run into this once with my NUC8i5BEH :-( How much the speed degraded?
Ι tried to transfer the same file (103Gb, scp transfer in a server in the same data center, Hetzner) before and after running this command..
103GB 67.7MB/s 25:58 this is before
the transfer rate is almost the half...
puuhhh ... but better then a crash I think
its not crashing.. it just loses its ethernet..
The bugs seems fixed in 5.2.2 ... but pve6 is on 5.0.x ... maybe the Proxmox guys could patch it themself in their kernel version? Maybe open an issue in their Bugtracker?
The bugs seems fixed in 5.2.2 ... but pve6 is on 5.0.x ... maybe the Proxmox guys could patch it themself in their kernel version? Maybe open an issue in their Bugtracker?
kernel 5.3 is already available in no subscription repo. (apt install pve-kernel-5.3)
I'm still getting the e1000 hangs with kernel 5.3
I'm still getting the e1000 hangs with kernel 5.3
I'll verifiy source in proxmox git in this commit is correctly reverted.
I'm still getting the e1000 hangs with kernel 5.3
Which pve-kernel-5.3 version ?
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。
