Hello!

I tried using the new SDN WireGuard Fabric in Proxmox VE 9.2.3. My setup consists of three PVE 9.2.3 nodes in a cluster with a shared SDN WireGuard Fabric (these nodes are not connected via L2). On top of that, I configured a VXLAN network with MTU 1370 (1500 base interface - 80WG - 50VXLAN).

The issue is that the WireGuard interfaces created by the Fabric are configured with the default MTU of 1500. With this configuration, traffic between nodes is unstable and I experience packet loss in the range of 20–30%.

As soon as I manually set the MTU of the WireGuard interfaces to 1420 (the commonly recommended value for WireGuard), the SDN VXLAN network becomes completely stable: no packet loss, no excessive latency, and no noticeable jitter. However, the PVE Fabric configuration currently does not seem to provide any way to specify the MTU for the underlying WireGuard interfaces. There is an MTU setting for VXLAN, but not for WG itself. I have tried setting the MTU manually and also adding it to /etc/network/interfaces.d/sdn. Both approaches work only until Proxmox regenerates or reapplies the configuration. This appears to happen at runtime, as the MTU eventually reverts to 1500 on its own.

Has anyone managed to run a WireGuard Fabric with VXLAN on top of it in a stable way? Or perhaps uses another simple and reliable setup using the new WireGuard Fabric?

Default base interface (with packet loss)

16: wg1sdn: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/none

And after manual MTU adjustment (no packet loss)

# ip link show wg1sdn
16: wg1sdn: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/none

Agree that this should be implemented, could you open a bugzilla entry at https://bugzilla.proxmox.com ?

I have tried setting the MTU manually and also adding it to /etc/network/interfaces.d/sdn. Both approaches work only until Proxmox regenerates or reapplies the configuration. This appears to happen at runtime, as the MTU eventually reverts to 1500 on its own.

As a workaround, in the meanwhile you can set it in /etc/network/interfaces directly:

iface wg0 inet manual
    mtu 1420

This should not get overwritten by the SDN stack.

On top of that, I configured a VXLAN network with MTU 1370 (1500 base interface - 80WG - 50VXLAN).

The issue is that the WireGuard interfaces created by the Fabric are configured with the default MTU of 1500. With this configuration, traffic between nodes is unstable and I experience packet loss in the range of 20–30%.

That seems odd, I've tested this locally with a VXLAN zone that has 1370 MTU and I did not experience any packet loss. Are you sending packets from inside a guest, I suppose? Did you double-check that the MTU of the network interface inside the guest is set to 1370?