Originally I really wanted to use SDN for this but that seems to not be possible -
We have a cluster of 3 machines running ceph and on it I have a bunch of VMs running a kubernetes cluster.
I want the kubernetes cluster to provision storage directly on the proxmox ceph and created a pool for it but that means that I also need to give the guests access to the ceph public network.
I now was given an additional vlan that spans all 3 nodes just for this purpose, so I can create a "storage network" but given what I have seen so far that would also expose the proxmox management UI on that network, using SDN for core services like this is as far as I understand not possible.
(One thing I think I can do now if I really want to start doing more network management is have a QinQ SDN, VXLAN would require more switch side configuration so I want to avoid that given how much time it took to even get some "purely internal vlans")