Foreword and update
The guide was long overdue for an update, but luckily yesterday i've managed to test a couple of things that allow me to be quite certain that the setup is stable and can be used in production. Several notes. The drivers are based on snuf drivers. From my experience with Western Digital, the current state of things is a train-wreck and they are not only not willing, but also probably incapable of properly maintaining the drivers. This guide applies to both kernels: 4.x (Proxmox 5) and 5.x (Proxmox 6). Pay attention to the drivers downloaded not to pick the wrong ones. All drivers are from my Dropbox repository. This will prevent issues with folder naming and allow for a single copy-paste action to deploy the drivers. This guide was tested for latest versions of Proxmox 5 and Proxmox 6.

Quick update #2: Just purchased a SX350 (ioDrive3) ioMemory card. A detailed instruction for deployment will be posted as soon as i figure it out. I know that we're all doing our part for the open source community, but wanted to thank everyone who has donated or requested paid support, since this really helps doing this digging and testing.

Quick update #3: Updated the guide for Proxmox 7.

–––

Hello, everyone!

Spent the past 24h figuring things out and decided to share the knowledge, since it's scarce on the subject.

Background
I was trying to figure out the optimal setup for running VMs on Proxmox. The main bottleneck always seemed to be IO related (was thinking IOPS & RW speed). I started investigating the appropriate solution and decided that the only way to know for sure is to actually test things out. I wanted to try 2 scenarios - using 4 SAS HDDs in a ZFS RAID10 pool with ZIL/SLOG on a PCIe SSD and simply run VMs off an LVM-thin pool off a PCIe SSD. Since i was using the Dell R620 old workhorse, the only suggestion that i've found from Dell regarding compatible PCIe SSDs was Fusion-Io iodrive2. So i got one of those. Turned out that it's not an easy thing to get it working on Proxmox.

The problems

1. Fusion-Io drivers were designed with Enterprise OSs in mind (RHEL, SLES, ESXi, etc). Later on Sandisk was kind enough to expand the drivers to other OSs (updated list is here: https://link-app.sandisk.com/Home/SoftwareDownload). But you cannot get these drivers to work directly on Proxmox and you need to recompile them from source for your kernel. Moreover, only kernels up to 4.x are supported.
2. The kernels do get updated and it would be pain to lose the drive with every update.

Mapping the solution

1. The very first thread i've encountered talking about the idea of using Fusion-Io with Proxmox was this one: https://www.reddit.com/r/homelab/comments/8lp8hw/fusionio_drivers_for_proxmox/
2. I understood that i would likely need the source of the drivers and the kernel headers.
3. The post referenced this GitHub repo: https://github.com/snuf/iomemory-vsl. The drivers there are not really necessary, since you can actually get the updated ones from SanDisk directly and actually compile them against latest Proxmox kernel (as you will later find out). What caught my eye was the DKMS approach to keep the drivers in line with the kernel updates.
4. I've noticed that simply installing the kernel headers didn't work properly when using apt install pve-headers so i discovered that the precise version needs to be indicated. The current kernel version is available via uname -r

The actual process

1. (This step applies only to Proxmox 5 instances) Copy this whole code to the Proxmox 5 console. It will automatically handle the drivers download, dependencies download, preparations and installation:
   

   Code:

   apt update && apt install --assume-yes pve-headers pve-headers-`uname -r` zip unzip gcc fakeroot build-essential debhelper rsync dkms && apt upgrade && apt autoremove --assume-yes && \
   mkdir /home/temp && cd /home/temp && \
   wget -O iomemory-vsl.zip https://www.dropbox.com/s/ktj2ive9elah04n/iomemory-vsl-4.20.1.zip?dl=1 && \
   wget -O fio-common_3.2.16.1731-1.0_amd64.deb https://www.dropbox.com/s/pd2ohfaufhwqc34/fio-common_3.2.16.1731-1.0_amd64.deb?dl=1 && \
   wget -O fio-firmware-fusion_3.2.16.20180821-1_all.deb https://www.dropbox.com/s/kcn5agi6lyikicf/fio-firmware-fusion_3.2.16.20180821-1_all.deb?dl=1 && \
   wget -O fio-sysvinit_3.2.16.1731-1.0_all.deb https://www.dropbox.com/s/g39l6lg9of6eqze/fio-sysvinit_3.2.16.1731-1.0_all.deb?dl=1 && \
   wget -O fio-util_3.2.16.1731-1.0_amd64.deb https://www.dropbox.com/s/57huby17mteg6wp/fio-util_3.2.16.1731-1.0_amd64.deb?dl=1 && \
   unzip iomemory-vsl.zip && cd /home/temp/iomemory-vsl && \
   cp -r /home/temp/iomemory-vsl/root/usr/src/iomemory-vsl-3.2.16 /usr/src/ && \
   mkdir -p /var/lib/dkms/iomemory-vsl/3.2.16/build && \
   ln -s /usr/src/iomemory-vsl-3.2.16 /var/lib/dkms/iomemory-vsl/3.2.16/source && \
   dkms build -m iomemory-vsl -v 3.2.16 && \
   dkms install -m iomemory-vsl -v 3.2.16 && \
   modprobe iomemory-vsl && \
   cd /home/temp && \
   dpkg -i fio-firmware-fusion_3.2.16.20180821-1_all.deb fio-util_3.2.16.1731-1.0_amd64.deb fio-sysvinit_3.2.16.1731-1.0_all.deb fio-common_3.2.16.1731-1.0_amd64.deb

2. (This step applies only to Proxmox 6 instances) Copy this whole code to the Proxmox 6 console. It will automatically handle the drivers download, dependencies download, preparations and installation:
   

   Code:

   apt update && apt install --assume-yes pve-headers pve-headers-`uname -r` zip unzip gcc fakeroot build-essential debhelper rsync dkms && apt upgrade && apt autoremove --assume-yes && \
   mkdir /home/temp && cd /home/temp && \
   wget -O iomemory-vsl.zip https://www.dropbox.com/s/n3a03ueumnjzbp8/iomemory-vsl-5.6.0.zip?dl=1 && \
   wget -O fio-common_3.2.16.1731-1.0_amd64.deb https://www.dropbox.com/s/pd2ohfaufhwqc34/fio-common_3.2.16.1731-1.0_amd64.deb?dl=1 && \
   wget -O fio-firmware-fusion_3.2.16.20180821-1_all.deb https://www.dropbox.com/s/kcn5agi6lyikicf/fio-firmware-fusion_3.2.16.20180821-1_all.deb?dl=1 && \
   wget -O fio-sysvinit_3.2.16.1731-1.0_all.deb https://www.dropbox.com/s/g39l6lg9of6eqze/fio-sysvinit_3.2.16.1731-1.0_all.deb?dl=1 && \
   wget -O fio-util_3.2.16.1731-1.0_amd64.deb https://www.dropbox.com/s/57huby17mteg6wp/fio-util_3.2.16.1731-1.0_amd64.deb?dl=1 && \
   unzip iomemory-vsl.zip && cd /home/temp/iomemory-vsl && \
   cp -r /home/temp/iomemory-vsl/root/usr/src/iomemory-vsl-3.2.16 /usr/src/ && \
   mkdir -p /var/lib/dkms/iomemory-vsl/3.2.16/build && \
   ln -s /usr/src/iomemory-vsl-3.2.16 /var/lib/dkms/iomemory-vsl/3.2.16/source && \
   dkms build -m iomemory-vsl -v 3.2.16 && \
   dkms install -m iomemory-vsl -v 3.2.16 && \
   modprobe iomemory-vsl && \
   cd /home/temp && \
   dpkg -i fio-firmware-fusion_3.2.16.20180821-1_all.deb fio-util_3.2.16.1731-1.0_amd64.deb fio-sysvinit_3.2.16.1731-1.0_all.deb fio-common_3.2.16.1731-1.0_amd64.deb

3. (This step applies only to Proxmox 7 instances) Copy this whole code to the Proxmox 7 console. It will automatically handle the drivers download, dependencies download, preparations and installation:
   

   Code:

   apt update && apt --assume-yes install zip unzip pve-headers pve-headers-`uname -r` && apt --assume-yes upgrade && apt --assume-yes autoremove && \
   mkdir /home/temp  && cd /home/temp  && \
   wget -O iomemory-vsl.zip https://www.dropbox.com/s/df06nuzvqndlvnk/iomemory-vsl-5.12.1.zip?dl=1 && \
   wget -O fio-common_3.2.16.1731-1.0_amd64.deb https://www.dropbox.com/s/pd2ohfaufhwqc34/fio-common_3.2.16.1731-1.0_amd64.deb?dl=1 && \
   wget -O fio-firmware-fusion_3.2.16.20180821-1_all.deb https://www.dropbox.com/s/kcn5agi6lyikicf/fio-firmware-fusion_3.2.16.20180821-1_all.deb?dl=1 && \
   wget -O fio-sysvinit_3.2.16.1731-1.0_all.deb https://www.dropbox.com/s/g39l6lg9of6eqze/fio-sysvinit_3.2.16.1731-1.0_all.deb?dl=1 && \
   wget -O fio-util_3.2.16.1731-1.0_amd64.deb https://www.dropbox.com/s/57huby17mteg6wp/fio-util_3.2.16.1731-1.0_amd64.deb?dl=1 && \
   unzip iomemory-vsl.zip && cd /home/temp/iomemory-vsl-5.12.1 && \
   apt update && apt --assume-yes install gcc fakeroot build-essential debhelper rsync dkms && \
   cp -r /home/temp/iomemory-vsl-5.12.1/root/usr/src/iomemory-vsl-3.2.16 /usr/src/ && \
   mkdir -p /var/lib/dkms/iomemory-vsl/3.2.16/build && \
   ln -s /usr/src/iomemory-vsl-3.2.16 /var/lib/dkms/iomemory-vsl/3.2.16/source && \
   dkms build -m iomemory-vsl -v 3.2.16 && \
   dkms install -m iomemory-vsl -v 3.2.16 && \
   modprobe iomemory-vsl && \
   cd /home/temp && \
   dpkg -i fio-firmware-fusion_3.2.16.20180821-1_all.deb fio-util_3.2.16.1731-1.0_amd64.deb fio-sysvinit_3.2.16.1731-1.0_all.deb fio-common_3.2.16.1731-1.0_amd64.deb

4. You may need to compile the drivers for the other kernels present on the system:
   ls /var/lib/initramfs-tools | sudo xargs -n1 /usr/lib/dkms/dkms_autoinstaller start or ls /lib/modules | sudo xargs -n1 /usr/lib/dkms/dkms_autoinstaller start (Proxmox 7)
5. You may need to reboot the OS and make sure the device is attached after the reboot by running fio-status -a
6. Format the device according to your needs and enjoy.

I'm still waiting to see how the device holds after the kernel update, just to make sure it's production ready. Other than that, seems to work properly. I've managed to create LVM-thin pool and import the VMs on it. The IO is very subpar (topic for another thread), but i was glad that a very decent hardware may still be used with the latest OSs.

Hope this helps a couple of souls out there. And you can buy me a beer if it saved you a couple of sleepless nights