How to recover ZFS after reinstalling proxmox

I recently fudged my production proxmox server after (stupidly) trying to join it to a cluster without first making sure both systems were updated to the same version of proxmox.

The production server was still running so I was able to backup the critical VMs to another proxmox host to get everything up and running again.

I reinstalled proxmox on my production server.

At this point I could have backed up and restored from the other proxmox host back to the production server, but the backup and restore took a long time and I knew that my ZFS data was still intact, so I figured it would be more efficient if I could just restore the ZFS pool.

Here is how I did it:

zfs list

Returned the zpool ID and a list of the VM disks.

zpool import -f <zpool ID>

The -f (force) flag was necessary because this was a "new" system.

zpool status

confirmed that the zpool was imported properly and was ONLINE.

In Storage under Datacenter I clicked Add, entered the zpool ID and selected my now ONLINE zpool from the dropdown list adding the zpool to the production server.

At this point, the data on the ZFS pool was available, but still no VMs. I could have recreated each VM. This seemed tedious. Since I had the machine running on the other proxmox host it seemed easier to just transfer over the VM configurations.

To do this, I copied all the .conf file in /etc/pve/qemu-server to my production proxmox node, then editted each .conf file adjusting the storage id to match the storage id on my production server. In my case this required changing:

scsi0: VM:vm-100-disk-0,size=60G

to

scsi0: local-zfs:vm-100-disk-0,size=60G

If both storage ids were the same, this would not be necessary.

"Magically" the VMs appeared. However, they would not boot. While the disks were attached to the VM, the VM did not see them at boot. Editing the VM's Options:Boot Order fixed this.