I realized last night that in my efforts to get ESXi up and running, I had installed the original release of ESXi 7.0. Since I had taken Friday off work to take care of a few things that needed to be done during the weekdays I decided to upgrade ESXi to Update 2. It also allowed me to test my theory from my previous post.
The update worked as planned (and boy-oh-boy are updates slow!) but was successful. As I expected the iSCSI share did not show up in ESXi. Let’s check the NAS and see if the targets are still there and if the ESXi server is connected. Yep, ESXi is shown as a host in DSM.
Okay, ssh into the ESXi box. Can I ping the NAS? Yes, working fine.
Log into the ESXi console (again, I’m not using vSphere). The software iSCSI sees the host and the target. But, no iSCSI devices are showing up under storage devices and obviously the datastores are not present.
Let’s try my theory from the last post. Try esxcfg-volume -l and get the UUIDs. Well now, iSCSI volumes are not showing up. Let’s try a few more pings to the NAS, rescan the devices. Nope, not working.
Time for some deep thinking. No panic this time as I know that the VMs are still in the iSCSI LUNs and I just have to get them mounted. And I have good backups!
Some more Googling and the first result from Reddit (vSphere 7.0 U2 iSCSI not working with older HP Lefthand SAN) seems to have the answer. It seems that the issue is around the IQN and how ESXi is handling this. I deleted the ESXi host on the NAS and re-created it. A rescan of the storage devices showed the iSCSI LUNs. I then did the esxcfg-volume -l and mounted, using esxcfg-volume -M UUID (with the “M” rather than the “m” option), the two UUIDs. Bang, the datastores reappears. While the article is for upgrades and not reboots, my gut is that this problem will persist until the bug is squashed. I haven’t tested a reboot because this is a pain in the arse and I have other things I have to do.
From the Reddit post, there is now a KB article from VMware on this issue (iSCSI adapter IQN may change during the upgrade of ESXi 7.0 U1 (84339)). I didn’t see the workaround (I’ll try it if a patch not available and I have to shutdown the ESXi or NAS) but here is that workaround example from the article:
To work around the issue:
Prior to the upgrade, use the esxcli get and set commands to set the generated iSCSI adapter IQN explicitly. As the IQN is a user setting it won’t change after the upgrade.
Get the IQN details:$ esxcli iscsi adapter get -A vmhba67
vmhba67
Name: iqn.1998-01.com.vmware:w1-hs3-n2503.eng.vmware.com:452738760:67
Set the IQN details:$ esxcli iscsi adapter set -A vmhba67 -n iqn.1998-01.com.vmware:w1-hs3-n2503.eng.vmware.com:452738760:67
EDIT
I couldn’t wait for the next time that I needed to reboot the ESXi server. That is probably a good idea given that I probably don’t want to have a number of things going on at the same time. I applied the VMware KB work around and then shutdown the VMs and rebooted the server.
SUCCESS! The iSCSI datastores automatically came up. A gin and tonic with lots of ice may be in order!
