VMware ESXi to XCP-NG

Given the pending Broadcom acquisition of VMware, I have been giving some thought about an alternative for my homelab virtualisation solution. Given Broadcom’s previous approaches to acquisitions, given I am using the free version of ESXi I have concerns whether the free version will continue.

These concerns are more than just for the homelab user. Organisations who are VMware customers are also having concerns. As Ed Scannell of TechTarget SearchDataCenter noted in his article Broadcom’s acquire-and-axe history concerns VMware users (retrieved 2022 July 10): “Broadcom’s history: Acquire and axe”. This make perfect business sense: sell off or stop the unprofitable/low profitability products, concentrate on your top customers and maximize shareholder return. That is your fiduciary responsibility.

However, that doesn’t help the homelab user. (Let alone the SMB market…) How this all plays out, well time will tell.

I’ve been a big advocate of VMware starting back in the very early 2000’s. Back in the days of VMware GSX (basically what VMware Workstation is now) and ESX. Back then, looking at server utilisation where everything was physical showed a massive underutilization. “Old fogies” like me will remember your NetServers and Proliants with average CPU utilizations around 25% (or 5-10%…).

Basically, I have been using VMware ESX and later ESXi both professionally and personally for about 20 years. (Wow! Where did the years go?)

Anyway, back to XCP-NG…

There are number of options to ESXi (or vSphere, really). There is XCP-NG, Proxmox VE, Hyper-V, KVM, etc. For a Microsoft shop, Hyper-V would be a logical choice and Hyper-V has come a long way over the years. For a Citrix shop, Citrix Hypervisor (formerly XenServer)) is an obvious option. Proxmox seems really neat. Xen is really cool because it is the basis for XenServer/Citrix Hypervisor (I’ll be sticking with XenServer as “Hypervisor” without the Citrix reads strange.) and XCP-NG with XCP-NG basically being the free, open source version of XenServer.

In the end, I have two potential options and if I’m going open source (free, as in free beer not a free puppy) what are my options? There really seems to be two: XCP-NG or Proxmox. This is a challenge given the maturity of ESXi.

I first tried Proxmox. The interface is really nice. However, after a week of trying to get iSCSI to work (with lots of YouTube videos and Google University) I finally gave up. The problem wasn’t so much with getting iSCSI to work but to work as I want it to work. I want it to work like ESXi where I create a datastore and have all my VMs reside on that datastore. I could easily get a VM to use all the datastore but I wanted a shared datastore; not a LUN (or LUNs) per VM. I tried to get LVM as the filesystem for VM use but for whatever reason I couldn’t get it to work. Basically, after a week I gave up.

The same thing almost happened with XCP-NG. However, after about five days (and more than a little help from Tom Lawrence’s YouTube videos :-)) I have XCP-NG up and running on ye olde DL360 Gen8. I had to replace the onboard P420i RAID controller with my old IBM 1015 in IT mode. The problem with the P420i is not that you cannot change it from RAID to IT mode. Rather, the DL360 Gen8 cannot boot from the P420i when in IT mode. I thought about buying a PCIe M.2 card and using an M.2 SSD I had lying around but I could not get confidence that I would be able to successfully boot from the PCIe card. I thought about finding an HP power/SATA cable but the price on eBay (plus shipping time) was not appealing. I did have to purchase a couple of SAS cables as the HP cables for the P420i weren’t long enough but $25 and 5 days shipping seemed reasonable. The only downside is that the HP drive lights do not work with the non-HP cables. Oh well…

The biggest challenge – as it would be with any transition – is figuring out how X is done on the new platform. I have most things figured out. My feeling is XOA – the management “console” – is not as mature as ESXi. You can go with the XCP version of XOA but without a subscription you cannot get the patches. I decided to go with Roni Väyrynen’s XenOrchestraInstallerUpdater running on Ubuntu (not Debian, because that is what I am familiar with). However, XOA in my opinion is not as mature and polished as ESXi. Maybe the new Xen Orchestra Lite (a/k/a XOA-lite) may fix this. Remember, the web-based ESXi management GUI was pretty rough at one time, too. Time will tell.

I have one “production” VM running on XCP-NG. Using Clonezilla I migrated the VM from ESXi to XCP-NG. Just remember to remove the open-vm-tools first (I use Ubuntu for most of my servers). About the only issue I have found so far is that the network interface names change during the migration from the “ensxxx” format to the “traditional” ethx format. No problem, just update Netplan on first boot.

One thing that I have to figure out is Synology iSCSI and redundancy. The Synology iSCSI for Linux seems to be single path only. I’m sure I’ll figure it out over the next few weeks. I only need another GigE PCIe card for the DL360 as I only have four GigE ports – one for XCP-NG, one for VMs, one for DMZ and one for iSCSI.

I also have to figure out how to deal with 10GigE for any migration. Right now ESXi is directly connected to my RS1221+ so I’ll need another dual port 10GigE port for the DL360 and a 10GigE switch and four DAC cables. That might take time.

Anyway, I do have options…

About Mike Pelley

Let’s see… A little about me… I’ve been around information technology since 1983 with computers such as DEC Rainbows (weird machine – the standard DOS couldn’t format its own floppy disks – remember them? – and I had to format them on a friend’s IBM PC) to Radio Shack TRS-80 to Apple ][e and Apple //c in the beginning. I have programmed in 8-bit assembly language on 6502, FORTRAN and COBOL on IBM System/370 (and I still hate JCL), VAX BASIC and COBOL (and a weird and massive WordPerfect 4.0 macro) on DEC VMS (Alpha), C/C++ on Digital Unix (ALPHA), and C/C++, Perl (it may be powerful but I still hate it), PHP on Linux (Red Hat, Centos, Ubuntu, etc.). I have work with databases such as Digital RDB (later to become Oracle RDB), Oracle DBMS, Microsoft SQL Server, MySQL and PostgreSQL on VAX, Alpha, Sun and Intel. Check out my professional profile and connect with me on LinkedIn. See http://lnkd.in/nhTRZe I still think that Digital created some of the best ideas in the world: VAX clustering, DSSI disks (forerunner to SCSI) and the Alpha processor (first commercial 64-bit processor – Red Hat screamed on an Alpha!). DEC just could not seem to be able to give air conditioners away to someone lost in the Sahara Desert! VMware is one of the best ways to get the most out of an x64 server. And I have tried Oracle VM, Virtual Box and Microsoft Virtual Server. Outside of that I am a huge military history buff starting in the early 20th century. I love Ford Mustangs (my ’87 Mustang GT was awesome) and if I had the money I would have a Porsche 928S4. If I had a lot of money I would have a Porsche 911 Turbo. I also play too much AmrA 3 Exile mod. Over 5,000+ hours... I have a wonderful son, Cameron. I have a long suffering (Do you really need all that computer junk?) wife, Paula. I live in Paradise, Newfoundland and Labrador.
This entry was posted in Uncategorized. Bookmark the permalink.

One Response to VMware ESXi to XCP-NG

  1. Rob Walker says:

    Looking forward to more news on this migration. I am also a fan of Tom Lawrence and prefer the XCP-ng way to Proxmox. I have some grand hopes for the ultimate workstation with GPU passthrough to windows, MacOS, and Qubes, with longterm hopes to purchase a Raptor Engineering PowerPC fully open workstation.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.