Exploring XCP-ng from a VMware User's Perspective

Welcome to the first of a few videos diving deep into VMware alternatives for your #homelab and your business. In this video I dive deep into the world of XCP-ng to uncover how it compares to VMware ESXi and evaluate it as a replacement from a VMware user's perspective. It's a long video and a lot of planning, learning, and effort went into it so let us know what you think!
*GET SOCIAL AND MORE WITH US HERE!*
Get help with your Homelab, ask questions, and chat with us!
🎮 / discord
Subscribe and follow us on all the socials, would ya?
📸 / 2guystek
💻 / 2guystek
Find all things 2GT on our website!
🌍 2guystek.tv/
More of a podcast kinda person? Check out our Podcast here:
🎙️ www.buzzsprout.com/1852562
Support us through the KZbin Membership program! Becoming a member gets you priority comments, special emojis, and helps us make videos!
😁 www.youtube.com/@2GuysTek/mem...
*TIMESTAMPS!*
0:00 Introduction
0:52 The history of XCP-ng
2:15 Comparing features in XCP-ng to ESXi
6:27 Comparing the consoles of XCP-ng to ESXi
8:47 Comparing Xen Orchestra to local ESXi host management - a tail of two GUIs
18:06 Is XCP-ng a good replacement for VMware ESXi?
18:36 What I don't like about XCP-ng
20:34 What I love about XCP-ng
21:38 Closing! Thanks for watching!

[@2GuysTek]

As of today (February 12, 2024) VMware ESXi free is no longer available. 😰 kb.vmware.com/s/article/2107518?lang=en_US

[@LAWRENCESYSTEMS]

Excellent video and very detailed comparison. Two things, their new UI is coming this year as is XO Lite which runs on the host and both address the more modern design look and feel. Also worht noting that not only can you move VM's between hosts in a resource pool, you can also move a VM between completely different resource pools. XO can connecting to many pools at the same time.

[@Garageland16]

I think for XCP-NG to be a real V-threat is partner collaboration with the big OEMs and certifications, you get community college professors pushing cert tracks- you become an OEM disrupter. - oh and hey Tom!

[@RazvanRosca]

Thanks, Lawrence, for your support!

[@hnmcclain]

Tom, I watch a lot of your videos. I appreciated your two cents, here. Will any of the new offerings from XCP-NG offer some level of self-service, do you know?

[@LAWRENCESYSTEMS]

@@hnmcclainit's all open source so you can run most all the features

[@razorr3751]

​@@LAWRENCESYSTEMS Hey! Thanks for your insight about XO lite, do you think it will be paid and support their 'DRS' like feature for free as well ?

[@awebster]

Great vid. Been using XCP-ng for many years, and like all the functionality. There is another GUI in the form of a fat client called XCP-ng center, which gives you much snappier response than the web web GUI of XOA. The parallel to this is the VMWare client of yesteryear. XCP-ng center can connect to multiple different unrelated pools to provide you with a single consolidated interface. So you can do pretty much everything with XCP-ng center and not have to run XOA, but XOA gives you the backup functionality. Finally, the only major issue I've run into is with purpose built virtual appliances for VMWare, migrating them to XCP-NG has proven unsuccessful in several instances.

[@NeilHyndman]

This video is VERY WELL DONE! I had some of the exact questions that were asked and answered in this specific video. A lot of different sites or vloggers talk about the licensing but they don't explain it as clearly as Rich does.

[@Ghandacity]

13:20 - When taking a VMware snapshot with memory, it will stun the VM at the beginning of the process. This can be quick enough you don't notice, but I have seen it make the system unavailable for up to 10 seconds.

[@rosonowski]

I'm glad someone else noted this. It can be an especially big problem for some VMs, we have to wait for a maintenance window at work.

[@thegrossmeyer]

This is something I came to comments looking for... ESXi has the same issue with snapshots with memory as it will stun the guest. This stun is absolutely enough to crash databases, although many applications won't notice or care.

[@milicsantiago]

great video. The last thing of VM import didn't realize until you mentioned. I had to rewind the video a couple of seconds, ha! Regards form Argentina

[@tylermartin2247]

This is an excellent video! I'm already planning to move from ESXi in my homelab to XCP-ng. I personally feel that XenServer is going to be the VMware replacement that companies are looking for, and it is great knowing that there is an open source "version" of it.

[@RazvanRosca]

Thanks for the support! We'll always have an Open-Source version available, we're not looking to change anything in the foreseeable future.

[@it4offices]

great video, worth noting that XCP-NG has a desktop application from the old Xen server - XCP-ng Centre that allows you to do a lot of the tasks & setup. You can also enable a XOA-Lite version by creating a simple html file. This provides a basic web interface.

[@KegRaider]

Hey great video. I only started using XCP-NG around February this year. I was an ESXi home user and decided I wanted to try something else since we use ESXi and vSphere at work and it's always good to keep the brain ticking away. I deployed their XOA guest shortly after the install, then quickly went to work rolling my own XOA on a separate server. The instructions online were not complete, with many libraries not part of the installs. A little googling around found and fixed those problems. So far, several months in I am happy with the stability and speed on XCP but yeh it's not for everyone as you say, the UI takes a lot of getting used to. The backups though! That is worth the learning curve.

[@hostilegoose9225]

Great and informative video 💪 I’m trying out Proxmox as a replacement for VMware at my job, so I’m looking forward to the VMware vs. Proxmox video 🙏

[@fleacero]

Wow, what a fantastic video. Well done, great work!

[@minigpracing3068]

Should probably mention that if you are hard core enough, you can do everything from command line on the host, that includes all the VM creation and migration stuff. Also XO has a lot less white space on low resolution monitors like the 1280x1024 in my server racks. I for one don't care how basic it looks, gets work done then I move on to other tasks. XO v6 will look more slick, preview some of this by putting XOlite on the host but know that you can't accomplish real work in lite yet. Give them time, updates are coming pretty fast for this.

[@kuhndj67]

Thanks for the overview... was very handy to give me a baseline understanding of XCP-ng and how it compares to VMWare. One thing prospective users should be aware of - no NVidia vGPU support for XCP-ng at the moment so if your environment needs vgpu support you're out of luck (unless you can live with passthrough)

[@jimlynch9390]

Great video. Thanks for the comparison. I tried to use Xen years ago but due to the very unfriendly and non-helpful user community I went elsewhere. I was forced to use VMware at a school I was volunteering at and found it a bit to complex for what it did for us. I had also installed it on my home systems for a while but gave it up for similar reasons. I just don't like it. The ui of XCP-ng reminds me of the OPNSense ui. It's not the greatest but it does get the job done.

[@akurenda1985]

One major drawback of XCP-NG and XOA is the 2 TiB VHD limit. If you have giant SQL or File servers with one or two giant disks in ESXi, you're gonna have a hard time migrating to XCP-NG and XOA.

[@phillynarrowstraight]

That does seem a bit 2010! Any ideas if there’s plans for an increased limit?

[@minigpracing3068]

They are switching visage imports away from native VHD, so I would expect a change to regular VMs in the near future.

[@Darkk6969]

Here is my take now that we're in 2024. Nobody should be running these insane 2TB VHDs. For a fileserver just use a small VM to run the OS and map the shares to another storage server that can properly protect the files like TrueNAS. This will make backup and recovery of the OS a snap. Let TrueNAS handle the backups like ZFS snaps and ship the snapshots to another TrueNAS server as backups. That is what I do for my NextCloud and Plex servers. I can see why admins would be stuck with large SQL database servers as a VM but you can use iSCSI to mount the shares and SQL won't have any issues accessing them as they will see it as dedicated storage. Just make sure the network speeds are 10 gigs or better if it's a busy SQL server.

[@PowerUsr1]

@@Darkk6969totally agree which is why I found the complaint a bit baffling. I can see spinning up a 160GB vhd but anything more I have a ISCSI connection to my SAN..,,which is best practice

[@RazvanRosca]

Here's some clarification on this matter. Problem: - SMAPI (Storage Management API) v1 uses VHD. - VHD (initially developed by Microsoft) has a 2TB disk limitation. - this is not an XCP-ng or XO limitation, but the format itself imposes it. Current workaround: use RAW. Downsides of using RAW: - RAW doesn't support copy-on-write. - This results in no snapshots -> no live storage motion -> no live migration between storages and no incremental backups. Other workarounds: - Use network-attached storage inside the OS or iSCSI. - Use multiple 2TB virtual disks and create a software RAID0 in the OS. Disks can be excluded from snapshots (snapshot the entire VM, but not the RAW drive). Solution: use SMAPIv3. About SMAPIv3: - SMAPIv3 is not production-ready yet. - Citrix isn't looking to work or cooperate on this, so Vates and the Community must address this. How is Vates addressing this: - Vates is aware of the issue and is actively addressing it. - SMAPIv3 integration will be ready for local drives and NFS until the end of 2024.We will build many drivers (i.e. for local SR, NFS SR, and such), and for each, we will choose a format. But it will be transparent for the user since functionally speaking it will be the same thing (snapshot, backups, live storage migrations, etc)

[@juebaitube]

Thank you. Well done ❤.

[@MSPcraps]

Thanks. I'm just looking for a hypervisor for home use and hadn't heard of xcp-ng until today.

[@liora2k]

Great video , as a 2 years heavy user of xcp-ng I can tell that only their UI is not modern and looks like an old fashioned open source, BTW also their new XO lite isn’t so modern and still need to have a massive polish

[@tehduke1776]

"Giant Beard" That was so cute. :)

[@RainMan52]

Gonna stock up on popcorn for the Hyper-V presentation 🙃 🍿

[@2GuysTek]

Does it need one…?

[@MrMcp76]

Isn't Hyper-v going away? I know Microsoft is at least ending the free version of it. Microsoft doesn't even use it for their own cloud service.

[@augurseer]

I'd like your break down on proxmox hyp-v and xcp-ng. Yea

[@Ghandacity]

​@@2GuysTek - I think you just did it.

[@RoelandJansen]

​@@MrMcp76not true

[@ronkelley1184]

XCP-NG works great until it doesn't. I used it a few years back and ran into two main problems: * Sometimes after rebooting the server, the mgmt interface would disappear - especially if the server was part of a cluster. No amount of cli commands would revive it. Time to reinstall (again!). * The VM disk files (VHD files) are written to a single directory using a UUID value. This means trying to manually recover a VM is very difficult since the VM name is not part of the VHD file. There is no per-VM subdirectory (like ESXi) - making it much more difficult to identify which VHD files correlate to which VMs (an especially important issue when a VM has multiple VHD files). My suggestion: Try XCP-NG, force it to break (hard power-off, remove a NFS share, etc), and then try to fix it. Only then will you appreciate the effort to get your server back. I am not saying XCP-NG is good/bad. Just make sure you know what you are getting into...

[@RazvanRosca]

Give it a try again, a lot has changed.

[@laurentmarandet4850]

Use backups with XenOrchestra (free when compiled from sources, which is easier than you could imagine, thanks to many scripts available to automate it).

[@xandrios]

Great introduction to XCP-NG, thanks. It indeed is painfully obvious that there are very few true enterprise options to replace VMWare outside of a home lab. How does Vates fit in this picture; they seem to offer a slightly different version of orchestra against a very different pricing model (Similar to VMWare Essentials Plus).

[@bstock]

I've been trying this out as a way to replace vmware for my homelab. I ran across 2 issues though: 1) Multipath for iscsi isn't really there in XCP-ng. You can do it but have to drop to the console and essentially configure it manually. That's not too bad on a small scale but could be problematic at a large scale. There's also no thin provisioning on iscsi storage. 2) PCI Passthrough is similarly not available on XCP-ng GUI. You can do it but it takes manual intervention at the CLI level. Again, not a big deal for small scale but it could be on a large scale, especially if you need to shut down and migrate VM's with PCI Passthrough regularly. If they can get these working well, I think they have a big potential to take over a ton of marketshare from vmware, but I think those are pretty big roadblocks for enterprise adoption.

[@JohnKissane]

I found getting the multipath iscsi a mess to deal with as well, it just works in ESXi but like you said there was a lot of manual tinkering via the CLI to get both paths available in XCP-ng.

[@GrishTech]

They are working on a new GUI. Hopefully development will speed up as companies jump ship and start giving their funds to vates.

[@2GuysTek]

I’d heard rumblings about the same thing, I hope they address some of my complaints with the new GUI. The irony is a friend of mine showed me the old Xen fat client (windows only unfortunately) and it’s miles better than the current XO UI. I don’t know how they went from having that as a template to the current incarnation of Xen Orchestra.

[@TITAN2K]

@@2GuysTek last I heard for a GUI, you can still use XenCenter to manage a XCP-NG host/hosts without XOA. Although it is still a windows only based app, and only installed on your workstation. So it does have some draw backs unlike having a local GUI, vCenter or XOA. Still great round up.

[@KaiHenningsen]

@@2GuysTek Licensing problems preventing them to opensource it and forcing them to start from scratch, combined with time pressure to get something out that works well enough?

[@jeanburgos3416]

Great video, and break down. That GUI... UGH!. Now we know why there are front end developers.

[@victorshane4134]

And here i am. The Proxmox guy :)

[@scanmikey]

As has been mentioned... XO-Lite is being developed as a built-in host management solution for XCP-NG but it is definitely not ready for prime time... yet. As far as the XOA versus (self compiled) XO XOA is the Vates supported, delivered for business, auto updating, easy to deploy version of XO. It is really geared toward businesses who pay for Support, and it has several features paywalled. Self-compiled XO takes away most of those paywalls (except for XOSAN) but Support is not available with it. I have used the self compiled version from the beginning in my homelab... as both a native VM and as a docker container. I also have XO running in an LXC container on my Proxmox host.

[@2GuysTek]

I understand Vates' rational over XOA vs. self-compiled XO in terms of business, I just don't think it makes much sense all things considered. I expect that if/when I decide officially which way to move my 'production' homelab, if it ends up being XCP-ng, I'd be running either the self-compiled or containerized version of it.

[@olivierlambert4101]

@@2GuysTek and that's perfectly fine :) XOA isn't mean for individuals. Source version is the way for home labs!

[@jfbeam]

It's nice to see Vates learned nothing from Citrix. Artificially paywalling features just to get people to buy the "starter" license is both stupid, and totally undermines why xcp exists in the first place.

[@michaelgregory903]

Sounds like a pretty good replacement. Though I think the VM essentially pausing during a snapshot with memory is a big downer at least for my environment. I hope they can figure out how to get around that in a update, otherwise I won't even be considering the product. Thanks for the direct visual comparison, it was really a good eye opener for this other platform that is available.

[@chrisbarth1887]

Reminder that it only pauses if you snapshot the active memory state.

[@JasonTaylor-po5xc]

Honestly, I've only played with ESXi from time to time. My current Hypervisor of choice is Proxmox since it does everything I need and support more of the capabilities I care about without extra cost than VMWare. I do use VMWare Desktop for local VMs and I was considering getting back into ESXi so could play with the Workstation-Host integration. I may try again but I need to see if it will run on my AMD hardware since my Intel boxes have e-cores which ESXi doesn't like. I've never played with Xen or Xcp-ng.

[@seantellsit1431]

I think it MUST be noted that XOA can only do backups if you have the full paid license or have compiled from sources.

[@xandrios]

This is a very important detail. Thanks!

[@rudypieplenbosch6752]

Yes, there is a good tutorial from Lawrence how to do that, its very easy.

[@KegRaider]

Compiling from source isn't all that difficult, they even have the "How-To" on their website's documentation. Spin up a Debian machine (physical or virtual) and follow the steps. If it errors on a step, read the error and address the failure.... or google it. Took me about an hour I guess to get my first one installed. It's worth the time especially if you want to have a backup solution that works so well.

[@laurentmarandet4850]

Compile it is very easy, I was happily surprised, due to scripts available on the net.

[@kevinalsop3772]

well done. thank you.

[@spyrule]

When we were planning on deploying ESXi to 8 hosts, the total licensing cost was north of $50k CDN / yr (and that WASNT the most feature rich license). We went with XCP-NG Enterprise, and got ALL features and support, and cost us $7k CDN / yr. We get ALL the features that were even locked out by ESXi. Other then deploying the client initially, everything else has been smooth sailing. Our only performance problem, is caused by VM's that can't utilize fast storage at native speed, due to a well known XEN problem (this goes deeper then XCP-NG, affects ALL Xen based platforms).

[@GabrievanZanten]

@spyrule would love to here more about that issue. Any links to that?

[@ashuggtube]

Brought here by Lawrence Systems. Thanks for this video!

[@2GuysTek]

Tom is the best! Thanks for stopping by!

[@ashuggtube]

@@2GuysTek well, he’s okay I guess. I’ll be back for the proxmox video, for sure!

[@JeremyMeeler]

You hinted at it, but lack of VLAN passthrough options is kinda critical for homelab folks or even larger companies wanting to run things like virtual routers. You can do it, but it's very clunky. Another fault I've seen, though not of XCP-ng or any other hypervisor, has been the lack of support for VMs running on anything other than ESXi. XCP-ng fully supports OVA imports and when you complete an import just to find out that the manufacturer/developer of said OVA places an "is this an ESXi host" check and denies use because it isn't ESXi, this can be very frustrating. Wasn't the point of the OVA standard to make VM placement flexible?

[@joebleed]

What do you mean by lack of vlan passthrough options? You can assign vlans to VMs so i'm not sure whats missing on this front? I'm working on learning xcp-ng as i'm hoping it will work for me at work to move away from vSphere. XOA's interface aside, it seems to do everything we need so far. though i do wish it had something like vmware's vSwitch setup for networking. It makes it easier to understand and follow. my hosts at work are rather simple and we don't use many vlans; but it's easy to follow loking at them in/on the vSwitch.

[@rudypieplenbosch6752]

Are you talking about xcp-ng ?, vlans work fine. And i run a virtualized Pfsense as well with 6 vlans, no problem assign, vlan to nic and vlan to vm and you are good to go.

[@KennethsTeknik]

Thankyou for the great video, I'm worried about the agents, having been on a XEN platform before, it was impossible to safely migrate some of the servers to Vmware (back when that was a thing), so how about the agents for xcp-ng, being based on the same hypervisor, I guess that's a big problem to migrate away from xcp-ng if first chosen?

[@Raima888s]

Is networking in XCP-ng like ESXi where it has network ports and network groups? Does it allow multiple VMs to utilise the same physical network port like in ESXi and does it support promiscous mode?

[@realandrewhatfield]

Subscribed!

[@shannonhull7243]

Do you know that anything like NSX-T abilities for XCP-ng

[@6downtown9]

Planning to learn virtualization in the future .just found out XO is more user friendly then exsi GUI

[@Catskeep]

hi, are XCP-NG have feature like VMware Site Recovery Manager (SRM) ?

[@jeremiefaucher-goulet3365]

I still mainly use XCP-ng Center and the command line. But i look forward to XO Lite. One day I'll deploy my k8s cluster, and then I'll run full XOCE

[@WillieHowe]

Great video -- the thing that I keep looking and and wondering is the 2TB file size limit with XCP-ng. Is that still a thing? It could be a deal breaker for some -- ProxMox doesn't have that limit. Thanks for great content!

[@RazvanRosca]

Here's some clarification on this matter. Problem: - SMAPI (Storage Management API) v1 uses VHD. - VHD (initially developed by Microsoft) has a 2TB disk limitation. - this is not an XCP-ng or XO limitation, but the format itself imposes it. Current workaround: use RAW. Downsides of using RAW: - RAW doesn't support copy-on-write. - This results in no snapshots -> no live storage motion -> no live migration between storages and no incremental backups. Other workarounds: - Use network-attached storage inside the OS or iSCSI. - Use multiple 2TB virtual disks and create a software RAID0 in the OS. Disks can be excluded from snapshots (snapshot the entire VM, but not the RAW drive). Solution: use SMAPIv3. About SMAPIv3: - SMAPIv3 is not production-ready yet. - Citrix isn't looking to work or cooperate on this, so Vates and the Community must address this. How is Vates addressing this: - Vates is aware of the issue and is actively addressing it. - SMAPIv3 integration will be ready for local drives and NFS until the end of 2024.We will build many drivers (i.e. for local SR, NFS SR, and such), and for each, we will choose a format. But it will be transparent for the user since functionally speaking it will be the same thing (snapshot, backups, live storage migrations, etc)

[@allesbelegt]

Xenserver would also be an interesting comparision. Since the beginning of 2023 it is no longer part of Citrix itself. The new Trial Edition of it, is no longer limited like it used to be, but comes with the same features as the Enterprise Edition. The only Limitation that is still in place for it, not for production/commercial use

[@martymccafferty7510]

If you need thin provisioning in ice ISCSI, your ISCSI server should provide the thin provisioning and SnapShot capability. You could use ZFS for your data store and ZFS gets you thin provisioning, snapshots, etc. These things might be advanced for some users, but worth learning.

[@LloydRandall]

nice, are you going to do a XCP-NG vs Proxmox series?

[@KarlderHeinz]

Please compare vCenter and XOA🙏 Great Video 👍

[@fxeconomist]

I'm just a regular guy using the free VMWare Player. What I found that sets it apart for me, is that it does something that both Hyper-V Manager and VirtualBox can't do : USB Passthrough. It has a simple mechanism to unplug ANY USB device from host and plug it to the guest. This brought me to my operational target, to get the host offline while the guest is online. So the Windows host practically becomes a gaming enabled type 1 hypervisor - something that Qubes cannot do. Is there any other solution out there - type 2 - that can do the same thing ?

[@haraldfielker4635]

I never ever saw a company explode that fast as VMWare.

[@2GuysTek]

Pretty much.

[@sullenfps]

need one of these against proxmox

[@2GuysTek]

It’s coming!

[@Darkk6969]

ProxMox admin here for both home and work. I've tried using XCP-ng to see how well it works and WebGUI workflow just doesn't make sense. Also I was turned off on the large pricing of the XOSAN. I understand they're changing the pricing model to make use of bundles. I like ProxMox as it just works and even purchased support subscription for all of the nodes at work. They don't force this on home labs which is a big plus.

[@dragonnoa1]

He literally said its coming next at the end of this video 😆

[@andibiront2316]

My experience. Before Broadcom: 3.5M in support and licese every 3 years. After Boardcom 4M every year.

[@JibunnoKage-cj2kz]

The original XenCenter (yes some time ago) was so unstable in our labs we dropped all XenServer evaluation years ago. Let's home the Xen-O is much better! The key issue... is the VM migration from hypervisor to hypervisor is it transparent as VMotion? Or does it has the typical VM freeze issue during migration that KVM still has today?

[@laurentmarandet4850]

No freeze unless you are using the deprecated dynamic memory option.

[@Diversifried]

The beauty of XCP being open source is, the complaints you have can be tweaked to your liking. You can manipulate everything you see in the environment. Excellent vid

[@112Haribo]

Yeah but companies looking to move away from VMWare are not looking for customizable. They want stuff to Just Work. That's why they paid VMware in the first place instead of using KVM and QEMU and libvirt etc.

[@en4ble773]

What about running LXC containers (to what proxmox have)?

[@believeinheroes]

Very helpful! Could you do a similar video about proxmox from a VMware user perspective?

[@2GuysTek]

It’s in the works now!

[@gabrielegaetanofronze6690]

Hey there! Thanks for the insights, always welcome! With my company we developed a new type-1 hypervisor based on kvm we call AtomOS. In fact it is rocky linux or alma linux (you can choose) running kvm-qemu plus a lot of smart features on top implemented as daemons. In addition it is autoscaling, that means it has no centralised services and a virtualisation cluster is an emerging behaviour from a bunch of servers within the same LAN (VLAN or VPN as well!). Big plus: it uses a VM definition protocol which we make compatible with all public clouds, in fact opening a world where hybrid-cloud is possible by design. And yes, we built in automatic GPU passthrough support! We would be delighted to get you some copies to get some feedback and, if you mind, for you to make a video!

[@2GuysTek]

Reach out to us via email or on our discord and let’s talk.

[@gabrielegaetanofronze6690]

@@2GuysTek done! Thanks 🤟🏼

[@petersachs2750]

We use Xenserver in production and overall it works well however we have had hosts crash without explanation and our workloads are mission critical. We've been looking at VMware but like many are concerned about rising prices.

[@RazvanRosca]

Give XCP-ng a try.

[@BoraHorzaGobuchul]

Proxxmox?

[@jani140]

Proxmox. We had scrapped our last xcp clusters years ago because of that. Since then running stable on Proxmox-ceph. Really. Skip XCP...

[@druxpack8531]

Both XCP-NG and Proxmox don’t support thin provisioning on iSCSI, only nfs…which makes it difficult if that’s what you currently are running with VMware.

[@Lodewyk1981]

But Citrix Xenserver Premium does. Busy with that project now + XOA, and it is also a fraction of the cost vmware.

[@RazvanRosca]

This will be fixed with SMAPIv3 soon.

[@jojobobbubble5688]

So now that both video's are out, which did you go with? My guess is that you went with XCP-ng.

[@iamagi]

How well does it handle a few dozen hosts?

[@josephstacklin3853]

Can you do hyperv and proxmox vs VMware including price?

[@succuvamp_anna]

Subscribed for the "Corporate Bullshit" quote. Been redoing all my studies, I was about to take Certification Test for VMWare but..... you don't get on a boat sinking.....

I used XEN with OVM a long time ago. It had a similar repositrory approach for storage, but it used OCFS2 for LUNs mounted on FC. What filesystem xcp is using for shared storage? Anyway OVM was crap compared to VMware, but you should make a video about the OLVM, which is its successor and it is also free. Maybe worth a try. As I know VMUG Advantage is still available(VMUG webinar on Friday). Those licences are good for homelabs.

[@2GuysTek]

I'll look into OLVM, thanks for the tip!

[@Darkk6969]

There is no telling what they will do with VMUG Advantage in the future. Probably low priority for now while they wreck the other pricing models.

[@alexandrohdez3982]

👏👏👏👏👏

[@RayAndress]

Why have functionality that has to be paid for which can be back door made free? Because here in the corporate world, it is worth paying for some things. You see a hurdle they are putting in to charge companies like the one I work for. I see functionality that when I need to redo a setup, I can pay to have set up for me and we both win, while you can do the compile method I don't want to do and test the functionality out. Keep up the video's. It was a great review.

[@SoliasSerene]

Vmware VVF? VCF? What do the acronyms stand for?

[@tomrberg]

Would you evaluate Oracle VM x86 and Oracle KVM as well?

[@nigratruo]

Forget Oracle, they are just copying Linux and not contributing anything good to it. The original KVM on Linux, part of the Linux kernel is highly optimized, it very good and Oracle has not made it better, but I fear worse. Go to the source, don't use a knock off copy. Also, Oracle is not known for a great opensource contributor, but a company focused on maximizing profit: No company with these motives has ever made opensource better.

[@louisdotarsenault]

What about vSan for xcp-ng?

[@tmitch120]

*IF* I had a real choice whether or not to migrate off of VMware, the lack of a VRC equivalent or ability to open the VM web console in its own window would be a deal-breaker. I have a minimum of two VM VRCs open on my desktop at all times. As it is, I like XCP-ng best of the VMware replacements that I have looked at, and will be looking for a workaround for the remote console issue.

[@nicholasl.4330]

Great video!! I would like to mention that VMware is not killing off VMUG yet, which is a great option for the homelab

[@gearboxworks]

He says he's not dead yet.

[@Foars989]

while i don't hate how XOA looks, sometimes i get lost lol, i hope their upcoming UI changes make it easier to navigate

[@Wayne_Mather]

I used xcpng for years, moved to esxi recently due to moving to synology (backups and restores reason for moving) Guess am having to go back or look at hyperv....

[@Wayne_Mather]

It should also be noted xcpng is lole centos was. It's an open source version of a commercial product - citrix xenserver. So while it maybe open source, it does have an upstream dependency

[@BoraHorzaGobuchul]

Why not proxxmox?

[@Wayne_Mather]

@@BoraHorzaGobuchul Yes am currently looking at what proxmox and proxmox backup can provide for me

[@svettnabb]

If you have xo-san, you don’t even need third party tools like veeam in order to do file restore. Vm backup is free in basic

[@MrMcp76]

Great video. I like xcp-ng, but for now I will continue to run my homelab on ESXi. The company I work for uses VMware, and we have discussed the Broadcom changes and how we may move forward. I brought up XCP-ng and got only laughes. They won't consider it because it is based on Citrix, and because of the short-comings of the support. We are a medium-sized business, and our infrastructure is 100% VMware locked in. Switching would take years, not even considering the years of planning and research needed to switch. I think in the end the majority of companies that will switch away from VMware are those that are being kicked out by Broadcom, or smaller companies that can't afford the new pricing structure.

[@hasanmujeeb8922]

Why is switching that hard? And take so long?

[@MrMcp76]

​@hasanmujeeb8922 Because all of our systems (in-house apps, SQL databases, backup solutions, network connectivity, storage arrangements, etc) were all built around VMware and it's capabilities and support structure. VMware does EVERYTHING but other type 1 hyoervisores do only some of those things. The reason the Broadcom changes are so devastating to many is because VMware is the foundation of their entire network and infrastructure and trying to change that would be more expensive than paying the extra to Broadcom. And Broadcom knows this.

[@Darkk6969]

Actually switching from VMware to another hypervisor like ProxMox isn't hard at all. I've done it with our VMware servers before this broadcom buyout. In fact VMware even provided the tools to allow easy migration from the host servers to another product. Just have to build a test environment and copy few VMs over to see how it works. It only took me three months to validate the VMs to ensure they will run on ProxMox with ZFS without issues. Then I did the full migration over the weekend. Users didn't notice anything different.

@@Darkk6969 How many hosts and VMs have you migrated to Proxmox?

[@druxpack8531]

I’m curious to see how your company handles the price increase when renewal quotes are in-hand. It’s easy to say all is cool in a meeting, it’s another when you actually have to cut the check.

[@autohmae]

One of the things I wondered: why do I see a lot of people saying: Proxmox instead of XCP-ng ? And why do all the VPS and cloud providers run Qemu/KVM instead of Xen ? (AWS being a good example of a company who moves from 1 to the other), which is obviously a big deal).

[@gerhardroediger8331]

because those people have no idea what they are talking about: proxmox is something different than xcp-ng. On regards to your second question: I guess because Qemu/KVM are considered type II hypervisors whereas Xen is a type I hypervisor, which means KVM/Qemu are hosted hypervisors in contrast to Xen(upload.wikimedia.org/wikipedia/commons/thumb/9/9e/Hyperviseur.svg/800px-Hyperviseur.svg.png ), which would be running on the "bare metal". I guess it just offers them more control/security over their own machines with KVM/QEMU.

[@nigratruo]

@@gerhardroediger8331 No that is false, totally false. You want to have an efficient, scalable and secure OS as the base for a hypervisor. VMware tried to write their own OS and they did a very bad job indeed (compare benchmarks between Proxmox (KVM running on Debian Stable) with VMware and Proxmox wins them all), which is not surprising: It is hard to compete with Linux today, Linux has been optimized for 20 years now, the Linux Kernel is extremely efficient. If you run on bare metal (which you can't there is nothing on there) then you still need an OS and that OS should be Linux, not some cooked up crap by a company that has no clue about how to program an OS. Even XCP-ng uses Linux, I can promise you that. Besides that: A Debian Linux that has no extra apps installed is as efficient as "bare metal", because you still need a Kernel, still need networking, you still need storage and that is what Linux provides. I think you guys don't quite understand how this all fits together and works and confuse things and don't understand what modularity means in Opensource and Linux: you can use any component with any other component, any part you don't need or like you can remove. AWS is the largest cloud provider on the planet, it runs an insane amount of VMs, them using KVM should really make you think why you prefer KVM (and Proxmox of course)

[@nigratruo]

@@gerhardroediger8331 Correction: Just looked it up, KVM is a type 1 hypervisor as well as VMware, they run directly on the hardware (via the Linux) Kernel. I don't think there are type 2 hypervisors anymore, that used to be a old technology before hypervisors become integrated into the Kernel of OS like Linux.

[@gerhardroediger8331]

@@nigratruo not sure about that one. You can't install kvm on bare metal, can you. Since it got part of kernel I guess it's kind of debatle how type1 kvm really is...

[@nigratruo]

@@gerhardroediger8331 Yes, you can install KVM on bare metal: Kernel Virtualization module is what it actually means, it is part of the Linux Kernel and the Kernel runs on bare metal, it is the very definition of bare metal, it is the system core that runs everything else. Today there are actually no more type 2 hypervisors, Linux is a Type 1 hypervisor, as the Hypervisor is so close to the metal as possible. Bare metal is not even possible, because you still need an OS that runs on it to provide any funtionality like storage, networking, ability to run apps. The hypervisor type 2 definition was from an older time when you had to install the virtualization hypervisor on the OS. It has been part of the Linux Kernel for over 10 years now and every Linux comes automatically with KVM, no install necessary.

[@servalous]

the difference is that XCP-ng has these infos prepared for you in the console, while vSphere ESXi requires you to put in commands to get them. One of the many critical feature requirement is to add/change vm resource with near no down time. If I would be required to shutdown a vm to expand the vmdk/vdisk, all that stuff would generate so much over head and over time. That most people would get a cobain headache... If a hypervisor cant change the vdisk while the vm is active, is a huge "no,no..."

[@Guichy]

I guess I’m biased since I’ve worked with vSphere for more than 10 years. I tried xcp-ng and could not get the Xen Orchestra using the 2 documented ways in their documentation and never worked. I gave up lol

[@peteratkin3788]

I'm still using ESXi, looked at XCP and honestly too much of a learning curve, the cost of XCP it's not free or cost approx by any means as I found out, unless I missed something, happy to be corrected. - that being said, I am planning to move to XCP, but I'm not in a hurry; thanks for the rundown. It's really pushing me; I'm just not there yet; I'm not a Linux fundi so XCP is a bit daunting with or without cost.

[@pmflav]

Xcp-ng can be free. Community xoa

[@Traumatree]

The Broadcom moves only benefits Microsoft Azure imo as people who are still on Vmware will now consider going on the cloud even more (sadly). Edit: The other major thing you forgot to add to the "things to hate" is it is RHEL-based which in my previous testing a few years ago, isn't really friendly to AMD based servers. I do hope things have changed, but that is still a red flag for me knowing that I mainly use AMD based servers. Edit2: Will you compare TrueNAS Scale to your list of options?

[@gmgaming2]

What about XO Lite for a built in host management GUI? 3:30

[@2GuysTek]

It's not built into the current version of XCP-ng yet, I understand they expect to bundle it soon in the later versions of XCP-ng though. From what I've read and been told so far, it's pretty rough around the edges still. Have you used it?

[@minigpracing3068]

​@@2GuysTekIt is also not even close to feature complete! It isn't scheduled to be ready until XO v6 which is still a bit away. The last version of XCP-NG Center us due out soon, but it is EOL after this.

[@gmgaming2]

@@2GuysTekNot yet, I’ve been looking into XCP-ng for my home lab, as I’d like to learn more about virtualization and get off Hyper-V. I’m okay with rough, or even not yet, as long as there is an understanding that it’s coming/improving.

[@kevinhilton8683]

@@minigpracing3068 Are you sure XCP-ng Center is EOL? I thought a new developer was announced and I'm aware there are nightly builds which work great.

[@minigpracing3068]

@@kevinhilton8683 You are correct, a new person has stepped up to try and keep it going.

[@gearboxworks]

I found the criticisms of the XOA UI interesting. I've only used VMware a small amount in my career, but I really dislike their UI. It feels dated, cramped, and hard to use. OTOH, when I saw XOA in your video just now, I found their UI refreshingly better. I have not actually used it yet, so that's just my first impression. Who knows how I will feel when actually using it. Any chance your criticism of the UI is because you are just used to VMware and not XOA? I know that I am certainly more comfortable with the things that I know well vs. alternatives that change how I need to approach working. 🤷‍♂️

[@brutester]

12:36 I think you can use RDP to get to the console remotely.

[@thecrazymouse7220]

What is the ability to expand drives greater than 2TB?

[@laurentmarandet4850]

You can combine many 2 TB disks, with Windows and Linux VMs to solve the issue.

[@TayschrennSedai]

ngl i'm judging you on that E5-2680 =D - Decent video, though the one thing I really agree with is that the vsphere 'console' hasn't changed literally since like... 3.5? Probably before honestly. But....... That's because it has a built in GUI =) and it's an HTML5 gui... Still, the time i spend in vmware console is next to never. Overall? It's decent looking but I'd still pick vmware any day of the week for business use. Broadcom needs to fix pricing or they're going to get a wakeup call.

[@rimantascalyj276]

Does it support PCI pass through?

[@RazvanRosca]

Yup.

[@VasiliyVolkov]

Отлично! Спасибо! А есть такое же детальное сравнение - vSphere vs oVirt? ;-)

[@mochouinard]

Setting up the ISO storage and uploading to it is annoying ! But I'm happy I switched to XCP-NG a few years ago now ! I went with this instead of ProxMox, because XCP-NG was much more stable !

[@nigratruo]

That is very true: VMs are an old (kinda obsolete) concept of app virtualization, a very inefficient one and the world is moving to container virtualization and large scale container infrastructures of course run Kubernetes (and guess on which OS that runs: Linux of course) I moved away from VMs a few years ago (I can usually see trends like that earlier than most people) and now am a Kubernetes Administrator.

[@mochouinard]

@@nigratruo I never could stand container system. It feel so limiting or just a waste of time to manage and setup. I feel container is just great for just trying a software quickly without the hassle to actually install it. But once you start actually do more with it than it base functionality, it just become more trouble than it worth. It really just feel like shareware type of things. For example, you need to put an additional container with a reverse proxy to share https port between multiple container... It just make things much more complex than it needed. Simple OS + package system to me make much more sense.

[@Garageland16]

Is Nutanix CE a thing?

[@2GuysTek]

Nutanix does have a community edition of AHV, is that something you’d be interested in seeing a video about?

[@joaopedroalbernaz]

Totally ​@@2GuysTek

[@Darkk6969]

@@2GuysTek Would love to see CE in action.

[@zparihar]

I'd love to see that in action as well

[@nigratruo]

@@2GuysTek You know Nutanix is on the short list of possibilities for many people (along with Proxmox) so yes, I'm sure lots of people are wondering how well that compares. But I would also do comparisons with Proxmox, because the choice is not VMware vs xzy, VMware is sadly (forced) out for most people, what is much more interesting is for example Proxmox vs XCP-ng and especially the KVM Engine that runs Proxmox and is also what AWS uses, it is highly optimized, even better than VMware actually: Just google VMware vs proxmox benchmarks and you will find that Proxmox beats VMware in in most of them, which should make a lot of people think. Because of course it is not really the steering wheel and instrument cluster (The Proxmox GUI) in the car that wins the race, but the Motor and drive train / wheels that is KVM and Linux with the highly optimized Linux Kernel.

[@thraxarioustailchaser158]

...except the install fails and boot loops on some new hardware. This is why I went with proxmox instead.

[@jamespadgett5761]

How about Harvester from Rancher.

[@lazerusmfh]

Xcp ngs lack of ability to run containers destroys its use case for me. Proxmox has been a major life improvement as a large majority of my virtual machines are Linux based. I can deploy a CT on proxmox in only a minute or two, and they reboot fully in about 10. Currently have well over 50 proxmox nodes

[@KegRaider]

ESXi doesn't have Dockers either though. Unraid has KVM and Dockers as well as a fairly decent storage solution. Does Proxmox require you to load disk drivers during Windows VM creation like Unraid? To b fair, he didn't compare ESXi to Unraid or Proxmox though.

[@lazerusmfh]

@@KegRaider unraid is simply just not mature or stable enough. After running multiple unraid nodes for over 10 years they are just super fragile and like to break in unknown ways and for stupid reasons. Proxmox is dead nuts stable and updating works every time.

[@IvarsRuza]

Every feature of vmware is payed and has to have vcenter on top of esxi. Vcenter is total mess after hyper-v or proxmox

[@autohmae]

20:25 that's easy: for businesses, it's not the homelab users which bring in the money.

[@minster586]

There is also XCP-NG center

[@doalwa]

Didn’t know that Xen is even still a thing. The more you know ™

[@ajhopton]

Would be nice if these projects could agree on a clustered file system solution that gives a similar experience to VMFS. This is a pretty big shortfall imo

[@berndeckenfels]

What about vSAN?

[@kevinwilliamson7158]

I'm wondering the same too.

[@RazvanRosca]

Great question. We have something named XOSAN.

[@-tineidae]

What all open source (except ovirt) is missing is shared block storage. If you use ESXI with a SAN, there is no alternative because all these open source solutions do not support snapshots/thin provsion on shared block storage. Thats a limitation of LVM.

[@lcarsos]

well this seems even more prescient now.

[@dennislindqvist5461]

What about automation? VMware supports an extensive library of commands in PowerShell. Anyone being serious about administrating virtual machines need to be able to automate most anything about the virtual resources.

[@2GuysTek]

From my research into XCP-ng, it has a rich API for automation as well.

[@rudypieplenbosch6752]

wow at 350 percore, i would be paying a handsome sum already, but with my soon to be genoa, i have to lower the amount of cores for sake of being able to run vmware😂. I'll stick to xcp-ng