Exploring XCP-ng from a VMware User's Perspective

[@2GuysTek]

As of today (February 12, 2024) VMware ESXi free is no longer available. 😰 kb.vmware.com/s/article/2107518?lang=en_US

[@LAWRENCESYSTEMS]

Excellent video and very detailed comparison. Two things, their new UI is coming this year as is XO Lite which runs on the host and both address the more modern design look and feel. Also worht noting that not only can you move VM's between hosts in a resource pool, you can also move a VM between completely different resource pools. XO can connecting to many pools at the same time.

[@Garageland16]

I think for XCP-NG to be a real V-threat is partner collaboration with the big OEMs and certifications, you get community college professors pushing cert tracks- you become an OEM disrupter. - oh and hey Tom!

[@RazvanRosca]

Thanks, Lawrence, for your support!

[@hnmcclain]

Tom, I watch a lot of your videos. I appreciated your two cents, here. Will any of the new offerings from XCP-NG offer some level of self-service, do you know?

[@LAWRENCESYSTEMS]

@@hnmcclainit's all open source so you can run most all the features

[@razorr3751]

​@@LAWRENCESYSTEMS Hey! Thanks for your insight about XO lite, do you think it will be paid and support their 'DRS' like feature for free as well ?

[@awebster]

Great vid. Been using XCP-ng for many years, and like all the functionality. There is another GUI in the form of a fat client called XCP-ng center, which gives you much snappier response than the web web GUI of XOA. The parallel to this is the VMWare client of yesteryear. XCP-ng center can connect to multiple different unrelated pools to provide you with a single consolidated interface. So you can do pretty much everything with XCP-ng center and not have to run XOA, but XOA gives you the backup functionality. Finally, the only major issue I've run into is with purpose built virtual appliances for VMWare, migrating them to XCP-NG has proven unsuccessful in several instances.

[@Ghandacity]

13:20 - When taking a VMware snapshot with memory, it will stun the VM at the beginning of the process. This can be quick enough you don't notice, but I have seen it make the system unavailable for up to 10 seconds.

[@rosonowski]

I'm glad someone else noted this. It can be an especially big problem for some VMs, we have to wait for a maintenance window at work.

[@thegrossmeyer]

This is something I came to comments looking for... ESXi has the same issue with snapshots with memory as it will stun the guest. This stun is absolutely enough to crash databases, although many applications won't notice or care.

[@NeilHyndman]

This video is VERY WELL DONE! I had some of the exact questions that were asked and answered in this specific video. A lot of different sites or vloggers talk about the licensing but they don't explain it as clearly as Rich does.

[@it4offices]

great video, worth noting that XCP-NG has a desktop application from the old Xen server - XCP-ng Centre that allows you to do a lot of the tasks & setup. You can also enable a XOA-Lite version by creating a simple html file. This provides a basic web interface.

[@KegRaider]

Hey great video. I only started using XCP-NG around February this year. I was an ESXi home user and decided I wanted to try something else since we use ESXi and vSphere at work and it's always good to keep the brain ticking away. I deployed their XOA guest shortly after the install, then quickly went to work rolling my own XOA on a separate server. The instructions online were not complete, with many libraries not part of the installs. A little googling around found and fixed those problems. So far, several months in I am happy with the stability and speed on XCP but yeh it's not for everyone as you say, the UI takes a lot of getting used to. The backups though! That is worth the learning curve.

[@hostilegoose9225]

Great and informative video 💪 I’m trying out Proxmox as a replacement for VMware at my job, so I’m looking forward to the VMware vs. Proxmox video 🙏

[@tylermartin2247]

This is an excellent video! I'm already planning to move from ESXi in my homelab to XCP-ng. I personally feel that XenServer is going to be the VMware replacement that companies are looking for, and it is great knowing that there is an open source "version" of it.

[@RazvanRosca]

Thanks for the support! We'll always have an Open-Source version available, we're not looking to change anything in the foreseeable future.

[@minigpracing3068]

Should probably mention that if you are hard core enough, you can do everything from command line on the host, that includes all the VM creation and migration stuff. Also XO has a lot less white space on low resolution monitors like the 1280x1024 in my server racks. I for one don't care how basic it looks, gets work done then I move on to other tasks. XO v6 will look more slick, preview some of this by putting XOlite on the host but know that you can't accomplish real work in lite yet. Give them time, updates are coming pretty fast for this.

[@akurenda1985]

One major drawback of XCP-NG and XOA is the 2 TiB VHD limit. If you have giant SQL or File servers with one or two giant disks in ESXi, you're gonna have a hard time migrating to XCP-NG and XOA.

[@phillynarrowstraight]

That does seem a bit 2010! Any ideas if there’s plans for an increased limit?

[@minigpracing3068]

They are switching visage imports away from native VHD, so I would expect a change to regular VMs in the near future.

[@Darkk6969]

Here is my take now that we're in 2024. Nobody should be running these insane 2TB VHDs. For a fileserver just use a small VM to run the OS and map the shares to another storage server that can properly protect the files like TrueNAS. This will make backup and recovery of the OS a snap. Let TrueNAS handle the backups like ZFS snaps and ship the snapshots to another TrueNAS server as backups. That is what I do for my NextCloud and Plex servers. I can see why admins would be stuck with large SQL database servers as a VM but you can use iSCSI to mount the shares and SQL won't have any issues accessing them as they will see it as dedicated storage. Just make sure the network speeds are 10 gigs or better if it's a busy SQL server.

[@PowerUsr1]

@@Darkk6969totally agree which is why I found the complaint a bit baffling. I can see spinning up a 160GB vhd but anything more I have a ISCSI connection to my SAN..,,which is best practice

[@RazvanRosca]

Here's some clarification on this matter. Problem: - SMAPI (Storage Management API) v1 uses VHD. - VHD (initially developed by Microsoft) has a 2TB disk limitation. - this is not an XCP-ng or XO limitation, but the format itself imposes it. Current workaround: use RAW. Downsides of using RAW: - RAW doesn't support copy-on-write. - This results in no snapshots -> no live storage motion -> no live migration between storages and no incremental backups. Other workarounds: - Use network-attached storage inside the OS or iSCSI. - Use multiple 2TB virtual disks and create a software RAID0 in the OS. Disks can be excluded from snapshots (snapshot the entire VM, but not the RAW drive). Solution: use SMAPIv3. About SMAPIv3: - SMAPIv3 is not production-ready yet. - Citrix isn't looking to work or cooperate on this, so Vates and the Community must address this. How is Vates addressing this: - Vates is aware of the issue and is actively addressing it. - SMAPIv3 integration will be ready for local drives and NFS until the end of 2024.We will build many drivers (i.e. for local SR, NFS SR, and such), and for each, we will choose a format. But it will be transparent for the user since functionally speaking it will be the same thing (snapshot, backups, live storage migrations, etc)

[@spyrule]

When we were planning on deploying ESXi to 8 hosts, the total licensing cost was north of $50k CDN / yr (and that WASNT the most feature rich license). We went with XCP-NG Enterprise, and got ALL features and support, and cost us $7k CDN / yr. We get ALL the features that were even locked out by ESXi. Other then deploying the client initially, everything else has been smooth sailing. Our only performance problem, is caused by VM's that can't utilize fast storage at native speed, due to a well known XEN problem (this goes deeper then XCP-NG, affects ALL Xen based platforms).

[@GabrievanZanten]

@spyrule would love to here more about that issue. Any links to that?

[@bstock]

I've been trying this out as a way to replace vmware for my homelab. I ran across 2 issues though: 1) Multipath for iscsi isn't really there in XCP-ng. You can do it but have to drop to the console and essentially configure it manually. That's not too bad on a small scale but could be problematic at a large scale. There's also no thin provisioning on iscsi storage. 2) PCI Passthrough is similarly not available on XCP-ng GUI. You can do it but it takes manual intervention at the CLI level. Again, not a big deal for small scale but it could be on a large scale, especially if you need to shut down and migrate VM's with PCI Passthrough regularly. If they can get these working well, I think they have a big potential to take over a ton of marketshare from vmware, but I think those are pretty big roadblocks for enterprise adoption.

[@JohnKissane]

I found getting the multipath iscsi a mess to deal with as well, it just works in ESXi but like you said there was a lot of manual tinkering via the CLI to get both paths available in XCP-ng.

[@scanmikey]

As has been mentioned... XO-Lite is being developed as a built-in host management solution for XCP-NG but it is definitely not ready for prime time... yet. As far as the XOA versus (self compiled) XO XOA is the Vates supported, delivered for business, auto updating, easy to deploy version of XO. It is really geared toward businesses who pay for Support, and it has several features paywalled. Self-compiled XO takes away most of those paywalls (except for XOSAN) but Support is not available with it. I have used the self compiled version from the beginning in my homelab... as both a native VM and as a docker container. I also have XO running in an LXC container on my Proxmox host.

[@2GuysTek]

I understand Vates' rational over XOA vs. self-compiled XO in terms of business, I just don't think it makes much sense all things considered. I expect that if/when I decide officially which way to move my 'production' homelab, if it ends up being XCP-ng, I'd be running either the self-compiled or containerized version of it.

[@olivierlambert4101]

@@2GuysTek and that's perfectly fine :) XOA isn't mean for individuals. Source version is the way for home labs!

[@jfbeam]

It's nice to see Vates learned nothing from Citrix. Artificially paywalling features just to get people to buy the "starter" license is both stupid, and totally undermines why xcp exists in the first place.

[@JasonTaylor-po5xc]

Honestly, I've only played with ESXi from time to time. My current Hypervisor of choice is Proxmox since it does everything I need and support more of the capabilities I care about without extra cost than VMWare. I do use VMWare Desktop for local VMs and I was considering getting back into ESXi so could play with the Workstation-Host integration. I may try again but I need to see if it will run on my AMD hardware since my Intel boxes have e-cores which ESXi doesn't like. I've never played with Xen or Xcp-ng.

[@JeremyMeeler]

You hinted at it, but lack of VLAN passthrough options is kinda critical for homelab folks or even larger companies wanting to run things like virtual routers. You can do it, but it's very clunky. Another fault I've seen, though not of XCP-ng or any other hypervisor, has been the lack of support for VMs running on anything other than ESXi. XCP-ng fully supports OVA imports and when you complete an import just to find out that the manufacturer/developer of said OVA places an "is this an ESXi host" check and denies use because it isn't ESXi, this can be very frustrating. Wasn't the point of the OVA standard to make VM placement flexible?

[@joebleed]

What do you mean by lack of vlan passthrough options? You can assign vlans to VMs so i'm not sure whats missing on this front? I'm working on learning xcp-ng as i'm hoping it will work for me at work to move away from vSphere. XOA's interface aside, it seems to do everything we need so far. though i do wish it had something like vmware's vSwitch setup for networking. It makes it easier to understand and follow. my hosts at work are rather simple and we don't use many vlans; but it's easy to follow loking at them in/on the vSwitch.

[@rudypieplenbosch6752]

Are you talking about xcp-ng ?, vlans work fine. And i run a virtualized Pfsense as well with 6 vlans, no problem assign, vlan to nic and vlan to vm and you are good to go.

[@kuhndj67]

Thanks for the overview... was very handy to give me a baseline understanding of XCP-ng and how it compares to VMWare. One thing prospective users should be aware of - no NVidia vGPU support for XCP-ng at the moment so if your environment needs vgpu support you're out of luck (unless you can live with passthrough)

[@MSPcraps]

Thanks. I'm just looking for a hypervisor for home use and hadn't heard of xcp-ng until today.

[@druxpack8531]

Both XCP-NG and Proxmox don’t support thin provisioning on iSCSI, only nfs…which makes it difficult if that’s what you currently are running with VMware.

[@Lodewyk1981]

But Citrix Xenserver Premium does. Busy with that project now + XOA, and it is also a fraction of the cost vmware.

[@RazvanRosca]

This will be fixed with SMAPIv3 soon.

[@sullenfps]

need one of these against proxmox

[@2GuysTek]

It’s coming!

[@Darkk6969]

ProxMox admin here for both home and work. I've tried using XCP-ng to see how well it works and WebGUI workflow just doesn't make sense. Also I was turned off on the large pricing of the XOSAN. I understand they're changing the pricing model to make use of bundles. I like ProxMox as it just works and even purchased support subscription for all of the nodes at work. They don't force this on home labs which is a big plus.

[@dragonnoa1]

He literally said its coming next at the end of this video 😆

[@Raima888s]

Is networking in XCP-ng like ESXi where it has network ports and network groups? Does it allow multiple VMs to utilise the same physical network port like in ESXi and does it support promiscous mode?

[@seantellsit1431]

I think it MUST be noted that XOA can only do backups if you have the full paid license or have compiled from sources.

[@xandrios]

This is a very important detail. Thanks!

[@rudypieplenbosch6752]

Yes, there is a good tutorial from Lawrence how to do that, its very easy.

[@KegRaider]

Compiling from source isn't all that difficult, they even have the "How-To" on their website's documentation. Spin up a Debian machine (physical or virtual) and follow the steps. If it errors on a step, read the error and address the failure.... or google it. Took me about an hour I guess to get my first one installed. It's worth the time especially if you want to have a backup solution that works so well.

[@laurentmarandet4850]

Compile it is very easy, I was happily surprised, due to scripts available on the net.

[@liora2k]

Great video , as a 2 years heavy user of xcp-ng I can tell that only their UI is not modern and looks like an old fashioned open source, BTW also their new XO lite isn’t so modern and still need to have a massive polish

[@RainMan52]

Gonna stock up on popcorn for the Hyper-V presentation 🙃 🍿

[@2GuysTek]

Does it need one…?

[@MrMcp76]

Isn't Hyper-v going away? I know Microsoft is at least ending the free version of it. Microsoft doesn't even use it for their own cloud service.

[@augurseer]

I'd like your break down on proxmox hyp-v and xcp-ng. Yea

[@Ghandacity]

​@@2GuysTek - I think you just did it.

[@RoelandJansen]

​@@MrMcp76not true

[@GrishTech]

They are working on a new GUI. Hopefully development will speed up as companies jump ship and start giving their funds to vates.

[@2GuysTek]

I’d heard rumblings about the same thing, I hope they address some of my complaints with the new GUI. The irony is a friend of mine showed me the old Xen fat client (windows only unfortunately) and it’s miles better than the current XO UI. I don’t know how they went from having that as a template to the current incarnation of Xen Orchestra.

[@TITAN2K]

@@2GuysTek last I heard for a GUI, you can still use XenCenter to manage a XCP-NG host/hosts without XOA. Although it is still a windows only based app, and only installed on your workstation. So it does have some draw backs unlike having a local GUI, vCenter or XOA. Still great round up.

[@KaiHenningsen]

@@2GuysTek Licensing problems preventing them to opensource it and forcing them to start from scratch, combined with time pressure to get something out that works well enough?

[@ronkelley1184]

XCP-NG works great until it doesn't. I used it a few years back and ran into two main problems: * Sometimes after rebooting the server, the mgmt interface would disappear - especially if the server was part of a cluster. No amount of cli commands would revive it. Time to reinstall (again!). * The VM disk files (VHD files) are written to a single directory using a UUID value. This means trying to manually recover a VM is very difficult since the VM name is not part of the VHD file. There is no per-VM subdirectory (like ESXi) - making it much more difficult to identify which VHD files correlate to which VMs (an especially important issue when a VM has multiple VHD files). My suggestion: Try XCP-NG, force it to break (hard power-off, remove a NFS share, etc), and then try to fix it. Only then will you appreciate the effort to get your server back. I am not saying XCP-NG is good/bad. Just make sure you know what you are getting into...

[@RazvanRosca]

Give it a try again, a lot has changed.

[@laurentmarandet4850]

Use backups with XenOrchestra (free when compiled from sources, which is easier than you could imagine, thanks to many scripts available to automate it).

[@gmgaming2]

What about XO Lite for a built in host management GUI? 3:30

[@2GuysTek]

It's not built into the current version of XCP-ng yet, I understand they expect to bundle it soon in the later versions of XCP-ng though. From what I've read and been told so far, it's pretty rough around the edges still. Have you used it?

[@minigpracing3068]

​@@2GuysTekIt is also not even close to feature complete! It isn't scheduled to be ready until XO v6 which is still a bit away. The last version of XCP-NG Center us due out soon, but it is EOL after this.

[@gmgaming2]

@@2GuysTekNot yet, I’ve been looking into XCP-ng for my home lab, as I’d like to learn more about virtualization and get off Hyper-V. I’m okay with rough, or even not yet, as long as there is an understanding that it’s coming/improving.

[@kevinhilton8683]

@@minigpracing3068 Are you sure XCP-ng Center is EOL? I thought a new developer was announced and I'm aware there are nightly builds which work great.

[@minigpracing3068]

@@kevinhilton8683 You are correct, a new person has stepped up to try and keep it going.

[@en4ble773]

What about running LXC containers (to what proxmox have)?

[@Catskeep]

hi, are XCP-NG have feature like VMware Site Recovery Manager (SRM) ?

[@WillieHowe]

Great video -- the thing that I keep looking and and wondering is the 2TB file size limit with XCP-ng. Is that still a thing? It could be a deal breaker for some -- ProxMox doesn't have that limit. Thanks for great content!

[@RazvanRosca]

Here's some clarification on this matter. Problem: - SMAPI (Storage Management API) v1 uses VHD. - VHD (initially developed by Microsoft) has a 2TB disk limitation. - this is not an XCP-ng or XO limitation, but the format itself imposes it. Current workaround: use RAW. Downsides of using RAW: - RAW doesn't support copy-on-write. - This results in no snapshots -> no live storage motion -> no live migration between storages and no incremental backups. Other workarounds: - Use network-attached storage inside the OS or iSCSI. - Use multiple 2TB virtual disks and create a software RAID0 in the OS. Disks can be excluded from snapshots (snapshot the entire VM, but not the RAW drive). Solution: use SMAPIv3. About SMAPIv3: - SMAPIv3 is not production-ready yet. - Citrix isn't looking to work or cooperate on this, so Vates and the Community must address this. How is Vates addressing this: - Vates is aware of the issue and is actively addressing it. - SMAPIv3 integration will be ready for local drives and NFS until the end of 2024.We will build many drivers (i.e. for local SR, NFS SR, and such), and for each, we will choose a format. But it will be transparent for the user since functionally speaking it will be the same thing (snapshot, backups, live storage migrations, etc)

[@xandrios]

Great introduction to XCP-NG, thanks. It indeed is painfully obvious that there are very few true enterprise options to replace VMWare outside of a home lab. How does Vates fit in this picture; they seem to offer a slightly different version of orchestra against a very different pricing model (Similar to VMWare Essentials Plus).

[@LloydRandall]

nice, are you going to do a XCP-NG vs Proxmox series?

[@tomrberg]

Would you evaluate Oracle VM x86 and Oracle KVM as well?

[@fxeconomist]

I'm just a regular guy using the free VMWare Player. What I found that sets it apart for me, is that it does something that both Hyper-V Manager and VirtualBox can't do : USB Passthrough. It has a simple mechanism to unplug ANY USB device from host and plug it to the guest. This brought me to my operational target, to get the host offline while the guest is online. So the Windows host practically becomes a gaming enabled type 1 hypervisor - something that Qubes cannot do. Is there any other solution out there - type 2 - that can do the same thing ?

[@Garageland16]

Is Nutanix CE a thing?

[@2GuysTek]

Nutanix does have a community edition of AHV, is that something you’d be interested in seeing a video about?

[@joaopedroalbernaz]

Totally ​@@2GuysTek

[@Darkk6969]

@@2GuysTek Would love to see CE in action.

[@zparihar]

I'd love to see that in action as well

[@jimlynch9390]

Great video. Thanks for the comparison. I tried to use Xen years ago but due to the very unfriendly and non-helpful user community I went elsewhere. I was forced to use VMware at a school I was volunteering at and found it a bit to complex for what it did for us. I had also installed it on my home systems for a while but gave it up for similar reasons. I just don't like it. The ui of XCP-ng reminds me of the OPNSense ui. It's not the greatest but it does get the job done.

[@milicsantiago]

great video. The last thing of VM import didn't realize until you mentioned. I had to rewind the video a couple of seconds, ha! Regards form Argentina

[@allesbelegt]

Xenserver would also be an interesting comparision. Since the beginning of 2023 it is no longer part of Citrix itself. The new Trial Edition of it, is no longer limited like it used to be, but comes with the same features as the Enterprise Edition. The only Limitation that is still in place for it, not for production/commercial use

[@KennethsTeknik]

Thankyou for the great video, I'm worried about the agents, having been on a XEN platform before, it was impossible to safely migrate some of the servers to Vmware (back when that was a thing), so how about the agents for xcp-ng, being based on the same hypervisor, I guess that's a big problem to migrate away from xcp-ng if first chosen?

[@autohmae]

One of the things I wondered: why do I see a lot of people saying: Proxmox instead of XCP-ng ? And why do all the VPS and cloud providers run Qemu/KVM instead of Xen ? (AWS being a good example of a company who moves from 1 to the other), which is obviously a big deal).

[@gerhardroediger8331]

because those people have no idea what they are talking about: proxmox is something different than xcp-ng. On regards to your second question: I guess because Qemu/KVM are considered type II hypervisors whereas Xen is a type I hypervisor, which means KVM/Qemu are hosted hypervisors in contrast to Xen(upload.wikimedia.org/wikipedia/commons/thumb/9/9e/Hyperviseur.svg/800px-Hyperviseur.svg.png ), which would be running on the "bare metal". I guess it just offers them more control/security over their own machines with KVM/QEMU.

[@gerhardroediger8331]

@@nigratruo not sure about that one. You can't install kvm on bare metal, can you. Since it got part of kernel I guess it's kind of debatle how type1 kvm really is...

[@MKjanek32]

@@nigratruo There are still type 2 hypervisors, KVM and bhyve being actually classical examples by their definitions. KVM as a type 1 hypervisor is a very debatable position given that it integrates with the host OS simply as a kernel module and runs VMs as a user processes, just like VirtualBox for example. And it has not even be on Linux, because KVM has been ported to FreeBSD, Illumos and maybe some other systems. On the other hand, XCP-ng and ESXi are both really type 1 hypervisors. Slightly misunderstanding from Rich at the beginning of the video, Xen itself is a microkernel in many ways similar to ESXi, directly controlling the hardware and running Linux host OS used for hypervisor management in a special, privileged kind of VM called dom0. Those two forms the XCP-ng hypervisor. In some scenarios when live VM management is not needed (maybe in IoT or automotive), it is possible to run just Xen with minimal overhead, entirely without dom0.

[@michaelgregory903]

Sounds like a pretty good replacement. Though I think the VM essentially pausing during a snapshot with memory is a big downer at least for my environment. I hope they can figure out how to get around that in a update, otherwise I won't even be considering the product. Thanks for the direct visual comparison, it was really a good eye opener for this other platform that is available.

[@chrisbarth1887]

Reminder that it only pauses if you snapshot the active memory state.

[@JibunnoKage-YouTube-Channel]

The original XenCenter (yes some time ago) was so unstable in our labs we dropped all XenServer evaluation years ago. Let's home the Xen-O is much better! The key issue... is the VM migration from hypervisor to hypervisor is it transparent as VMotion? Or does it has the typical VM freeze issue during migration that KVM still has today?

[@laurentmarandet4850]

No freeze unless you are using the deprecated dynamic memory option.

[@martymccafferty7510]

If you need thin provisioning in ice ISCSI, your ISCSI server should provide the thin provisioning and SnapShot capability. You could use ZFS for your data store and ZFS gets you thin provisioning, snapshots, etc. These things might be advanced for some users, but worth learning.

[@petersachs2750]

We use Xenserver in production and overall it works well however we have had hosts crash without explanation and our workloads are mission critical. We've been looking at VMware but like many are concerned about rising prices.

[@RazvanRosca]

Give XCP-ng a try.

[@BoraHorzaGobuchul]

Proxxmox?

[@jani140]

Proxmox. We had scrapped our last xcp clusters years ago because of that. Since then running stable on Proxmox-ceph. Really. Skip XCP...

[@shannonhull7243]

Do you know that anything like NSX-T abilities for XCP-ng

[@fleacero]

Wow, what a fantastic video. Well done, great work!

[@SoliasSerene]

Vmware VVF? VCF? What do the acronyms stand for?

[@iamagi]

How well does it handle a few dozen hosts?

[@andibiront2316]

My experience. Before Broadcom: 3.5M in support and licese every 3 years. After Boardcom 4M every year.

[@jeremiefaucher-goulet3365]

I still mainly use XCP-ng Center and the command line. But i look forward to XO Lite. One day I'll deploy my k8s cluster, and then I'll run full XOCE

[@jojobobbubble5688]

So now that both video's are out, which did you go with? My guess is that you went with XCP-ng.

[@louisdotarsenault]

What about vSan for xcp-ng?

[@thecrazymouse7220]

What is the ability to expand drives greater than 2TB?

[@laurentmarandet4850]

You can combine many 2 TB disks, with Windows and Linux VMs to solve the issue.

[@justanotherhuman-d6l]

Can you do hyperv and proxmox vs VMware including price?

[@Diversifried]

The beauty of XCP being open source is, the complaints you have can be tweaked to your liking. You can manipulate everything you see in the environment. Excellent vid

[@112Haribo]

Yeah but companies looking to move away from VMWare are not looking for customizable. They want stuff to Just Work. That's why they paid VMware in the first place instead of using KVM and QEMU and libvirt etc.

[@Melpheos1er]

The only two caveats I found but they are related to xen and not xcp-ng itself compared to esx are migration which are a tad slower and with lot of memory means a longer vm stun and memory included snapshot which definitely stun the VM where it is transparent on esx

[@brutester]

12:36 I think you can use RDP to get to the console remotely.

[@tehduke1776]

"Giant Beard" That was so cute. :)

[@Guichy]

I guess I’m biased since I’ve worked with vSphere for more than 10 years. I tried xcp-ng and could not get the Xen Orchestra using the 2 documented ways in their documentation and never worked. I gave up lol

[@victorshane4134]

And here i am. The Proxmox guy :)

[@6downtown9]

Planning to learn virtualization in the future .just found out XO is more user friendly then exsi GUI

[@Wayne_Mather]

I used xcpng for years, moved to esxi recently due to moving to synology (backups and restores reason for moving) Guess am having to go back or look at hyperv....

[@Wayne_Mather]

It should also be noted xcpng is lole centos was. It's an open source version of a commercial product - citrix xenserver. So while it maybe open source, it does have an upstream dependency

[@BoraHorzaGobuchul]

Why not proxxmox?

[@Wayne_Mather]

@@BoraHorzaGobuchul Yes am currently looking at what proxmox and proxmox backup can provide for me

[@haraldfielker4635]

I never ever saw a company explode that fast as VMWare.

[@2GuysTek]

Pretty much.

I used XEN with OVM a long time ago. It had a similar repositrory approach for storage, but it used OCFS2 for LUNs mounted on FC. What filesystem xcp is using for shared storage? Anyway OVM was crap compared to VMware, but you should make a video about the OLVM, which is its successor and it is also free. Maybe worth a try. As I know VMUG Advantage is still available(VMUG webinar on Friday). Those licences are good for homelabs.

[@2GuysTek]

I'll look into OLVM, thanks for the tip!

[@Darkk6969]

There is no telling what they will do with VMUG Advantage in the future. Probably low priority for now while they wreck the other pricing models.

[@michaelshes9562]

One note regarding XCP-ng is that it does not support install on USB/SD-Card device as VMware/Proxmox

[@KarlderHeinz]

Please compare vCenter and XOA🙏 Great Video 👍

[@svettnabb]

If you have xo-san, you don’t even need third party tools like veeam in order to do file restore. Vm backup is free in basic

[@berndeckenfels]

What about vSAN?

[@kevinwilliamson7158]

I'm wondering the same too.

[@RazvanRosca]

Great question. We have something named XOSAN.

[@Traumatree]

The Broadcom moves only benefits Microsoft Azure imo as people who are still on Vmware will now consider going on the cloud even more (sadly). Edit: The other major thing you forgot to add to the "things to hate" is it is RHEL-based which in my previous testing a few years ago, isn't really friendly to AMD based servers. I do hope things have changed, but that is still a red flag for me knowing that I mainly use AMD based servers. Edit2: Will you compare TrueNAS Scale to your list of options?

[@tmitch120]

*IF* I had a real choice whether or not to migrate off of VMware, the lack of a VRC equivalent or ability to open the VM web console in its own window would be a deal-breaker. I have a minimum of two VM VRCs open on my desktop at all times. As it is, I like XCP-ng best of the VMware replacements that I have looked at, and will be looking for a workaround for the remote console issue.

[@jamespadgett5761]

How about Harvester from Rancher.

[@rimantascalyj276]

Does it support PCI pass through?

[@RazvanRosca]

Yup.

[@IvarsRuza]

Every feature of vmware is payed and has to have vcenter on top of esxi. Vcenter is total mess after hyper-v or proxmox

[@thraxarioustailchaser158]

...except the install fails and boot loops on some new hardware. This is why I went with proxmox instead.

[@jeanburgos3416]

Great video, and break down. That GUI... UGH!. Now we know why there are front end developers.

[@servalous]

the difference is that XCP-ng has these infos prepared for you in the console, while vSphere ESXi requires you to put in commands to get them. One of the many critical feature requirement is to add/change vm resource with near no down time. If I would be required to shutdown a vm to expand the vmdk/vdisk, all that stuff would generate so much over head and over time. That most people would get a cobain headache... If a hypervisor cant change the vdisk while the vm is active, is a huge "no,no..."

[@autohmae]

20:25 that's easy: for businesses, it's not the homelab users which bring in the money.

[@ashuggtube]

Brought here by Lawrence Systems. Thanks for this video!

[@2GuysTek]

Tom is the best! Thanks for stopping by!

[@ashuggtube]

@@2GuysTek well, he’s okay I guess. I’ll be back for the proxmox video, for sure!

[@mochouinard]

Setting up the ISO storage and uploading to it is annoying ! But I'm happy I switched to XCP-NG a few years ago now ! I went with this instead of ProxMox, because XCP-NG was much more stable !

[@mochouinard]

@@nigratruo I never could stand container system. It feel so limiting or just a waste of time to manage and setup. I feel container is just great for just trying a software quickly without the hassle to actually install it. But once you start actually do more with it than it base functionality, it just become more trouble than it worth. It really just feel like shareware type of things. For example, you need to put an additional container with a reverse proxy to share https port between multiple container... It just make things much more complex than it needed. Simple OS + package system to me make much more sense.

[@Foars989]

while i don't hate how XOA looks, sometimes i get lost lol, i hope their upcoming UI changes make it easier to navigate

[@lazerusmfh]

Xcp ngs lack of ability to run containers destroys its use case for me. Proxmox has been a major life improvement as a large majority of my virtual machines are Linux based. I can deploy a CT on proxmox in only a minute or two, and they reboot fully in about 10. Currently have well over 50 proxmox nodes

[@KegRaider]

ESXi doesn't have Dockers either though. Unraid has KVM and Dockers as well as a fairly decent storage solution. Does Proxmox require you to load disk drivers during Windows VM creation like Unraid? To b fair, he didn't compare ESXi to Unraid or Proxmox though.

[@lazerusmfh]

@@KegRaider unraid is simply just not mature or stable enough. After running multiple unraid nodes for over 10 years they are just super fragile and like to break in unknown ways and for stupid reasons. Proxmox is dead nuts stable and updating works every time.

[@peteratkin3788]

I'm still using ESXi, looked at XCP and honestly too much of a learning curve, the cost of XCP it's not free or cost approx by any means as I found out, unless I missed something, happy to be corrected. - that being said, I am planning to move to XCP, but I'm not in a hurry; thanks for the rundown. It's really pushing me; I'm just not there yet; I'm not a Linux fundi so XCP is a bit daunting with or without cost.

[@pmflav]

Xcp-ng can be free. Community xoa

[@RayAndress]

Why have functionality that has to be paid for which can be back door made free? Because here in the corporate world, it is worth paying for some things. You see a hurdle they are putting in to charge companies like the one I work for. I see functionality that when I need to redo a setup, I can pay to have set up for me and we both win, while you can do the compile method I don't want to do and test the functionality out. Keep up the video's. It was a great review.

[@-tineidae]

What all open source (except ovirt) is missing is shared block storage. If you use ESXI with a SAN, there is no alternative because all these open source solutions do not support snapshots/thin provsion on shared block storage. Thats a limitation of LVM.

[@juebaitube]

Thank you. Well done ❤.

[@believeinheroes]

Very helpful! Could you do a similar video about proxmox from a VMware user perspective?

[@2GuysTek]

It’s in the works now!

[@MrMcp76]

Great video. I like xcp-ng, but for now I will continue to run my homelab on ESXi. The company I work for uses VMware, and we have discussed the Broadcom changes and how we may move forward. I brought up XCP-ng and got only laughes. They won't consider it because it is based on Citrix, and because of the short-comings of the support. We are a medium-sized business, and our infrastructure is 100% VMware locked in. Switching would take years, not even considering the years of planning and research needed to switch. I think in the end the majority of companies that will switch away from VMware are those that are being kicked out by Broadcom, or smaller companies that can't afford the new pricing structure.

[@hasanmujeeb8922]

Why is switching that hard? And take so long?

[@MrMcp76]

​@hasanmujeeb8922 Because all of our systems (in-house apps, SQL databases, backup solutions, network connectivity, storage arrangements, etc) were all built around VMware and it's capabilities and support structure. VMware does EVERYTHING but other type 1 hyoervisores do only some of those things. The reason the Broadcom changes are so devastating to many is because VMware is the foundation of their entire network and infrastructure and trying to change that would be more expensive than paying the extra to Broadcom. And Broadcom knows this.

[@Darkk6969]

Actually switching from VMware to another hypervisor like ProxMox isn't hard at all. I've done it with our VMware servers before this broadcom buyout. In fact VMware even provided the tools to allow easy migration from the host servers to another product. Just have to build a test environment and copy few VMs over to see how it works. It only took me three months to validate the VMs to ensure they will run on ProxMox with ZFS without issues. Then I did the full migration over the weekend. Users didn't notice anything different.

@@Darkk6969 How many hosts and VMs have you migrated to Proxmox?

[@druxpack8531]

I’m curious to see how your company handles the price increase when renewal quotes are in-hand. It’s easy to say all is cool in a meeting, it’s another when you actually have to cut the check.

[@VasiliyVolkov]

Отлично! Спасибо! А есть такое же детальное сравнение - vSphere vs oVirt? ;-)

[@nicholasl.4330]

Great video!! I would like to mention that VMware is not killing off VMUG yet, which is a great option for the homelab

[@gearboxworks]

He says he's not dead yet.

[@succuvamp_anna]

Subscribed for the "Corporate Bullshit" quote. Been redoing all my studies, I was about to take Certification Test for VMWare but..... you don't get on a boat sinking.....

[@minster586]

There is also XCP-NG center

[@zyghom]

what happened to Proxmox? died? or not comparable?

[@luis449bp]

Proxmox still exist and it's improving with every release

[@2GuysTek]

It's alive and well and we'll be comparing that in the next installment video!

[@Darkk6969]

ProxMox admin here for home lab and work. It's alive and well. They're now on 8.1 series and keeps getting better over time.

[@gabrielegaetanofronze6690]

Hey there! Thanks for the insights, always welcome! With my company we developed a new type-1 hypervisor based on kvm we call AtomOS. In fact it is rocky linux or alma linux (you can choose) running kvm-qemu plus a lot of smart features on top implemented as daemons. In addition it is autoscaling, that means it has no centralised services and a virtualisation cluster is an emerging behaviour from a bunch of servers within the same LAN (VLAN or VPN as well!). Big plus: it uses a VM definition protocol which we make compatible with all public clouds, in fact opening a world where hybrid-cloud is possible by design. And yes, we built in automatic GPU passthrough support! We would be delighted to get you some copies to get some feedback and, if you mind, for you to make a video!

[@2GuysTek]

Reach out to us via email or on our discord and let’s talk.

[@gabrielegaetanofronze6690]

@@2GuysTek done! Thanks 🤟🏼

[@CodeX-f6b]

Would be nice if these projects could agree on a clustered file system solution that gives a similar experience to VMFS. This is a pretty big shortfall imo

[@gearboxworks]

I found the criticisms of the XOA UI interesting. I've only used VMware a small amount in my career, but I really dislike their UI. It feels dated, cramped, and hard to use. OTOH, when I saw XOA in your video just now, I found their UI refreshingly better. I have not actually used it yet, so that's just my first impression. Who knows how I will feel when actually using it. Any chance your criticism of the UI is because you are just used to VMware and not XOA? I know that I am certainly more comfortable with the things that I know well vs. alternatives that change how I need to approach working. 🤷‍♂️

[@dennislindqvist5461]

What about automation? VMware supports an extensive library of commands in PowerShell. Anyone being serious about administrating virtual machines need to be able to automate most anything about the virtual resources.

[@2GuysTek]

From my research into XCP-ng, it has a rich API for automation as well.

[@kevinalsop3772]

well done. thank you.

[@whiteyau88]

XCP is a good start but it has some oddities/issues which you only really encounter with use. No USB passthrough support No SAS shared storage support If the first time you attach an iso is after booting the VM you have to reboot it again to have it fully install Guests without management tools installed can't have their network changed while running Some linux VM's can't be live migrated (notably Citrix Netscalers) No ability to browse datastores, even via the CLI You need to create a separate ISO SR You can delete a disk from a running VM without any warnings or file locks The free built in backup is not differential XOA consumes several GB of ram, which can be a detriment on very small deployments No easy ability to see what VM's are on what networks Very poor alerting, eg out of the box it will not send emails that a storage path is down. (seemingly they can only be turned on with xcp center)

[@TayschrennSedai]

ngl i'm judging you on that E5-2680 =D - Decent video, though the one thing I really agree with is that the vsphere 'console' hasn't changed literally since like... 3.5? Probably before honestly. But....... That's because it has a built in GUI =) and it's an HTML5 gui... Still, the time i spend in vmware console is next to never. Overall? It's decent looking but I'd still pick vmware any day of the week for business use. Broadcom needs to fix pricing or they're going to get a wakeup call.

[@tekviper9]

Thank you for this comparison but from your information and other research out on XCP & Prox Mox both technologies show promise but are not mature enough to support an enterprise level. Ihope they continue to mature.

[@sbme1147]

Actually from the first post LawrenceSystems ( his media arm of his IT company ). They do backups for Netflix, he specializes in XCP-mg and 95% that’s exactly what they use. Been watching his channel for a while now

[@tekviper9]

@@sbme1147 Not picking an argument Im speaking of real enterprise, 200+ clusters 1000+ hosts 30+ vcenters, 40,000 VMs (GM, Ford, Chrysler run these massive enviorments) Netflix Backups is one thing massive commercial enterprise is a different league. These two tech show promise but have a bit to go. In 2005 VMware went from ok to greatness. Prox Mox and XCP are the VMware of 2005 they just need polish.

[@RazvanRosca]

It is *very* production-ready. :)

[@BoraHorzaGobuchul]

Driver his is proxmoxx not production ready?