Making The Case for Open Source Hypervisors with Tom Lawrence!

[@2GuysTek]

Be sure to leave your comments for Tom and me below. In a few weeks, we'll answer them on the live show!

[@Rood67]

I thought my work would stay because we have such a large VMware structure. But Tuesday, the head VMware admin said he is evaluating option because it will be too expensive to stay where we are.

[@2GuysTek]

You’re not alone.

[@christopherpeterson6004]

$6,400 CAD /year

[@cheebadigga4092]

Yeah at work my colleague is currently waiting for a licensing offer from Broadcom and depending on that he has to decide whether to stay or not. For us it boils down to "does moving to Proxmox or other options, and the cost that comes with moving to it and maintaining a new platform hold up in contrast to the licensing offer we'll get". I personally hope for Proxmox. We're gonna evaluate Proxmox's capability for SANs as a side project to see how it holds up compared to the ease of use of VMware/ESXi regardless of Broadcom's offer.

[@BPL-Whipster]

I'm a hyper-v shop (despite being employed as a VMware admin), but my partner program distributors have all been departnered by broadcom apart from one. Appalling. We can't even service small businesses that want VMware outside of Azure or AWS now, effectively.

[@nadtz]

Went from vmware/xen then later xcp-ng at work, when I finally got around to adding a vm host at home proxmox did take some getting used to but it wasn't that bad for a single host homelab. Having a working, easy to use gui without jumping through hoops was also nice *but* if my goal had been to replicate what I use or run into at work I'd have run something else, I specifically set out to learn proxmox because I wasn't that familiar with it but I kept hearing how good it had gotten and I'm happy I did.

[@ipstacks11]

Why would you have to run something else? Where did proxmox fall short? I am curious because I’m not, a VMWare expert and when I started using it at work, all I could think about is how proxmox doesn’t have all the clunky-ness of VMWare. I was thinking maybe the virtual switching was better? I would love to see a real comparison of VMWare on the same hardware, doing the same workloads against proxmox. Where Inwork the VMWare pros say proxmox couldn’t fill the need, but never say why in a reasonable way.

[@nadtz]

@@ipstacks11 Not understanding the "Why would you have to run something else? ". For my use at home proxmox has been great and I have no plans on changing hypervisors.

[@PowerUsr1]

damn this interview was gooooood. Lots of gems if you play close attention. How OSS projects work and the little intricties of it. Seriously this was good stuff. Its worth a second watch actually. Lots of game to soak up.

[@akurenda1985]

This will just mean that Vmware Professional Certs will be niche. Which will mean that you get more money for supporting it =). I'm glad this will allow companies to branch out to something other than vmware, though. XCP-NG and Proxmox are fantastic.

[@Ingeanous]

I think an exodus from VMware from mid size companies will flood the market with VMware admins that don't have the skillset to quickly transition to another hypervisor platform... I remember something similar happening 20 years ago when there was a transition from analog circuit multiplexers to everything to over IP routing...

[@geneann8286]

You were talking about backups, but never mentioned Proxmox Backup Server for some reason.

[@Darkk6969]

ProxMox Backup Server been solid for us. I use it for home and work.

[@locusm]

Its a great product - it will also quiesce your SQL and AD databases using VSS. Its perplexing why Citrix dropped VSS support.

[@BagerTube]

Yea, love PBS too

[@InterFelix]

Well, it does backups just fine... But is it comparable to a modern backup solution in any way? No, not at all. And that's the problem. There's a lot of functionality missing, especially around application aware backup and restore, like guest single file restore or application item restore. If all you want to do is backing up and restoring entire VMs, it's perfectly fine. But many of my clients use features like application item restore and guest single file restore extensively, so support of enterprise backup solutions is crucial for them. I'm thrilled to see Veeam announce their intent on supporting Proxmox in the future, and I would like to see other backup vendors follow suit. Also, I'd love for them to do the same for XCP-NG, as it's the more enterprise-ready hypervisor platform imho.

[@geneann8286]

@@InterFelix Yes, it is not application aware but you can restore single file(s).

[@MaxTheDog167]

I love to use proxmox ve and proxmox backup server on my homelab. It helped me learn some Linux which I am happy to branch out of Windows and Apple. I am enjoying it a lot. I may look into other hypervisors to setup for fun though.

[@gdchance1914]

It would be interesting to hear more comparing oVirt and/or Linux VM. But kudos from those of us that are moving away from VMware.

[@ssyd780]

At 34:47 you missed that Proxmox supports LXC containers natively in the hypervisor. I use it for some of my simpler services like a Pihole. Hell, even my fileserver runs in a container that I pulled down from within Proxmox where they include a bunch of Turnkey containers for all sorts of tasks, ready to run. Edit: Whoops got keyboard rage and then Tom mentioned it a minute later lol

[@Jordan-hz1wr]

We switched from VMware to XCP-ng and couldn’t be happier with everything from XOA to the team on the forums.

[@2GuysTek]

When did you cut over? How was your transition process?

[@Jordan-hz1wr]

@@2GuysTek We started slowly cutting over November of last year and just finished last month. Two 4-server clusters running ~100 vm's using the V2V tool. It went very smoothly. I reached out to support once just as a test drive (you can utilize their support with the trial license btw) and they had the "issue" resolved before VMware would've ever accepted the ticket. We're a mid-size local city government.

[@LAWRENCESYSTEMS]

Awesome!

[@robrjones99]

I appreciated this video. Both of you have great perspectives and are reasonable in your approaches. We still don't know what we are going to do with our small environment of 6 ESXi hosts and about 360ish vm's. The backup "issue" is an issue. We have invested a lot in a solution that works well with VMWare. So, we'll have to figure that part out too. But thanks again for this and keep putting out the good stuff!

[@david_kinyua]

Proxmox has a pretty solid backup server perhaps check that out too. Migration to proxmox isn't as smooth in proxmox as it is in XCP-NG I'd say but worth checking out the two and evaluate.

[@brodriguez11000]

Comes down to, are there any alternatives to all things VMware? License is secondary since this bad news didn't leave much time for changeover. Past that the open vs close becomes more important.

[@kuhndj67]

I had a quote for another three years of sns on my small cluster but didn’t get it ordered in time so needed to refresh the quote post-acquisition… quote went up 25% to over $200k, so now I’m evaluating alternatives and Broadcom will need to recover another $150k from the whales just to break even.

[@jasonklems8584]

@kuhndj67 just curious, how big your environment is/was ? size of cluster, number of VMs , etc. thanks in advance

[@thetux0815]

As a former vmware staff member I'm running my home lab on proxmox (and nest some vsphere labs on it if needed). But on a professional level I'm not too sure. How're you guys approaching the issue that basically no major server vendor (Dell, HPE, Lenovo, Cisco, ...) offers official OS support for Debian/Proxmox? That's a hard sell for business critical use cases.

[@Darkk6969]

As more and more customers move off of vmware to other open source like ProxMox this will give them more money to pay for more developers on their platform. This is also happening with XCP-ng. Both are winners in this.

[@Kevinmulhalljr]

The nonprofit sector has been suffering as well. VMware’s philanthropic program was killed last October and the community has been so reliant on it. Would love to connect to Tom to understand more about cutting over to XCP-NG or Proxmox

[@itlackie]

I've been nudging my boss to move over to XCP-NG for work. He mentioned how crazy some of the price hikes have been and how expensive it is.

[@bryonadams5526]

I appreciate the support being explored more in the conversation. Proxmox will struggle to be proper enterprise ready until they revisit their support options. 2 hour SLA during business hours is pretty bad, and limited support tickets for certain tiers is a bad look. It really only works when you don't need support, which for some is a perfectly fine business decision to make. The two prior videos for XCP-NG and Proxmox were great, though one thing that was missing for me was nested virtualization support for things like CML (Cisco Modeling Labs). Any idea on how well these work on recent releases?

[@mrljvb]

No detailed mention of Harvester.... I've been curious as it will integrate well with Rancher which I use for Kubernetes.

[@Ristridin2]

It actually does, we‘re evaluating the solution right now

[@pvalpha]

Yeah, I keep on meaning to give harvester a try in my homelab. I'm kinda torn between XCP-NG, Proxmox and Harvester for playing around with. I don't think I'd deploy it for our on-prem at work, but it is certainly worth a look.

[@highrzr]

It use to be CA was where technology went to die. Broadcom said hold my beer, and they're now where technology goes to die.

[@2GuysTek]

Facts

[@williamp6800]

In the past,it was said that Symantec was where software goes to die.

[@Mitchell7790]

One benefit of Veeam and others is application-aware support for things like Active Directory, SQL and other systems that a good chunk of enterprises rely on. Still to see that elsewhere.

[@2GuysTek]

The good thing for us Veeam users is that we can always switch to agent-based backup and get the same features as VM-based backup, and once XCP-ng and Proxmox finally supports Veeam, we can just switch over to that and maintain our backup chains.

[@Tetrastructural]

@@2GuysTekyes but then you probably need to pay for extra Veeam workload licenses to manage the individual backup agent jobs. 🙄

[@johnharrison712]

We are still stuck on VMware NSX since we haven't found anything that is opensource or come close to NSX. Any thoughts if XCP will be coming down with more tools or features sets?

[@damiendye6623]

Nsx can be done with open vswitch

[@garymackay611]

As mentioned several very good options to run server vm’s. One piece I do not see much on is Virtual Desktops. What do I do with my VMware Horizon View clients? Microsoft’s Azure HCI is an option but crazy expensive. Is there anything in the Xcp-ng or Proxmox or ???? world to handle this?

[@LAWRENCESYSTEMS]

There is not really any good open source replacement for VMware Horizon

[@garymackay611]

@@LAWRENCESYSTEMS Any solutions? Open Source or not?

[@LAWRENCESYSTEMS]

@@garymackay611 Nothing good, Citrix is probably the most popular

[@y0jimbb0ttrouble98]

Unfortuantely the only real alternative to Horizon is Citrix CVAD, Parallels RAS and then MS RDS farm. Citrix CVAD runs just fine on XCP-NG but is not supported on Proxmox although Nutanix is supported (its pretty good on Nutanix).

[@Br0llyLSSJ]

I read that there is a 2TiB file size Limit for the virtual Disk (VHD format) in xcp-ng. Is there any chance, that it now gets faster updated to bigger sizes? Are they maybe adopting the vhdx format, which has a limit of 64TiB? 2TiB is too small to be considered for us as we got many file servers with bigger disks (some of them got even bigger than the 16TiB allowed by Proxmox).

[@kristopherleslie8343]

Is that the only option you have considered

[@APHATMOUSE]

Where do we find the backup writeup that Tom mentioned having recently completed?

[@andrewjohnston359]

I'd like to know this too....what's the setup for multi-tenenacy (for MSPs) - managing multiple clients, backing up to the cloud and confirming the restore was successful, and hooking that into a ticketing system =)

[@syedsulaimankajamohideen6056]

I really hope XCP-NG will have good HCI solutions especially for virtual SAN solution like Vsan from VMware. Proxmox has native support for CEPH for this

[@thecrazymouse7220]

Can you provide feedback on XCP-NG limits to 2TB drive space. I am VMWare and have systems with storage 3+TB. Would this not work in XCP-NG? Promox states it can handle 16TB. Please let me know what your experience is.

[@donaldwilliams6821]

The maximum file size limitation for XCP-ng's VHD format is 2 TiB, which means that a VM disk can't be larger than 2 TiB. However, some have used multiple disks with spanning to create files larger than 2 TiB. Others have used raw volumes for VMs, which can exceed the 2 TiB limit but no snapshots

[@edwinphilips5212]

This (Open vs Closed) is an interesting conversation, particularly where RedHat and Google appear to be slowly closing their source now. I personally find myself torn. There are, I think, concerns about sustainability. If lazy people can grab the code and spam the market with cheap knockoffs, without contributing to the actual development, the organization actually doing the work starves. How do you protect against that? And I get that having the source open can protect against it getting shelved, and allows more eyes to look for bugs, etc..., but CAN and WILL are two entirely different things. Seems like it can create a false sense of security in some instances.

[@2GuysTek]

Sure, that's a possibility. Tom made a good point in the video about the 'Just fork it' mentality and it being easier said than done. I think that when you have two very strong open source projects like XCP-ng and Proxmox as active players in the space, it's harder for copycats who would use their code to gain a foothold in the space because users will naturally move towards the main projects first. Thanks for the comment!

[@gg-gn3re]

redhat hasn't and can't close any source. It's still all open source

[@edwinphilips5212]

@@gg-gn3re Perhaps I am misinterpreting what I am hearing - my understanding was they were making at least some of their code inaccessible without signing up. You can sign up for free, but they can revoke access. While none of that closes the source, it does seem to be a step on the direction of building the means to do so in the future (And they did rant about freeloaders as a response to those who complained, which suggests it is on their mind).

[@levskilevov4888]

@@gg-gn3re??? It is open, but you don't have options to download, only their clients who pay for support! Red Hat = IBM

[@damiendye6623]

That's what VMware did when it was smaller, it used Linux with not contributions at all. It still does as vcentre is a Linux appliance

[@RonnieRedd]

Glad I learned Linux KVM in the home lab at the same time as needing to move to vmware at work (I did it when only vmware did windows vm's) (yes, it was EARLY). Too bad that's maintained by RedHat 😮

[@LampJustin]

Why too bad? RedHat is a great steward of OpenSource and without it many great projects would suffer and we wouldn't be where we're at... They need to make money somehow and CentOS stream is 95% rhel. It's the direct upstream. Every change will be in there first. It's not just black and white

[@RonnieRedd]

@@LampJustin yeah, RedHat used to be all of that. IBM is ruining that. CentOS has already become irrelevant

[@LampJustin]

@@RonnieRedd this is not IBM doing it. RedHat works autonomously...

[@damiendye6623]

​@@LampJustinnot really it has to meet it's masters targets as such not independent

[@ericwadebrown]

There was talk of open sourcing VMware's ESXi. However, it was considered a large effort to clean up the code to be presentable for the public and redo all the build, CI/CD, etc. Ultimately management teams saw no benefit in the act.

[@nyccontrabass3489]

I don’t think people are mad enough. We all should be pissed.

[@glmchn]

I am. F* Broadcomm, f* them so much

[@magicker8052]

I love the basic interface on XO and the windows client for that matter

[@einardivision]

Great video, btw I like the beard! what products do you use?

[@2GuysTek]

All natural, baby!

[@acmaysnetworker]

Question, between all the of open source vm's xcp-ng is the lead candidate to replace vmware at most companies, there will be an immediate need for more customer business & technical communication, NOT troubleshooting, asking and responding to business and technical wants and needs. Their has to be a training program for Engineers and Admins. although it is very similar to vmware,, business want and need assurance for the competent technical staff. I see nothing about technical certs or proof of training???? Vmware just handed over MOST of it customers mid size and small customers. maybe even some of the large/mid size customers as the pricing is insane unless you are a datacenter.

[@jhippl]

i love the idea of xcp-ng and have gotten quotes for it but the only thing stopping me from migrating to it is the 2tb vdisk limit. All my NVR's have large vdisks and I cant do iscsi direct mapping because they are built and running so I cant move them over from vmware. Once they solve that issue I can move till then I am going to have to go to Hyper-V.

[@thecrazymouse7220]

I am concerned about the 2TB. I have storage 3+TB. Promox shows up to 16TB

[@arigornstrider]

You're gonna have downtime migrating from your existing VMWare systems to any other solution; plan in time to migrate these large volumes to NFS (preferred) or iSCSI storage.

[@RogerioPereiradaSilva77]

While I can agree to some extent that not having good CLI skills may not exactly be the most important factor when evaluating a candidate for a job in IT in whatever capacity, experience suggests that such candidates usually are the worst performers in every IT team I have worked at because they just cannot grasp simple concepts such as automation and they just panic and don't know what to do when dropped onto a command line (which happens more often than not whether you care to admit it or not) so I at the very least expect them to display willingness to learn. If they don't, that's fine. No harm taken but I am content to part ways at that point if I am the one calling the shots. Make of that what you will.

[@bigjaydogg3384]

As one of the guys that called him out on this (though not in the way he stated) I 100% agree. My point in the original video was not that “everyone in IT should know the command line” but instead “if Proxmox is the platform you’re running, you should put forth a minimum effort to grasp basic troubleshooting concepts.” I’d say the same if it was a domain admin complaining about having to use ADSI edit, or a MDM admin figuring out enrollment processes. As professionals we have to have some level of competency of our chosen tool. If pretty colors makes or breaks a hypervisor for you, that’s fine, choose something else. But you can’t complain about not having troubleshooting tools when you have a terminal blinking in front of you. It’s preposterous.

[@joezaino4674]

Nice deep conversation. Great job!

[@kevinrineer5356]

What was that podcast Tom was listening to about navigating Bloodhound? Id like to add that to my listen list

[@LAWRENCESYSTEMS]

The Risky Business Podcast.

[@bitferret-rx5rn]

So is there a 1:1 comparable application similar to VMware

[@n3m3f3]

Why am I finding info related to xcp-ng using older code or versions ??

[@youtux2]

Thanks for a very interesting and informative video!

[@ipstacks11]

I would l like to see a comparison of these platforms at scale. Each configured how best meets their needs, maybe VMWare on a Dell Rail stack, just fully configured. The others the same way. Proxmox on whatever hardware makes sense which with Debian probably means there are tons of choices. Then compare them even if they were all on the same hardware. We use VMWare where I work and I am no VMWare expert but my experience with it is it seems pretty clunky and I have always wondered if proxmox could fill the demand. I know the support isn’t the same but just like kubernetes, if it works even though you have to build up a lot of “environment” around it, people still use it. The VMWare guys I know say proxmox can’t do the job but don’t explain why. I get the corporate problems of being the one taking the risk isn’t good. I guess I will wait and see. If there really is a vacuum here then whatever the real contender is will get forked, change, get financing, bought or whatever to meet the need. It makes me wonder if proxmox is being run at scale in a large install, I just don’t know.

[@napalmsteak]

Where is this Zen Orchestra Fat Client you speak of?

[@LAWRENCESYSTEMS]

There is XCP-ng Center

[@magicker8052]

Still use it now and then.. it's quite good

[@topdigitalgaming]

Think with the VMWARE's ESXI server issues that the other free Type 1 Hypervisors out there is going to develop more and try to be the next number 1 free Hypervisor out there. Welcome to open source free Type 1 Hypervisor war.

[@autohmae]

3111 are these projects really open source projects if their is no developer community ?

[@carlostavaresjr958]

VMware closing off the free software is a lost a lot of people to learn the software to recommend it. Vmware wrote their own death. I deployed and supported VMware forever in both single ESXI hosts and clusters. Now I can say I can do the same thing with XCP-NG and KVM. In my opinion, Docker and LXC belong in VM's. It's safer and keeps troubleshooting at the guest os. Keep the hypervisors doing what they do which is management of resources and migrations if something goes wrong.

[@DS-ou7xm]

Proxmox all the way

[@subsequent]

Great video, thanks 🎉

[@LampJustin]

As to Kubernetes: It's kinda obvious that both of you don't really work with it much. It's a very important field to support though. Especially in the world of multi cloud. Saying that having no Kubernetes integration could ne seen as a net positive is just not understanding what it is you're talking about. Having Kubernetes integration does not mean being able to control the cluster from within the GUI, but the other way around. What i mean specifically is the csi driver for example. To provide storage into Kubernetes you'll need a csi driver. Having a driver for proxmox and or xcpng would be great. It's a big miss right now. At least the proxmox community seems to be creating that right now

[@bigpod]

what about openstack

[@andrewkamoha4666]

[Title] *Open Source Hypervisors* Why not virt-manager with minimal headless linux install??

[@kristopherleslie8343]

Wish they would just merge and form the company we need

[@calculusentropy]

Broadcom the high tech metals recycler.

[@napalmsteak]

Unraid does have native Docker support in the OS, installing containers in just a few clicks in the UI.

[@falazarte]

Unless you have a tech KZbin channel, don't expect the same treatment or support as Tom does from XCPng.

[@LAWRENCESYSTEMS]

That is simply not true.

[@falazarte]

@LAWRENCESYSTEMS how can you know? Have you been on the other side of the equation? Your channel is the biggest advertising for XCPng. If they don't treat you well, then I cannot even imagine what my experience would have been like.

[@LAWRENCESYSTEMS]

@@falazarte First, I have never needed their support, second we have been doing consulting for years for many companies using XCP-ng and many of our clients have support contracts with Vates and they consistently say how great they are.

[@brett_rose]

I've been doing IT for a couple of decades and I've never touched Linux. I write code, C#, Python, and others. I don't want to be required to use a command line for my VM stack management. I configure my firewalls via GUI if possible. Why would I not if all the same options are there? We do exist out there in the IT world. Thanks for the video.

[@tylercoan]

Small virtualization environment and I'm in the same boat. I can say for sure I've ruled out VMware, but I'm leaning XCP-NG. I've looked at Scale and Nutanix and I'm not a fan of locked in hardware.

[@falazarte]

Give proxmox a try

[@kristopherleslie8343]

You can coat the motherboard to be water proof 😂

[@VanceMorris]

KVM? Hello?

[@shammyh]

It's only far and away the most popular hypervisor on Earth...

[@alphabanks]

You've tested all these solutions but you totally skipped over HyperV most organizations with an Enterprise agreement will most likely move to HyperV.

[@2GuysTek]

I am working on a Hyper-v video as we speak. The focus of this discussion was around open source hypervisors, and that excludes Hyper-V

[@LAWRENCESYSTEMS]

I don't have a lot of confidence Microsoft won't kill Hyper-V as a move to force people into using Azure more.

[@JasonsLabVideos]

ME FIRST!!!

[@TheDropForged]

Me second