Why I Stopped Self-Hosting

Рет қаралды 23,211

NeetCodeIO

Күн бұрын

Пікірлер: 85

@jaketus 8 ай бұрын

Haven't ever heard somebody call VM from public provider as self-hosting, since you literally aren't self-hosting.

@dinoscheidt 8 ай бұрын

He’s talking about VPS not VM. I.e. a Virtual Private Server as a fractional part of the CPU, RAM, Drive etc which you need to self set-up to get to the point to host anything. A Virtual Machine on the other hand is essentially “just” software. It can be just an operating system (so you still need to set-up and manage things to be able to host in production) or open the door to incrementally handing off more hosting responsibilities i.e. via containers. It’s a general effort gradient compared to bare metal cost and self management and not is or isn’t. Which is also the reason why this video, without more details on what was actually done, isn’t so useful. I.e. one would nowadays rarely self-host a website. But processing video files can easily make sense for dedicated hardware just doing that.

@jaketus 8 ай бұрын

@@dinoscheidt VPS and VM doesn't make (almost) any difference from operating point of view, if it's hosted by someone else. And certainly makes absolutely no difference for the point of this video. What I'm questioning, is calling a server, VM or otherwise, hosted by someone else, as "self-hosted". As it literally is not. It's someone else buying the hardware and handling the upkeep. Which is important difference compared to actually self-hosted, where at minimum, you own the hardware, and preferably the physical environment the hardware is ran in.

@regnam503 8 ай бұрын

@@jaketusIn the literal sense of the word, you’re right, but here „self-hosted” usually means rolling out using a VPS as opposed to individual cloud services.

@jaketus 8 ай бұрын

@@regnam503 I'd argue that no, it doesn't usually mean VPS. And especially, when we're talking in the context of it-companies the size of Netflix, self-hosted obviously means actual Netflix owned hardware. And contradictory to this video, Netflix isn't 100% cloud. They have their own hardware points-of-presence in ISP data centers as cache, so that they don't have to pay the huge data transfer costs of AWS. And that's actual self-hosting.

@Phaceial 8 ай бұрын

@@regnam503it literally doesn’t. VPS is not self hosted.

@Bryan-eg7si 8 ай бұрын

This sounds more like managed vs unmanaged services

@mosti72 8 ай бұрын

I personally think the title should be "Why I stopped self-managing my cloud hosting" since in both of your case it is hosted on the cloud, just managed differently. The points you made I mostly agree with though.

@mike2ykme 8 ай бұрын

I think an important point is that Netflix isn’t using AWS for a majority of their video content being transferred to users. From older posts they were putting things closer to the user by working with ISPs directly to help out the congestion issues.

@kelsey_roy 8 ай бұрын

That’s what edge computing does

@MykhayloS 8 ай бұрын

That is called CDN

@bkucenski 7 ай бұрын

They may be using S3 as the master storage and then they have servers machines at ISPs all over the place that probably get their files from that central S3 data store. Linus Tech Tips has their own multi-petabyte storage but LTT only has 2PB and Netflix is using 8PB. S3 isn't that absurdly expensive and using it includes all the labor and hardware costs ensuring your data isn't lost or offline. I think LTT required about a million worth of hardware for their 2PB machine and Netflix is getting 8PB reliably for 1 million per year. It's compute that costs a fortune in the cloud.

@HumanoidTyphoon91 8 ай бұрын

My understanding of the scaling of DHH's company is that it's a B2B company where signing large contracts take time and it's easy to plan for more compute power for new large clients, and small clients joining doesn't make much of a difference from the server's point of view.

@enzocrespillo4899 8 ай бұрын

It sounds like you probably set up the server with quite a bit of technical debt, which is understandable given the short time it took you to set it up initially (those 8 hours). You could have taken the opportunity to learn how to properly set up the server so that subsequent changes would be less significant. Anyway, what I'm getting at is that this doesn't seem to be a problem with self-hosting per se, but rather with inexperience.

@NeetCodeIO 8 ай бұрын

Frequent changes and redeployments are time consuming, and a maintenance burden regardless of how long they take in my opinion. Why would anyone ever want to manually do something that can be handled for them? Why would I ever wanna self-host my database? Personally, I would never choose to do that. If a problem can be solved by throwing a reasonable amount of money add it, that's always preferred than using dev hours for it imo.

@Socsob 8 ай бұрын

Yeah I agree. If you want to trade money instead of learning how to do something yourself, that is totally fine and normal though. Example being as a game dev, you could learn how to produce your own art or outsource that part and focus/learn on the parts you are already skilled at. I think it is a matter of prioritization and deciding what you think actually benefits you in the long term.

@astral6749 8 ай бұрын

@@NeetCodeIO I get your point. Not everyone has the time to spend on things they'd rather have someone else do. However, I think your reasoning also discredits those who are into these. Why go through the trouble of self-hosting? Because it's fun to tackle a challenge, and not everyone has the money to throw at the problem. If we let ourselves slide down the slippery slope, we end up with: Why even do all these in the first place? Why not be a couch potato? It's certainly easier.

@diamondkingdiamond6289 8 ай бұрын

@@NeetCodeIOif you had a data intensive application, self hosting your database might be the only solution otherwise it would be too expensive.

@comradepeter87 8 ай бұрын

Most polite way to say "skill issues"

@kishankokal 8 ай бұрын

why not use docker containerised deployments with github actions???

@soltomoon3620 7 ай бұрын

still doesn't solve his issue of the time consumption of management. i think the main realization he is having is a business mindset vs a software engineer.

@lolikpof 8 ай бұрын

Just use docker for development, and deploy in docker to production. No hassle, no problems

@multivitamin7 8 ай бұрын

Idk about no problems. Docker for local dev can be kinda slow when ur waiting for things to rebuild and such. That’s been my experience anyways

@lolikpof 8 ай бұрын

@@multivitamin7 why would you be rebuilding? you only need to rebuild when changing ur Dockerfile/compose and pretty much never else. Look into devcontainers for greater convenience developing in docker

@kevgoeswoof 8 ай бұрын

@@multivitamin7 You can have a development image that, for example, in JavaScript that you run with hot reloading (e.g. npm run dev instead of npm run build within the container). Then the cost of using Docker becomes just the initial container startup, which is often under a second.

@multivitamin7 8 ай бұрын

@@kevgoeswoof I've seen this work pretty well, but it depends some on what your stack is like.

@theshashank11 8 ай бұрын

how is this self hosting? he is hosting in cloud, not on premise.

@blackplaydoh3522 8 ай бұрын

?? Not mutually exclusive. You host a service yourSELF vs using a managed solution.

@astral6749 8 ай бұрын

Are you really self-hosting if you're hosting it on the cloud tho?

@alexanderfournier7292 8 ай бұрын

I'm old enough to remember working at companies that didn't use the cloud. Holy fuck was it annoying waiting to get a literal box installed in your datacenter somewhere in the EU. That alone is worth the extra money.

@potaetoupotautoe7939 6 ай бұрын

you could do a hybrid where you develop with flexibility in mind. have cloud ready for peak, the usual bandwidth is handled with self-hosting

8 ай бұрын

Clickbait title, I don't even think you can call this self-hosting since you don't even host the things, they are hosted on cloud although you manage them.

@runofthemillnpc 8 ай бұрын

Not everything is about unhealthy hyper-growth. I don't think DHH here was trying to compare themselves to Slack/MS Teams levels of scale + complexity with this move or telling that they can do a better job than those folks by not doing cloud.

@tanchwa3740 7 ай бұрын

"I needed to install those python dépendances, that took a couple hours" Bruh, you're using someone else's tool and you aren't using their docker image?

@dejangegic 4 ай бұрын

fair point

@nadtz 8 ай бұрын

This is more like managed vs unmanaged, both are 'in the cloud', one just has all the knobs and tunables done for you. I'm a sysadmin not a dev so I feel exactly the opposite because when stuff breaks on the backend I'd be one of the people troubleshooting and fixing it anyway but I can understand your perspective as well. I've worked with companies that have moved their infrastructure to the cloud and others who after reaching a certain size have either moved stuff on prem or lease colo space and it's all situational. In the end if you can get your work done faster and easier no harm no foul.

@MrSurfsAlot 8 ай бұрын

Well if your website isn't making any money and you don't have any users it doesn't make sense to pay money for a cloud provider. I was just wasting like $70 a month on azure bills for the cheapest tier DB and Windows server .. i'd like to self host but would need to spend $100 to my internet provider for a static IP and haven't had the time to look into work arounds

@kevinsantana876 8 ай бұрын

Interesting video! Like many things in engineering the answer of whether or not to self host is dependent on what the end goal is and which trade-offs your willing to make. I think DHHs post and your video are great example of what this thought process looks like in action vs using a technology/stack/approach/whatever because the cool kids (netflix, dhh, google, amazon, etc...) are using/doing it.

@JohnDoe-vb3ks 7 ай бұрын

I think you make several assumptions, that are not always true: 1. Linear scale for infrastructure maintenance time cost, it's not always the case. I'll argue that in some cases, it'll plateau and can be not that big. 2. That engineers are productive every waking hour. Well, I want to believe that, but practice shows otherwise, so I won't necessary count maintenance time as a loss. But as always in engineering - it depends, and should be evaluated on a case by case basis.

@adityamwagh 8 ай бұрын

I used this video as a monologue in the background. Helped me focus. 😊

@wisdomlounge4452 8 ай бұрын

Whaaat! You used this guy's rambling techno babble soapbox rant for background sounds? - Nice!👍 I'm going to play it at my job site, all his yapping played incessantly on a repeating loop, mixed with soothing classical piano music, like Chopin's Nocturnes. The people at my job site may not have one clue what he's pontificating about but they're gonna love it anyways, lol! We're in this friendly competition as to who can provide the most "interesting" (if not annoying) background music -Lol!😁👍

@RemotHuman 8 ай бұрын

Maybe there is a way to self host with less time commitment? Maybe something like dokku/caprover/coolify would help?

@bmdavis419 8 ай бұрын

Such a great video, pretty much my thoughts as well

@multivitamin7 8 ай бұрын

Ben Davis spotted

@regnam503 8 ай бұрын

David seems to care a lot about being independent and self-owned, growth is not necessarily required for that. Once you embrace this mindset, his behavior actually makes sense.

@anantmishra6783 Ай бұрын

I love your videos, but for this one, I want to give you an alternate approach, how about dockerizing the application? could that have solved your problem while staying self hosted?

@olanrewajuadam7499 4 ай бұрын

How do you do all the writing on the blackboard sort of thing, which software is that

@jacobitosuperstar 8 ай бұрын

What the fuck that this take man... there are plenty of tools to do continous delivery in your VM (i don't know why this guy thinks that a VM in the cloud is not part of the cloud????) that you just didn't use. And the guy of the article said him self many times, that the cloud is good to rent, but not to base your business around it, because it will drain you from your profits (also, I am seeing that you don't know who that guy is or what app he has, that allowed him to do this change, but hey, who am I to tell you anything???). And finally, Netflix is not "full in AWS", like the guy in the article that you skipped through, they use AWS for services that they don't want to replicate.

@mrkostya008 8 ай бұрын

okay the analogy with ppl building their pcs is a weird one. if you want to get yourself a pc you usually build one yourself as prebuilt options are just meh, unless you have the buck to get a custom one.

@gowtham4943 8 ай бұрын

i wonder how much do you pay for your firebase? is billing okay compared to deploying own backend?

@HoloPanio 8 ай бұрын

I swear, you are the Dr. K of Software Engineering, and I am here for it!

@MM-24 Ай бұрын

Ehat about from a privacy/ security perspective?

@duckgoesquaack 4 ай бұрын

I gotta say the title is clickbait, a VPS is not self hosted, running it on your own hardware is. Unmanaged vs managed is probably the better term to use in the argument here. I agree with certain things like the value of time and getting the product out and such but man, 40+ hours just to add packages to Python on docker is... quite the skill issue. The assumption that self managing a backend being a matter of pride could apply to some people but generalizing it is quite asinine. Besides that, how do you think SRE's, DevOps engineers, and the likes learn the skillset they need to run the stuff you wrote at Google at scale effectively? I suppose in your case it would be SRE's because Google.

@almuaz 6 ай бұрын

VPS ain't self-hosting.

@ordinarygg 8 ай бұрын

I think you misinterpreted term VPS and self-hosting. I have own home-lab with self-hosted software. Probably you are talking about VPS -> cloud. It will take some time till you came back when you learn that all those companies do the same but wraps everything in UI.

@raishelmy7176 7 ай бұрын

only jr dev face this issue. i never spend more than hours setting up my self hosted just for 1 item. More like sales person trying to make small thing big.

@31redorange08 8 ай бұрын

You have to realize that US engineers are extremely overpaid. An hour of your time would be much less worth outside the US.

@wisdomlounge4452 8 ай бұрын

The cost of living is also usually less elsewhere, so the difference is often a wash.

@31redorange08 8 ай бұрын

@@wisdomlounge4452 It is, but not to that extent.

@hobbit125 7 ай бұрын

Skill issues all around. And you weren't even self-hosting to begin with.

@AndreiTelteu Ай бұрын

Your comparison with people who build their own PCs is wrong because a lot of system integrators that sell prebuilt PCs cheap out on power supplies or SSD quality or standard components. This components don't matter at first but will matter very much when you want to upgrade anything inside your pre-built PC. Also if something goes bad you have a higher chance of debugging the problem yourself because building your PC makes you gain knowledge just by researching the components ! I can bring the same argument to self hosting. If something goes wrong with your Judge 0 integration and the company is busy with bigger clients than you, your startup is entirely screwd because you said it's critical. Also what happens when Judge 0 builds a new products and does not offer support for your version ? You are entirely dependent on them. And with every little feature that you decide to outsource and pay for a hosted version instead of self-hosting, you add a lot of external dependencies just for a few hours of extra work you had to do to self-host. When any of your outsourced features have an outage (they do happen every once in a while) you have an outage as well. Does this worth a few extra hours of work ? Everyone should decide that for themselves. It sould be in reverse: The external modules that are very critical to your business, you should know very well and self host if possible. If you need to change this in the future it will have a big impact on your operation and customers. What is not critical and very easy to replace (support ticket system, live chat suport, payment processor, invoicing software, license management) you should absolutely pay for a hosted solution. If you need to change this, it would not make that much of an inpact on your operation and customers. More arguments here: dev.to/andreitelteu/should-you-self-host-open-source-services-or-pay-for-hosted-versions--4ga3

@xrehpicx 8 ай бұрын

its pride and also u get to learn wt ur code is actually running on more than just at the surface level

@botobeni 7 ай бұрын

I thought self hosting was on your own pc, not a rented one

@markzuckerbread1865 8 ай бұрын

I have to self host jenkins at work and god i hate it so much, i wish we just used a normal managed CI instead

@deeprajofficial1729 8 ай бұрын

Why is there to hate self hosting Jenkins?

@markzuckerbread1865 8 ай бұрын

@@deeprajofficial1729 to quote their own docs, it's not cloud native, it uses the file system for storing all it's data, a pain in the ass to backup, furthermore, they have this convoluted plug in system that turns a simple update into dependency hell.

@earthling_parth 8 ай бұрын

I love this channel but at just 1:08 mark, the entirety of this video loses the point for me. We other's have mentioned you stopped self managing your 'cloud 'resources.

@xrehpicx 8 ай бұрын

phonepe selfhosts most things on a huge insane dc, i work here smh managing is cheaper at the cost of my mental