Cloud vs Local GPU Hosting (what to use and when?)

[@mangaart3366]

KZbin is a mine and sentdex's videos is gold!

[@dpchppr]

are*

[@mangaart3366]

@@dpchppr I thought about this a lot and I wasn't sure what's the right answer so I ended up leaving it like this.

[@mattizzle81]

I play around as a hobby and I went all in and built an expensive PC with a RTX-2080 ti, as well as a high end gaming laptop with a GTX1060. I barely use it enough to justify the cost of the GPU at all. Should have stuck with Paperspace. ;) Just the psychological factor of 'paying $2 per hour' on the cloud for something I don't own makes me nervous every time though, and is less convenient than having your own physical machine. Just the cost of a gaming GPU these days is a little painful for me. Like you say though, it's useful to have something local just to work on and go to cloud when you need. I just won't upgrade my local GPU in the future, if I need an upgrade, cloud it is.

[@JGrimm71]

Thank you for helping us with your knowledge. Please do not be dissuaded by the few who act strangely. Thanks again.

[@utkarsh1874]

Waiting for the data science beast workstation review :)

[@sentdex]

Going to wait til I've had it for a month. Should be end of Sept/Early Oct.

[@uh6537]

I am happy that Sentdex is sharing his brainpower in the KZbin cloud.

[@tophan5146]

Hi Edward Snowden

[@TheRaghavboyz]

I'm facing bottlenecks with my 2080ti running rnns and cnns using keras and tf-gpu(I've double checked that GPU is used during training), the only advice i get on internet is to increase my batch size, but i couldn't find better results in terms of cpu v/s gpu training speed. I would really appreciate you making a video on Deep Learning using local gpu.

[@bennguyen1313]

Since a bit of time has passed since this video.. does it still make sense to buy one of older GPUs mentioned (Titan RTX, 1080 TI , 970 TI), or would the cost/performance be worth starting with the newer RTX 3080, 3070, 3060 Ti and 3090 , AMD RX 6800 and 6800 XT? I'd like to practice locally before doing the cloud.. linode , aws, digitalocean,etc.

[@jdhshs12]

Great Topic thx!!

[@nklasio]

hey sentdex, thanks for your videos, i really enjoy them but I can't really seem to find a way of learning ai stuff for myself as I can't come up with a "real-world scenario". maybe you could do some videos on real world stuff? Like what would you use ai for on mobile apps or really anything that is somewhat "real life" Thanks for everything! You are great

[@kitagrawal3211]

Have you tried Kaggle? Lots of actual projects, datasets, and competitions available. "Real-world scenario" sounds very vague at least to me. What do you consider to be "not real-world"?

[@mattizzle81]

For me, the 'real world scenario' is physically tracking inventory in a warehouse using object detection. It is not as easy as it sounds, because 2D object detection alone is not good enough, everything needs to be tracked in 3D. I am learning a ton from just having a goal of creating such a thing. Find something that is very tedious and boring and needs to be automated, and simply can't be done using traditional programming techniques because it requires computer vision, etc.

[@minma02262]

I like your sharing. Do you think this advice still stands for 2021? Anything you would change your mind on?

[@theunboxingexperience3480]

Nice breakdown, thank you so much!

[@nemonomen3340]

KZbin saw that I was considering blowing 3 weeks worth of income on a nice pc setup to train ai as a hobby. Now it specifically put this in my recommendations feed to stop me. Excellent video. Now excuse me while I silently cry in a corner.

[@Doggieluv25]

Thank you - really helpful!!

[@prabhavkaula9697]

Quick question would you recommend getting MBP 14/16 M1 Pro for decent local computational resources and great battery + Cloud vs Chunky big boi windows machine with smol 3080/4080 etc.

[@milostean8615]

WHAT were you laughing at? 0:00

[@joseortiz_io]

Great insight Sentdex. Thank you for sharing. Anyone, what has your experience been working with hosted services? 🤔

[@rendermanpro]

GPUs that you buy, later you can sell it. With current market even have profit from selling old GPUs or sell at the same price like 2-3 years ago. The main issue with local GPUs on scale is power delivery and powerline at your place, but for most cases it suppose to be ok'ish. This is like subscribtion based licensing, you paid a little bit less but do not own anything, just giving away money.

[@TheKrazzyStudios]

Recently found Q Blocks: They are like the Airbnb of computing. Got a dedicated 2080Ti for less than 30 cents/hour.

[@stetsonb7915]

Excuse me if this has already been asked but, I am curious if you have ever talked with other KZbinrs (Linus Tech Tips comes to mind specifically) about creating a collaboration video or testing benchmark system for GPU testing with Neural Networks? I feel like there could be tremendous value for individuals or organizations to be able to compare their compute needs with many possible computer specs. Clearly, the more VRAM and Higher clock speeds will perform better. But finding the marginal cost to marginal output could be helpful and very interesting. While your video makes it clear cloud computing is super great and likely to be the best option for most individuals, for people like me who are really excited about Neural Networks & also enjoy playing games on PC/streaming/video editing, it could prove to be quite valuable in terms of choosing our own upgrade path.

[@ablanchi]

Just messing around with the first couple Fast Ai projects, it took me about 25 minutes to run a res-net 50 (~15 minutes for the res-net 34) with their cat/dog(~7,300 images) data set on a old 1050Ti 4G. Not very fun to double the course length just from training the smallest dataset xD

[@CaleMcCollough]

I ended up with one 3090 24gb and two 3060 12gb cards. I'm going to upgrade to a dual 3090 with water cooling. If I had the money I would be an H100 or A6000 and I would upgrade my 3060 to a 4060 TI 16GB. You're way better off making a monolith webserver with a 128-core Threadripper or Sapphire Rapids dual CPU with 2TB RAM and six A6000 or H100 with water-cooling than paying for cloud resources. You can spend $millions on something like that. Do you really want to fund the free tier? No. You want to save $millions on your server bill and pocket the money. For most people, this scale doesn't make sense. They would never be able to use that much computer power, but I can chew through it.

[@hekun830930]

I learnt a lot from it, thank you very much for sharing! ^_^

[@ialimijororakotoniaina3274]

I have to be honest with myself like you said, I just wanted to game hahaha

[@jimmytorres2658]

same here... do you think going for a RTX 3070 would be the right choice for a new ML student ?

[@ialimijororakotoniaina3274]

@@jimmytorres2658 yes it’s one of the best choices depending on your budget, just make sure you have a NVIDIA GPU

[@charitarthchugh]

@@jimmytorres2658 I think that the upcoming 3060 non ti is going to help you more because of the extra vram( if you can get your hands on one).

[@jimmytorres2658]

@@charitarthchugh awesome! Thanks a lot for your response :)

[@ericsonnen5248]

Awesome advice!

[@varunjain8981]

With AWS the moment I start the notebook, it start to charge on hourly basis. Even though we are only writing the code and not using GPU to its capacity. In my case, I take probably 50% to 60% of time to write code. If I am on could I will paying for this time as well. Whereas if I do most of the work on local then I don't pay for it. Do you think it makes sense to do development locally and perform final training on cloud?

[@miguelescalantemilke7204]

That’s just what I plan to do when trying AWS resources soon. I like to train a very simple model before running my models on Google cloud.

[@rendermanpro]

Cheaper in the cloud, it's like with some software and services, at the beginning they offers you much lower cost with no profit at all, and after you used to use it, hooked, they will up price.

[@mattmryczko3608]

Great Video!

[@schrodingerscat3912]

dude this sentdex coffee mug game is sick

[@karangadgil9847]

Hello, I have a laptop with pretty good specs ( i7 , 16 gb ram, 4.2 ghz, 2 gb nvidia mx 350 - entry level 256ssd , 1 tb hdd) . i think the only drawback is less graphics and the fact that its a lower end MX series. Would you say it'd be easier to fill that gap easily with cloud gpu? How practical is it to use cloud gpu against local? I am particularly thinking from Data Science perspective.

[@charitarthchugh]

I would not expect to run much on an MX350. You are going to be severely hampered. In your case, stick to CPU only for local or go full cloud.

[@Ehabmxd]

Hi, I am very beginner I have an inquiry if I purchase a server to support my main computer Do I install programs that require a lot of effort on the server system, or do I install them on my main device In other words, can you take advantage of the power of the server processor on the programs on the primary computer?

[@WisePrediction]

Great insights! If I am not mistaken, I haven't seen you to mention Google's TPUs for training models. In some presentations, they even mention that it is free. (I don't know though to which degree it is free. :) ) Would be interesting to hear your thoughts on Google's TPUs...

[@danielgn6227]

Google Colab let's you run GPUs and TPUs for free. But I think there's a time limit of one hour.

[@esysss]

it's time to deep learning tutorial!!!!

[@botlifegamer7026]

Is there and Update to this video?? If not can you make one if you still rent out gpu power.

[@kanuckadisk]

At what point would you say then the cost of one of these GPUs would need to fall in comparison to the rates? Would a rate at 1,500 hours still leave you in the cloud camp? What about 1,000? Where would you put your rule of thumb?

[@sentdex]

Between 1 and 4k hours, depending on future prospects

[@rauhan_sheikh]

I've just gotten into AI and ML, I'm complete beginner so I was thinking of buying an expensive laptop for that and now I'm just going to buy m1 MacBook Air 😅, and when i need I'll just use cloud 😅.

[@sagicarmel5988]

Damnn... :D So helpful :)

[@jeff4877]

Ok, ok, I want to buy cool stuff...

[@omeraiman2983]

I am thinking about buying a used server and complete it with gpu!

[@vipulkaushal1597]

is that gun and a bong in the back?I hope you are still good!

[@danielgn6227]

It's Elon Musk's flamethrower

[@100timezcooler]

what if you also play video games?

[@prabhavkaula9697]

My king you speak 📠📠📠📠📠

[@cargouvu]

Is that a gun over your right shoulder? Never seen anything like it. Is it a paintball gun?

[@anthonyrodriguez5080]

cargouvu look up boring company not a flame thrower, it’s a flamethrower made by Elon musk

[@ironmantooltime]

Like most people don't need a 64 core threadripper, 8 core is nice, I don't need a £1000 card but for dev £150 is fine 🙂

[@j.k.ravshanovich]

Can I use cloud GPU to play games locally?

[@TynOng]

Can you also include the cost of data storage for next cloud Vs local ML comparison please? Thanks.

[@prakashyadav008]

can u do a video on which is better Paperspace, Amazon or google cloud? and in Paperspace does the cost means .. how much the machine is on ?

[@kitagrawal3211]

I have tried GCP and AWS both. They are both pretty good and you only pay for when you actually have the machines in use. I prefer AWS over GCP (but do use Google Collab pretty often for quick testing and model building). Paperspace seems similar although I am not sure about their architecture and data storage. S3, Redshift and GCP cloud storage are very powerful and extremely cheap and splitting the storage and computation cost is a very cost-effective solution. I love AWS for ML primarily because of their AutoML and Sagemaker features. The architecture is very component independent and customizable. I also like the fact that how easy it is to take anything to production once the development is reasonable by creating one/ multiple endpoints. I will like to know more about this others perspective on this topic too.

[@maximeellerbach4190]

Can't afford cloud gpu so I'm using my old gtx970, I will maybe consider using google Collab in the future

[@sentdex]

Are you sure? 8GB GPU on paperspace is $0.5/hr for example.

[@Redeyejedi808-u9g]

@@sentdex You can't play gta5 on cloud gpu tho :)

[@sentdex]

@@Redeyejedi808-u9g Wrong. Not only can you, I have. I even streamed some of the Python plays GTA V from paperspace.

[@Redeyejedi808-u9g]

Well... I shall go off and eat my hat sir... and maybe watch it

[@maximeellerbach4190]

It's cheap but still I don't gain any money with my deeplearning, I'm still a student and for me it's simpler to just run locally. Specially when I'm not at home and have a bad internet for example. But cloud GPU is definitely the right way to go and I agree with that !

[@ryanpunamiya]

sentdex, when are you making a computer rig update?

[@ayush0477]

wow zero dislikes.

[@sentdex]

The children were still in school. Give it time. The buffoons *always* show up. I learned to stop caring about downvotes.

[@hewhointheearthlydomainsee1272]

Amazon G5 seems the best performance per cost.

[@joxa6119]

I don't know, but you look glowed up.

[@gameplaying9099]

what about rtx 3090?

[@noisycarlos]

What? Members only??? Just kidding. It was pretty interesting.

[@harshprajapati9251]

Hi sentdex ✋

[@sentdex]

Hello

[@mikerhinos]

Cool video, I wonder why it's not members only though... '^^ With tools like Google Colab, for me the future is clearly Cloud GPU. I don't have the money to buy a powerful system dedicated to ML, I don't have the money to maintain it buying newest hardware regularly etc... With internet connections going faster and faster, in a few years we'll only have some sort of cheap Raspberry Pi laptops and we'll all do things on the cloud :) ML, gaming, etc...

[@saajjgames3433]

Member only! U subbing right now! Jk

[@777Syarif]

I wonder if this cloud GPU can run predicting autonomous model in CARLA...If so, please give us a tutorial :)

[@emmanueljoshua9972]

This is that I have in mind as well. I hope he can make a tutorial on running CARLA on a cloud GPU

[@MalaiischeIntelligez]

basically I don't understand but I sub on your channel so I can learn more....good VD anyway

[@ankk98]

My heart wants to buy the best gpu for gaming and ml but my mind is not allowing it 😭

[@arianvc8239]

Nice haircut

[@sentdex]

Ok

[@m.talibsameerkhan8369]

Hi Snowden

[@bralis2]

One should not aim for RTX immediately , one can always buy cheaper GPU for learning purposes - Nvidia Quadro T600 with 4GB GDDR6 memory let's say.

[@sentdex]

It's not really RTX or not. It's a function of enough memory, and then the speed of the card. Skip the 4GB GPU unless you have one already, 4GB is far too little memory to be working with. If you're just learning about DL, CPU/RAM can be enough.

[@YouTubeFunHandle]

You accidentally reveal your money stash by your t shirt lol

[@GNARGNARHEAD]

$2 an hour for $4000 of GPU... *a math teacher somewhere cries*

[@illusivegirl]

Not for members only? dislike unsub

[@KaranSingh-or7yy]

*_why re-upload the video man ???_* 😓😓

[@lolagepwned]

He explained in a community post and another video Basically a KZbin bug

[@KaranSingh-or7yy]

@@lolagepwned 👍

[@seenbefore1560]

Jesse Pinkman!

[@Swirtl3]

Haven't you uploaded this three times now?

[@sentdex]

Not to my knowledge

[@rant4la]

Firts

[@amarnamarpan]

There's just so much inconvenience of using cloud GPUs with a slow/unreliable internet connection.

[@MuslimibnAqeel]

"Hey man i will unsub you because of ONE unintentionally mistake. I am triggered and offended by YOU, even though you did tremendous work for us." --- Generation Ungrateful

[@MuslimibnAqeel]

I honestly started machine learning by watching your videos and the amount of just saying "thank you for your work" can not be counted by myself. I just wanted to tell you man that there are loyal subscriber and we love your work and we REALLY appreciate it. Cheers mate

[@shimshon9271]

can you stop with those cloud vs local gpu discussion??? i'm so pissed

[@eyeborg3148]

well then stop being pissed

[@TynOng]

Don't watch the video then

[@mattizzle81]

Pissed over what?