Parallel Computing in R

[@danielribes423]

Thanks so much Matt for this amazing video. You changed my point of view when it comes to parallel computing. I thought it was only possible running some instances of R at the same time, but you showed it was not necessary to do so in order to reduce the running time of a model. The exercise I want to do now is to find out the running time to run 10000 iterations with as many instances as the number of cores of the model and compare it with the time required with a single instance with doParallel cluster.

[@MattBirch]

Try a few ways! And have fun with it! It is an awesome tool that I hadn't known before the last year or so!

[@LectorFurtivo]

Thanks for the video! It's interesting to see that three cores performed quite similarly to seven cores.

[@MattBirch]

My pleasure! And yeah, this code was simple enough that there was not much to be gained by going to more cores. I am working on a project at work that has hundreds of lines of code within each iteration of the foreach loop, including training multiple ML models and doing a bunch of calculations. That is complex enough that every core seems to count. I run it in a cluster with 62 cores. More might even be good, but that is the best I have access to. It all just depends.

[@Aim4sixmeals]

Could u do a video for beginners in R?

[@MattBirch]

I will over time! There are a few in this playlist that may be helpful to you: kzbin.info/aero/PLWd1brOYtkZXYV7MEus3mtp77fyuHDwFy

[@tarasst6887]

🎉🎉🎉

[@MattBirch]

Haha, glad you liked it!