Parallelizing R code with the furrr package: Accelerating a 16 hour analysis (CC057)

[@SoraidaGarcia-e9x]

I wanted to say how much I appreciate your videos. I finally can understand some of the functions because you have great detail on how to use them with actual examples. I also work on a mac and have urban ecology data. You are so savvy and I would like to code like you one day. You think of solutions right on the spot. I don't have this type of help while I am working on my PhD so thanks for making me a better coder.

[@Riffomonas]

Remember that you only see me coding for ~20 minutes. After I've already put a couple of hours into each video and done some editing. Also, don't compare your chapter 1 to my chapter 20 - keep practicing!

[@DYT32]

I've been using plyr and DoMC for parallel processing in RStudio. In the past, I experienced RAM issues while using nest, mutate, and map functions. Since the plyr package got retired, I've been wondering whether I should try purr and furrr again. However, so far, the plyr workflow has been working perfectly fine for me. I believe the main advantage of the furrr approach lies in its flexibility for parallel processing on the fly.

[@olgaalieva3653]

A very nice explanation, thank you!

[@Riffomonas]

Thanks Olga! 🤓

[@hansmeiser6078]

How can I register an external class-object for furrr::future_map_dfr(), which is used inside the triggred routine?

[@Riffomonas]

Thanks for watching! You should be able to run it like the non-parallel map functions. I usually use it within a mutate statement in a pipeline and feed it a column from the data frame

[@roneyfraga]

Thanks, nice material!

[@Riffomonas]

Hi Roney- glad you enjoyed it - thanks for watching!

[@delt19]

Wish I had 16 cores in my local machine!