Using the {arrow} and {duckdb} packages to wrangle medical datasets that are Larger than RAM

[@tomfenn4]

Really useful presentation, and timely for me. Personally I find data.table statements are greatly improved with just a little whitespace.

[@prashanthb6521]

Thanks a lot sir for this introductory video about Arrow.

[@musicspinner]

Masterful deployment of the "Kobayashi Maru" reference. 🖖

[@carvalhoribeiro]

Very good presentation. Thanks for sharing this

[@tdawry]

A neat question to answer. I'm using the duckplyr library and it's nice to not have to think about anything. It does make a strong argument for having a fast hard drive (an SSD is an order of magnitude faster than a traditional HDD, an M2 is an order of magnitude faster than that, and modern nvme drives are even faster).

[@winspyre]

Wow. perfectly narrated.

[@higgi13425]

For further learning, here are the links from the next to last slide: Arrow cheatsheet: raw.githubusercontent.com/rstudio/cheatsheets/master/arrow.pdf video intro: kzbin.info/www/bejne/hWWVfYijf7-DrpI full workshop from useR!: arrow-user2022.netlify.app DuckDB website: duckdb.org R package: cran.r-project.org/web/packages/duckdb/index.html data.table website: rdatatable.gitlab.io/data.table dtplyr (a data.table translator): dtplyr.tidyverse.org

[@tmuffly1]

This talk blew my mind. Thank you very much!

[@matthewson8917]

Perfectly summarizes my big data journey. Really good!

[@VictorOrdu]

Wow, thank you for this illuminating presentation.

[@gueyenono]

Great presentation.

[@torbjornstorli2880]

Loved your presentation. Well done Sir!😊

[@JohnoScott]

Great talk. Concise and to the point.

[@porlando12]

Excellent presentation!

[@multitaskprueba1]

You are a genius! Fantastic video! Thanks!

[@ZachRenwickData]

great video and interesting analysis use case!

[@arunabhbarua1924]

How about just using duckdb and SQL?

[@higgi13425]

Certainly reasonable for the SQL-fluent. The {duckplyr} package (which came out after this talk) has made it easier for the dplyr-fluent to use DuckDB.