RStats tip #3 use pipe connections

[@maqdala]

You said see you guys next time, but it's been a while ever since. Could you please come with another gem video tutorial again?

[@oldachm]

Great screencast - 17min of pure knowledge here!!! Okay so "7z" > "pigz" for compression but is that the "best" on the market in terms of I/O? I read a blog a few years back (2017) from Karl Browman talking about sqlite, feather, and fsd; are these only needed if working with distributed storage systems (e.g. SQLite and MongoDB)? I seem to remember benchmarking at one time and realizing that feather only writes data frames and can't store nested data frames so I was resorting to .Rds 🤷 Thank you again, and please respond here or DM @MattOldach if you can share any insights

[@JimHester4]

The fastest widely used compression format for both reading and writing I am aware of is [lz4](github.com/lz4/lz4), though its compression ratio is lower than most other formats in the default settings. [Zstandard](facebook.github.io/zstd/) provides a good tradeoff of compression speed and ratio. Binary formats like you mention are a completely different topic, and in general I prefer keeping my data in text based formats that are easier to work with using standard command line tools. However if you do want to use binary formats for R the two packages I would look into are [qs](cran.r-project.org/package=qs) and [fst](cran.r-project.org/package=fst)

[@datasciencezj3303]

this only works on *nix system and not out of the box right?

[@JimHester4]

No, it works anywhere R does.

[@ravikumar-vr6zm]

or taklu kya haal h