Rust as a Tool For Data Scientists?

Rust as a Tool For Data Scientists? | My Experience

Рет қаралды 15,131

Күн бұрын

Пікірлер: 27

@professornumbskull5555 2 жыл бұрын

You might have known these already, but following are a few tips, that might speed up your code: 1. use cargo clippy to lint your code, it can sometimes provide a way to improve your code performane 2. Compile your code with --release flag, this, removes many useless runtime checks, that rust does in the debug mode, to provide you with more concrete data

@EmilNicolaiePerhinschi 2 жыл бұрын

you might not know but most of the math packages in python are written in C or Fortran or C++ and optimized to hell and back, and you only see the Python wrapper, so when the data is small the python overhead shows better

@abstractqqq 2 жыл бұрын

that's why. Thanks

@abstractqqq 2 жыл бұрын

well, true. Definitely not saying Rust can replace NumPy or SciPy. Just want to point out that using rust in some edge computational cases / non-computation heavy lifting might be good ideas.

@EmilNicolaiePerhinschi 2 жыл бұрын

@@abstractqqq Rust can definitely replace python, only somebody needs to take the time to write the packages

@balen7555 2 жыл бұрын

@@EmilNicolaiePerhinschi yeh no. rust will never replace python. Leave that for something like Julia. People who program in Python do not want to worry about memory management, explicit error handling, memory usage, etc..., at the cost of less productivity and more terseness. It's not just about more man-power. Rust's philosophy is inherently not suited for the things python is currently "good" at

@techpriest4787 2 жыл бұрын

@@balen7555 you are confusing Rust with C/C++. There is no manual memory management/usage. That is the whole idea behind Rust. If you are referring to optimization then also no. You do not have to write optimized Rust code but still will be faster than Python.

@KenoticMuse 2 жыл бұрын

This is a good perspective. You're differentiating between what it takes to build and run a model, versus building the data pipeline to feed into the model (preprocessing) and process the outputs of the model (postprocessing).

@fraternitas5117 5 ай бұрын

I work at an enterprise company getting into model development, how can I contact you for a discussion?

@cipsikolakilit1220 2 жыл бұрын

That was great. Thank you.

@simdimdim 2 жыл бұрын

I wonder, is it youtube hiding/deleting my comments or are they being human moderated... I was just trying to say there are bindings for tensorflow and pytorch and there's a site from the 'are we game yet' series called arewelearningyet one can visit for other science related crates. And as a side note, one Can run rust in Jupyter with the help of evcxr (gotta say tho, that's one awfully hard to remember name.)

@abstractqqq 2 жыл бұрын

Yes! I've tried the Rust with Jupyter and it is cool, but to me using Jupyter is almost always for some quick analysis or ad hoc request where performance doesn't matter that much... Well, that is just a big part of day to day work (quite boring but true in my company).. I will look into the bindings thanks for sharing

@chucky29949 2 жыл бұрын

i had to sort the comments by 'newest' to see your comment. evcxr sounds interesting, as ultimately python will suffer from being designed as a more scripting/data science language vs general wide/fast language one like rust/c/etc.

@adamhenriksson6007 2 жыл бұрын

I would love to see native rust implementation of tools like pandas and xlxswriter.

@simdimdim 2 жыл бұрын

@@adamhenriksson6007 have you checked out 'polars'?

@adamhenriksson6007 2 жыл бұрын

@@simdimdim wow nice! Looks very promising. No xlxswriter built in but still tho... CSV will have to do

@DailyProg Жыл бұрын

Broooooo, this is so good

@belikegrey 2 жыл бұрын

Thanks a lot bro very nice video

@Cyrodiili 2 жыл бұрын

So what about Go in this case? Much faster than Python with easy syntax.

@abstractqqq 2 жыл бұрын

Sry I do not follow GO that much. But yea, if GO fits in the lifecycle of the product well, then GO for it! But if you're absolutely going for speed, then C and Rust should probably be the top choices. I am biased towards Rust because I think I am too young to learn C lol. Lots of server side benchmarks are done on other channels and you can find those on youtube. Just type "Rust vs GO".