Make Python code 1000x Faster with Numba

  Рет қаралды 446,290

Jack of Some

Jack of Some

Күн бұрын

Пікірлер: 684
@gdsvsrv
@gdsvsrv 3 жыл бұрын
finally, my print("hello world") can run at the speed of light!
@felicytatomaszewska
@felicytatomaszewska 3 жыл бұрын
That's not true
@MartinoxxHD
@MartinoxxHD 3 жыл бұрын
@@felicytatomaszewska no way
@dv_interval42
@dv_interval42 3 жыл бұрын
This meme is much closer to the truth than people would think.
@hanchen2355
@hanchen2355 3 жыл бұрын
You can’t because it’s IO
@aryanmn1569
@aryanmn1569 3 жыл бұрын
@@dv_interval42 how?
@klioseth4336
@klioseth4336 3 жыл бұрын
Seriously this is brilliant. By applying this to the 3 main functions of a 1500 lines project I went from a 24 minutes runtime to about 20 seconds. that's 70X faster for a day's work (it'd be faster now that I learned the more complicated details). The C++ equivalent of the code does it in about 15 seconds.
@MichaelKudlatheInterested
@MichaelKudlatheInterested 3 жыл бұрын
Amazing, amazing video. As a PhD student, I'm sure you've saved me much more than 8 hrs of compute.
@JackofSome
@JackofSome 3 жыл бұрын
I expect a co-author ship on all your papers now 👀
@sorry4all
@sorry4all 3 жыл бұрын
Lol
@senwang1982
@senwang1982 3 жыл бұрын
@@JackofSome fair enough😂
@alexp6013
@alexp6013 3 жыл бұрын
I've once had code run for 48h, so I fully believe that it could be true
@username42
@username42 3 жыл бұрын
@@JackofSome i think he will thank you in the acknowledgement part in his thesis for sure :)
@ger15100
@ger15100 3 жыл бұрын
Numba really is great. Here is my first experience with it: I have some code that I estimated would take up to half a year to run. With some minor changes to the code for compatibility with Numba, the run time for the same conditions was up to 1.5 hours. This was a game-changer for me as I needed to run this script many times. One of my colleagues also implemented it into his machine learning code and managed to obtain some significant speed bumps.
@tronali5703
@tronali5703 4 жыл бұрын
One thing to note is that Numba really optimizes nested loops. If you say have to call a function a 100 or million times, better to have an outerloop instead that iterates within the function. With parallel turned on, you can see a 1000x increase.
@JackofSome
@JackofSome 4 жыл бұрын
This actually makes a lot of sense. Thanks for the comment!
@chriskeo392
@chriskeo392 2 жыл бұрын
This is crazy! Awesome
@AlexanderHyll
@AlexanderHyll Жыл бұрын
Its one of the only ways to run the notoriously slow nested loops in Python (whenever it can’t be vectorized). We had a simulation to run that went from 81 days pure python to 2h (~1000x).
@valente.victor
@valente.victor 3 жыл бұрын
This is one of the best programming-related video tutorials I've ever seen. No-nonsense style, very direct and still computationally rigorous. Awesome stuff!
@irok1
@irok1 3 жыл бұрын
Finally, this is what Python has needed
@raycao88124
@raycao88124 3 жыл бұрын
Just use C🤣
@JackofSome
@JackofSome 3 жыл бұрын
cPython is already "just use C" so not quite
@tsunekakou1275
@tsunekakou1275 3 жыл бұрын
python doesn't need to be faster, it designed to be simple not efficient. there are plenty other programming languages that designed to be efficient like C, C++ that has proved themself. sure they are hard to learn but trying to get python faster is just like reinvented the wheel, learning C++ from scratch in my opinion is much more beneficial in long term than trying to make python faster because essentially you are learning the same things but you won't get far with python.
@tando6266
@tando6266 3 жыл бұрын
@@tsunekakou1275 Couldn't disagree more, for a lot of science we are paid to do things, not write c++ libraries to do things. As much as I like C++ it is woefully inadequate for a research setting with deadlines.
@tsunekakou1275
@tsunekakou1275 3 жыл бұрын
@@tando6266 which points?
@saurabhpareek3310
@saurabhpareek3310 3 жыл бұрын
I am in tears to see my code run so fast, thank you so much for this video
@joshuatyler4657
@joshuatyler4657 3 жыл бұрын
I implemented it in my code, and it took around an hour worth of computation and put it down to 30 seconds. Amazing stuff.
@JackofSome
@JackofSome 3 жыл бұрын
Nice nice!
@joshuatyler4657
@joshuatyler4657 3 жыл бұрын
@@JackofSome One funny thing that I found is that you can’t create numpy.linespace or numpy.arrange arrays with anything besides integers. I had to create an integer array, then divide it by 2 to make it a float32, and then populate it with a for loop to get the actual array I wanted. Also, numpy.zero arrays have to be initialized outside of the function and passed as an argument instead of being initialized in the array. But the extra work is worth the speed up.
@preyasurakshit3405
@preyasurakshit3405 3 жыл бұрын
I cannot thank you enough for this video. I made a simulation of Gray Scott's Reaction diffusion in python, but it was painstakingly slow. After using numba, it now runs quite fast. Earlier it took nearly about 1 hour to render just 100 frames at a mere 400x400 resolution, now it renders 100 frames in 45 seconds, in 1280x720 resolution. Thank you very much for this video.
@JackofSome
@JackofSome 3 жыл бұрын
That's amazing to hear! Glad it could be helpful. Would love to see the result.
@JackofSome
@JackofSome 3 жыл бұрын
If you're just trying to do convolutions then you might want to look into numba's own way for doing them. I forget what it's called but it's optimized specifically for the convolution operator
@kaandonbekci6414
@kaandonbekci6414 3 жыл бұрын
Similar to other commenters, I have to say this is one of the best KZbin tutorials on writing fast Python code. Great buildup to the mic drop moment, amazing progression. Jack, I hope you make it big in the CS-tutorials KZbin space and have a great day.
@JackofSome
@JackofSome 3 жыл бұрын
Thanks for the kind words. Videos have been recommended a lot lately so fingers crossed.
@jmnunezd1231
@jmnunezd1231 4 жыл бұрын
One of the most awesome Python tutorials I have seen. The narratives, the examples and the way you explain it is just awesome, 5 starts to you man! keep on doing it!
@journalperen
@journalperen 4 жыл бұрын
True
@mpilosov
@mpilosov 3 жыл бұрын
This was wonderfully clear and helpful! Thank you! My PhD involved running a lot of these forward problems under uncertainty, Lost much time trying to parallelize physics simulations and fighting with the GIL. Sometimes it is just worth waiting.
@muhammadanas1201
@muhammadanas1201 2 жыл бұрын
I had to submit a project in 2 days (that I had months to do but waited until the last minute) and in one piece of code, I needed to clean data of 6 million tweets. Regular python code showed runtime of 400+ hours (I used tqdm library for it). I got cold feet thinking I’m going to miss the deadline, but finally got the idea to search for ways to make the code faster. Found this library out on the internet and using it, I was able to run that piece of code in JUST 10 MINUTES. This library saved my project that day 👌.
@JackofSome
@JackofSome 2 жыл бұрын
Numba: enabling procrastinators since 2012 😂
@syoudipta
@syoudipta Жыл бұрын
After being skeptical about this library for a while, I've gotten used to it. I've watched many tutorials and read many articles and documentation. After weeks of determination to learn this technique properly,... I have become, comfortably Numba. 🤟
@gabrielcesar568
@gabrielcesar568 3 жыл бұрын
Grad students from around the world are thankfull for this video.
@sirosala
@sirosala 2 жыл бұрын
Excellent video, in one day I implemented 80% of the code and strategies you explain. Very good ! Greetings from Rosario - Argentina.
@frodobaggins3974
@frodobaggins3974 4 жыл бұрын
11:45 - try this: np.zeros_like(input_list, dtype = np.int64, order='C') My experience with numba - it works great, but types have to be very precisely defined for every single object. Otherwise nopython will fail, and where numba sucks is generating traceback messages - in the case of complex functions it will take forever to debug. So type whatever you can - it is not a 'Pythonic' way of thinking and writing, but in this particular case it is necessary. Anyway, great introduction!
@JackofSome
@JackofSome 4 жыл бұрын
Thanks for the tip, and best of luck on your journey to mount Doom. We're all counting on you.
@0.Andi.0
@0.Andi.0 2 жыл бұрын
Hi, I have a question, how can I declare like a type so numba can compile it? For example I have a vector3 class that cannot compile with numba (class with x,y,z components)
@mariomuysensual
@mariomuysensual 2 жыл бұрын
@@0.Andi.0 use numpy arrays instead
@MisterFresh022
@MisterFresh022 3 жыл бұрын
What an amazing video. I'm a student in Computer Science and we just did a week ago an introduction to Numba. And you sir not only explained to me the concepts in a positive astonishing way, but also gave me the desire and the excitement to apply it on all my old and future codes. Thanks you ! PS : Is there more videos where you're explaining the others topics ( convolution, neural networks .. )
@JackofSome
@JackofSome 3 жыл бұрын
Great. I hope you share it with your class :) Unfortunately no other videos on Numba
@id104335409
@id104335409 3 жыл бұрын
I nod my head with approving noises as if I understand what is going on.
@SomeFreakingCactus
@SomeFreakingCactus 3 жыл бұрын
These are some seriously impressive results. I wonder if this could give Python a niche in indie game development.
@jafarjuneidi7585
@jafarjuneidi7585 4 жыл бұрын
I already had a background in Numba but I thoroughly enjoyed this video and learned some useful details as well. I'd be glad to see some more python related videos from you
@MVHiltunen
@MVHiltunen 4 жыл бұрын
numba is really sometimes absolutely stunning in it's acceleration per extra work.
@JackofSome
@JackofSome 4 жыл бұрын
That it is friend.
@dadogh
@dadogh 3 жыл бұрын
That's an awesome video. Thank you! I've been running orekit scripts which took me ~4h, and with numba there are taking ridiculously 2min!
@JackofSome
@JackofSome 3 жыл бұрын
Oh wow those are amazing gains! Glad I could help
@shahnawazazam
@shahnawazazam 3 жыл бұрын
Just make sure your cpu cores arnt too angy at you
@trizzzzlez
@trizzzzlez 2 жыл бұрын
I came back to say a huge thank you. currently doing my Phd. my code was to run 300k iterations it took like 3days+. your short video changed everything, i took few hours to rewrite the code to suit numba as you explained. boom 2hrs i have results on the screen instead of 3days+
@JackofSome
@JackofSome 2 жыл бұрын
That sounds amazing. Please be very careful and do consistency checks between the two codes. Don't wanna find out there was an issue when you're writing your dissertation 😅
@trizzzzlez
@trizzzzlez 2 жыл бұрын
@@JackofSome yes I will jack thank you.
@Meodoc
@Meodoc 2 жыл бұрын
I really liked that you stayed real and just said that you aren't qualified to comment on why the python loop with the numpy array took so much longer to execute than just saying something with the risk of talking bullshit and spreading misinformation in order to sound competent, a behaviour I have seen way to often online and which bothers me quite much. Very great, clear tutorial, truly one of the best python tutorials I have seen so far on KZbin. Keep it up! :)
@JackofSome
@JackofSome 2 жыл бұрын
Nice. Yeah it's always better for me to just say "I don't know" when I don't know :)
@Meodoc
@Meodoc 2 жыл бұрын
@@JackofSome Love it 🙌
@mithgil-cool
@mithgil-cool 3 жыл бұрын
Thanks for your introduction! This really helps me. I have been working with Numpy but it is too time-consuming. Working with Numba is really fantastic, my Python module runs more than 1000x faster than the original one.
@Xaminn
@Xaminn 3 жыл бұрын
Recursive Fibonacci will truly get ya.
@drepi57
@drepi57 3 жыл бұрын
Jack!!! Congrats for this fast, clear and insightful video. This is a game changer for me (I'm a Fortran 77...20** programmer now starting with python). Live long and prosper \\//
@boyuanchen4997
@boyuanchen4997 2 жыл бұрын
Simple, effective, to-the-point! Encountering a topic modeling problem that requires a lot of steps of training, I was about to try cython which would have taken a huge amount of time. Numba turned out to be ENORMOUSLY SIMPLER and equally effective. Many thanks to numba developers and Jack!!
@graemeford1333
@graemeford1333 3 жыл бұрын
Ah yay. Thanks for this. I must say Numba is very impressive. What I am most interested in though is applying it to a research setting which you helped a lot with as well! I am just happy that it was designed for scientific applications as well!
@MrRyanroberson1
@MrRyanroberson1 3 жыл бұрын
this is absolutely perfect because i just found this while running a really oddly intensive random number process (something like one million terms per second? i need billions) and numba sped it up by a factor of 100 halfway through the video, and with njit it didn't actually speed up any further, it just got a more consistent time (randomly jit is 10% faster or slower depending i guess on the random values
@jackrdye
@jackrdye 4 жыл бұрын
Why didn't I find this sooner. I am still so glad I found it will 100% use this in the future. Literally had a situation identical to ur example a month ago needed to simulate over 160,000 situations. Was taking 11+ hours to run. I assumed there were better ways but I haven't had a lot of experience. Thanks for this very clear explanation
@nahiyanalamgir7056
@nahiyanalamgir7056 Жыл бұрын
Amazing tutorial and Numba is a great software. I hope Python officially will one day ship with a compiler because the community got so huge that people often write performance-intense code in Python that goes against its nature. Moreover, it'd be amazing to ship a Python product statically compiled without gigabytes of libraries!
@thestralspirit
@thestralspirit 3 жыл бұрын
Why wasn't this video in my KZbin recommendations a year ago? Nonetheless, this is an amazing tool for python. I wouldn't have found it without you!
@peterbonnema8913
@peterbonnema8913 3 жыл бұрын
Both Numba and this video are amazing. The video was really really clear and concise. I'm definitely gonna remember Numba for future projects.
@TimmyTam
@TimmyTam 3 жыл бұрын
I really thought the title was clickbait. It wasn't. This is honestly amazing, thanks for showing us this :D
@brockobama257
@brockobama257 3 жыл бұрын
There’s no way this delivers as promised. I’m about to implement it and will update soon.
@JackofSome
@JackofSome 3 жыл бұрын
You're on! (feel free to DM me on reddit/twitter if you're running into issues)
@szymonozimek
@szymonozimek 3 жыл бұрын
let us know
@profoundgenius9726
@profoundgenius9726 3 жыл бұрын
Let us know
@lucianomachado9271
@lucianomachado9271 3 жыл бұрын
Yup, would gladly want know
@linear9185
@linear9185 3 жыл бұрын
so how did it work?
@sasimitra5871
@sasimitra5871 3 жыл бұрын
I don't know why I need this. I'm still a High School student doing python for fun. But imma keep this tucked away in my mind. I have a feeling I'll need it soon enough.
@CrazyFanaticMan
@CrazyFanaticMan 3 жыл бұрын
When your'e working on projects that needs the program to do things fast, you want C like performance because low level languages like C/C++ are about as fast as you can get. Python is not a low level language. It's a high level dynamically typed language. This means it's pretty slow. So even though it's pretty fast to write a python program because it's basically the same as writing in English, you're giving up the actual execution speed (performance) in order to keep that fast typing speed (developer performance). Numba solves this problem by keeping the Python typing that we all love and getting the C like performance that is needed for projects where speed is very important.
@eskees9498
@eskees9498 3 жыл бұрын
@@CrazyFanaticMan so I'm also in high-school just doing python for fun. Is number useful for smaller projects or only for large projects. My biggest code is only about 80 lines of code, but it uses my Webcam to track hands. Would it be worth it to implement numba for this low amount of code?
@CrazyFanaticMan
@CrazyFanaticMan 3 жыл бұрын
@@eskees9498 It doesn't matter how many lines of code you have, whether it's 80 lines or a billion lines. What matters is the problem you are trying to solve. If for example you say "I want my webcam to not only accurately detect the location of my hands but I also want my webcam to track my hands if I move them really fast". I would imagine a webcam tracking hands that are moving really fast would require very fast performance speed in order to keep up. So in this case, numba may help you a lot. It all really depends on what you want and what your expectations are. The size of your program does not matter, numba can you help you whether you're program is big or small
@eskees9498
@eskees9498 3 жыл бұрын
@@CrazyFanaticMan thank you, I appreciate the info
@dmgeo
@dmgeo 3 жыл бұрын
Thank you for your video. I work with satellite image processing and we are used to spend hours waiting for a particular processing task to finish... I for sure will be trying to use Numba
@jakubkahoun8383
@jakubkahoun8383 2 жыл бұрын
This is the best Numba video, short , good examples etc.
@hunt9556
@hunt9556 3 жыл бұрын
Nice tutorial, I'm gonna try numma in my plasma physics simulations
@nitsanbh
@nitsanbh 3 жыл бұрын
This is one of the best intro/tutor videos I’ve ever watched on any subject
@RealLionTion
@RealLionTion 3 жыл бұрын
YES, I LOVE PYTHON AND THIS IS PERFECT THANK YOU MATE
@dinoscheidt
@dinoscheidt 3 жыл бұрын
Awesome video. Simply forwarded this to two of my junior devs and they completely got it now. THANK YOU!
@PandemicGameplay
@PandemicGameplay 3 жыл бұрын
This is revolutionary. I always wanted a library to allow these but without the hassle of installing something like Cython, etc. Just a good ol' pip lib that can allow my code to speed up orders of magnitude. A nice bonus is the ability to release the GIL.
@vibhu8024
@vibhu8024 Жыл бұрын
I love how "incredibly fast" python code is pretty much about going back to c/c++ syntax.
@ecielyang9719
@ecielyang9719 2 жыл бұрын
I have struggle for Numba for 3 days. This Vedic is really clear
@AnthonyThomas_Ant
@AnthonyThomas_Ant 3 жыл бұрын
Liked and subscribed. The length and pace of the video were spot on, especially as you’ve shared the notebook.
@Rotem_S
@Rotem_S 3 жыл бұрын
As someone who tried numba and was disappointed I appreciate you showing possible causes and solutions for that
@lordtejas9659
@lordtejas9659 4 жыл бұрын
You deserve my like. Nice work sir. Now I can show to my friends how a optimizated and we'll written code looks like. Numba.
@vishawashkumar5159
@vishawashkumar5159 3 жыл бұрын
This needs much more views.. This is a revolution..
@evertonsantosdeandradejuni3787
@evertonsantosdeandradejuni3787 2 жыл бұрын
Holy shit, I'm sold. Examples on point, mic indeed touched the ground. Thanks for the tip
@janewan1196
@janewan1196 3 жыл бұрын
Thank you so much! I was trying and failing to turn to C++ for number crunching, but this has just saved me!
@eddantes_
@eddantes_ 3 жыл бұрын
Great video. The fact that you are sharing the jup notebook is amazing. Thanks a lot. Please consider making a follow up video with more complex applications of numba. For example, integrations!
@dydufjfbfnddhbxnddj83
@dydufjfbfnddhbxnddj83 3 жыл бұрын
I didn't understand anything as a beginner to Python (after learning C,C++ and Java) but it looks really cool man. I will use this for python, especially in competitive programming
@ViralKiller
@ViralKiller 2 жыл бұрын
4:06, so why can't you copy and paste this cached code into a new function, so it executes fast even on first run?
@Mzkysti
@Mzkysti Жыл бұрын
Thank you, learned ton of new about Numba even that I have actually used it few years back :). Currently I've been using more Cython since I recall I was always in trouble to getting the functionality out of the class which had it but do you think it's no time to try Numba again?
@raccoonteachesyou
@raccoonteachesyou 4 жыл бұрын
Damn, I love your Python videos ! Very interresting, you talk about stuff other don't. Thx man !
@AJ-et3vf
@AJ-et3vf 3 жыл бұрын
Heyy man! This is a very nice tutorial for numba! There are only few numba tutorials out there and I hope you could make a bit more. For example, I became interested in using numba to speed up my Runge-Kutta Algorithms that I coded myself, but I couldn't make it work. I would try to apply it on the for-loop part but it would throw an error that numba doesn't now the class function, something like that. Hopefully, an extensive numba tutorial would be made in the future.
@Epiphonication
@Epiphonication 3 жыл бұрын
that's fantastic, I used to resort to using my uni's HPC cluster to get some more speed but now it's so much faster even on my old thinkpad, thank you so much!
@JackofSome
@JackofSome 3 жыл бұрын
Wow ... If you're able to go from HPC cluster to a laptop those are ... Those are some massive gainz
@Epiphonication
@Epiphonication 3 жыл бұрын
@@JackofSome truly massive! I'm going to try numba + HPC soon, let's see how fast that goes
@JackofSome
@JackofSome 3 жыл бұрын
Them: nooo you can't write heavy workloads in pure python You: haha, numba go brrrrrr
@Epiphonication
@Epiphonication 3 жыл бұрын
@@JackofSome exactly my thought :D
@pranavramesh7052
@pranavramesh7052 3 жыл бұрын
thank you i had a code which took like a whole 5 minutes to start running and now it is much faster
@jongmassey
@jongmassey 3 жыл бұрын
Damnit, why did I only discover this 6 days before my PhD submission deadline?!
@eskees9498
@eskees9498 3 жыл бұрын
How was your submission deadline?
@jongmassey
@jongmassey 3 жыл бұрын
@@eskees9498 horrendous, my stomach is still in knots!
@eskees9498
@eskees9498 3 жыл бұрын
@@jongmassey well good luck 👍
@lemurmad
@lemurmad 3 жыл бұрын
The fact i got this in my recomended when i made a screencapture to table that was at 1 - 4 fps at 480p This is gonna help a lot!
@JackofSome
@JackofSome 3 жыл бұрын
Do let me know if it helps (or even if it doesn't). If your app/project is public I would love to take a look at it.
@lemurmad
@lemurmad 3 жыл бұрын
Ill test tomorrow as its kinda late. Its not currently public as its just something specific, but I dont mind sharing it. I do really hope it makes it faster!
@chinmayk8004
@chinmayk8004 3 жыл бұрын
This was the voice of god. Just today I was running a slow heavy code
@jamesmarchetti3305
@jamesmarchetti3305 2 жыл бұрын
My new fav channel. Have you done a video talking about your Linux setup? If not I would love to see it/could you share some info?
@adityanjsg99
@adityanjsg99 Жыл бұрын
Presentation level is ingenious!
@danial_amini
@danial_amini 2 жыл бұрын
The first delay thing is not just the compile overhead for numba. It also takes a long long time apparently for "warm up" of the SIMD or AVX instructions. Even if the code is precompiled, if you change the matrix & recalculate it's gonna take a whole long time again for the first run. In applications when you need only a few iterations this is really not worth it. If you are doing several iterations on each matrix like more than 10 times then yes it's worth it.
@JackofSome
@JackofSome 2 жыл бұрын
This hasn't been my experience. I use numba in production and beyond the first run the code always runs at the same speed even with changed inputs
@MichaelBrown-gt4qi
@MichaelBrown-gt4qi 3 жыл бұрын
This is amazing. This completely changes python for me.
@michaelc3650
@michaelc3650 3 жыл бұрын
Damn this is really cool, just came across it and it should definitely help in speeding up machine learning or ai projects
@yashmanwani6232
@yashmanwani6232 3 жыл бұрын
Man. I must have been living under a rock to have missed numba for so long
@CoolDude911
@CoolDude911 2 жыл бұрын
10:10 Why are numpy arrays slow to dynamically allocate? -> The basic answer is that it is a C++ vector underneath and that vector which can be given extra capacity to append elements but at a certain point it will have to ask the operating system for more contiguous memory and has to be reallocated (this is very slow). The solution is to assign the numpy array a certain length beforehand with the np.empty() function. Python is designed to be fast to allocate an object of any amount of memory and a uses a linked list for its List type. This does have a cost in terms of defragmentation and garbage collection and it is slower to iterate through a linked list as well. Thank you for this informative video.
@JackofSome
@JackofSome 2 жыл бұрын
Python's lists are not linked lists, they are Implemented somewhat similar to vectors. They also have O(1) lookup, something you can't have with a linked list. This is a pretty common misconception I wonder where it comes from. Maybe a really old version used to have linked lists? Here's a link in the python docs talking about the basic implementation docs.python.org/3/faq/design.html#how-are-lists-implemented-in-cpython
@CoolDude911
@CoolDude911 2 жыл бұрын
@@JackofSome Oh cool, that's interesting. I assumed that because I found it faster to append a list than a numpy array. Maybe the numpy array is using a vector without a smart way of having extra capacity so it has to be reallocated all the time. I assume that a list in python has to contain pointers to a generic memory buffer objects so I guess the is linking going on there, as it has to allow fast generic memory allocation. It's funny how the inference time and allocation is measured but that does not take into account garbage collection and fragmentation.
@Blooddarkstar
@Blooddarkstar 3 жыл бұрын
Very well explained and saved me from some mistakes. You deserve way more subscribers!
@dishapandey96
@dishapandey96 4 жыл бұрын
Your way of explanation is very nice. Excellent video :)
@stxnw
@stxnw 3 жыл бұрын
couldn’t understand what this video was saying 4 months ago. i’m back and i finally understand.
@JackofSome
@JackofSome 3 жыл бұрын
Fantastic
@guy-dev
@guy-dev 3 жыл бұрын
I'm a Java dev but I'm saving this for later, don't mind me.
@hectorminator4
@hectorminator4 3 жыл бұрын
Dude, I'm an aerospace engineer student currently taking my masters studies, and I have to program with python and some programs take some minutes to calculate, because I have to solve complex nonlinear equation systems, and that takes some time that makes me desperate, I'll definitely try this, thanks a lot
@robosergTV
@robosergTV 3 жыл бұрын
this is golden. You explain stuff very clearly and easy to understand. Please do more vids :) ps. do you still playing around with Reinforcement Learning? Does your job relate to it in any way?
@JackofSome
@JackofSome 3 жыл бұрын
Unfortunately that's a no on both counts. I do lots of deep learning work in my job but it's all related to object detection.
@IndellableHatesHandles
@IndellableHatesHandles 3 жыл бұрын
I bet my friend who called Python a "fake language" is soiling himself right now.
@JackofSome
@JackofSome 3 жыл бұрын
Ehh, let him call it a fake language. You can instead occupy yourself with making awesome things instead :)
@JackofSome
@JackofSome 3 жыл бұрын
That's exactly how you say it in English :). Thank you for your kind words
@overloader7900
@overloader7900 3 жыл бұрын
I mean, its technically just translates python to actual machine code... So, as usual, the best way to make python fast, is to not use python
@JackofSome
@JackofSome 3 жыл бұрын
Exactly. Because when you compile C++ you officially stop using C++, right?
@overloader7900
@overloader7900 3 жыл бұрын
At this point its about the language that is the easiest to convert to machine code
@shkhrvarshney
@shkhrvarshney 3 жыл бұрын
Awesome tutorial! Thank you so much. I shared the video with my old Intro to Parallel computing professor. Hopefully he sees the email coming from non UW email id lmao. Video would be pretty helpful to any new(&old) students of the course.
@andrewglick6279
@andrewglick6279 3 жыл бұрын
Me: Watching videos about optimizing Python code to procrastinate studying for a systems and controls exam Video: Talks about systems and controls applications of Numba to mock me :)
@JackofSome
@JackofSome 3 жыл бұрын
Bwahahahaha. And I even used a dry friction example which is probably useless for your exam 😈
@andrewglick6279
@andrewglick6279 3 жыл бұрын
​@@JackofSome Oh no, it was not at all useful on the exam. 😂 Thankfully, the exam went well! Also, this is a super great video! I'm currently in the process of settling on a workflow of using either Cython or Numba for a research project. From what I can tell, Cython is more powerful, but Numba is cleaner and easier to make work. After all the struggling I've gone through with getting Cython to work (and how all linters/highlighters more or less give up with Cython syntax), this video made Numba look really nice.
@JackofSome
@JackofSome 3 жыл бұрын
Pretty much my experience with Cython, though I do intend to get good at it at some point. Know that C++ (or even Rust) is a very viable option for speeding bits of code. I have a recent video on how to interface C++ with pyhton.
@andrewglick6279
@andrewglick6279 3 жыл бұрын
@@JackofSome Unfortunately, after trying to implement Numba into my code, it seems that Numba isn't equipped to handle what I'm trying to do. The code I had written, while heavily NumPy dependent, is also heavily class dependent. Numba's jitclasses are not quite ready for use and seem to completely break with attempting to create a list of jitclasses. I don't really want to restructure the code as non-object oriented just to work with Numba; I may try posting something on StackOverflow later, and unless you have a solution to this, I'm thinking I'm still stuck with Cython. 🙃
@JackofSome
@JackofSome 3 жыл бұрын
Aww. Cython would be a better option. I use Numba a fair bit in object oriented code but through abstraction of heavy functions in a numba friendly way, but that's not always an option
@wenhanzhou5826
@wenhanzhou5826 3 жыл бұрын
Initially, it took Python 25min to run, but now it took 2min just by adding one line of code! You have definitely saved more than 8h for me! Edit: it actually took 2.5 sec.
@JackofSome
@JackofSome 3 жыл бұрын
In addition to positive comments I also accept co-authorships as compensation 👀
@wenhanzhou5826
@wenhanzhou5826 3 жыл бұрын
@@JackofSome of course!
@andres_io2931
@andres_io2931 3 жыл бұрын
Thank you! ... Do you think that this would be useful (and compatible) in a Raspberry environment, or it needs a laptop/PC ?
@JackofSome
@JackofSome 3 жыл бұрын
It works fine on arm
@mkthakral
@mkthakral 2 жыл бұрын
Fist visit on this channel, but you are the best ;-)
@andylib
@andylib 4 жыл бұрын
This was awesome !! Sooo interesting, straight up going into this
@ranraj100
@ranraj100 Жыл бұрын
Great work man! Can you kindly make a video showing where the numba takes more time than conventional python and where use of cython is more efficient than numba. Thank you.
@steamlink4803
@steamlink4803 2 жыл бұрын
does the first time compilation need to be done for every new set of arguments, or is it a one-time thing in general?
@djudju8047
@djudju8047 2 жыл бұрын
This video was very interesting for me. I just understood why Python is so slow.
@sumit_singh_live
@sumit_singh_live 2 жыл бұрын
Your video was that amazing and well executed that I was hypnotized to hit subscribe and like. Keep it going.
@mengzhu8385
@mengzhu8385 2 жыл бұрын
Amazing video! I'm starting to think to give numba a try.
@anon10101
@anon10101 3 жыл бұрын
This is, as DJ Khaled would say, a major key. wow.
@iosbase
@iosbase 3 жыл бұрын
"Another one!" - Numba Fan Number 1
@fadillahmaani7413
@fadillahmaani7413 4 жыл бұрын
Great video, thank you very much ! I hope that you will make numba tutorial for class too.
@unflexian
@unflexian 7 ай бұрын
mate you're a jupyter magician
@Firigion
@Firigion 3 жыл бұрын
Okay, I'm hooked. Now i just need this for DDEs.
@VictorRodriguez-zp2do
@VictorRodriguez-zp2do 3 жыл бұрын
Basically it makes you have to things the way you would in C or C++, in my opinion just using a faster typed language would make more sense. The only problem is that right now the majority of libraries and work for scientific work is being done in python when it should be done in fast, typed, compiled languages.
@JackofSome
@JackofSome 3 жыл бұрын
The majority of libraries that are used for scientific work in python are written using C or C++, with thin python wrappers (e.g. numpy, scipy, opencv, tensorflow, etc) Numba code deviates ever so slightly from python norms. It in no way looks as tedious as performant C or C++ code (especially considering you can use numpy code in numba functions)
@ImtihanAhmed
@ImtihanAhmed 3 жыл бұрын
Numba might just have fixed my biggest bottleneck on the computation for my master's thesis :)
@b992152
@b992152 4 жыл бұрын
Great video mate. Really enjoyed it
@marcusdudebro9426
@marcusdudebro9426 3 жыл бұрын
This is big. How did I not know this existed.
@erkanturan7215
@erkanturan7215 2 жыл бұрын
Perfect video! Are you really writing that fast or do you talk over the fast-forwarded video? I would go for the second one but then I saw you going like "this one" or "that one" with the cursor and I am not sure anymore :)
@JackofSome
@JackofSome 2 жыл бұрын
It's probably the magic of editing. I'm against wasting the viewer's time so I edit like a crazy person. I can navigate code pretty fast though so it's possible some stuff was actually real time
4 жыл бұрын
What about running on GPU? It would be interesting to see some performance comparisons about that. Check out CuPy as well, which enables GPU with Numpy.
4 жыл бұрын
I just saw a related comment :)
@JackofSome
@JackofSome 4 жыл бұрын
Soon :)
@a4e69636b
@a4e69636b 2 жыл бұрын
Have you ever used pypy? I would be interested in seeing how Numba and PyPy compare to each other.
Python Development in Spacemacs
9:14
Jack of Some
Рет қаралды 31 М.
Tutorial: CUDA programming in Python with numba and cupy
45:43
nickcorn93
Рет қаралды 81 М.
Motorbike Smashes Into Porsche! 😱
00:15
Caters Clips
Рет қаралды 23 МЛН
ТЮРЕМЩИК В БОКСЕ! #shorts
00:58
HARD_MMA
Рет қаралды 2,2 МЛН
Compiled Python is FAST
12:57
Doug Mercer
Рет қаралды 116 М.
Python is NOT Single Threaded (and how to bypass the GIL)
10:23
Jack of Some
Рет қаралды 110 М.
25 nooby Python habits you need to ditch
9:12
mCoding
Рет қаралды 1,8 МЛН
1000x faster data manipulation: vectorizing with Pandas and Numpy
26:39
How Much FASTER Is Python 3.13 Without the GIL?
10:00
ArjanCodes
Рет қаралды 176 М.
Numba Explained in 30 Minutes - The Python and Numpy Compiler
30:29
Turn Python BLAZING FAST with these 6 secrets
5:01
Dreams of Code
Рет қаралды 49 М.
Motorbike Smashes Into Porsche! 😱
00:15
Caters Clips
Рет қаралды 23 МЛН