Five Amazing Python Libraries you should be using!

  Рет қаралды 76,098

Jack of Some

Jack of Some

Күн бұрын

Пікірлер: 120
@jaydouken
@jaydouken 3 жыл бұрын
Included the libraries and what they do at the start of the video to not waste your time. Instant like for me.
@JackofSome
@JackofSome 4 жыл бұрын
Timestamps for mobile users: Argh - Painless CLI Generation (00:43) tqdm - Easy Progresbars (05:10) msgpack - Like Binary JSON (08:44) schedule - Python's cron (12:43) Redis Simple Cache - Memoization (14:11) Very insightful advice about the potential dangers of redis simple cache from a Reddit user: www.reddit.com/r/programming/comments/flww79/intro_to_5_amazing_python_libraries_argh_tqdm/fl18qpp?
@ozonesama
@ozonesama 2 жыл бұрын
Alas the insightful advice has been deleted.
@040040919
@040040919 3 жыл бұрын
I wanted to tell you I pressed like at 00:42, when I realised I'm watching one of the best summaries ever. The cotent as well was great. Very good job, mate.
@yashsinha9779
@yashsinha9779 3 жыл бұрын
Holy shit the last one (Redis Cache Library) that's my fork of the main library lmao
@JackofSome
@JackofSome 3 жыл бұрын
Ayoooo. Thank you so much for making the fork! Did you also upload one of the pypi packages?
@yashsinha9779
@yashsinha9779 3 жыл бұрын
Nah, I haven't uploaded any pypi package ever, lol.
@SGF08Y
@SGF08Y 3 жыл бұрын
argh would've saved me sooooo much time had I known about it sooner! Definitely improves code structure right from the onset and I hadn't appreciated how messy your code can get when working with argparse.
@asiergaliciamartinez7449
@asiergaliciamartinez7449 4 жыл бұрын
Nice video!! I have just made use of tqdm in one of my university projects :D. Thanks for discovering it to me!!!
@JackofSome
@JackofSome 4 жыл бұрын
My pleasure :)
@undisclosedmusic4969
@undisclosedmusic4969 4 жыл бұрын
Came for the suggestion, stayed for the insightful software engineering commentary. ❤️
@JackofSome
@JackofSome 4 жыл бұрын
This is the nicest comment I've ever gotten.
@scriptkiddie6151
@scriptkiddie6151 3 жыл бұрын
Your videos are just awesome, I truly enjoy them. Please don't ever stop making them.
@JackofSome
@JackofSome 3 жыл бұрын
My last serious video was a year ago 😅 But I'm making more :)
@fizipcfx
@fizipcfx 3 жыл бұрын
i believe if you add "Beginning 0:00" to the beginning of the timestamps in the description, youtube will divide the video bar to the partitions of it. edit:typo and grammar mistakes
@JackofSome
@JackofSome 3 жыл бұрын
Thanks! I finally did that and the chapters are there now.
@markusbuchholz3518
@markusbuchholz3518 4 жыл бұрын
Your channel and videos are really amazing. Great to continue with AI/ML/Reinforcement and other CS subjects. I am convinced that your are going to have a lot of subscribers. Good luck
@JackofSome
@JackofSome 4 жыл бұрын
Thank you for your kind words. Please consider sharing the videos with others that might be interested.
@markusbuchholz3518
@markusbuchholz3518 4 жыл бұрын
No worries. As lover to open source SW I always share the state-of-the-art work. Thanks for your effort!
@hunterap23
@hunterap23 2 жыл бұрын
Not sure if you’ve become aware of it since making this video, but Python 3 has the built-in `functools` library which has both the `cache` and `lru_cache` functions that can be used as decorators. `lru_cache` takes an argument `maxsize` to specify how many combinations of unique input arguments should have their results cached, and the `cache` decorator is the same but assumes an infinite number of catchable results. This has the benefit of not requiring redis nor an external plugin, but the downside is the cache only exists in that runtime of python.
@JackofSome
@JackofSome 2 жыл бұрын
I made a separate video on this. These in memory caching decorators have been around forever but the benefit of Redis Simple Cache is that it can survive a runtime restart.
@hunterap23
@hunterap23 2 жыл бұрын
@@JackofSome oh awesome! I just found your channel (through your Numba JIT video) and have been going through all the videos I'm interested from oldest to newest, so I guess I just haven't gotten there yet. Love what I've been seeing so far!
@amrohendawi6007
@amrohendawi6007 3 жыл бұрын
the name tqdm derives from the Arabic word "تقدم: taqaddum" which translates to progress.
@yashindane2844
@yashindane2844 3 жыл бұрын
Wonderful
@jbs3144
@jbs3144 3 жыл бұрын
I use tqdm on a daily basis. It’s my canary when crunching number or looping. It also works in CL and Jupyter NBs.
@Very_SpicyUser
@Very_SpicyUser 2 жыл бұрын
Everything you need to know about coding new string methods is perfect for you, the person who is reading right now will have a luck tomorrow!
@ronaldokun
@ronaldokun 4 жыл бұрын
Awesome content, some python gems I didn't know. Thank you! Subscribed.
@elvisdelahoz3004
@elvisdelahoz3004 3 жыл бұрын
Has something changed in msgpack? I downloaded the file from the git repo and ran it in python3 and python3.9 and I get a traceback as Traceback (most recent call last): File "c:\temp\ytvideo_python_lib_favs\five_amazing_python_msgpack.py", line 62, in msgpack_example_1() File "c:\temp\ytvideo_python_lib_favs\five_amazing_python_msgpack.py", line 43, in msgpack_example_1 back_from_msgpack = msgpack.unpackb(ftmp) File "msgpack/_unpacker.pyx", line 195, in msgpack._cmsgpack.unpackb ValueError: int is not allowed for map key
@JackofSome
@JackofSome 3 жыл бұрын
Interesting. I haven't tried msgpack with newer versions. Lemme check
@IncomeCountry
@IncomeCountry 3 жыл бұрын
I got the same error with msgpack-1.0.2 and patched it like this : back_from_msgpack = msgpack.unpackb(f.read(), strict_map_key=False) Hope this helps!
@romainmartinez6421
@romainmartinez6421 4 жыл бұрын
Hi Jari and thanks for the great content. Apparently you're using a tiling window manager. Have you considered doing an "abs (min (...))" video on that? By the way, your videos really improved my dev workflow. You introduced me to org-mode, spacemacs, fzf, autojump and vspacecode and, from now on, I could not code without them. You should set up a patreon, I will support you with great pleasure.
@JackofSome
@JackofSome 4 жыл бұрын
Hi. I get a lot of requests for my setup so I guess I should do a video soon. I used to have a patreon but my output is really sporadic so I took it down. If you want to donate you can do that here www.buymeacoffee.com/Ej8qUm5 Or alternatively you can donate to a charity like Humanity First or the Red Cross
@anthonyaouad4190
@anthonyaouad4190 3 жыл бұрын
Ever heard of APscheduler? It's similar to the scheduler library but you add jobs to a scheduler object instead. I find it pretty reliable
@JackofSome
@JackofSome 3 жыл бұрын
Heard it from another commenter here. Will try it out
@SkitoOwn
@SkitoOwn 3 жыл бұрын
What's the lib/command you're using to see that nice history menu on 2:50? :o Nice content, btw :)
@JackofSome
@JackofSome 3 жыл бұрын
Fzf. I have a video on that.
@johndong2880
@johndong2880 3 жыл бұрын
Great mentor, please keep keep on, although found a year later.
@MultiCraftTube
@MultiCraftTube 3 жыл бұрын
4:44 nobody can keep me from passing my settings dict through 50 functions!
@mathgeniuszach
@mathgeniuszach 3 жыл бұрын
_Oh_ lol
@somebodystealsmyname
@somebodystealsmyname 3 жыл бұрын
Is there any gain from using APScheduler over schedule?
@ceasnov
@ceasnov 3 жыл бұрын
0:54 - I feel a Korra reference
@dziwneumniedziaa1752
@dziwneumniedziaa1752 4 жыл бұрын
Thanks, a really cool video! I will definitely try some of these libraries out soon. As for the redis cache, do you think there's an advantage to using it over sth like lru_cache?
@JackofSome
@JackofSome 4 жыл бұрын
As far as I know lru_cache is gone once your process exits. redis cache doesn't have that limitation.
@dziwneumniedziaa1752
@dziwneumniedziaa1752 4 жыл бұрын
@@JackofSome Oh yeah, thanks, that makes sense.
@Whiz8
@Whiz8 3 жыл бұрын
Hi can someone explain when did we realistically need argparse and tqdm becaue from my perspective it seems to not be that useful because python is mainly for backend development right?
@JackofSome
@JackofSome 3 жыл бұрын
If you're working on web, then probably yes python is probably only useful for backend work. To the rest of the world it's used for everything.
@dennisbrand5269
@dennisbrand5269 3 жыл бұрын
Which desktop-environment is he using here? 4:38
@thingsiplay
@thingsiplay 4 жыл бұрын
tqdm is fantastic.
@EvanGillespie
@EvanGillespie 3 жыл бұрын
I've used Click + setuptools for CLI functions before. How does argh compare?
@JackofSome
@JackofSome 3 жыл бұрын
Click is just as good (or possibly better) than argh
@unionsafetymatch
@unionsafetymatch 3 жыл бұрын
I tried to use tqdm to implement a file loading progress bar but somehow it just wasn't working out. I could figure out how to pass the total size of the file and see how much has been accessed or read to the package. I'm not even sure what the file type was anymore but if anyone could give an example or some insight, that'd be great. btw @jack, your videos really ARE great. I just came across them today randomly and I love them!
@deepslave
@deepslave 3 жыл бұрын
i have sed lru_cache from functools for cache memory, but you use redis simple cache, i understand that the recently used value it save in redis database, is it true? but is this the case its more powerfull for big data!
@fakeaccount4563
@fakeaccount4563 3 жыл бұрын
What about pypi.org/project/cachetools/ instead of redis-cache since it looks abandoned?
@JackofSome
@JackofSome 3 жыл бұрын
Aren't these all in-memory though? I've been meaning to make a python 3 compatible fork of redis cache and maintaining it. Maybe now is the time edit: I reviewed the codebase for cachetools and it's a very nice library. I really like it. But it _is_ still in memory. Even the TTL cache does not persist to disk in any way.
@nicwanavit5463
@nicwanavit5463 3 жыл бұрын
why would one use msgpack over json+ gzip?
@Kiarash_
@Kiarash_ 4 жыл бұрын
One point about floating points getting truncated, the meaningful part of a double is the 16 numbers, the numbers that are truncated are just artifact of the format of floating-point and would not be exactly as given, you can test it with entering this snippet:"a = float('0.1111111111111111111'); print("{:.20f}".format(a)) " Which on my machine give the following output: 0.11111111111111110494 For more information about the issues rising working with floating points, check here: www.topcoder.com/community/competitive-programming/tutorials/representation-of-integers-and-reals-section-1/
@lwjasons
@lwjasons 4 жыл бұрын
I really like the font you used! Could I know the name of it?
@JackofSome
@JackofSome 4 жыл бұрын
Mononoki
@ExDarkx3
@ExDarkx3 3 жыл бұрын
I love this, this is going to revolutionise my code! However, for msgpack, there is the native pickle library. How do these compare?
@JackofSome
@JackofSome 3 жыл бұрын
pickle is great but it's python specific. msgpack allows you to use the data in other languages (e.g. nodejs, something I do alot).
@MB26363qwerty
@MB26363qwerty 4 жыл бұрын
When are you doing your policy optimization lviestream! I am looking forward to it
@JackofSome
@JackofSome 4 жыл бұрын
I'm trying to figure out a good time. With the shelter in place order right now I can't go to the office to do the stream which is what I used to do. Maybe I'll do tutorials first and then do the livestreams for Q&A
@atursams6471
@atursams6471 3 жыл бұрын
You are a great guy! You forgot keyboard and mouse for python macro manipulation.
@JackofSome
@JackofSome 3 жыл бұрын
Gotta leave some things for future videos :D
@ukaszwasowicz2956
@ukaszwasowicz2956 3 жыл бұрын
What would u recommend for prod in case of redis cache it?
@JackofSome
@JackofSome 3 жыл бұрын
That's a very complicated and situation specific question. For my production environment, for example, I use a combination of in memory caching, disk based caching, and redis based caching, all hand written.
@Belissimo-T
@Belissimo-T 3 жыл бұрын
Why not use pickle as data serialization?
@JackofSome
@JackofSome 3 жыл бұрын
Pickle is python specific and some times stores more than what you need. Very useful in some cases, but not as a general format in my experience.
@Belissimo-T
@Belissimo-T 3 жыл бұрын
@@JackofSome Thanks for the fast response! I actually didn't know that.
@beebakrizzle
@beebakrizzle 3 жыл бұрын
Argh is not really maintained, and I don't see the advantage of redis simple cache over functools' lru_cache. The msgpack homepage has some issues with showing the available implementations as well. Other than that it's pretty interesting! I can recommend APScheduler for a more flexible and powerful scheduler for Python.
@JackofSome
@JackofSome 3 жыл бұрын
Not maintained yes but still perfectly functional, but yes it's probably better to use click or something. As far as I know lru_cache gets scrubbed as soon as the process exits. Is that not the case? Redis cache is persisted to disk. I'd be interested in reviewing the issues with msgpack. I know in the early days they had some problems but haven't encountered any myself. It was a huge boon to a project where I needed a python server to send lots of data over websockets to a webpage.
@beebakrizzle
@beebakrizzle 3 жыл бұрын
@@JackofSome Cheers for the reply. Yes click does indeed seem more on top of things! You are right that lru_cache is not persisted across runs, neither is 3.9's unlimited "cache" afaik. But on the other hand it doesn't require any external dependencies, so it's worth a mention I think. Regarding msgpack, I'm just referring to their homepage not displaying anything in the bottom section titled "Languages" and "API". Other than that I think it's a solid option for serializing data that doesn't need human reading.
@Decstasy
@Decstasy 3 жыл бұрын
What about lru_cache from functools? Is it so much better?
@JackofSome
@JackofSome 3 жыл бұрын
I made another video about why this isn't the same thing and has a different use case. I recommend giving it a watch
@Decstasy
@Decstasy 3 жыл бұрын
@@JackofSome thank you very much!
@paweszczepanski6738
@paweszczepanski6738 Жыл бұрын
WHOA
@fourmatt
@fourmatt 3 жыл бұрын
Would you be able to do an example of msgpack with sockets?
@JackofSome
@JackofSome 3 жыл бұрын
I actually use msg pack for communication between a tornado server and a webapp written using typescript. Check out safijari/3viz if you're curious
@fourmatt
@fourmatt 3 жыл бұрын
@@JackofSome Okay, thanks for the quick reply! Much appreciated
@simplegamer6660
@simplegamer6660 3 жыл бұрын
What desktop environment are you using?
@JackofSome
@JackofSome 3 жыл бұрын
xfce with i3 as the window manager
@leonhma
@leonhma 3 жыл бұрын
Instead of redis you can Just use cache from functools (works the Same but without extra Installation)
@JackofSome
@JackofSome 3 жыл бұрын
I just put out a video showing why that's not equivalent
@divelix2666
@divelix2666 2 жыл бұрын
How does msgpack differ from protobuf?
@JackofSome
@JackofSome 2 жыл бұрын
Protobuf needs a defined schema. Msgpack does not. Msgpack is just a more storage efficient json.
@b34-d5j
@b34-d5j 4 жыл бұрын
Your using xubuntu, right? What desktop manager are you using? That makes your windows split like that?
@JackofSome
@JackofSome 4 жыл бұрын
i3
@yabeonwrath
@yabeonwrath 3 жыл бұрын
Whats the difference between msgpack and pickle?
@JackofSome
@JackofSome 3 жыл бұрын
Pickle is specific to python whereas msgpack is more like json. Many languages implement the spec and so the data can be easily shared (e.g. I have a 3d visualizer that runs on the web and can communicate with a local process over websockets and send data using msgpack)
@orbyfied
@orbyfied 3 жыл бұрын
isnt msgpack just NBT
@JJSogaard
@JJSogaard 3 жыл бұрын
Whaaat. I haven’t really done a cli because it just seemed to cumbersome for me to even bother. I guess there is no reason not to now...
@Galactipod
@Galactipod 3 жыл бұрын
The inverse for me, I guess. The vast majority of my programs are CLI-only because I just didn't want to deal with a GUI.
@antoniogoncalves705
@antoniogoncalves705 3 жыл бұрын
One library i like a lot is pretty_downloader by deadsec-security
@primekrunkergamer188
@primekrunkergamer188 3 жыл бұрын
Cant u just import cache instead of using redis cache?
@JackofSome
@JackofSome 3 жыл бұрын
Cache is in memory for the current process no? Or does it persist to disk?
@aminghafoori6496
@aminghafoori6496 3 жыл бұрын
tqdm is awesome
@JackofSome
@JackofSome 3 жыл бұрын
Amen to that
@hugocosta9017
@hugocosta9017 3 жыл бұрын
For cache use lru_cache from functools which should already come with your python installation, that one does the exact same thing.
@JackofSome
@JackofSome 3 жыл бұрын
Nope. Doesn't persist which is what's different here. I seem to have done a bad job of explaining this because I get this comment a lot. Maybe I should make a video
@yuu-kun3461
@yuu-kun3461 4 жыл бұрын
So you moved to doom now? Funny how I didnt like spacemacs but I am actually using doom emacs rn :)
@JackofSome
@JackofSome 4 жыл бұрын
Nope. Still Spacemacs, theme and modeline are Doom.
@UnspeakableCreature
@UnspeakableCreature 3 жыл бұрын
Argh vs click? Who will win?
@JackofSome
@JackofSome 3 жыл бұрын
Click since argh is kind of dead. Unless...
@yoramyoram4846
@yoramyoram4846 3 жыл бұрын
what is you font?
@JackofSome
@JackofSome 3 жыл бұрын
Mononoki
@yoramyoram4846
@yoramyoram4846 3 жыл бұрын
@@JackofSome thanx, also can you make video about secure data serialization, like pickings or Json over socket.
@deadeye1982a
@deadeye1982a 3 жыл бұрын
Other cool random libraries: Typer, Rich, Loguru, Diskcache, pyzmq, Datasette, FastAPI .... just continue the list :-)
@kirillolar7614
@kirillolar7614 3 жыл бұрын
Rich is really cool for building beautiful CLI apps, and if I am not mistaken it has it's own progress bars. Also looked for Typer and Loguru, they seem super easy to use
@mattmess1221
@mattmess1221 3 жыл бұрын
click is better than argh. And typer adds support for type annotations.
@Yoku5
@Yoku5 4 жыл бұрын
Argh has been abandoned for years, I think Click is a better option
@JackofSome
@JackofSome 4 жыл бұрын
Fair point. Though it's not like there's anything particular that needs maintenance. I'll check out click
Make Python code 1000x Faster with Numba
20:33
Jack of Some
Рет қаралды 447 М.
Python is NOT Single Threaded (and how to bypass the GIL)
10:23
Jack of Some
Рет қаралды 111 М.
Cat mode and a glass of water #family #humor #fun
00:22
Kotiki_Z
Рет қаралды 42 МЛН
黑天使被操控了#short #angel #clown
00:40
Super Beauty team
Рет қаралды 61 МЛН
Sigma Kid Mistake #funny #sigma
00:17
CRAZY GREAPA
Рет қаралды 30 МЛН
Top 18 Most Useful Python Modules
10:50
Tech With Tim
Рет қаралды 935 М.
The Fastest Way to Loop in Python - An Unfortunate Truth
8:06
mCoding
Рет қаралды 1,4 МЛН
VSCode's Python Interactive mode is AMAZING!
6:58
Jack of Some
Рет қаралды 353 М.
All Top 40 Python Libraries EXPLAINED in 20 minutes
22:04
25 nooby Python habits you need to ditch
9:12
mCoding
Рет қаралды 1,8 МЛН
Python Decorators in 15 Minutes
15:14
Kite
Рет қаралды 455 М.
Python dataclasses will save you HOURS, also featuring attrs
8:50
Ray: Faster Python through parallel and distributed computing
9:41
10 Python Tips and Tricks For Writing Better Code
39:21
Corey Schafer
Рет қаралды 1,3 МЛН
Cat mode and a glass of water #family #humor #fun
00:22
Kotiki_Z
Рет қаралды 42 МЛН