15 POWERFUL Python Libraries You Should Be Using

[@ArjanCodes]

💡 Learn how to design great software in 7 steps: arjan.codes/designguide.

[@marcelogrim]

A video about asynchronous programming would be very nice. Showing how to organize the code clearly and efficiently. And of course, talk about any available libraries as well.

[@ADBraimah93]

That video was already made on his channel I believe.

[@andydataguy]

Agreed! 💯 There are so many aspects to async especially when considering fastAPI using or automations design.

[@k98killer]

I made a library called "sqloquent" that includes both synchronous and async implementations of the ORM and query builder. I also have been working on a mesh networking system for micropython (blocked atm by lack of ed25519 signature functionality; I added TweetNaCl to the firmware with runtime bindings, but it was far too slow), and it makes use of async extensively. If you are writing an async library, you organize the code exactly as you would if it was a synchronous library. For an app/script that uses async libraries, the main thing to keep in mind is to not use `asyncio.run` or something similar within a running event loop, but the only other difference is that your program entry point will have to call a high level asyncio function to run the async code.

[@Casey_W]

I always appreciate pointers to useful libraries! I would throw in the standard "shelve" library that provides a simple ORM using the normal dict methods almost transparently with no external dependencies.

[@ArjanCodes]

Great library suggestion!

[@thinhhoang7361]

duckdb is my go-to module for quick analyzing a specific tables or just reading txt, csv, parquet file.

[@MichaelCampbell01]

json, too.

[@pablogonzalez7959]

Great video. I'd love one about connecting python and C/C++

[@DrGreenGiant]

I'd like a video on this too! My go-to now is pybind11, much prefer it to boost. It's still a pain to setup though the moment you have to do something real world that's not covered by the documentation examples.

[@k98killer]

There is a package "cffi" that is used a lot for that. Could be a good topic for a video.

[@brunotic]

I use pybind11 for this (C++). Highly recommended

[@pablogonzalez7959]

@@k98killer ohh, I didn't know that. But isn't python written in C, if I understand correctily we shouldn't need an external library for it

[@k98killer]

@@pablogonzalez7959 maybe if you want to extend the python interpreter itself with a native extension, but if you want to make a package that runs in any python interpreter, you will probably want to use a cffi wrapper.

[@Aucacoyan]

Yes please! I want the video about the return package!

[@osjerick]

Please, more about Returns!

[@wowzers94]

A pygame video would be a little different and neat.

[@ArjanCodes]

Noted!

[@deanban]

This video may be aimed at beginners/juniors, but I'd like to throw in a few of my favorites useful for a bit more advanced workload: Celery(useful if you need to queue tasks), Dedupe(saved me 3 days of coding utility functions), Dynaconf(env vars are so 2014), Returns(rust style return), Expressions(functional, if you want), Loguru(better logs make better programmer), Polars(try loading 40M rows into a Pandas df), Poetry,(pkg management) Pypika(cool ORM), and Streamlit(build data apps). The following may be unsolicited advices towards juniors, but please learn itertools. Also, you may be a good coder, but if you are bad at architecture, no amount of good libs is going to make you a good engineer. So please, also learn some system design.

[@agailloty]

Thanks

[@Sibula]

Polars is pretty cool. If your data doesn't fit into memory or you otherwise need or want to distribute the computation, there's also Dask.

[@deanban]

@@Sibula Dask is fantastic, albeit a real pain to set up properly in k8s as a distributed system. I'd go with PySpark instead for that kind of workload.

[@daviddeclercq9998]

I recently learned about this tenacity library which is great for retrying when something has failed!

[@raphaelproux1364]

It looks amazing!! Thanks!

[@marcreaume8689]

Really cool thank you

[@dankprole7884]

Nice, looks good for calling LLMs

[@raphaelproux1364]

Very nice video! I love hearing about great libraries 😊 one I came across recently is xarray, which you can see as a multidimensional numpy array or dataframe with coordinates and clever data selection.

[@dankprole7884]

xarray is way too big brained for me! 😂

[@kittentikkamasala69]

Would be nice to get some information on how to create your own templates for Cookicutter. I imagine it would be good for microservice architecture where you potentially have many, many projects with similar structure.

[@southpole76]

check the manual ... it comes with templates you can modify

[@DaveParr]

Arjan games is a series I would binge.

[@guitoo1918]

Sympy works really well in a Jupyter notebook. It can properly display the formulas.

[@AK-vx4dy]

PyGame is amazing for simple GUIs like dashboards, etc

[@ac130kz]

returns, especially the future_safe decorator, is just something that was always missing from my Python code after doing a decent bit of Rust. Insanely good for writing safe libraries without random side effects

[@masihaahmadi7663]

Thank you for such an amazing video It was really helpful

[@ademhilmibozkurt7085]

Your contents are awesome! Please Arjan, make a large scale real life program with Python. This could include database processes, file operations, performance improvement, computing and web. I will write this comment every video of you I watch :). Greetings

[@blismatic]

Would love to see a more in-depth video on NiceGUI!

[@dankprole7884]

Looks like a good library for my local chatbot!

[@papa5101]

Please, more about Pybind with c++!

[@insanecbrotha]

Coolest package I recently came across: typeguard!

[@johntamplin]

Quietly typing away with my coffee with Arjan in the background. Suddenly 13:28 , "... all that nasty stuff you have on there, you sick bastards!" **Coffee snorted through nose onto keyboard.**

[@r.walid2323]

Hello, why you haven't tried the 21:16 #15 Screenshot-to-code on yuor local machine, is this for safety sakes ?

[@AmosAAnderson]

It would be good to see a video on PyGame and Pyglet both.

[@215Leon215]

I do think the Returns package is quite interesting since it gives you a way to handle errors in a Error as Value approach which could be nice if your backend is a combo for Go and Python. Would make the error handling more similar across the codebase

[@djaydavidz]

Thanks for sharing and especially not leaving out the ones that may or may not be a good idea after all. Other question: Do you have any suggestions on libraries that document based on the code to eliminate manual documenting as much as possible?

[@rangabharath4253]

Awesome as always. Thanks for the useful content. PS: You look similar as Stranger Things Jim Hopper. 😊

[@igor28lg]

A video about Python Monorepos (possibly using Poetry) would be amazing!!

[@norden-am-werk]

I love the "Box" thing. That was new to me :P Instead of using Protocols, I'd rather do dataclasses then

[@AloisMahdal]

It's cool but it seems that if we care about types, something like pydantic would just do that and better (with type annotations, validations, constructs like tagged unions and all...) Use cases where the sheer simplicity of Box could shine are just when we don't care about types, ie. in really trivial scripts where we also happen to have enough control of the data that we can afford to just trust the keys....

[@archibald-yc5le]

Don't think it's too necessary, only if you're sick and tired of typing brackets and quotes. And it breaks types, although a generic Box[T] parameter would be a nice to have

[@_balancy_]

there is no need in "Box", since we have TypedDict, dataclasses, pydantic, msgpec which support the same functionality with some extra features

[@vladimirtchuiev2218]

Omegaconf that comes as a dependency for hydra has the DictConfig object which basically does the same thing. If you already know exactly what you need in your object dataclasses are better for it though, if you're a ML researcher and you play with architecture better just work with dict and their variations

[@VojtechMach]

@@_balancy_ It is not the same in my understanding. With pydantic et al you have to predefine the schema. With Box you can use an arbitrary dict with no overhead.

[@josephbolton8092]

pydantic, duckdb, streamlit, ruff, mypy, locust, ipython, wat

[@dankprole7884]

Nice. I think I'm gonna move to jinja2 for the prompting. F-strings just aren't cutting it when you need to do iteration and such.

[@mberlinger3]

Have you used Fugue? It's an abstract layer between single node and distributed processing. I'd be interested in a showcase about how to use it and benchmarks

[@alansnyder8448]

I'm interested in a Pygame video, just for some weekend fun. My area of application is to make a simple Kerbal-like game to explore the rocket equation and get to space from different planets like the moon or Mars. Pygame would be good for quicker feedback and being able to change things and diagram which orbits are reachable with what sized payloads.

[@jh0ker]

Recently found the `returns` package and it seems interesting, but I had the same concerns as you mentioned. Would watch a video about it.

[@g2david]

any suggestion on CLI args handling?

[@dbwalker0min]

Some of my favorites: poetry, google fire, pendulum, loguru, hexdump, icecream, pipx

[@southpole76]

poetry pretty much obsolete now because of uv

[@dbwalker0min]

@@southpole76 I didn’t know about uv. I’ll have to check it out.

[@matthieurochette]

I think Cookiecutter is a good tool if you need to templatize the way you/your org creates project (for example, if you're a freelance who creates one project per client but always with the same technologies, or if you work in a company with microservices and new ones pop up all the time)

[@jjxed]

I think the one I use for every single project I work on is Python Fire. It takes care of all my CLI needs

[@DV-13]

I like humanize, provides way to easily convert various types to human-readable text. Numbers, datetimes, timedeltas, etc.

[@ravenecho2410]

the type system, is like another language in itself. having the lap enabled, will show you how many errors i did not know iterator vs iterable, how the meshed with yield, for flatten, alias[*T] for nested, recursives, non defined type aliases for packages not doing the plumbing and saying everything is any or dynamic, etc i might throw up a quick repo for ``` flatten flatmap pipe compose ``` with full type hints as its something i seem to do on every project ive spent more than a month in, one thing i find cumbersome is piping or composing with one or more arguments, some sort of starmatch might be nice

[@d3stinYwOw]

I would add 'sched' python library - thread-safe, nice helper for repetitive task execution on fixed time slices.

[@JohnMatthew1]

Looking good these days @ArjanCodes :)

[@ArjanCodes]

Thanks John! :)

[@smithdoesstuff]

+1 for a Pygame video!

[@Iamsuperb4]

It would be a lot of fun to see you make a pong or brick breaker type game in Python!

[@AlanBerman]

I recommend covering the "click" package at some point

[@macerdough]

he did kzbin.info/www/bejne/fIjElJSkqNGZfK8si=oefIgu9vIbN2JYUl

[@nigh_anxiety]

He did a video on click 3 months ago. kzbin.info/www/bejne/fIjElJSkqNGZfK8si=26qUno0ozJaw6DC9

[@zeppelin0110]

The pipe library is especially cool. I wish it was built into python.

[@nater122]

great vid, thank you!!

[@Gigusx]

A PyGame video would be pretty cool. I'm sure you'd make it interesting, too! 😉

[@luigi_cuervini]

Great video! I guess that the main use of Box is that you dont have to do the typing annotation, right? If we wanted to do typing we could use pydantic instead. Or does anyone see it another way?

[@RanaUniverse]

I am just Start Watching Your Videos in This Month 😎😎😎 Its Boost my Python learning capabilities 🙃🙃🙃

[@utayasurian419]

Thanks Arjan...

[@vladimirtchuiev2218]

Pydantic with OpenAI API and gpt 4o does wonders. I also like lightning, fire, hydra, instructor. Poetry is kinda cool as well but i have to get used to it still.

[@murphygreen8484]

Don't often need it, but when I do I like fuzzywuzzy

[@fierce1340]

Yes do a pygame game!

[@glorytoarstotzka330]

can you make a video covering self-automation tools like pywinauto, pyautogui, pynput, pygetwindow, etc.? having solutions to new problems is nice, but what about faster solutions for existing problems that already need to be solved

[@tjanos88]

Do you know a good UML tool, that can quickly visualize classes and dependencies inside a project/module/file? Or some kind of code analysis tool if I would like to know how many lines long are my functions or files on average?

[@alexl4447]

Thank you for this overview. Do you guess, that pandas is not as usefull as sql?

[@oleksiyyeryomenko205]

PIXI package - awesome replacement for conda/mamba and poetry.

[@d3stinYwOw]

Just slap uv ;)

[@edgeeffect]

Back when I was trying to get a handle on functional programming Pickle was a brief cause of confusion because I had to learn that Python pickling isn't FP pickling.

[@hubstrangers3450]

Thank you...

[@Sibula]

That example of pipe just seems like a worse way to do a list comprehension: [x * 2 for x in numbers if x % 2 == 0]. For more complex uses there's also map() and filter() for example.

[@FernandoCordeiroDr]

Anjan be like: please please give me an excuse to create games! 😂

[@xiggywiggs]

pygame pygame pygame please please please!!!!! I know python isn't really it for game development, but with my limited use of pygame I have found it's a pretty quick way to through together a custom UI without having to learn a while new thing like with most of the UI libraries out there.

[@rafiullah-zz1lf]

Pdfplumber made me a hero 😂

[@2010edward1978]

at first glance it looks useful, but later on I got confused how you choose the items to observe. As for me some are useful and well known but some really very specific. Anyway I enjoyed watching the video.

[@davidemiani]

Why do you think that the "returns" approach is more valid than the standard python "exceptions" one?

[@ArjanCodes]

I don’t. Exceptions are built into Python and probably make more sense there. Monadic error handling has the advantage that you need to be explicit about errors that your functions can result in, which helps write code that’s more robust. But IMO you get the most out of it if it’s built into the language, like with Rust.

[@TheBprs]

mypy and pre-commit should be on that list imho.

[@edgeeffect]

Yes... if you want to use monadic errors in an exception based language, you're going to make a mess. I still remember when PHP moved from (what the hell could you call it?) "C style error handling"(?) to exceptions the mix-up of the two was pure chaos. Having tried to use monadic errors in JS and Python, it was that same chaos again.

[@jakobkristensen2390]

So Pipe is similar to streams in Java?

[@ondrejnovak339]

Which Rust ORM would you recommend? sea-orm?

[@ArjanCodes]

I haven't used an ORM yet in my Rust project. When I do, I'll share my favorite one :)

[@ac130kz]

Diesel

[@ayhamsaffar8407]

uv seems new but cool. Its trying to replace pip & conda & poetry with a single blazingly fast rust backend.

[@_balancy_]

It's cool, fast and weights less than poetry, but it's not the python library)

[@oleksiyyeryomenko205]

It will be interesting to force Arjan to review and compare uv vs pixi, both on rust 😂

[@DAHUME]

I would like to see a prototype game on pygame in the form of a corridor shooter similar to doom 3d or an rpg like fallout 1. Naturally, it should be just a prototype.

[@mayconfrr]

Try using a TypedDict with Box

[@SentinelaCosmica]

Thanks for your advice for data engineers to learn about sorting, quick searching, and merge sorting... now I am focused on map, reduce, filter, and functional programming, SQLAlchemy, Itertoos, functools, Relational Algebra, Math set theory.

[@Hacker-at-Large]

Yes, please, on pygame.

[@pudicio]

Video about optimizing performance and profiling

[@inhahe]

I actually wrote something that does the same thing as Box many years ago xD

[@mikeofs1304]

Rust have the Deisel, which can compete with Alchemy

[@edgeeffect]

At work, I'm trapped in the miserable existence that is PHP... we have ONE nice ORM like SQLAlchemy (that hardly anyone uses) and a bunch of others that are truly pestilential. The ORM haters might change their feelings coming from my world and then giving SQLAlchemy a try.

[@DaveParr]

18:15 totally hate box for that reason

[@evlezzz]

Using pickle to transfer data between "different systems" sounds _very_ dangerous. If code on both sides is not in sync, the result could be unpredictable.

[@edgeeffect]

It's a bit sad that many of these don't do nice type annotations.... it's something I'd like to see more package maintainers add... I suppose this is the price we pay for adding types to our languages so late... abandoning types back in the 1990s was such a huge mistake.

[@klmcwhirter]

An OR/M should never be a first choice - in any language. A business case must be raised to justify the need. In Python, please ALWAYS prefer SqlAlchemy Core. It has mostly the same feature set, without the downsides. OR/M's almost never give a robust production quality (performant) solution. I have encountered this truth in every language / OR/M I have used - C, Java, C#.NET, JS, etc. +1 for sympy - not perfect - but wow! Totally agree with the Python 2 pedigree of this package. Come on sympy project, please give us type hints. But it is really useful for prototyping the implementation complex formula and in some cases the series terms end up so large that a pure Python solution is just the most cost effective, risk averse approach to take. I do wish it could make use of GPU. But that will take more time ... pretty early yet. GPU libraries are still pretty raw and low-level (e.g., cupy).

[@Joao-pl6db]

I found out that pygame is great package to deal with audio.

[@pjmoran42]

Coolist... streamlit!

[@RagHelen]

Last time I tried, Beautiful Soup didn't have Xpath.

[@DrGreenGiant]

My fave package at the moment is "rich." It's fantastic for making very nice logging to stdout. Combines well with the logging library so you can have prettified stdout but simple logfiles.

[@yomajo]

Some people like ORMs, others like injections.

[@Soltaiyou]

I feel like having type checking switched off is like going commando. Not for everybody but so liberating! Sorry Arjan 😢

[@ramon421]

Circus Clown here, let's see how this goes

[@PaoloEmilioSerra]

Reflex for web dev supporting React

[@mydetlef]

Maybe we have to ask an AI to get the right template for Cookicutter.

[@alexanderpoplawski577]

It would be easier to ask the ai to setup the project.

[@mydetlef]

numrizer works only with english words. 😞

[@cetilly]

mypyc

[@serenditymuse]

That typing gets in the way of functional code and especially higher order functions is why I am glad it is optional and not in forced in python. If that changes then python will no longer be my goto language.

[@SkaChild]

uv

[@macerdough]

Its sooo good

[@pawew1060]

Streamlit

[@Mjhapp]

Yep. If I am building a tool for myself that seems easier to handle by building a UI vs command-line, I go to Streamlit.

[@macerdough]

It should be mandatory for a python library to have type annotations. Like i understand not wanting to use annotations in you're own projects, but this is code you intend to share with others. I have to turn off type analysis or set it to basic every time i am using a third party library cause people don't want to annotate their code. Like at least add a stub(.pyi) file. Its so annoying

[@JohnWalz97]

Rule of thumb: Never use pickle!

[@mattayres9748]

Where is your mouse