8 things in Python you didn't realize are descriptors

Descriptors are everywhere in Python, but you may not have realized it.
They underlie functions, properties, staticmethods, and more! In this video we take a look at 8 Python descriptors and explain the descriptor protocol.
― mCoding with James Murphy (mcoding.io)
Source code: github.com/mCodingLLC/VideosS...
descriptor docs: docs.python.org/3/howto/descr...
staticmethods vid: • Python staticmethod an...
super vid: • super/MRO, Python's mo...
slots vid: • Python __slots__ and o...
SUPPORT ME ⭐
---------------------------------------------------
Patreon: / mcoding
Paypal: www.paypal.com/donate/?hosted...
Other donations: mcoding.io/donate
Top patrons and donors: Jameson, Laura M, Dragos C, Vahnekie, Neel R, Matt R, Johan A, Casey G, John Martin, Mutual Information
BE ACTIVE IN MY COMMUNITY 😄
---------------------------------------------------
Discord: / discord
Github: github.com/mCodingLLC/
Reddit: / mcoding
Facebook: / james.mcoding
CHAPTERS
---------------------------------------------------
0:00 Intro
0:17 What are descriptors?
2:08 #1 functions
3:10 #2 properties
5:27 #3 classmethods and staticmethods
6:47 #4 slots
7:42 #5 _dict_
8:30 #6 SQLAlchemy models
9:06 #7 validators
10:16 #8 super lookups
10:46 Interactions with _getattr_
14:02 Thanks

[@moneyluser5711]

One of the most useful videos in existence for understanding python objects more deeply. The delivery is so good. Only what you need, no time wasted.

[@xan1716]

Another example is cached_property: it's implemented as a non-data descriptor. It is accessed on the first call; the result is then computed and saved into the instance dictionary. Since the instance dictionary has greater priority than a non-data descriptor, successive lookups just return the value from the instance dictionary.

[@purevert]

Sometimes it feels like he can read my mind and make tutorials on things I will need. Thanks for always making tutorials that are so detailed.

[@redmastern576]

KZbin algorithms and your interest are tuned to this. He’s making these videos, KZbin Ml just knows you too well.

[@glorytoarstotzka330]

amazing timing, literally the other day I spent quite a while to read the whole python official docs on descriptors. often times I see the "I was just needing this, thanks for the tutorial", but now I can finally relate to this kind of moment

[@Veptis]

The more videos I watch on your channel the more I learn. But I also feel more and more alien to using python for 3+ years and not ever needing or noticing most of the stuff.

[@isaacingleby8771]

Most of the stuff on this channel you shouldn't be doing in my opinion. I watch this channel to patch some holes in my understanding of what may be happening under the hood, but applying this knowledge to production code is a bad idea. Most of it is just complication for complications sake. Object attribute setting/getting described in this video is something most devs will need to tweak at some point. However, if a dev in my team tried to do most of what this guy shows off I would tell them in no uncertain terms to rewrite the code. Such a dev is creating problems down the line that will have to be picked up by some sorry soul for some wothless brownie points from less experienced devs who don't have the experience/confidence in Python to call out BS when they see it.

[@japedr]

Awesome. Finally I got to understand the descriptor protocol magic. Thank you so much.

[@loere]

it took me 2 years to finally find a python video that still teached me stuff

[@rb1471]

This video made me realize that despite trying to learn more python through your videos, there's still a tone I'll never get lol. Me: "How does this work?" Video "Oh I made a video on that" Repeat x10

[@PanicAtTehCisco]

Numpy also uses this for matrix transposition when you do m.T, implemented as a property

[@gregorymorse8423]

This is valuable content. Learning more advanced technical nuanced aspects of Python is very interesting and useful 👍

[@dennismuller1141]

everytime I watch one of your videos I feel like I know nothing about python xD

[@Joooooooooooosh]

This would be good for some of the things you'd typically use implicit conversions for in C#. Like OrderNumber -> String.

[@sharkinahat]

dict._ _dict_ _['_ _dict_ _'] isn't a dict? Mind blown.

[@JustRollin]

No need to be a dict about it. Sorry, I'm a dicted to puns.

[@aaronm6675]

This was good to hear

[@MrDowntemp0]

Is it possible to use the descriptors for field validation in a class that uses slots?

[@AntonioZL]

The python master!

[@oida10000]

Also with @property.setter you have the advantages of a proper setter method, which might does some checks without having to call a setter method directly.

[@royler8848]

Amazing video

[@xan1716]

I'm not sure 12:40 is entirely correct: if you have `class Foo: d = Desc()` where `Desc` is some descriptor class that for example always returns a constant (say, 42), and if we further have `class Bar(Foo): d = 12` for instance, then `b = Bar(); print(b.d)` prints the class var, not the non-data descriptor, since it precedes it in the mro.

[@mCoding]

In your example you have overwritten the descriptor with the value 12, which is not a descriptor, so the logic goes "12 not a data descriptor, next. instance variable not found in dunder dict, next. 12 is not a non-data descriptor, next. return the class variable 12. The first step in the process is always to look up the class variable in MRO, but that variable is only _returned_ if it is not a data descriptor, nothing was found in the object's dict, and it is not a non-data descriptor. Here is the official doc's pure Python implementation where you can see the series of fallbacks (docs.python.org/3/howto/descriptor.html?highlight=attribute#invocation-from-an-instance). Hopefully this explains it better than I did.

[@xan1716]

@@mCoding oh I see (hadn't looked this up in the docs before -- thanks for the link!). So basically, i) only the first hit in the mro counts, ii) the order is: data descriptor > instance dict > whatever else. If "whatever else" happens to be a descriptor (necessarily non-data) invoke it, else just return it (in which case it's just a class variable)

[@zefciu]

I still don’t get, why staticmethod is a descriptor. After all defining a method as static is about preventing its binding. So it shouldn’t require any special access logic. Could you explain this more?

[@alexandergoryunov8021]

Umm, I don't get the per-class vs per-attr distinction. The get/set methods are also defined on an instance, so you can get the exact same behavior by defining __getattr__ on the Descriptor class instead of __get__.

[@nnirr1]

`_getattr_` wouldn't work that way when defined on the descriptor itself. When i call instance.descriptor, only the instance's `_getattr_` will be (potentially!) called and will return the instance of the descriptor itself. In order to call the descriptor's `_getattr_` i would have to do something like instance.descriptor.x. However, by defining `_get_` on the descriptor itself, python will do that extra step and instead of just returning the descriptor, it would invoke the descriptor's _get_.

[@alexandergoryunov8021]

@@nnirr1 Oh, okay - that makes sense. It's still too niche for this late at night though ) For instance the fact that both _get_ and _set_ appear to work on an instance of the class but only _get_ seems to work on the class itself.

[@nnirr1]

@@alexandergoryunov8021 Hmmm didnt think about it but it it IS kinda inconsistent in that way. I can see why they did that but i can also understand why it is confusing as well

[@lawrencedoliveiro9104]

Remember, every function is a descriptor -- it has a ‗‗get‗‗ method, but no ‗‗set‗‗, ‗‗delete‗‗ or ‗‗set‗name‗‗, which makes it a non-data descriptor.

[@marcomaiocchi5808]

I thought I knew python till I saw this video. Goddamnit.

[@HitBoxMaster]

What font are you using?

[@gloweye]

In `__getattribute__`, in the (very rare) case I use it, i'd use super().__getattribute__ instead of object.__getattribute__ Ugh, youtube comments format everything weird and I can't get it right. But I think my meaning comes through.

[@osolomons]

Petition for markdown comment formatting!

[@d00dEEE]

@@osolomons Yeah, I'd +1 that except that youtoob comments are a black hole and after you've made some effort towards intelligent content and formatted it nicely, only three people will ever see it. Better to have people like James have their own forums/discourse/whatever, where we can go comment and have those comments discoverable.

[@lawrencedoliveiro9104]

‗try‗ using ‗this‗ character instead. Best I can do ...

[@anon_y_mousse]

@@osolomons They actually do to a tiny extent. If I use an *asterisk* around a word and the content creator has it enabled, it'll make it bold, for instance.

[@CrapE_DM]

I think the reason for data and non-data descriptors actually has more to do with being able to override functions on an instance rather than the caching idea, but it's all speculation.

[@__mrmino__]

No, you cannot just use function calls in places of properties: imagine having an instance field in your public API. Now try to make that field dynamic - if not for properties, you couldn't do so without breaking backwards compatibility.

[@sadhlife]

if you want your api to always be 100% backwards compatible without properties, you could just make all the attributes private and define a get_x() for every self._x attribute. But yeah, properties are way cleaner

[@__mrmino__]

@@sadhlife yes, hence all of the getters and setters boilerplate in Java, which, thank god, Python developers avoided.

[@init1508]

Cool

[@GertCuykens]

just when I think I start to understand python, youtube is saying hold my beer ...

[@JusticeNDOU]

hold on, what are descriptors ?

[@l4luchi123]

Excellent video as always! How about a video about slices? Especially for Numpy they play a big role, e.g. for numpy.roll. ;-)

[@anthonydesouza9983]

Double Dundee Dict is my stripper name

[@trag1czny]

discord gang 🤙🤙

[@bereck7735]

aye tragic

[@bereck7735]

You are cute

[@trag1czny]

@@bereck7735 no u

[@bereck7735]

​@@trag1czny cutie, how are you ;)

[@dotmashrc]

i didn't need to see this today

[@rosscousens6263]

Please write a book!

[@smb1397]

can i ask you why you pronounce tuple like that? i wanna do it like that but google says im wrong i need some sources to show my friends

[@drdca8263]

I’ve heard it both ways

[@isodoubIet]

I mean it's not spelled tupple

[@flicko0]

not discord gang

[@bereck7735]

Flicko, tragic is trag1c . ;)

[@tapu_]

We meet again

[@flicko0]

We do meet once more

[@LambdaJack]

Almost C++.

[@nnirr1]

Im writing a pickling library and when testing its behaviour with descriptors i accidentally found a bug within python's core pickle library. The c implemented classmethod descriptors (such as datetime.__dict__["now"]) were pickled into a wrong set of instructions that invoke the descriptors, returning the underlying value upon unpickling (instead of the descriptor itself)

[@Daniel_Zhu_a6f]

as somebody who programs python i wish descriptors, custom attributes, setters and getters never existed. practice proves that they are very tricky to get right. much like python's inheritance and exception systems, they are just a terrible mistake. after all, if you really need an object with variable amount of attributes, you probably just need something else (like a dictionary). it's not normal when literally any expression you write can do anything: thread suspension, caching, IO, process spawn, arbitrary flow control jumps (when throwing exceptions). it would be good to have some language that is 100% compatible with python but optimized for immutability of objects and types, with interfaces instead of inheritance and with normal metaprogramming techniques like templates and sanitized macros.

[@Daniel_Zhu_a6f]

@@a2sbestos768 kotlin is quite different from java, but they are 100% compatible. same with zig and c. why can't there exist a language that has comptime-immutable types and is compiled to same bytecode as python?

[@naghs2726]

when i started coding I loved your python videos but now that ive learned more languages i wish you would do videos on more of them instead of just python like C++ and Java

[@scopedealer]

Can someone pls help me? Just reply to this comment. So I'm only 11 and I just got the python book and I'm learning if statements and I've made it all but I don't know how to run it or whatever for example, if a made a code looking like this: if mood == tired Hit_snooze_button = true and then I do print (Bob is tired. He hits the snooze button.) And then u press enter and nothing happens, it doesn't say correct or incorrect, it doesn't even give me a error or anything. So can anyone help me with this pls?

[@lawrencedoliveiro9104]

Try Jupyter. It’s probably the easiest way to learn Python.

[@scopedealer]

@@lawrencedoliveiro9104 ok I'll try it thank you

[@NostraDavid2]

If you're already using vscode, add a line with "# %%" to a .py file to create a jupyter cell in text more. I like that more then creating a jupyter notebook. The you can use shift-enter and ctrl-enter to run a cell (just experiment with it)

[@lawrencedoliveiro9104]

@@scopedealer And don’t accept poor substitutes.

[@Jaime.02]

gang 🤙discord

[@bereck7735]

Hello mithic

[@Jaime.02]

@@bereck7735 i'm not mithic xD

[@bereck7735]

​@@Jaime.02 oh my bad, your pgp kinda reminded me of him.

[@vaulttectradingco8438]

Discord Gang

[@anon_y_mousse]

Did you copy the python executable into your PyCharm projects folder? Does it not respect the PATH variable or are you just being weird?

[@mCoding]

As is standard industry practice, I use virtual environments so that my different Python projects with different dependency requirements do not affect each other. See docs.python.org/3/library/venv.html for more information about virtual environments.

[@p_square]

discord gang but late

[@fuexfollets5755]

Discord gang

[@lior_haddad]

🤔

[@SkyFly19853]

Pythonformers... Here they come... 🤔😏

[@noahcollin9532]

Coming from C#, it baffles me that getters and setters aren't used more in python

[@kdaubtf2]

Disgusting boilerplate 🤮

[@__mrmino__]

Why would you want them? What purpose does a getter on a simple instance field serve?

[@noahcollin9532]

@@__mrmino__ for getters in particular in python, I'd imagine it's useful in teams. A get method could return all customer information except payment, unless it's coming from a paymentInfo class instance. So only certain objects could get certain attributes. And one coder can be assured that other classes don't get information they shouldn't. I'm honestly not working in production python, so I don't know how, if at all, they're used in industry. But it's a central tenant of C# on top of public and private objects and classes.

[@__mrmino__]

@@noahcollin9532 you just described one of the usecases of @property :).

[@noahcollin9532]

@@__mrmino__ wow I've learned something from a KZbin comment. I thought the comments section was only a place to scream into the void. Thanks!

[@i007c]

969 LOL :)))

[@timseguine2]

Double dunder dict? I hardly know her

[@markcuello5]

HELP

[@hanabimock5193]

All this weird stuff just to achieve what a typed compiled language already achieves way faster, way simpler. Python is great for prototyping but if you need to do stuff performant and handle abstraction better, just use something else.

[@bunchofcells3333]

Discord Gang

[@bereck7735]

Heya, fellow rustacean, try out samarium ;)

[@saveuyghurmuslims2354]

its not tech with tim its mcoding bruh

[@markcuello5]

HELP

[@lunarmagpie4305]

Discord Gang