Building A Custom Context Manager In Python: A Closer Look

Context managers in Python allow you to robustly control setup and teardown of resources. In this video, I'll show you how to use them and I'll cover several ways to create your own.
The code I worked on in this video is available here: github.com/ArjanCodes/2022-co....
💡 Get my FREE 7-step guide to help you consistently design great software: arjancodes.com/designguide.
💻 ArjanCodes Blog: www.arjancodes.com/blog
🎓 Courses:
The Software Designer Mindset: www.arjancodes.com/mindset
The Software Designer Mindset Team Packages: www.arjancodes.com/sas
The Software Architect Mindset: Pre-register now! www.arjancodes.com/architect
Next Level Python: Become a Python Expert: www.arjancodes.com/next-level...
The 30-Day Design Challenge: www.arjancodes.com/30ddc
🛒 GEAR & RECOMMENDED BOOKS: kit.co/arjancodes.
👍 If you enjoyed this content, give this video a like. If you want to watch more of my upcoming videos, consider subscribing to my channel!
💬 Discord: discord.arjan.codes
🐦Twitter: / arjancodes
🌍LinkedIn: / arjancodes
🕵Facebook: / arjancodes
👀 Code reviewers:
- Yoriz
- Ryan Laursen
- Sybren A. Stüvel
- Dale Hagglund
🎥 Video edited by Mark Bacskai: / bacskaimark
🔖 Chapters:
0:00 Intro
0:53 Explaining the example
2:15 What are context managers?
3:42 Creating a context manager using a class
6:40 Creating a context manager using a decorator
9:37 Asynchronous context managers
10:07 Aiosqlite
13:50 Final thoughts
#arjancodes #softwaredesign #python
DISCLAIMER - The links in this description might be affiliate links. If you purchase a product or service through one of those links, I may receive a small commission. There is no additional charge to you. Thanks for supporting my channel so I can continue to provide you with free content each week!

[@ArjanCodes]

💡 Get my FREE 7-step guide to help you consistently design great software: arjancodes.com/designguide.

[@HeyThisIsAndrew]

It’s like your reading my mind. I wish I had a channel this good to help me improve with my understanding of Git CI/CD workflows and pipelines.

[@ArjanCodes]

Glad you're enjoying it! I'll probably cover topics like this in the future (I'm working on a more generic DevOps video at the moment).

[@LuvMyChemicalRomanc3]

I feel the same way, Git CI/CD workflows and documentation is not always beginner friendly

[@bigdaddy5303]

Really glad I came across this channel. Been dabbling in python for about 4 months and have made some really good progress. Feel confident I can create everything I want, but I'm a chemical engineer, not a coder....but I want to create good habits and make stuff that a coder will not roll their eyes reading. This is the first channel that just skips the same repetitive basic coding stuff 99% of channels focus on and just focuses on being a smart coder. And it's all delivered so well. I find I learn more in a 10 minute video from you than some of these bigger channels don't even do in a 2 hour video. Plus your fast typing is so soothing lol.

[@peterwelkenbach5902]

i totally agree :-)

[@AustinWitherspoon]

I knew about the class version of a context manager- but never knew about the function decorator version! Super cool.

[@djl3009]

Nicely done! concise and thorough. I looked into custom context managers a few months ago and like many, settled on the class version, but after this video I think I'll be leaning towards the decorator.

[@vasilijestosic8331]

Today I actually worked with custom context manager, dealing with FTP connections. This video is really good for deeper understanding of context mangers. Well done! 👏👏

[@golammuhaimeen2825]

Soo good man! Great stuff! Love it!

[@1337Hammersmith]

It's amusing that this was the exact thing i stopped on this week, and thought, there has to be an easier way to manage contexts :) Insightful and well paced as always

[@jordansilke3629]

Helpful video with clear, practical examples as always!

[@ArjanCodes]

Glad it was helpful!

[@davidbundgaard]

Interesting stuff, where was this when I needed this. Moved on to other languages for my app development.

[@graveetone]

thank you for covering this topic, specially for the async part of working with context managers!

[@ArjanCodes]

I'm glad you found the video helpful!

[@xaco56]

Thanks for a fine explanation of context managers. A small quibble: I don't think "with" makes a new scope, because I have code that assigns a buffer to a new variable within a "with" and then accesses the variable outside of the "with" just fine.

[@ibrahimtprdk6622]

I downloaded the document and didn't even realize I finished it in one go. Thank you for your thoughts and valuable information

[@ArjanCodes]

Thanks so much Ibrahim, glad the content is helpful!

[@kemalcihanbastak8476]

Since you used yield function in connection part, can you make a video about when to use yield instead of return

[@ArjanCodes]

Thanks for the suggestion!

[@pranavnyavanandi9710]

Yes generators, iterators and descriptors, I have a hard time grasping them.

[@AlpNixTech]

Cool! It's nice to know there are much more uses cases of context managers than opening and closing files 😅

[@ishtiaquehussain]

Learned something new, as always! Thank you! (btw, also learned how to pronounce your name, I was wrong the whole time) :D

[@rakesha9176]

Your content is excellent and the quality of the videos are Top Notch. You deserve alot of views and subs ❣️🔥

[@ArjanCodes]

Thank you so much 😄

[@j.reddick5585]

Man! Every clip always has some depths! Love all the series! BTW, is there any chance that we can see UNCLE BOB explains how the INFAMOUS thread works in Python? lol

[@pranavnyavanandi9710]

Yes, about the GIL and python not supporting multithreading.

[@tobiastriesch3736]

Cool video! I learn something new every time :) I also have a suggested topic/question: What's your take on normal SQL queries vs object-relational mappings, e.g. using SQLalchemy? Do or do not? How can I make best use of context managers here?

[@joaquimt.duarte2538]

I think these days I feel like trying we will have the best transpilers with AI. Like you write it in a simple library like tortoise orm or Django and it turns into more native and faster code.

[@virtualraider]

Love the video! Just a small nitpick: I wouldn't put the commit inside finally, because you don't want to save things in a dirty state. I would put it on the else condition, or better yet, in a transaction context manager 🤓

[@ArjanCodes]

Good point!

[@rrwoodyt]

Very timely video, considering what I'm working on today. I actually prefer the class-based context manager to the decorator version, though that is probably because I haven't used generators that much. Anyway, nice work on the video.

[@felipealvarez1982]

Hi Roy. I don't think you need to 'understand' the generator function. They way you _use_ it does not change. It is an implementation detail which can be ignored.

[@rrwoodyt]

@@felipealvarez1982 You are absolutely right. As far as the implementation goes though, I find the class-based version more intuitive, and that's what I would do. I definitely appreciate the elegance of the generator version.

[@miguelvasquez9849]

I already knew this, but what a good video

[@ArjanCodes]

Thank you Miguel, glad you liked the video!

[@qbrteam6398]

I would love to see a real life implementation of the hexagonal architecture using python. I haven't been able to see anything similar on KZbin.

[@siriusraja]

Hi Arjan, How do we implement connection pooling in this case? Do we need to move the connection creation line outside of the function?

[@temik26]

Thanks for videos Arjan, they're helpful. First time I got acquainted with context managers is when I read Python Tricks book by Dan Bader

[@ArjanCodes]

Thank you - glad you like the videos!

[@monooo]

Nice video

[@manonthedollar]

In production code, would you put the entire `with` block inside a `try` block for cases where a connection can't be made (maybe application.db can't be opened and so the `finally` block would throw an exception too)? That's where I start to lose confidence in my design, because especially when I'm doing more than just a couple of things with an open db connection or file handle, most of my code is indented several levels inside some sort of massive function->try->with structure and it "feels" wrong but I don't know what I could be doing better.

[@felipealvarez1982]

Ajan codes discord channel can help. Share your code examples there 😊

[@lawrencedoliveiro9104]

I try to design my cleanup code to never throw an error. So for example if a connection was never opened, then the connection-closing logic should automatically become a noop in this case. That helps to keep things simple.

[@zenmaster24]

do any of the ways shown have performance repercussions? would one way be more performant than another?

[@tuverentetil]

Hi Arjan, can you also add snippets how to test code you're writing ? Seems that test are largely overlooked i.e i'd like to learn how to use pytest with context manager.

[@ipelezikis]

This is a video I was looking for! A question. Why commit is included in a finally block and not in an else block. Is there a certain logic on why one is prefered than the other? Thanks in advance

[@thejedijohn]

Great video. Loved the much steadier camera angles. And I learned about the context manager decorator. I didn't know we could write our own context managers without explicitly writing the dunder functions. Thanks!

[@ali-om4uv]

Super helpfull video! I have hower some problems putting this to an actual usage. Lets say i want to have a databasemanager class that that handles all conection. Can create a new db on a server create new tables on existing dbs.....delete those . Further more it should be able to execute parameterized querrys were the querry is a variable and so on .... How would something like this been done professionally? Especially if we are dealing with a remote lets say sql server instance?

[@rahulprasad2318]

Tbh, i like the class version better.

[@flightmansam]

I like your style

[@ade_niko]

What will be the return type for open_db method in this case?

[@kopytko998]

I have impression you are looking into my daily activity and send me suggestions how to improve my code :) Thank you

[@nateriver8261]

Make please a video about descriptors and their role in a python ecosystem

[@ShlomoGottlieb]

Video idea: Python coroutines and how it's used with 'yield' and 'yield from'. No matter how many times I look it up, I never manage to wrap my head around it.

[@poncelas91]

Great video as always. It would be nice if you make a video explaining generators and coroutines in detailed, just as an idea 😊

[@lawrencedoliveiro9104]

I sum this up in something I call Van Rossum’s Triangle:

[@lawrencedoliveiro9104]

ww

[@lawrencedoliveiro9104]

w.

[@lawrencedoliveiro9104]

devi

[@lawrencedoliveiro9104]

antart.

[@Khushpich]

Cool video and nice beard Arjan

[@ArjanCodes]

Haha, thanks!

[@aadithyavarma]

At min 9:00, how does the try work with only a finally and without an except. Shouldn't you always have a except for the try block?

[@DrGreenGiant]

Is there a way of using a context managed object at class scope rather than function scope. For example, acquiring the resource in __init__ then releasing it when the object gets destroyed. Hypothetical example; a logger class that opens a file when it is created, closes when destroyed. For context, I have a very performant function which extracts data from an FPGA, does some maths then needs to be logged. I don't like the idea of having to do a "with open(fn) as f" in every single loop for performance. (This is running at 2kHz and the "while" isn't even in this function contained by this class)

[@DmitryKorDimKorzunin]

What is the code hinting when you start the coding, is it some vscode extension?

[@ArjanCodes]

I’m using GitHub Copilot here for code suggestions.

[@brianpack369]

Using non invisible characters to define code blocks? We've gone full circle, everyone.

[@stefans8632]

Hi there, could you please me do download and install these 2 packages logging and sqlite3 ? Beause i tried from pycharm and comand prompt but without success : pip install logging ERROR: Could not find a version that satisfies the requirement logging (from versions: 0.4.9.6) ERROR: No matching distribution found for logging pip install sqlite3 ERROR: Could not find a version that satisfies the requirement sqlite3 (from versions: none) ERROR: No matching distribution found for sqlite3 Thanks,

[@channalbert]

I might be missing the point here, but what is the difference between putting “connection.close()” after a try-catch clause and adding it inside “finally”?

[@estebanartavia7827]

If you put it outside and your code raises an exception, your connection will be still open depending on the error because your execution might finish before touching that line. Finally executes regardless of a failure or exception.

[@channalbert]

@@estebanartavia7827 wow! I knew there had to be a reason. Thank you.

[@richard1598]

A bit out of context (no pun intended ;-): why I need to declare a function explicitely as "async" (as in async def main()" when using an async-y keyword or calling some async function makes it cristal clear, that the function itself is per definition async? Saw this in multiple languages already and always asked myself "wth" is this boilerplate for?

[@maroben225]

How the fuck i did not fi'd your channel earlier !!! Man my youtube is all about tech and python stuff how come never found you !!! Please try promoting ur channel somehow I am learning great stuff feom you especially we are hungry for more design patterns and new python stuff. Also of you can make videos about driven events in python...or probably also covering fastapi with its pydantic models and dive deeper ... You can go deeper in dependency injection and pydantic models sqlchemy ...please consider this. And like man seriously im sooo fucking happy fouding a senior like you 🎉🎉🎉🎉🎉

[@ArjanCodes]

Thanks so much! Glad that you’re here and thanks for the suggestion!

[@farazahmed1668]

Why we need to close the database connection every time?

[@ArjanCodes]

Databases will have a maximum number of connections. If you don't close the connection, the number of open connections just goes up and the database won't accept new connections once you reach the maximum and then your application stops working.

[@farazahmed1668]

​@@ArjanCodeswow its so simple. so that's why we need to close the connection every time. thank you MR. Arjan.

[@ArjanCodes]

You’re welcome!

[@csoviii]

afaik with statement doesn't create a scope

[@danielcrompton7818]

I still don’t get why you yield not return…

[@ekebro]

I just have to protest. SQLute is an objectively better name for the class. When in doubt, go for the musical alternative.

[@ArjanCodes]

Haha, I was definitely fighting with my keyboard there 😂

[@quillaja]

@@ArjanCodes I appreciate watching you mess up typing in your videos. It makes me feel like I'm not the only fat fingered keyboard klutz programmer.

[@nathantoulbert4406]

I really hope I'm a seagull in my next life. Just imagine living life as the shitter, and not the shittee!!!

[@k.s.chaithanya121]

First comment

[@ArjanCodes]

Well done :)

[@art4eigen93]

Tapped in for a seagull manager video but totally disappointed to see no seagulls. Disliked :p