David Beazley - Python Concurrency From the Ground Up: LIVE! - PyCon 2015

[@hezzytheninja]

The fact that he walked in and did this demo from scratch while keeping the packed room engaged was incredible.

[@brefaccion]

and without syntax highlighting and completion õ.o

[@VietVuHunzter]

His talk are amazing. I was most impressed with the vault talk. This guy is Tom Hanks on an island, but instead of Wilson, he got Python!

[@cheaterman49]

@@VietVuHunzter It actually sort of happened to him, he mentioned how he was consulting for a bank or something, and they gave him a laptop with no internet or IDE or any sort of tools, but there was Python and a basic text editor IIRC.

[@kevindong9672]

@@cheaterman49 kzbin.info/www/bejne/iIuXhKFjjpx0hpo

[@kevindong9672]

@@cheaterman49 Actually it's proof discovery in a legal process, patent lawsuit or something

[@penciltwirler5829]

This is a performance by someone who has achieved the highest level of their craft.

[@stabgan]

Facts

[@ChristophZwerschke]

A fantastic, entertaining and highly educational talk. It always bothers me that I can't play the piano and talk at the same time (my wife usually asks me things while I'm playing). But David can even type concurrent Python code in Emacs in Allegro vivace speed and talk about it at the same time. An expert in concurrency in every sense of the word. How enviable!

[@crimsun7186]

+Christoph Zwerschke But when you're coding is easier. He's just talking what he's doing on the code, which is not that hard to do.

[@UniqueOneShot]

+Christoph Zwerschke Practice pair programming and it'll become easier

[@phisherPhisher]

his ability to explain his actions while programming fast and accurate without any real tool support (not even syntax highlighting) is pretty much amazing.

[@MrQsam]

watching this after 10years... still relevant knowledge!!!

[@Roycocup]

This presentation is legendary. Not only is David extraordinarily savvy on the inner workings of Python, but also I doubt that anybody can write code and present ideas AT THE SAME TIME! Sourcery!

[@SaiJeevan4]

dang.. I keep coming here every now and then ..such a brilliant talk !

[@chmod0644]

The Jimi Hendrix of Python

[@bobocsabin]

Good one ! :))

[@nareshgb1]

Hendirx + Robin Williams.

[@Spoons4Hands]

Actually he's more of a Bob Ross of Python I think

[@djanthony6662]

This guy is a legend.

[@KevinJingQiu]

Probably my all-time favourite conference talk :)

[@memoriasIT]

Video quality isn't the best, he's writing everything from scratch... But that doesn't even matter! His explanations are great and has the perfect balance between humor and educational. One of the best talks I've ever seen for real.

[@luckyboy20021]

A lot of wisdom here, to me, it take 3 times watching this to understand the fully concept of concurrency, nice presentation with useful practice, thank a lot.

[@Mindoza94]

This speech is actual even for today! The best explanation of async programming it Python I've ever seen

[@joseville]

27:06, i've never seen a list (albeit empty) on the left side of an assigment statement. What does it do? The simpler `[] = 1` throws `TypeError: 'int' object is not iterable`. Oh ok, 42:17, it was a bug/typo, but somehow it still worked. Btw, awesome talk. He's a great speaker.

[@djl3009]

Thank you! This brilliant talk/live-code-session finally answered all my unanswered questions about concurrency and coroutines in Python. This is absolutely the most legendary live coding with concurrent explanation, I've ever seen.

[@cenntraru]

This talk made me move from Python to Golang. Thank you David! 🙂

[@gaatutube]

I would probably have to watch this 20 times over to really understand most (if not all) of his demo. It's like watching a Ninja hack away at the keyboard.

[@danielstromberg]

Using coroutines in this way is an example of "cooperative multitasking"; if all your code yields the processor when it should, it can work quite well, but a single uncooperative code element can make a mess of things.

[@asdfasdfasdf383]

This is a phenomenal performance. I learned a lot

[@shanee7511]

David Beasley's videos always make ideas so much clearer for me. Such a great teacher!

[@kapucko1]

the best talk I've seen so far

[@mtp1376]

PyCon 2015, where I actually first learned coding in Python and started loving it!

[@taiminyang4236]

Amazing, the best and smoothest live coding demo I have ever seen!

[@aaronhall8039]

At 6:16, I'm wondering what he wrote there, where it is blocked by the inner video frame.

[@ashwch]

Aaron Hall Its Thread(target=fib_handler, args=(client,), daemon=True).start() :P github.com/dabeaz/concurrencylive/blob/master/server.py

[@aaronhall8039]

Ashwini Chaudhary I just realized I never thanked you. Thanks. :D

[@dukiwave]

God bless you, man

[@FullMetal-nu8bj]

This is so interesting, David is informative and hilarious at the same time. I didn't know python supports threading in this way! Awesome!

[@terminalrecluse]

Dave is one of the best presenters of all time

[@bbruvey]

stackoverflow.com/questions/30147165/why-does-assigning-to-an-empty-list-e-g-raise-no-error if you want to know why 42:30 works

[@solomonxie5157]

"You know, every great talk, is started with Socket Programming."

[@rockapedra1130]

Super simple and well explained! Excellent presentation!

[@landwarrior348]

David is Mozart of Python world.

[@aziz9488]

Great talk, well that's what I call an experienced python programmer!

[@HimothyOHooligan]

Probably should've gained up the audio a bit. I have my volume maxed out to be able to hear him at a nominal level.

[@randixlai9216]

The only thing I learn from the video is he's legend.

[@yomajo]

Zero experience in sockets and concurrency in Python. I could not even come up with threaded performance monitor let alone all tne rest. Incredible.

[@lokeshagrawal8240]

What is the code line written to watch a thread at 6:16?

[@Alexnotjones]

have the same question. it works fine with ... args=(client,)).start()

[@GodefroyClair]

Someone understands why he says ""the load is essentially offloaded somewhere else, you're not fighting with the GIL or anything like that..." 15:48 ? kzbin.info/www/bejne/g3TWZoKsnbtsiZY The GIL is still limiting the task that are sent to the pool of threads right? Where are those task submitted otherwise ?

[@chaoticguud]

If you are trying to code along and are getting socket already in use when you are submitting a job to the process pool, put fib_server(('', 25000)) in server.py in a `if __name__ == '__main__'`

[@watercollector]

Thanks a lot! I just got this error and deliberately searched in the comments if somebody already published the answer. Thanks!

[@danielstromberg]

yield from was introduced in CPython 3.3.

[@JuanPabloReyesSepulveda]

Awesome Talk. Impressive.

[@ruipeng5515]

Great Talk! Learned a lot from it!

[@birrafondaio]

Values unpacking works on lists as well: >>> def returns_a_list_of_lists(): >>> return 1, [2, 3, 4], [5, 6, 7] >>> a, [b, c, d], [e, f, g] = returns_a_list_of_lists() >>> a == 1 True >>> g == 7 True The third and last argument must have been an empty list.

[@goshinbi44]

I think he might actually be one of those mythical 10x developers lol.

[@muntoonxt]

I think that's a ridiculous myth -- that the 10x developer is a myth. Every field has people that are incompetent, people that are average, and people that are exceptional. The average people do important work. But exceptional folks are capable of generating solutions that are simply 10x better than what an "average person" would ever have come up with.

[@narayanbala995]

Commenting a bit late. Agree that this is a spectacular talk. Has anybody tried the multithreading code - the first example? I can never get 2 clients working at the same time. The server responds to only one client.

[@ashwch]

Code: github.com/dabeaz/concurrencylive/

[@filiphron3147]

+Ashwini Chaudhary Thanks

[@FlavioCoelhoC]

Amazing talk. for pythonistas, of course...

[@bunc11]

GODLIKE!!!!

[@antonkot6250]

7 years ago, omg. now it's mainsteam anywhere. NodeJS, Kotlin, Spring Reactor

[@kukas1043]

Would it be bad practice (or helpful?) to write fib() as C module which releases the GIL? How much would that add overhead?

[@Thebarbiturat]

No slides at speakerdeck and 404 for github page =(

[@medtiw7457]

Anyone tried this on windows and got some different behavior? (eg: the requests per second didn't drop when I asked the server to compute the fib number of 35 like in: @11:18)

[@medtiw7457]

On the other hand the perf1.py test is affected by the computation of the fib number of 35

[@vrajeshkumar]

Awesome talk!

[@randyt700]

is there a screencast of this available like his talk "built-in super heroes" ?

[@De4sher]

turns out even in python 2.7, you can do this: `[] = []` BUT `{} = {}` will not work.... says it's a SyntaxError "can't assign to literal". What do you know?

[@Water_Soul]

There David talking about CPU process, but will it works with process which use GPU if we have only one video card and something what take almost all GPU's memory? Or better wrap this process with threading.Lock()?

I feel sorry for the 5 people who 👎 this talk.

[@HelioMeiraLins]

Still not old in 2022.

[@cypherphage4294]

my mind got destroyed way before that

[@stoneshou]

I'm not sure I've got the point. It was a great talk and I think I've learned something new about threading and async io, but the two problems being presented: 1.performance impact due to the GIL, 2. performance cliff due to python prioritizing CPU intense jobs, was there a hinted solution to any of these problems? Especially for the second problem, all I learned from the video was by combining async io with ProcessPool, we can achieve similar performance cliff as the threading approach. But the performance cliff is still here?

[@chris8443]

Right, his point was that there are no easy solutions to those problems and that asyncio doesn't make them go away.

[@prateemmandal743]

I think the take away is that when there is massive number of concurrent tasks, co-routines is the way to go - you can mix in multiprocessing to ensure all cores get used - Posix threads are not going to scale for such requirements.

[@b7smart926]

where his writen code could be found

[@kodampradeep]

I had to watch twice to understand what he is typing and telling at the same time. Lol . How can we become like him :-(

[@Mahorir]

Frigging amazing!

[@tetamusha]

When I first had to implement a small distributed processing program in Python, I skipped Threads and went right to the multiprocessing package, which actually creates different processes for each object you start. This behavior is much more on par with the behavior of other languages and what is expected from a concurrency perspective. After all, why would you implement concurrent workers if you can't take advantage of more processing power? Might as well do a script that works sequentially.

[@marcalanstreeter]

Douglas De Rizzo Meneghetti Because sometimes what you're doing doesn't require more processing power just more processing time. This especially includes IO work where the processor gets tied up waiting for a response and is doing nothing in the meantime. Now imagine you had to make multiple IO connections each waiting on the other. In cases like these it is better to wait on all concurrently, and that's why it's important to understand the nature of your program. Is it IO bound or CPU bound? There are other tradeoffs to consider and obviously from watching this talk it could be that neither multiprocessing or threads are necessary or desirable in the end.

[@kevindong9672]

Even with GIL, multi thread is ok for IO-bound tasks.

[@林玮-k4c]

Great talk

[@chenlonglee9511]

where I can download code?

[@HybridIntuition]

Teclado brought me here!!

[@abdelrhmanahmed1378]

your future_wait will grow very fast because we dont clear it i think its better to do this 'def future_done(future): tasks.append(future_wait.pop(future)) future_notify.send(b'x')

[@jmoz]

The De Niro & Pacino of Python.

[@BGSone]

Best talk !!!

[@mehdisaffar]

it bothers me when people dont use syntax highlighting..

[@jriggatx]

What he said at 43:46. Thanks!

[@Spoons4Hands]

this guy pythons!

[@lsslock4856]

why not yield future? where from about if what equal future?

[@michaelwu9238]

nice

[@eotfofiw2748]

TL;DR; Wouldn't it be good if Python had a concurrency model like Node.

[@human-011]

Python docs have very poor asyncio documentation #python_improve_asyncio_docs

[@FirstNameLastName-fv4eu]

This is called "intellectually"

[@vishcdac07]

Tom Hanks!

[@Acid31337]

that's why some switched to go

[@linan714]

Amazing talk, I really enjoy it. The only problem I have with it is he and the python community pretend NodeJS not exists. Isn't the non-blocking + callbacks the whole idea of nodejs?

[@zacharynathoo2537]

node cannot really handle multi core and so is not great, Erlang, go are much better

[@stabgan]

Too complicated for me

[@mickavelimckenzy3689]

oh God! I don't know SHIT in python

[@bradleyrintoul2644]

This is pretty great, but just the fact that threads are still a "hot topic" in Python gives me pause... It's 2016.

[@rustytrd8245]

Thanks david for amazing talk. For steps go to: github.com/codeAshu/concurrency

[@ttrss]

goodbye GIL

[@artintelligence2682]

Down is stupid, even less opened emacs editor. Large font and on the screen 10 lines only visible, constantly scrolls down/top

[@sindarin1350]

Can't handle the spit noises arg

[@AlqGo]

Spaghetti code

[@luqmansen]

Watching this in 2024, amazing talk 🫡👏👏