David Beazley - Python Concurrency From the Ground Up: LIVE! - PyCon 2015

[@hezzytheninja]

The fact that he walked in and did this demo from scratch while keeping the packed room engaged was incredible.

[@brefaccion]

and without syntax highlighting and completion õ.o

[@VietVuHunzter]

His talk are amazing. I was most impressed with the vault talk. This guy is Tom Hanks on an island, but instead of Wilson, he got Python!

[@cheaterman49]

@@VietVuHunzter It actually sort of happened to him, he mentioned how he was consulting for a bank or something, and they gave him a laptop with no internet or IDE or any sort of tools, but there was Python and a basic text editor IIRC.

[@kevindong9672]

@@cheaterman49 kzbin.info/www/bejne/iIuXhKFjjpx0hpo

[@kevindong9672]

@@cheaterman49 Actually it's proof discovery in a legal process, patent lawsuit or something

[@penciltwirler5829]

This is a performance by someone who has achieved the highest level of their craft.

[@stabgan]

Facts

[@ChristophZwerschke]

A fantastic, entertaining and highly educational talk. It always bothers me that I can't play the piano and talk at the same time (my wife usually asks me things while I'm playing). But David can even type concurrent Python code in Emacs in Allegro vivace speed and talk about it at the same time. An expert in concurrency in every sense of the word. How enviable!

[@crimsun7186]

+Christoph Zwerschke But when you're coding is easier. He's just talking what he's doing on the code, which is not that hard to do.

[@UniqueOneShot]

+Christoph Zwerschke Practice pair programming and it'll become easier

[@phisherPhisher]

his ability to explain his actions while programming fast and accurate without any real tool support (not even syntax highlighting) is pretty much amazing.

[@chmod0644]

The Jimi Hendrix of Python

[@bobocsabin]

Good one ! :))

[@nareshgb1]

Hendirx + Robin Williams.

[@Spoons4Hands]

Actually he's more of a Bob Ross of Python I think

[@Roycocup]

This presentation is legendary. Not only is David extraordinarily savvy on the inner workings of Python, but also I doubt that anybody can write code and present ideas AT THE SAME TIME! Sourcery!

[@KevinJingQiu]

Probably my all-time favourite conference talk :)

[@djanthony6662]

This guy is a legend.

[@SaiJeevan4]

dang.. I keep coming here every now and then ..such a brilliant talk !

[@luckyboy20021]

A lot of wisdom here, to me, it take 3 times watching this to understand the fully concept of concurrency, nice presentation with useful practice, thank a lot.

[@memoriasIT]

Video quality isn't the best, he's writing everything from scratch... But that doesn't even matter! His explanations are great and has the perfect balance between humor and educational. One of the best talks I've ever seen for real.

[@djl3009]

Thank you! This brilliant talk/live-code-session finally answered all my unanswered questions about concurrency and coroutines in Python. This is absolutely the most legendary live coding with concurrent explanation, I've ever seen.

[@danielstromberg]

Using coroutines in this way is an example of "cooperative multitasking"; if all your code yields the processor when it should, it can work quite well, but a single uncooperative code element can make a mess of things.

[@cenntraru]

This talk made me move from Python to Golang. Thank you David! 🙂

[@Mindoza94]

This speech is actual even for today! The best explanation of async programming it Python I've ever seen

[@gaatutube]

I would probably have to watch this 20 times over to really understand most (if not all) of his demo. It's like watching a Ninja hack away at the keyboard.

[@shanee7511]

David Beasley's videos always make ideas so much clearer for me. Such a great teacher!

[@MrQsam]

watching this after 10years... still relevant knowledge!!!

[@solomonxie5157]

"You know, every great talk, is started with Socket Programming."

[@mtp1376]

PyCon 2015, where I actually first learned coding in Python and started loving it!

[@kapucko1]

the best talk I've seen so far

[@asdfasdfasdf383]

This is a phenomenal performance. I learned a lot

[@FullMetal-nu8bj]

This is so interesting, David is informative and hilarious at the same time. I didn't know python supports threading in this way! Awesome!

[@taiminyang4236]

Amazing, the best and smoothest live coding demo I have ever seen!

[@birrafondaio]

Values unpacking works on lists as well: >>> def returns_a_list_of_lists(): >>> return 1, [2, 3, 4], [5, 6, 7] >>> a, [b, c, d], [e, f, g] = returns_a_list_of_lists() >>> a == 1 True >>> g == 7 True The third and last argument must have been an empty list.

[@chaoticguud]

If you are trying to code along and are getting socket already in use when you are submitting a job to the process pool, put fib_server(('', 25000)) in server.py in a `if __name__ == '__main__'`

[@watercollector]

Thanks a lot! I just got this error and deliberately searched in the comments if somebody already published the answer. Thanks!

[@ashwch]

Code: github.com/dabeaz/concurrencylive/

[@filiphron3147]

+Ashwini Chaudhary Thanks

[@goshinbi44]

I think he might actually be one of those mythical 10x developers lol.

[@muntoonxt]

I think that's a ridiculous myth -- that the 10x developer is a myth. Every field has people that are incompetent, people that are average, and people that are exceptional. The average people do important work. But exceptional folks are capable of generating solutions that are simply 10x better than what an "average person" would ever have come up with.

[@joseville]

27:06, i've never seen a list (albeit empty) on the left side of an assigment statement. What does it do? The simpler `[] = 1` throws `TypeError: 'int' object is not iterable`. Oh ok, 42:17, it was a bug/typo, but somehow it still worked. Btw, awesome talk. He's a great speaker.

[@landwarrior348]

David is Mozart of Python world.

[@terminalrecluse]

Dave is one of the best presenters of all time

[@HimothyOHooligan]

Probably should've gained up the audio a bit. I have my volume maxed out to be able to hear him at a nominal level.

[@bbruvey]

stackoverflow.com/questions/30147165/why-does-assigning-to-an-empty-list-e-g-raise-no-error if you want to know why 42:30 works

[@rockapedra1130]

Super simple and well explained! Excellent presentation!

[@aziz9488]

Great talk, well that's what I call an experienced python programmer!

[@danielstromberg]

yield from was introduced in CPython 3.3.

[@FlavioCoelhoC]

Amazing talk. for pythonistas, of course...

[@aaronhall8039]

At 6:16, I'm wondering what he wrote there, where it is blocked by the inner video frame.

[@ashwch]

Aaron Hall Its Thread(target=fib_handler, args=(client,), daemon=True).start() :P github.com/dabeaz/concurrencylive/blob/master/server.py

[@aaronhall8039]

Ashwini Chaudhary I just realized I never thanked you. Thanks. :D

[@dukiwave]

God bless you, man

[@narayanbala995]

Commenting a bit late. Agree that this is a spectacular talk. Has anybody tried the multithreading code - the first example? I can never get 2 clients working at the same time. The server responds to only one client.

[@JuanPabloReyesSepulveda]

Awesome Talk. Impressive.

[@bunc11]

GODLIKE!!!!

[@yomajo]

Zero experience in sockets and concurrency in Python. I could not even come up with threaded performance monitor let alone all tne rest. Incredible.

[@randixlai9216]

The only thing I learn from the video is he's legend.

[@antonkot6250]

7 years ago, omg. now it's mainsteam anywhere. NodeJS, Kotlin, Spring Reactor

[@10monthspregnant]

This guy is the John Goodman of Python :D

I feel sorry for the 5 people who 👎 this talk.

[@kodampradeep]

I had to watch twice to understand what he is typing and telling at the same time. Lol . How can we become like him :-(

[@abdelrhmanahmed1378]

your future_wait will grow very fast because we dont clear it i think its better to do this 'def future_done(future): tasks.append(future_wait.pop(future)) future_notify.send(b'x')

[@ruipeng5515]

Great Talk! Learned a lot from it!

[@Thebarbiturat]

No slides at speakerdeck and 404 for github page =(

[@tetamusha]

When I first had to implement a small distributed processing program in Python, I skipped Threads and went right to the multiprocessing package, which actually creates different processes for each object you start. This behavior is much more on par with the behavior of other languages and what is expected from a concurrency perspective. After all, why would you implement concurrent workers if you can't take advantage of more processing power? Might as well do a script that works sequentially.

[@marcalanstreeter]

Douglas De Rizzo Meneghetti Because sometimes what you're doing doesn't require more processing power just more processing time. This especially includes IO work where the processor gets tied up waiting for a response and is doing nothing in the meantime. Now imagine you had to make multiple IO connections each waiting on the other. In cases like these it is better to wait on all concurrently, and that's why it's important to understand the nature of your program. Is it IO bound or CPU bound? There are other tradeoffs to consider and obviously from watching this talk it could be that neither multiprocessing or threads are necessary or desirable in the end.

[@kevindong9672]

Even with GIL, multi thread is ok for IO-bound tasks.

[@De4sher]

turns out even in python 2.7, you can do this: `[] = []` BUT `{} = {}` will not work.... says it's a SyntaxError "can't assign to literal". What do you know?

[@GodefroyClair]

Someone understands why he says ""the load is essentially offloaded somewhere else, you're not fighting with the GIL or anything like that..." 15:48 ? kzbin.info/www/bejne/g3TWZoKsnbtsiZY The GIL is still limiting the task that are sent to the pool of threads right? Where are those task submitted otherwise ?

[@HelioMeiraLins]

Still not old in 2022.

[@cypherphage4294]

my mind got destroyed way before that

[@kukas1043]

Would it be bad practice (or helpful?) to write fib() as C module which releases the GIL? How much would that add overhead?

[@jmoz]

The De Niro & Pacino of Python.

[@vrajeshkumar]

Awesome talk!

[@randyt700]

is there a screencast of this available like his talk "built-in super heroes" ?

[@Water_Soul]

There David talking about CPU process, but will it works with process which use GPU if we have only one video card and something what take almost all GPU's memory? Or better wrap this process with threading.Lock()?

[@Mahorir]

Frigging amazing!

[@stoneshou]

I'm not sure I've got the point. It was a great talk and I think I've learned something new about threading and async io, but the two problems being presented: 1.performance impact due to the GIL, 2. performance cliff due to python prioritizing CPU intense jobs, was there a hinted solution to any of these problems? Especially for the second problem, all I learned from the video was by combining async io with ProcessPool, we can achieve similar performance cliff as the threading approach. But the performance cliff is still here?

[@chris8443]

Right, his point was that there are no easy solutions to those problems and that asyncio doesn't make them go away.

[@prateemmandal743]

I think the take away is that when there is massive number of concurrent tasks, co-routines is the way to go - you can mix in multiprocessing to ensure all cores get used - Posix threads are not going to scale for such requirements.

[@林玮-k4c]

Great talk

[@lokeshagrawal8240]

What is the code line written to watch a thread at 6:16?

[@Alexnotjones]

have the same question. it works fine with ... args=(client,)).start()

[@BGSone]

Best talk !!!

[@b7smart926]

where his writen code could be found

[@mehdisaffar]

it bothers me when people dont use syntax highlighting..

[@human-011]

Python docs have very poor asyncio documentation #python_improve_asyncio_docs

[@eotfofiw2748]

TL;DR; Wouldn't it be good if Python had a concurrency model like Node.

[@HybridIntuition]

Teclado brought me here!!

[@jriggatx]

What he said at 43:46. Thanks!

[@medtiw7457]

Anyone tried this on windows and got some different behavior? (eg: the requests per second didn't drop when I asked the server to compute the fib number of 35 like in: @11:18)

[@medtiw7457]

On the other hand the perf1.py test is affected by the computation of the fib number of 35

[@chenlonglee9511]

where I can download code?

[@Spoons4Hands]

this guy pythons!

[@michaelwu9238]

nice

[@FirstNameLastName-fv4eu]

This is called "intellectually"

[@vishcdac07]

Tom Hanks!

[@linan714]

Amazing talk, I really enjoy it. The only problem I have with it is he and the python community pretend NodeJS not exists. Isn't the non-blocking + callbacks the whole idea of nodejs?

[@zacharynathoo2537]

node cannot really handle multi core and so is not great, Erlang, go are much better

[@Acid31337]

that's why some switched to go

[@lsslock4856]

why not yield future? where from about if what equal future?

[@mickavelimckenzy3689]

oh God! I don't know SHIT in python

[@rustytrd8245]

Thanks david for amazing talk. For steps go to: github.com/codeAshu/concurrency

[@stabgan]

Too complicated for me

[@bradleyrintoul2644]

This is pretty great, but just the fact that threads are still a "hot topic" in Python gives me pause... It's 2016.

[@ttrss]

goodbye GIL

[@sindarin1350]

Can't handle the spit noises arg

[@AlqGo]

Spaghetti code

[@artintelligence2682]

Down is stupid, even less opened emacs editor. Large font and on the screen 10 lines only visible, constantly scrolls down/top

[@luqmansen]

Watching this in 2024, amazing talk 🫡👏👏