Larry Hastings - Removing Python's GIL: The Gilectomy - PyCon 2016

[@LorenaNicole]

This was such a great talk! I'm revisiting this so I can go read more about the gilectomy project!!!

[@adamperry7294]

Small note: Rust doesn't have a tracing GC or any GC for that matter. It does have reference-counted box types (Rc, Arc), but they're opt-in.

[@ChrisLeeX]

Wonderful. Would love to see more talks like this.

[@pgtrots]

Dude speaks really well for an ultra clever geek :-)

[@StefanNuxoll]

Rust doesn't use garbage collection like the presenter states when comparing python to other languages. Rust object lifetimes are statically determined by the compiler and are deterministically freed when they are no longer used, with the exception of objects contained in the Rc or Arc boxed types.

[@hmm-.-]

Stefan Nuxoll who cares about how rust works, he was just throwing some examples out there. This information is completely irrelevant.

[@hmm-.-]

Jake He talked about Rust for less than a second, and all you see in comment section is "rust, rust, rust, and rust". Point out something interesting and valuable, some ideas, don't talk about something completely irrelevant.

[@GoncaloLuis89]

17:05 -> Would someone please explain to me how is that not breaking C extensions? Isn't it possible to create with a system that implies an "auto-lock arround c extensions" (29:53) on the current C extensions but would allow future non-GIL safe extensions to run freely. This would be the best of both worlds.

[@adammccurdy7481]

Valid question at the end. Use a tool to it's strengths.

[@nandoflorestan]

That's illogical. The gilectomy is about expanding Python's strengths. You're talking about using Python, he's talking about making a better Python.

[@drasticfred]

Instead of locking whole interpreter, better of lock variables/objects, upside down isolation for threads can be achieved. Introduce a bitmap table for vars/objs mark them locked/unlocked etc. Don't mess with message passing either.

[@waats]

He said that Rust use a tracing garbage collection ( 9:25) . I thought one of the main features of Rust is not using garbage collection. Does someone know more about this ? It's quite confusing ^ ^

[@HoeBlogggs]

Rust GC works somewhat similar to reference counting, when an object moves out of scope it's dereferenced. This works by having the concept of ownership. It definitely doesn't use a tracing GC. //edit - Holy shit some people have actually built a GC for it, github.com/Manishearth/rust-gc

[@manoelnt0]

I love the question on the end :v (which actually is basically a implicit statement). 31:29

[@noli-timere-crede-tantum]

Great, insightful talk from almost 5 years ago. However, just like today, it doesn't solve the problem people are really after: make Python take advantage of all CPU cores in a sane way.

[@Zeturic]

While I appreciate the idea of the initmodule and nogil_initmodule (that is: opt-in, so you don't break code), I'm not really a fan of it. Mainly because, ideally, at some far-off point in the future, the GIL should be a distant memory. Having separate builds (with GIL and without) should eventually not be a thing; at that point, the only entry point would not have the obvious name (it'd be called nogil_initmodule, even though the GIL had been long abandoned and was no longer a thing).

[@josedanielgomezrodriguez6163]

This slide is not present on speakerdeck.com/pycon2016

[@TheJesseWu]

Cool, DS Dad is a programmer.

[@grhayes]

99% of the issues he is describing are related to trying to keep the stupid reference counting. Create a resource management system that keeps track of variables and their scope. Destroy them when they go out of scope or are manually destroyed. Speed it up by using a pre-allocated memory space and assign space to variables from it so you don't need to use memory allocation as much. It would be very similar to what we do when creating console games and trying to manage resource space. It also increase performance. As for the GIL being efficient. The last video says you did a check every 100 instruction where you jump out and back and check a flag to see if anyone is waiting on it. That's 10 added instructions for every 100 roughly instantly killing 10% of over all performance. What part of that is efficient? Looking at his benchmark results and listen to him talk he lacks a good deal of understanding how multi-threading works. Fib is a horrible demonstration for multi-threading. There is no real explanation of how he implemented it. Vs kzbin.info/www/bejne/sIavhml-ndVnaNk Frankly, their reliance on C is a bit much. Don't get me wrong I love C. However, If they move to C++ 14 or above they could fix a lot of this easier.

[@pycz]

Hm, how can we determinate the scope of every variable in python? Python variables are not like C variables, it is just pointers to actual variable objects, which could be pushed around whole program and be assigned to multiple variable names.

[@SianaGearz]

10% overhead can be considered small in Python, since CPython (mainline implementation) is about 100 times slower than C and C++. So everything that doesn't move the needle that much can be considered "fast" or "efficient" in the context. Why do you think C++14 is beneficial for implementation of a language interpreter? There exists no operating system primitive that you can access from C++ that you cannot access via C. Not necessarily ANSI C, but C with platform extensions. And all it does is manipulate low-level data structures and arranging dynamic dispatch around that, advanced C++ features simply don't hep with that at all. Fib is what could be considered one of worst cases, but until worst case doesn't get THAT much worse, like not orders of magnitude slower, you definitely haven't achieved performance goals. So it is a valid benchmark, even though it's obviously not representative.

[@HoeBlogggs]

Considering the level of the talk the question at the end was utterly awful. Just... Why would you ask that?

[@ManuelGarridoPena]

I almost went up to the microphone and asked "What is the GIL?"

[@john46923]

I thought the question was a great joke. There was time only for one, so somebody in the audience of some 900 people comes forward ... the burden of asking the single question is on him ... and he asks something to which we can boo and laugh. At least the guy who asked (he was sitting a few rows behind me) took it in good spirit. Another great thing about the question is also that he's right. :) Python is suitable for an amazing array of tasks, but you can't force it to be good for everything without risking that it becomes good for nothing.

[@eddie2378]

Im giving Julialang a try.

[@tamasgal_com]

Do it! ;)

[@wtfvids3472]

"Just use another tool" is equivalent to saying "just spend months learning another tool". Why is it so hard for people to see this? If removing the gil works and multithread becomes better, then it's a win. Really, it's as simple as that.