David Bau PhD Defense

Dissection of Deep Neural Networks - David Bau's PhD Defense at MIT, August 24 2021
Do deep networks contain concepts?
One of the great challenges of neural networks is to understand how they work. Because a deep network is trained by an optimizer, we cannot ask a programmer to explain the reasons for the specific computations that it happens to do. So we have traditionally focused on testing a network's external behavior, ignorant of insights or flaws that may hide within the black box.
But what if we could ask the network itself what it is thinking? Inspired by classical neuroscience research on biological brains, I introduce methods to directly probe the internal structure of a deep convolutional neural network by testing the activity of individual neurons and their interactions.
Beginning with the simple proposal that an individual neuron might represent one internal concept, we investigate the possibility of reading human-understandable concepts within a deep network in a concrete, quantitative way: Which neurons? Which concepts? What role do concept neurons play? And can we see rules governing relationships between concepts?
Following this inquiry within state-of-the-art models in computer vision leads us to insights about the computational structure of those deep networks that enable several new applications, including "GAN Paint" semantic manipulation of objects in an image; understanding of the sparse logic of a classifier; and quick, selective editing of generalizable rules within a fully trained StyleGAN network.
In the talk, we challenge the notion that the internal calculations of a neural network must be hopelessly opaque. Instead, we strive to tear back the curtain and chart a path through the detailed structure of a deep network by which we can begin to understand its logic.
Dissertation, slides, demos, code, and data at dissection.csail.mit.edu/

[@MEHRAN986]

"You need to hold on tight to your optimism. Because if you don't, You're never going to figure out how to do the hard things to answer the hard questions and make it really work"

[@peterw.5700]

This presentation has blown my mind. Thank you for making this publicly available!

[@ililil]

Congratulations Dr. Bau! It is a great and systematic research project and a very important one with many practical implications. I am sure it could be a great step towards a transparent general AI if you start combining and interconnecting such separate networks together at the interaction levels. I am fascinated! Thank you very much! It really inspiring!

[@squarehead6c1]

Wow, Dr. Bau is really pedagogical and is pleasant to listen to.

[@ervinperetz5973]

Great watch. Thanks for sharing this, David. - Ervin

[@MyMrChill]

Great job! I really appreciate what you have achieved.

[@mumbaicarnaticmusic2021]

Congrats! This was really interesting!

[@ninirema4532]

Dear prof Dr sir Thank you very 🙏 much 🙏

[@KonstantinMedyanikov]

Really cool results !

[@uyaseen]

Thanks for sharing, it was a great watch! Can you please comment on: 1. How do you search for "concept neurons" in giant models efficiently? 2. You mentioned in another comment that your group is working on interpreting GPT-X models, can you briefly comment on the concepts your group is trying to find out in GPT-X based models?

[@gyeonghokim]

the video has been greatly intriguing, and really enjoyed watching it. thanks for sharing 58:16

[@rexf5152]

Congratulations!

[@DavidBau]

Thank you!

[@katelingley3869]

I can't join the chat without creating a channel, which I don't have time to do - but I wanted to drop in and congratulate you, Dr Bau!!

[@DavidBau]

Thank you Kate!!!

[@vikramkaviya96]

I am at 27 minutes mark and I am impatient to ask this question with out completing this video that does this method scale up to models which have millions and billions of parameters

[@DavidBau]

Yes. We have found that the largest models, trained for a long time on massive data sets, tend to have very rich interpretable structure. Developing interpretable methods for massively parameterized models such as GPT-X is the topic of ongoing work in my group, and we have found that large models are a very target-rich environment. Oddly enough, one of the more difficult problems is to clarify is how interpretable structure emerges in the very simplest toy settings, trained on small problems, where the emergent structure is less obvious.

[@MarinaArtDesign]

"PhD Defense at MIT" and for most people, he is "the amazing maze guy". Due to pandemics and KDP explosion, his mazes are now in demand. A lot of people is trying to figure out how to make solutions script.

[@DavidBau]

Check out this link - web.mit.edu/PostScript/obfuscated-1993/labyrinth.ps - it is my submission to the obfuscated postscript contest web.mit.edu/PostScript/obfuscated-1993/WINNERS and it generates random mazes together with a solution, with all the computation done on the postscript printer. Edit the postscript as a textfile to change the options.

[@DavidBau]

By the way, if you or somebody you know is considering a PhD, I am looking for students! (For Fall 2022.) Check out our papers davidbau.com/research/, apply to the Khoury school www.khoury.northeastern.edu/apply/phd-apply/, and drop me a note if you are interested.

[@dibalokechanda478]

Dr. Bau , what is the best way to reach you ?

[@iamr0b0tx]

Nice work. Bushy eyebrow kids isn't something I thought I would hear today 😅