How to check if a neural network has learned a specific phenomenon?

🤔 In this video, Ms. Coffee Bean and I explain how "probing" neural networks (in NLP) works. In other words, how we check if a neural net trained on task A is also able to perform task B.
We are also touching upon a very nice paper, recently published by Elena Voita and Ivan Titov. Since I strongly recommend anyone to read it, here is the reference to the explained paper:
Information-Theoretic Probing with Minimum Description Length on arxiv.org/pdf/...
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
🔥 Optionally, pay us a coffee to boost our Coffee Bean production! ☕
Patreon: / aicoffeebreak
Ko-fi: ko-fi.com/aico...
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
🔗 Links:
KZbin: / @aicoffeebreak
Twitter: / aicoffeebreak
Reddit: / aicoffeebreak
#AICoffeeBreak #MsCoffeeBean

[@user-or7ji5hv8y]

What is the difference between probing and fine-tuning in transfer learning?

[@AICoffeeBreak]

Not that big, very well observed. It is basically the same procedure. The difference lies in the reasons to do "transfer" and the amounts of data. Probing is usually done with little training data and not for other downstream tasks, but for finding out what the network has picked up.

[@DavenH]

Informative video, I'm gonna watch all your content now! Also thank you for getting 'phenomenon' and 'phenomena' right. A pet peeve, but nearly all native English speakers mess it up!

[@AICoffeeBreak]

Thanks! I guess it belongs to the perks of being a non-native speaker: even the common mistakes are different. 😅

[@chinmayapani2831]

Hello Letitia Your explanations are excellent. It keeps the leaner like me watching your videos all the way through. One doubt i have related to this video. You mentioned, bobs pass the data in a compressed form to alice, alice decodes the message and reconstruct it to get the label and if alice able to get the right label then she gives credit to bob. The question is how alice get to know whether she has decoded correctly?Where is the true label to compare ?

[@AICoffeeBreak]

Hi and thanks for the question! It is such a long time since I made the video. As far as I remember, the true label is in the training data.

[@Youkouleleh]

Hello, I have some question: 1) 3:30 , you remove the layers that were responsibles to classification et replace it with probling layer, but how do you know how much layer to remove, is there a rules of a method to precede ? 2) is it the same than transfer learning or is it difference btw what you present and transfert learning ? For the overfitting: - if you divide your set with a train/test, will that be enough to check if their is overfitting ? Thansk for the video

[@AICoffeeBreak]

Hello, thanks for dropping by! :) 1) It depends on the architecture, so let's take a very simple example: In a classical CNN architecture, you have a sequence of convolutional layers, followed by a couple of fully connected (FC) layers. These FC are interpreted as classification layers and are removed. The convolutional layers beforehand count as feature extraction layers. 2) Yes, probing has a lot in common with transfer learning, since all the weights from the feature extraction layers are kept (frozen) and in this sense transferred for new probe-specific classification layers. Overfitting: Yes and no. Overfitting is a very tricky phenomenon. If you have a good test set, then the train/test gap can already hint to the presence of overfitting. But sometimes the test set distribution is so close to the training one, that no strong alarm signals are raised. You think that everything is fine but you see your model fail in applications you would had never expected it to. For probing: it is hard to decide when to stop training, even if you have a good test set.

[@TheRelul]

very nice Romanian accent! Subscribed

[@AICoffeeBreak]

Welcome, thanks for the sub! Love your reason 🤣

[@gergerger53]

Just stumbled across your channel. Very impressed with your videos! Happy to subscribe :)

[@AICoffeeBreak]

Happy to have you here! :)

[@kfliden]

Thanks, first video on probing that makes sense to me. But just wondering if probing is just for diagnostics or it's actual option for fine tuning in production?

[@AICoffeeBreak]

When the representations of the model are really good, it might happen that probing (tuning just a linear head at the end) is enough. But most of the time, in production you need new model abilities and knowledge, so fine-tuning is often the option.

[@safaelaat1868]

Thank you very much, your videos are very helpful for me!! Could you activate the automatic translation please?

[@AICoffeeBreak]

Hey thanks for the comment. But is the automatic translation something I can activate? I did not know that, I thought it's a user setting.

[@safaelaat1868]

@@AICoffeeBreak Thank you for your replay. Really I don t know but i can activate it for other videos but not for this one. It tell me that it is not available

[@AICoffeeBreak]

Ah, now I realize I haven't uploaded subtitles for this one. This might be the reason. Will do so in the next days. Thanks for pointing this out. 🤝

[@safaelaat1868]

@@AICoffeeBreak Thank you very mutch !

[@AICoffeeBreak]

@@safaelaat1868 Just added the captions. It should work in 20-30 minutes or so after KZbin has processed them. :)