The Exponential Family (Part 1)

[@mCoding]

Can't wait for part 2!

[@BlaqueT]

This explained the concept and inner workings of the exponential family so much better than my lecturer did. Keep up the amazing work!

[@vivekveer3272]

This video is so amazing. Great visuals, really nicely explained.

[@SamuelLiJ]

The expression for the expectation and variance in terms of log(Z) shows up again in statistical mechanics. There the energy and energy fluctuations are derivatives of log(partition function), precisely because the distribution over microstates is in the exponential family with t(x)=E and theta=beta. Great stuff!

[@marcegger7411]

Here because of mCoding and honestly I’m baffled. Amazing content, amazing lecture, your channel is going to blow up. All the best!

[@Mutual_Information]

You mCodings folks are likely half my followers! What a bump :)

[@py5050]

I am late to the party. But you deserve so much more credit and attention. This answered all the questions I had and then some.

[@Mutual_Information]

The party is just getting started! This topic is one of my favorites, but it's quite technical, so there were never any expectations of getting a lot of attention. But those familiar with stats definitely appreciate it

[@xLyndo]

Thanks, DJ! I was coincidentally trying to learn about these in Bishop but my eyes just glazed over. These visualizations are great!

[@Mutual_Information]

Thanks! Glad it helped and more coming!

[@SohailKhan-zb5td]

Your explanation is the best on internet

[@Mutual_Information]

That is quite a compliment - thank you very much

[@梁文-r9i]

Thank you for delivering the concept so clearly!!!

[@Mutual_Information]

Happy to! Nice to see this very technical topic getting some love

[@PeacefulAnxiety]

The pace for the bullet points was perfect, it's clear the effort to get it to flow so well!

[@Mutual_Information]

Ha yea getting the timing right is a challenge. Still a work in progress, but glad people are noticing g

[@Dan-xl8jv]

This is amazing. So well explained

[@kimchi_taco]

Is DJ practicing boxing? He naturally guards up constantly.

[@Mutual_Information]

Lol I don't like my old videos because they have this awk-as-hell hand shit. I was listening to some garbage 'talk with your hands' advice for KZbinrs.. ugh!

[@twistedsector]

@@Mutual_Information I think it's great!

[@Thefare1234]

If math stats books had added intuitive explanations like this we would have been able to solve climate change by now.

[@Mutual_Information]

That's quite a compliment - thanks!

[@dhinas9444]

You are a cool guy for this explanation, mr. Mutual Information!

[@parthkdoshi]

Lovely video! Thanks a lot. Waiting for part 2..

[@JR-iu8yl]

Glad I found this channel

[@isleep8519]

eager to see part 2.

[@roshinroy5129]

Brother, love from India!

[@chrisha313]

This is amazingly done, thank you so much!

[@roydouek]

Your content is gold!

[@trinidadcisneros4339]

Wow this was crystal clear! looking forward to watching your other videos thank you

[@Mutual_Information]

And this is an old one! Glad you like it. I'm actually thinking about reshooting it..

[@trinidadcisneros4339]

@@Mutual_Information I'll be looking out for it if you do, keep up the great work!

[@ilyboc]

can't wait for part 2 :D

[@Mutual_Information]

Thank you very much - please do!

[@jaewonlee8147]

Keep up the great work man!

[@vyrgill]

Crazy good video, really! Thank you so much

[@Mutual_Information]

I love it when someone likes my harder videos!

[@Dupamine]

The details are crazy! But i guess the exponential family is not completed yet. I might come back here a few months later

[@kylekuang1052]

mannn this is phenomenal

[@Mutual_Information]

Thank you Yi! The exponential family is a beautiful thing

[@hamzaelotmani]

woww , what a video ,goood job

[@frecklematt]

bruh how do you have so few views???? amazing videos!!

[@Mutual_Information]

lol KZbin just can't handle me

[@pabloagogo1]

Hi, in one of the integrals where you have something like the integral of : exp( x-x^2), how is it that after integration one gets Logarithmic functions coming out. Hope you can explain how the Logarithm appears.

[@parthkdoshi]

Can you please also do a video on deep exponential families?

[@abdulrhmanaun]

Thank you for your help

[@user-or7ji5hv8y]

Wow, never would have guessed that they were somehow related. Is this the main family that matters or are many more?

[@Mutual_Information]

This one matters a lot! And I’m not familiar with any other classes of distributions that come close. A lot of theorems/tools are designed for the exponential family, just b/c they have such convenient properties

[@benjaminpedersen9548]

I think you can technically form uniform distributions within the family, it is just uninteresting since you have to choose your set beforehand and the distribution would be effectively parameterless: Choose h to be an indicator function of some set A and t to map everything to 0 or, even simpler, be zero-dimensional. This gives you probability density 1/nu(A) 1_A.

[@bluearctik3980]

Does it make sense to analogize the exponential family to overloaded functions in computer science? Something like: “given some inputs that determine the search space and sufficient statistics, return the appropriate distribution.” This a very lucid explanation, by the way - I’ve bookmarked the video for future reference!

[@Mutual_Information]

Thank you! And to answer your question, I don't quite see the analogy your referring to in the first sentence, but the second sentence seems fair to me. If the inputs are the decisions for t(x), h(x) and v(x) and the function your referring to searches for the theta* according to the data, then returning theta* would amount to returning "the appropriate distribution". So I think you're right

[@toducanh]

Thank you

[@II_superluminal_II]

HOLY SHIT UR CHANNEL IS AMAZING BROTHER, just a CS grad interested in the secrets of AI. P=NP tho

[@desjajjaden49]

U r my HERO!!!!

[@Mutual_Information]

lol should I wear a cape next time??

[@hitashasharma2178]

Such a nice video

[@junhanouyang6593]

Really good video that help me understand the concept of exponential family. However I just have one small confusion. In a normal distribution case, why t(x) needs to be x and x^2? Because to me having an x will tell us the x^2 value and thus knowing x should give us the probability distribution via theta. I know to form the PDF for normal distribution you need x and x^2. I just can’t find a good logic behind this.

[@Mutual_Information]

I see you're point. The way I would think about x and x^2 is they are measures of the same thing.. such that their probability depends (basically) **linearly** on these measures, using a fixed parameter vector. Let's say you want to only use x. Could you write the normal density as a linear function of *only* x?? That's what you can't do... and that's why x^2 is needed as well.

[@camila_braz]

Thanks a lot!!!

[@SiaHranova]

woah, this was awesome..

[@Mutual_Information]

It's nice when someone can appreciate the harder videos.

[@Mohamed_Salah8]

Sometimes it takes only one video to make people subscribe 😂.... thank you

[@Mutual_Information]

I was actually thinking of re-shooting this one. I'm glad it still works for some!

[@AIMJL-j2i]

GOAT!

[@SuperGanga2010]

Nice lesson, but 4K please! Even if your camera is 1080p, the rendered math would look sharper.

[@Mutual_Information]

Yes, that was a more recent change. I was less familiar with camera best practices when I shot this.

[@OmegAtlAnt]

Suggestion: Set your playback speed to 0:75 if you don't like to feel like a dumb idiot

[@Mutual_Information]

ha yea these older videos are too fast. Amateur mistake on my part, but the new stuff is better paced!

[@OmegAtlAnt]

@@Mutual_Information I was feeling frustrated and commented a bit too harshly. It was a great explanation overall.

[@Jacob011]

Is that a Gaussian process regression model on your desktop background? I appreciate the high-quality content and the visuals. I just wouldn't call this p(x|\theta) = 1/Z(\theta) h(x) exp(t(x)\theta) an equation, because asking for a solution of this doesn't make any sense.

[@Mutual_Information]

Yep, that’s a GP and in fact that’s my next video coming out (sometime near the end of this week). And yea, you’re right. It’s an expression, not an equation. I’ll keep that in mind.

[@martinschulze5399]

That gestures xD

[@Mutual_Information]

haha Old terrible habit.. Rest assured, I feel shame

[@lethalavidity]

3Blue1Brown's little brother ;)

[@keggluneq]

Love your content, but it would be nice if you could sloooooow it down a bit. You feed us a shit-ton of information in 15 minutes. I find that noob creators tend to unnecessarily zip through their material. I personally like the pace of 3b1b & Trefor Bazett.

[@jacobcaurdy2997]

I love your content and explanations but i find you being side-by-side with the blackboard explanations very distracting, I think it’s gotta do with the hand gestures combined with intonations. Maybe thats just me. All love though, your content is superb.

[@Mutual_Information]

I agree in fact. My new format (see my most recent video) makes the latex more front and center, and that's how things we'll be going forward. It's all a work in progress

[@undisclosedmusic4969]

Cool video and all, but why are you wasting precious energy by having two monitors and a computer on idle while filming?

[@Mutual_Information]

Ha never thought of that. If it makes any difference to you, I turn the AC and fridge off since they make noise. So maybe that nets out :)

[@undisclosedmusic4969]

@@Mutual_Information I don’t mind, perhaps your wallet and/ or the planet do 😅

[@welcomethanks5192]

how to map this form en.wikipedia.org/wiki/Exponential_family#:~:text=This%20yields%20the%20canonical%20form with your equation?

[@Mutual_Information]

eta = theta and log Z(theta) = A(eta)

[@welcomethanks5192]

@@Mutual_Information why not subtract A(eta)? How you get 1/A(eta)?? exp(x-a) = exp(x)/exp(a)