Variational Inference: Foundations and Innovations

[@whaleshark8700]

Wonderful Talk~~~~ 7:45 start 10:49 GMM model example 13:37 LDA example 22:42 Conditionally conjugate models 28:22 ELBO 30:52 Mean-field VI 37:27 Stochastic VI 48:07 Black box VI 1:00:47 Reparameterization and amortization

[@pauloabelha]

“Great question! I wish this talk was over so I could go and think about it”

[@jiachenlei1489]

amazing! brief presentation but gives deep insights

[@bnglr]

Check Blei's latest talk on this topic: www.cs.columbia.edu/~blei/talks/Blei_VI_tutorial.pdf kzbin.info/www/bejne/epLUf4GCnsmmraM kzbin.info/www/bejne/jZWag5KPjZmDmbM And the 2016 NIPS tutorial talk: www.cs.columbia.edu/~blei/talks/Blei_VI_tutorial.pdf kzbin.info/www/bejne/pZjHp5JsmcepjLM

[@ewfq2]

28:08 On the "bad properties of KL divergence" and alternative measures of divergence; Does anyone have any things to point to? Very interesting

[@citiblocsMaster]

7:45 This has to be true

[@조성민-y9n]

so damn true

[@monart4210]

I understand we measure the distance between two distributions using KL divergence, but am still very confused. How do we know whether we are getting closer to the actual posterior distribution if we do not know the posterior distribution?

[@prafful1723]

Please someone answer this!!

[@superhanfeng]

because c=a+b and c is a constant, and a is the KL is the divergence between posterior and variational distribution. By maximizing b, you minimizes a.

[@rhettscronfinkle3106]

It is a mathematical thing. You could check out Sergey Levine's Lectures. RAIL CS182 Latent Variable Models. It is part of a larger lecture series on DL. It's there on KZbin.

[@j2schmit]

You're exactly right, we don't know that actual posterior distribution, so it at first seems intractable to try to minimize a KL-divergence involving the actual posterior distribution. This is where the ELBO (the evidence lower bound) comes into play. This quantity can be used to bound the KL quantity of interest from below, and we can maximize this quantity without knowing the actual posterior, thereby finding the minimum of the KL. For a nice summary of this see Section 2.2. of this paper, arxiv.org/pdf/1601.00670.pdf

[@carlossouza5151]

Amazing talk!!!

[@sandeepreddy6295]

Great lecture !

[@martindelgado4834]

Can we have access to the slides please?

[@rdflrlz]

Martin Delgado You can get the slides if you check Dr. Blei's website.

[@harshnigam3385]

DAmn!

[@rickferreira3146]

Groovy