Variational Inference: Foundations and Modern Methods (NIPS 2016 tutorial)

[@yididiyayilma900]

30:00 The most intuitive explaination for stochastic optimization I have ever heard so far.

[@alexeygritsenko9955]

At 44:43 - why does the score function have expectation of zero?

[@mataneyal]

\begin{equation} \begin{aligned} \mathbb{E}_q [ abla_ u g(z; u)] &= \mathbb{E}_q [ abla_ u \log p(x,z) - abla_ u \log q(z; u)]\\ &= - \mathbb{E}_q [ abla_ u \log q(z; u)] ~ \textrm{($\log p(x,z)$ is not a function of $ u$)}\\ &= - \int q(z; u) abla_ u \log q(z; u) \\ &= - \int abla_ u q(z; u)~\textrm{(Log Derivative Trick)}\\ &= 0 ~\textrm{($q(z; u)$ is a continuous probability distribution)} \end{aligned} \end{equation}

[@maxturgeon89]

Chain rule and dominated convergence theorem

[@Filaaaix]

At 1:27:30 => I didn't really get how you derive the Auxiliary variational bound. Is there a good source where it's explained more thoroughly?

[@nicodetullio]

lo