Stanford CS236: Deep Generative Models I 2023 I Lecture 7 - Normalizing Flows

For more information about Stanford's Artificial Intelligence programs visit: stanford.io/ai
To follow along with the course, visit the course website:
deepgenerativemodels.github.io/
Stefano Ermon
Associate Professor of Computer Science, Stanford University
cs.stanford.edu/~ermon/
Learn more about the online course and how to enroll: online.stanford.edu/courses/c...
To view all online courses and programs offered by Stanford, visit: online.stanford.edu/

[@dohyun0047]

i hope someone answer my question... in inference phase (not training) we can just pick z form standard gaussian distribution because of in KL divergence in training time? because KL has made latent variables(z) to be distributed in standard gaussian distribution (p(z) , we are assuming simple gaussian)

[@CPTSMONSTER]

8:00 Without KL term, similar to a stochastic autoencoder which takes an input and maps it to a distribution over latent variables 8:30 Reconstruction to resemble Gaussian, KL term encourages latent variables generated through encoder to be distributed similar to the prior distribution (Gaussian in this case) 10:00? Trick decoder 12:50? q also stochastic 14:10 Both p and q generative models, only regularizing latent space of an autoencoder (q) 15:10 Marginal distribution of z under p and under q seems like a possible training objective, intractable integrals 24:10? If p is a powerful autoregressive model, then z is not needed 32:05? Sample p of z given x, invert generative process, find z's likely under that posterior, intractable to compute 34:25? Sample from conditional, not selecting from most likely z 53:50 Change of variables formula 56:40 Mapping unit hypercube to parallelotope (linear invertible transformation) 59:10 Area of parallelogram is determinant of matrix 59:50 Parallelotope pdf 1:08 Non-linear invertible transformation formula, generalized to determinant of Jacobian of f. Dimension of x and z are equal, unlike in VAEs. Determinant of Jacobian of inverse of f is equal to inverse of determinant of Jacobian of f. 1:15:00 Worked example of non-linear transformation pdf formula 1:17:45 Two interpretations of diffusion models, stacked VAEs and infinitely deep flow models 1:21:20 Flow model intuition, latent variables z don't compress dimensionality, views data from another angle to make things easier to model