An Instability in Variational Methods for Learning Topic Models

Andrea Montanari, Stanford University
simons.berkele...
Optimization, Statistics and Uncertainty

[@tengyuanliang3400]

typo? lambda_min(X) maybe lambda_min(-lambda X + I) on 22'20

[@andreamontanari6475]

Right! It is \lambda_min(Hess F)

[@tengyuanliang3400]

Nice talk Andrea. Two questions. (1) The fix using TAP, or the AMP methods, if compared to the iterative procedure in variational Bayes (now do consider the algorithmic part), comparing these two iterative methods, is there an intuitive explanation why TAP fixes the problem (what additional information it uses, is it some interaction terms)? (2) Am I understanding correctly, somehow these problems your calculation informs us spectral dominates Variational Bayes? Since below spectral threshold VB produce false discovery, and above spectral threshold -- spectral solves it fast. Thanks!

[@tengyuanliang3400]

Another minor question, for the Z2 synchron problem, can the Bayes optimal/information boundary be a direct implication of the phase transition in top eigenvector in the rank one spike Wigner model?

[@andreamontanari6475]

Hi Tengyuan, thanks for the feedback and the questions. Perhaps it is better to discuss by e-mail, or I can send you the paper as soon as it is finished (hopefully soon).