Dr. Andre Wibisono | Optimal score estimation via empirical Bayes smoothing

Speaker: Dr Andre Wibisono (Yale University)
Date: 15th Jul 2024
Title: Optimal score estimation via empirical Bayes smoothing
Event: (DMLW01) International workshop on diffusions in machine learning: foundations, generative models, and optimisation
Abstract: We study the problem of estimating the score function of an unknown probability distribution $
ho^*$ from $n$ independent and identically distributed observations in $d$ dimensions. Assuming that $
ho^*$ is subgaussian and has a Lipschitz-continuous score function $s^*$, we establish the optimal rate of $\tilde \Theta(n^{-\frac{2}{d+4}})$ for this estimation problem under the loss function $\|\hat s - s^*\|^2_{L^2(
ho^*)}$ that is commonly used in the score matching literature, highlighting the curse of dimensionality where sample complexity for accurate score estimation grows exponentially with the dimension $d$. Leveraging key insights in empirical Bayes theory as well as a new convergence rate of smoothed empirical distribution in Hellinger distance, we show that a regularized score estimator based on a Gaussian kernel attains this rate, shown optimal by a matching minimax lower bound. We also discuss the implication of our theory on the sample complexity of score-based generative models. Joint work with Yihong Wu and Kaylee Yang.
👉 Want to learn more about the talks in this stream?
🔗| Seminar Link: www.newton.ac....
🔗| Workshop Link: www.newton.ac....
-------------------
FOLLOW US
🌐| Website: www.newton.ac.uk
🎥| Main Channel: / @isaacnewtoninstitute
🐦| Twitter: / newtoninstitute
💬| Facebook: / newton.institute
📷| Instagram: / isaacnewtoninstitute
🔗| LinkedIn: / isaac-newton-institute...
SEMINAR ROOMS
🥇| INI Seminar Room 1: / @iniseminarroom1
🥈| INI Seminar Room 2: / @iniseminarroom2
🛰️| INI Satellite Events: / @inisatellite
ABOUT
The Isaac Newton Institute is a national and international visitor research institute. It runs research programmes on selected themes in mathematics and the mathematical sciences with applications over a wide range of science and technology. It attracts leading mathematical scientists from the UK and overseas to interact in research over an extended period.
👉 Learn more about us and our events here: www.newton.ac.uk