How to train a GAN, NIPS 2016 | Soumith Chintala, Facebook AI Research

NIPS 2016 Workshop on Adversarial Training
Soumith Chintala, Facebook AI Research
github.com/soumith/ganhacks
While research in Generative Adversarial Networks (GANs) continues to improve the fundamental stability of these models, we use a bunch of tricks to train them and make them stable day to day.
Here are a summary of some of the tricks.

[@noirmist3777]

1. Normalize the input [-1.+1] 2. Modified Loss function :min(log(1-D) to max(D) 3. Use Spherical Z rather than Uniform one - Sampling Generative Networks 4. Do not mix Real and Fake data by BatchNorm 5. Avoid Sparse Gradients (ReLU to Leaky ReLU, Maxpool to Average Pool 6. Label Smoothing 7. DCGANS/Hybrid models (KL + GAN, VAE+GAN) 8. Use RL stochastic tricks 9. ADAM 10. Trank Failure early- check loss

[@vaibhav6431]

github.com/soumith/ganhacks

[@FireSonix]

For those strangers who wanna go a little further: 11. Don't balance via loss statistics (Do not use D/G Losses for any hyperparameter tuning) 12. If you have labels, use them (Use Additional information from labels, e.g. in AC-GAN. CGAN etc) 13. Add noise to inputs, decay over time 14. Train discriminator more, sometimes 15. Batch Discrimination 16. Discrete Variable (Use additional information other than labels)

[@iansullivan8]

'things are fucking up' , I like that