Improved Techniques for Training Score-Based Generative Models

This research paper focuses on improving score-based generative models (SBGMs) to produce high-quality, high-resolution images. The authors identify limitations of existing SBGMs, specifically their inability to scale to higher resolutions and occasional training instability. They propose new theoretical analyses and techniques for selecting noise scales, an efficient method for incorporating noise information, and a process for configuring annealed Langevin dynamics, all of which are crucial for successful training and sampling in high dimensions. Additionally, they introduce the use of an exponential moving average (EMA) of model weights to enhance stability during training. The culmination of these improvements, dubbed NCSNv2, demonstrates the capability of generating high-fidelity images across various datasets and resolutions previously challenging for this class of models.