A solvable model of learning generative diffusion: theory and insights
Hugo Cui, Cengiz Pehlevan, Yue M. Lu
在这份手稿中,我们考虑了学习由双层自动编码器进行参数化的流动或基于扩散的生成模型的问题,该模型通过在线随机梯度下降训练,在具有底层低维流形结构的高维目标密度上。 我们对学习模型生成的样本分布的低维投影进行了严格的渐近表征,特别是确定其对训练样本数量的依赖。 基于此分析,我们讨论了模式崩溃如何产生,并导致模型在生成模型在生成的合成数据上重新训练时崩溃。
In this manuscript, we consider the problem of learning a flow or diffusion-based generative model parametrized by a two-layer auto-encoder, trained with online stochastic gradient descent, on a high-dimensional target density with an underlying low-dimensional manifold structure. We derive a tight asymptotic characterization of low-dimensional projections of the distribution of samples generated by the learned model, ascertaining in particular its dependence on the number of training samples. B...