NoiseRater: Meta-Learned Noise Valuation for Diffusion Model Training

Authors: Fang Wu, Haokai Zhao, Da Xing et al.

arXiv · PDF

Summary

NoiseRater is a meta-learning framework that assigns importance scores to individual noise samples in diffusion model training, rather than treating all noise uniformly. A parametric "noise rater" network conditions on the data and timestep to weight each noise realization, and is trained via bilevel optimization to improve downstream validation loss. The authors then deploy a two-stage pipeline: soft weighting during meta-training, then hard noise selection during standard training. Experiments on FFHQ and ImageNet show that prioritizing informative noise improves both training efficiency and generation quality.

Main takeaways:

• Standard diffusion training treats all injected noise samples as equally informative; NoiseRater challenges this by learning per-instance noise importance.
• A parametric rater assigns scores conditioned on data and timestep; it's trained via bilevel optimization (meta-learned to improve validation after inner diffusion updates).
• A decoupled two-stage pipeline transitions from soft reweighting (meta-training) to hard noise selection (standard training) for efficiency.
• Empirically, not all noise is equal—prioritizing high-value noise improves both training speed and final image quality on FFHQ and ImageNet.

Relevance

Not directly related to my persona/midtraining work—this is about diffusion model training. It's only tangentially relevant if I ever think about example weighting or curriculum learning during midtraining or fine-tuning, but that's outside my current research scope.

Abstract

arXiv:2605.08144v1 Announce Type: new Abstract: Diffusion models have achieved remarkable success across a wide range of generative tasks, yet their training paradigm largely treats injected noise as uniformly informative. In this work, we challenge this assumption and introduce NoiseRater, a meta-learning framework for instance-level noise valuation in diffusion model training. We propose a parametric noise rater that assigns importance scores to individual noise realizations conditioned on data and timestep, enabling adaptive reweighting of the training objective. The rater is trained via bilevel optimization to improve downstream validation performance after inner-loop diffusion updates. To enable efficient deployment, we further design a decoupled two-stage pipeline that transitions from soft weighting during meta-training to hard noise selection during standard training. Extensive experiments on FFHQ and ImageNet demonstrate that not all noise samples contribute equally, and that prioritizing informative noise improves both training efficiency and generation quality. Our results establish noise valuation as a complementary and previously underexplored axis for improving diffusion model training. Our code is available at: https://anonymous.4open.science/r/NoiseRater-DEB116.