Closing the ELBO Gap in Diffusion Models for Speech Enhancement and Dereverberation

We examine the Evidence Lower Bound (ELBO) within diffusion models (DMs) applied to speech enhancement (SE) and dereverberation (SD). We focus in particular on the interplay between the ELBO and Gaussian noise schedule (GNS), and the choice of practical loss functions. We hypothesize that the suboptimal performance of DM-based SE and SD can arise from the absence of a well-calibrated GNS. We therefore refine the noise schedule design by controlling the minimum and maximum noise variances. Additionally, we introduce the Importance of Condition as a novel metric that quantitatively assesses the influence of noise variance on the model behavior during reverse diffusion processes. Our analysis reveals that changing the GNS configuration substantially affects the reliance of the model on the input condition, thereby impacting the overall performance. Furthermore, we demonstrate that conventional loss functions used in DMs inherently impose a performance ceiling that prevents convergence to the theoretical optimum of the ELBO, resulting in suboptimal SE and SD outcomes. We propose a two-stage training framework to alleviate this limit. First, a score-based DM uses an optimized GNS to perform initial enhancement. Second, a dedicated refinement model is trained to further improve the ELBO and enhance speech quality. Our comprehensive experimental validation demonstrates the effectiveness of the proposed framework on both SE and SD tasks.