FastVoiceGrad: One-step Diffusion-Based Voice Conversion with Adversarial Conditional Diffusion Distillation
Takuhiro Kaneko Hirokazu Kameoka Kou Tanaka Yuto Kondo


Inferring Cosmological Parameters on SDSS via Domain-Generalized Neural Networks and Lightcone Simulations
    [
    September 9, 2024
==========================================================================================================



§ ABSTRACT 
  Diffusion-based voice conversion (VC) techniques such as VoiceGrad have attracted interest because of their high VC performance in terms of speech quality and speaker similarity. However, a notable limitation is the slow inference caused by the multi-step reverse diffusion. Therefore, we propose FastVoiceGrad, a novel one-step diffusion-based VC that reduces the number of iterations from dozens to one while inheriting the high VC performance of the multi-step diffusion-based VC. We obtain the model using adversarial conditional diffusion distillation (ACDD), leveraging the ability of generative adversarial networks and diffusion models while reconsidering the initial states in sampling. Evaluations of one-shot any-to-any VC demonstrate that FastVoiceGrad achieves VC performance superior to or comparable to that of previous multi-step diffusion-based VC while enhancing the inference speed.[Audio samples are available at <https://www.kecl.ntt.co.jp/people/kaneko.takuhiro/projects/fastvoicegrad/>.]




§ INTRODUCTION



Voice conversion (VC) is a technique for converting one voice into another without changing linguistic contents.
VC began to be studied in a parallel setting, in which mappings between the source and target voices are learned in a supervised manner using a parallel corpus.
However, this approach encounters difficulties in collecting a parallel corpus.
Alternatively, non-parallel VC, which learns mappings without a parallel corpus, has attracted significant interest.
In particular, the emergence of deep generative models has ushered in breakthroughs.
For example, (variational) autoencoder (VAE/AE) <cit.>-based VC <cit.>, generative adversarial network (GAN) <cit.>-based VC <cit.>, flow <cit.>-based VC <cit.>, and diffusion <cit.>-based VC <cit.> have demonstrated impressive results.



Among these models, this paper focuses on diffusion-based VC because it <cit.> outperforms representative VC models (e.g., <cit.>) and has a significant potential for development owing to advancements in diffusion models in various fields (e.g., image synthesis <cit.> and speech synthesis <cit.>).
Despite these appealing properties, its limitation is the slow inference caused by an iterative reverse diffusion process to transform noise into acoustic features (e.g., the mel spectrogram[For ease of reading, we hereafter focus on the mel spectrogram as a conversion target but other acoustic features can be equally applied.]) as shown in Figure <ref>(a).
This requires at least approximately five iterations, typically dozens of iterations, to obtain sufficiently high-quality speech.
This is disadvantageous compared to other deep generative model-based VC (e.g., VAE-based VC and GAN-based VC discussed above) because they can accomplish VC through a one-step feedforward process.

To overcome this limitation, we propose FastVoiceGrad, a novel one-step diffusion-based VC model that inherits strong VC capabilities from a multi-step diffusion-based VC model (e.g., VoiceGrad <cit.>), while reducing the required number of iterations from dozens to one, as depicted in Figure <ref>(b).
To construct this efficient model, we propose adversarial conditional diffusion distillation (ACDD), which is inspired by adversarial diffusion distillation (ADD) <cit.> proposed in image synthesis, and distills a multi-step teacher diffusion model into a one-step student diffusion model while exploiting the abilities of GANs <cit.> and diffusion models <cit.>.
Note that ADD and ACDD differ in two aspects: (1) ADD addresses a generation task (i.e., generating data from random noise), while ACDD addresses a conversion task (i.e., generating target data from source data); and (2) ADD is applied to
images, while ACDD is applied to acoustic features.
Owing to these differences, we (1) reconsider the initial states in sampling (Section <ref>) and (2) explore the optimal configurations for VC (Section <ref>).

In the experiments, we examined the effectiveness of FastVoiceGrad for one-shot any-to-any VC, in which we used an any-to-any extension of VoiceGrad <cit.> as a teacher model and distilled it into FastVoiceGrad.
Experimental evaluations indicated that FastVoiceGrad outperforms VoiceGrad with the same step (i.e.,  one-step) reverse diffusion process, and has performance comparable to VoiceGrad with a 30-step reverse diffusion process.
Furthermore, we demonstrate that FastVoiceGrad is superior to or comparable to DiffVC <cit.>, another representative diffusion-based VC, while improving the inference speed.

The remainder of this paper is organized as follows:
Section <ref> reviews VoiceGrad, which is the baseline.
Section <ref> describes the proposed FastVoiceGrad.
Section <ref> presents our experimental results.
Finally, Section <ref> concludes the paper with a discussion on future research.



§ PRELIMINARY: VOICEGRAD



VoiceGrad <cit.> is a pioneering diffusion-based VC model that includes two variants: a denoising score matching (DSM) <cit.>-based and denoising diffusion probabilistic model (DDPM) <cit.>-based models.
The latter can achieve a VC performance comparable to that of the former while reducing the number of iterations from hundreds to approximately ten <cit.>.
Thus, this study focuses on the DDPM-based model.
The original VoiceGrad was formulated for any-to-many VC.
However, we formulated it for any-to-any VC as a more general formulation.
The main difference is that speaker embeddings are extracted using a speaker encoder instead of speaker labels, while the others remain almost the same.

Overview.
DDPM <cit.> represents a data-to-noise (diffusion) process using a gradual nosing process, i.e., x_0 →x_1 →…→x_T, where T is the number of steps (T = 1000 in practice), x_0 represents real data (mel spectrogram in our case), and x_T indicates noise x_T ∼𝒩(0, I).
By contrast, it performs a noise-to-data (reverse diffusion) process, that is, x_T →x_T-1→…→x_0, using a gradual denoising process via a neural network.
The details of each process are as follows:

Diffusion process.
Assuming a Markov chain, a one-step diffusion process q(x_t | x_t-1) (t ∈{1, …, T }) is defined as follows:

  
  q(x_t | x_t-1) = 𝒩(x_t; √(α_t) x_t-1, β_t 𝐈),

where α_t = 1 - β_t.
Owing to the reproductivity of the normal distribution, q(x_t | x_0) can be obtained analytically as follows:

  
  q(x_t | x_0) = 𝒩(x_t; √(α̅_̅t̅) x_0, (1 - α̅_̅t̅) 𝐈),

where α̅_t = ∏_i=1^t α_i.
Using a reparameterization trick <cit.>, Equation (<ref>) can be rewritten as

  
  x_t = √(α̅_̅t̅) x_0 + √(1 - α̅_̅t̅) ϵ,

where ϵ∼𝒩(0, 𝐈).
In practice, β_t is fixed at constant values <cit.> with a predetermined noise schedule (e.g., a cosine schedule <cit.>).

Reverse diffusion process.
A one-step reverse diffusion process p_θ(x_t-1 | x_t) is defined as follows:

  
  p_θ(x_t-1 | x_t) = 𝒩(x_t - 1; μ_θ(x_t, t, s, p), σ_t^2 𝐈),

where μ_θ indicates the output of a model that is parameterized using θ, conditioned on t, speaker embedding s, and phoneme embedding p, and σ_t^2 = 1 - α̅_t - 1/1 - α̅_̅t̅β_t.
Unless otherwise specified, x_0, s, and p are extracted from the same waveform.
Through reparameterization <cit.>, Equation (<ref>) can be rewritten as

  
  x_t - 1 = μ_θ(x_t, t, s, p) + σ_t z,

where z∼𝒩(0, I).

Training process.
The training objective of DDPM is to minimize the variational bound on the negative log-likelihood 𝔼 [ - log p_θ (x_0) ]:

  
  ℒ_DDPM(θ)
  = 𝔼_q(x_1:T | x_0) [ - logp_θ(x_0:T)/q(x_1:T | x_0) ].

Using Equation (<ref>) and the following reparameterization

  
  
  μ_θ(x_t, t, s, p) = 1/√(α_t) ( x_t - 1 - α_t/√(1 - α̅_̅t̅) ϵ_θ(x_t, t, s, p) ),

Equation (<ref>) can be rewritten as follows:

  
  ℒ_DDPM(θ) = ∑_t=1^T w_t 𝔼_x_0, ϵ [ ‖ϵ - ϵ_θ(x_t, t, s, p) ‖_1 ],

where ϵ_θ represents a noise predictor that predicts ϵ using x_t, t, s, and p.
See <cit.> for detailed derivations of Equations (<ref>)–(<ref>).
Here, w_t is a constant and is set to 1 in practice for better training <cit.>.
In the original DDPM <cit.>, the L2 loss is used in Equation (<ref>); however, we use the L1 loss according to <cit.>, which shows that the L1 loss is better than the L2 loss.

Conversion process.
When ϵ_θ is trained, VoiceGrad can convert the given source mel-spectrogram x_0^src into a target mel-spectrogram x_0^tgt using Algorithm <ref>.
Here, we use the superscripts src and tgt to indicate that the data are related to the source and target speakers, respectively.
In this algorithm, a target speaker embedding s^tgt and a source phoneme embedding p^src are used as auxiliary information.
To accelerate sampling <cit.>, we use the subsequence { S_K, …, S_1 } as a sequence of t values instead of { T, …, 1 }, where K ≤ T.
Owing to this change, α_S_k is redefined as α_S_k = α̅_S_k/α̅_S_k-1 for k > 1 and α_S_k = α̅_S_k for k = 1.
σ_S_k is modified accordingly.
Note that VC is a conversion task and not a generation task; therefore, x_0^src is used as an initial value of x (line 1) instead of random noise x_T ∼𝒩(0, I), which is typically used in a generation task.
For the same reason, the initial value of t is adjusted from T to S_K < T (line 2) to initiate the reverse diffusion process from the midterm state rather than from the noise.





§ PROPOSAL: FASTVOICEGRAD





 §.§ Rethinking initial states in sampling



In Algorithm <ref>, the two crucial factors that affect the inheritance of source speech are the initial values of (1) x and (2) t.

Rethinking the initial value of x.
When the initial value of x is set to x∼𝒩(0, 𝐈) (a strategy used in generation), no gap occurs between training and inference; however, we cannot inherit the source information, that is, x_0^src, which is useful for VC to preserve the content.
In contrast, when x_0^src is directly used as the initial value of x (the strategy used in VoiceGrad), we can inherit the source information, but a gap occurs between training and inference.
Considering both aspects, we propose the use of a diffused source mel-spectrogram x_S_K^src, defined as

  
  x_S_K^src =  √(α̅_S_K) x_0^src + √(1 - α̅_S_K) ϵ.

In line 1 of Algorithm <ref>, x_S_K^src is used instead of x_0^src.
The effect of this replacement is discussed in the next paragraph.



Rethinking the initial value of t (i.e., S_K).
As S_K is closer to T, x can be transformed to a greater extent under the assumption that it contains more noise, but can corrupt essential information.
As this is a nontrivial tradeoff, it is empirically investigated.
Figure <ref> shows the relationship between S_K and DNSMOS <cit.> (corresponding to speech quality) and that between S_K and speaker verification accuracy (SVA) <cit.> (corresponding to speaker similarity).
We present the results for two cases in which x_0^src and x_S_K^src are used as the initial values of x.
K was set to 1; that is, one-step reverse diffusion was conducted.
We observe that SVA improves as S_K increases because x is largely transformed toward the target speaker in this case.
When x was initialized with x_0^src, DNSMOS worsens as S_K increases.
In contrast, when x was initialized with x_S_K^src, DNSMOS worsens once but then becomes better, possibly because, in the latter case, the gap between training and inference is alleviated via a diffusion process (Equation (<ref>)) as S_K increases.
Both scores decreasd significantly when S_K = 1000, where x was denoised under the assumption that x is noise.
These results indicate that the one-step reverse diffusion should begin under the assumption that x contains the source information, albeit in extremely small amounts.[Note that, if K is sufficiently large, adequate speech can be obtained even with S_K = 1000 at the expense of speed.]
A comparison of the results for x_0^src and x_S_K^src indicates that x_S_K^src is superior, particularly when considering the SVA.
Based on these results, we used x_S_K^src with S_K = 950 in the subsequent experiments.
Figure <ref> shows the results for this setting.



 §.§ Adversarial conditional diffusion distillation



Owing to the difficulty in learning a one-step diffusion model comparable to a multi-step model from scratch, we used a model pretrained using the standard VoiceGrad as an initial model and improved it through ACDD.
Inspired by ADD <cit.>, which was proposed for image generation, we used adversarial loss and score distillation loss in distillation.

Adversarial loss.
Initially, we considered directly applying a discriminator to the mel spectrogram, similar to the previous GAN-based VC (e.g., <cit.>).
However, we could not determine an optimal discriminator to eliminate the buzzy sound in the waveform.
Therefore, we converted the mel spectrogram into a waveform using a neural vocoder 𝒱 (with frozen parameters) and applied a discriminator 𝒟 in the waveform domain.
More specifically, adversarial loss (particularly least-squares GAN <cit.>-based loss) is expressed as follows:

  
  ℒ_adv(𝒟)
      = 𝔼_x_0 [ (𝒟(𝒱(x_0)) - 1)^2
    + (𝒟(𝒱(x_θ)))^2],
  

  ℒ_adv(θ)
      = 𝔼_x_0 [ (𝒟(𝒱(x_θ)) - 1)^2 ],

where x_0 represents a mel spectrogram extracted from real speech.
x_θ represents a mel spectrogram generated using x_θ = μ_θ(x_S_k, S_K, s, p) (one-step denoising prediction defined in Equation (<ref>)), where x_S_K is the S_K-step diffused x_0 via Equation (<ref>).
The adversarial loss is used to improve the reality of x_θ through adversarial training.

Furthermore, following the training of a neural vocoder <cit.>, we used the feature matching (FM) loss, defined as

  
  ℒ_FM(θ)
  = 𝔼_x_0[ ∑_l = 1^L 1/N_l ‖𝒟_l(𝒱(x_0)) - 𝒟_l(𝒱(x_θ)) ‖_1 ],

where L indicates the number of layers in 𝒟.
𝒟_l and N_l denote the features and the number of features in the l-th layer of 𝒟, respectively.
ℒ_FM(θ) bears x_θ closer to x_0 in the discriminator feature space.

Score distillation loss.
The score distillation loss <cit.> is formulated as follows:

  
  ℒ_dist(θ)
  = 𝔼_t, x_0 [ c(t) ‖x_ϕ - x_θ ‖_1 ],

where x_ϕ is one-step denoising prediction (Equation (<ref>)) generated by a teacher diffusion model parameterized with ϕ (frozen in training): x_ϕ = μ_ϕ(sg(x_θ, t), t, s, p).
Here, sg denotes the stop-gradient operation, x_θ, t is the t-step diffused x_θ via Equation (<ref>), and t ∈{ 1, …, T }.
c(t) is a weighting term and is set to α_t in practice to allow higher noise levels to contribute less <cit.>.
ℒ_dist(θ) encourages x_θ (student output) to match x_ϕ (teacher output).

Total loss.
The total loss is expressed as follows:

  
  ℒ_ACDD(θ)     = ℒ_adv(θ) + λ_FM ℒ_FM(θ) + λ_dist ℒ_dist (θ),
  

  
  ℒ_ACDD(𝒟)     = ℒ_adv(𝒟),

where λ_FM and λ_dist are weighting hyperparameters set to 2 and 45, respectively, in the experiments.
θ and 𝒟 are optimized by minimizing ℒ_ACDD(θ) and ℒ_ACDD(𝒟), respectively.



§ EXPERIMENTS





 §.§ Experimental settings



Data.
We examined the effectiveness of FastVoiceGrad on one-shot any-to-any VC using the VCTK dataset <cit.>, which included the speeches of 110 English speakers.
To evaluate the unseen-to-unseen scenarios, we used 10 speakers and 10 sentences for testing, whereas the remaining 100 speakers and approximately 390 sentences were used for training.
Following DiffVC <cit.>, audio clips were downsampled at 22.05kHz, and 80-dimensional log-mel spectrograms were extracted from the audio clips with an FFT size of 1024, hop length of 256, and window length of 1024.
These mel spectrograms were used as conversion targets.

Comparison models.
We used VoiceGrad <cit.> (Section <ref>) as the main baseline and distilled it into FastVoiceGrad.
A diffusion model has a tradeoff between speed and quality according to the number of reverse diffusion steps (K).
To investigate this effect, we examined three variants: VoiceGrad-1, VoiceGrad-6, and VoiceGrad-30, which are VoiceGrad with K = 1, K = 6, and K = 30, respectively.
VoiceGrad-1 is as fast as FastVoiceGrad, whereas the others are slower.
For an ablation study, we examined FastVoiceGrad_adv and FastVoiceGrad_dist, in which score distillation and adversarial losses were ablated, respectively.
As another strong baseline, we examined DiffVC <cit.>, which has demonstrated superior quality compared to representative one-shot VC models <cit.>.
Based on <cit.>, we used two variants: DiffVC-6 and DiffVC-30, that is, DiffVC with six and 30 reverse diffusion steps, respectively.

Implementation.
VoiceGrad and FastVoiceGrad were implemented while referring to <cit.>.
We implemented ϵ_θ using U-Net <cit.>, which consisted of 12 one-dimensional convolution layers of 512 hidden channels with two downsampling/upsampling, gated linear unit (GLU) activation <cit.>, and weight normalization <cit.>.
The two main changes from <cit.> were that speaker embedding s was extracted by a speaker encoder <cit.> instead of a speaker label, and t was encoded by sinusoidal positional embedding <cit.> instead of one-hot embedding.
We extracted p using a bottleneck feature extractor (BNE) <cit.>.
We implemented 𝒱 and 𝒟 using the modified HiFi-GAN-V1 <cit.>, in which a multiscale discriminator <cit.> was replaced with a multiresolution discriminator <cit.> that showed better performance in speech synthesis <cit.>.
We trained VoiceGrad using the Adam optimizer <cit.> with a batch size of 32, learning rate of 0.0002, and momentum terms (β_1, β_2) = (0.9, 0.999) for 500 epochs.
We trained FastVoiceGrad using the Adam optimizer <cit.> with a batch size of 32, learning rate of 0.0002, and momentum terms (β_1, β_2) = (0.5, 0.9) for 100 epochs.
We implemented DiffVC using the official code.[<https://github.com/huawei-noah/Speech-Backbones/tree/main/DiffVC>]

Evaluation.
We conducted mean opinion score (MOS) tests to evaluate perceptual quality.
We used 90 different speaker/sentence pairs for the subjective evaluation.
For the speech quality test (qMOS), nine listeners assessed the speech quality on a five-point scale: 1 = bad, 2 = poor, 3 = fair, 4 = good, and 5 = excellent.
For the speaker similarity test (sMOS), ten listeners evaluated speaker similarity on a four-point scale: 1 = different (sure), 2 = different (not sure), 3 = same (not sure), and 4 = same (sure), in which the evaluated speech was played after the target speech (with a different sentence).
As objective metrics, we used UTMOS <cit.>, DNSMOS <cit.>, and character error rate (CER) <cit.> to evaluate speech quality.
We used DNSMOS (MOS sensitive to noise) in addition to UTMOS (which achieved the highest score in the VoiceMOS Challenge 2022 <cit.>) because we found that UTMOS is insensitive to speech with noise, which typically occurs when using a diffusion model with a few reverse diffusion steps.
We evaluated speaker similarity using SVA <cit.>, in which we verified whether converted and target speech are uttered by the same speaker.
We used 8,100 different speaker/sentence pairs for objective evaluation.
The audio samples are available from the link indicated on the first page of this manuscript.foot:samples



 §.§ Experimental results



Table <ref> summarizes these results.
We observed that FastVoiceGrad not only outperformed the ablated FastVoiceGrads (FastVoiceGrad_adv and FastVoiceGrad_dist) and VoiceGrad-1, which have the same speed, but was also superior to or comparable to VoiceGrad-6 and VoiceGrad-30, of which calculation costs were as six and 30 times as FastVoiceGrad, respectively.
Furthermore, FastVoiceGrad was superior to or comparable to DiffVCs (DiffVC-6 and DiffVC-30) in terms of all metrics.[On the Mann–Whitney U test (p-value > 0.05), FastVoiceGrad is not significantly different from VoiceGrad-30/6 and DiffVC-30 but significantly better than the other baselines for qMOS, and FastVoiceGrad is significantly better than all baselines for sMOS.]
For a single A100 GPU, the real-time factors of mel-spectrogram conversion and total VC (including feature extraction and waveform synthesis) for FastVoiceGrad are 0.003 and 0.060, respectively, which are faster than those for DiffVC-6 (fast variant), which are 0.094 and 0.135, respectively.
These results indicate that FastVoiceGrad can enhance the inference speed while achieving high VC performance.



 §.§ Application to another dataset







To confirm this generality, we evaluated FastVoiceGrad on the LibriTTS dataset <cit.>.
We used the same networks and training settings as those for the VCTK dataset, except that the training epochs for VoiceGrad and FastVoiceGrad were reduced to 300 and 50, respectively, owing to an increase in the amount of training data.
Table <ref> summarizes the results.
The same tendencies were observed in that FastVoiceGrad not only outperformed VoiceGrad-1 (a model with the same speed) but was also superior to or comparable to the other baselines.



§ CONCLUSION



We proposed FastVoiceGrad, a one-step diffusion-based VC model that can achieve VC performance comparable to or superior to multi-step diffusion-based VC models while reducing the number of iterations to one.
The experimental results demonstrated the importance of carefully setting of the initial states in sampling and the necessity of the joint use of GANs and diffusion models in distillation.
Future research should include applications to advanced VC tasks (e.g., emotional VC and accent correction) and an extension to real-time implementation.



§ ACKNOWLEDGEMENTS

This work was supported by JST CREST Grant Number JPMJCR19A3, Japan.

IEEEtran


