Latent Watermarking of Audio Generative Models
Robin San Roman Pierre Fernandez Antoine Deleforge Yossi Adi Romain Serizel







Latent Watermarking of Audio Generative Models
    
Robin San Roman
Meta, FAIR 

Inria Nancy
Pierre Fernandez
Meta, FAIR
 
Inria Rennes
Antoine Deleforge
Inria Nancy

Yossi Adi
Meta, FAIR 

Hebrew University of Jerusalem
Romain Serizel
Inria Nancy



    Received XXXX; Accepted YYYY
===========================================================================================================================================================================================================



§ ABSTRACT 
The advancements in audio generative models have opened up new challenges in their responsible disclosure and the detection of their misuse. In response, we introduce a method to watermark latent generative models by a specific watermarking of their training data.
The resulting watermarked models produce latent representations whose decoded outputs are detected with high confidence, regardless of the decoding method used. This approach enables the detection of the generated content without the need for a post-hoc watermarking step.
It provides a more secure solution for open-sourced models and facilitates the identification of derivative works that fine-tune or use these models without adhering to their license terms.
Our results indicate for instance that generated outputs are detected with an accuracy of more than 75% at a false positive rate of 10-3, even after fine-tuning the latent generative model. 




watermarking, audio, generative




§ INTRODUCTION


Sophisticated generative models are impacting various audio modalities: environmental sounds <cit.>, music <cit.>, and speech <cit.>. 
These models produce outputs increasingly indistinguishable from real data <cit.>. Their rapid proliferation and quality raise concerns about misuse (e.g. creation of deepfakes) and respect for intellectual property. 
These concerns are heightened when models are open-sourced, since they can be easily accessed and used by anyone, including malicious actors.
Consequently, regulators suggest watermarking to label and detect generative model outputs (refer to the https://artificialintelligenceact.eu/EU AI Act, https://www.whitehouse.gov/briefing-room/presidential-actions/2023/10/30/executive-order-on-the-safe-secure-and-trustworthy-development-and-use-of-artificial-intelligence/White House executive order, and http://www.cac.gov.cn/2023-07/13/c_1690898327029107.htmCAC measures).

Watermarking is a technique that slightly alters the audio after its generation, in a way that is inaudible for humans but identifiable by specific detection algorithms.
The state-of-the-art methods are based on deep neural networks <cit.> that are trained end-to-end to embed and detect watermarks in audio signals, even after audio compression or editing.
Such methods are for instance employed to safeguard APIs for public model demonstrations <cit.>.
However, while post-hoc watermarking has proven effective in certain scenarios, it is not as effective for protecting open-sourced models, as malicious users could potentially extract the output before the watermarking stage (for example by commenting out the code responsible for watermark embedding).

In the image domain, some methods <cit.> fine-tune decoders to output watermarked images directly, to make it compliant with open-sourcing.
However, in the audio domain, it is common and cost-effective to train decoders (also called vocoders) that convert latent representations to waveforms <cit.>.
Watermarking can thus be easily bypassed by using non-watermarked vocoders. 
Therefore, in this article, we propose to watermark the latent generative model that creates the latent representations.

We focus on MusicGen <cit.> due to its performance and adoption.
It consists of an auto-encoder EnCodec <cit.> that compresses audio into discrete representations (tokens) and a single-stage transformer (audio Language Model, LM) that predicts the next tokens and decodes them into a music stream.
We train watermark generator/detector models to be robust to EnCodec.
Intuitively, this makes both the audio and encoded tokens watermarked.
We then train the LM on tokens derived from audios that were preemptively marked.
The resulting LM produces tokens whose decoded outputs are watermarked, irrespective of the LM conditioning or decoding algorithm.
In other terms, as long as the watermarking algorithm withstands the audio tokenization, the watermark transfers from the training data to the generative model outputs.

In short, 
(1) we introduce a way to watermark audio generative models at the latent representations level, 
(2) we demonstrate that it makes generations detectable with high confidence while having almost no influence on the model performance, 
(3) we demonstrate the robustness of the watermark to model-level changes, namely, switching the decoding algorithm and fine-tuning the audio LM.






§ RELATED WORK


Audio generation is a challenging task because audio signals are high-dimensional and have complex temporal dependencies.
Early autoregressive deep-learning-based approaches like WaveNet <cit.> were quickly followed by GAN-based models <cit.>.
Inspired by progress in text generation <cit.>, audio language models (LM) have recently emerged as state of the art for most audio generative tasks such as text-to-speech <cit.>, music <cit.> or sound <cit.> generation.
They make audio modeling more tractable by compressing audio into discrete tokens using models like EnCodec <cit.>, SoundStream <cit.>, or DAC <cit.>. 
Additional tokens coming from text, melody, phoneme, speaker embedding, etc. may serve as conditioning to generate audio with user-specific characteristics.
Then a transformer-based model <cit.> generates audio by predicting the next tokens and decoding them.

In parallel to audio LMs, latent diffusion models have also been largely studied in recent works on audio generation. Those models can sample in an non autoregressive way from the training data distribution and have recently shown great generative habilities on different audio modalities such as speech <cit.>, music <cit.> or general audio <cit.>. 




Invisible audio watermarking
has evolved from using domain-specific features in the time/frequency domain of audios <cit.> to deep learning methods that employ encoder/decoder architectures <cit.>. 
Notably, AudioSeal <cit.> introduces localized audio watermarking with a detector producing time-step-level logits. This method also allows for a watermarking robust to neural compression models which is a necessary element for our work.


Generative model watermarking is attracting renewed interest thanks to its potential to improve detection of AI-generated contents.
In this context, the aforementioned methods apply watermarking post-hoc after audio generation, unlike more recent methods which do so in-model.
Examples include watermarking: image GANs by training a hyper-network model <cit.>, latent diffusion models by a quick fine-tuning of their decoder <cit.>, and HiFi-GAN decoders that take mel-spectrograms and output waveforms <cit.>. 
Unlike the last two approaches, our method operates one step earlier at the latent representation level.
It draws inspiration from research demonstrating that watermarks embedded in images or texts may propagate from the training data of generative models to their outputs <cit.>. 
We apply this concept to audio generative models and target audio language models.

















§ AUDIO MODEL WATERMARKING




 §.§ Problem statement


We consider providers training an audio-generative model on a large proprietary dataset. 
They aim to release the model publicly but worry about misuse and unauthorized redistribution. 
To mitigate these concerns, they watermark the model during training to enhance the detection of generated content or unauthorized API usage. 
We describe this watermarking process below and provide a step by step overview in Figure <ref>.



 §.§ Audio watermarking


We first build an audio watermarking model based on AudioSeal <cit.>.
It jointly trains a watermark generator G and a watermark detector D. 
G takes a signal s ∈ℝ^T and generates an additive watermark δ_w, that is made imperceptible through perceptual losses ℒ_percep(s, s+δ_w).
The watermarked audio s+δ_w is augmented into s'. 
Augmentations include padding the audio with 0, replacing intervals of watermarked audio with non-watermarked audio from the same batch, or dropping the δ_w.
s' is fed to the detector, which is trained to output which part of a waveform is watermarked via a localization loss ℒ_loc(D(s'), y'), 
where y' ∈{0,1}^T indicates the watermarked intervals in s' (1 for watermarked, 0 otherwise).

We make the following changes with regard to AudioSeal's recipe.
We remove the message encoder to only focus on watermark detection.
Furthermore, it is important to remember that the audio LM will be trained on tokens, not directly on audio. 
Hence, the LM will not retain watermark information if it is absent at the discrete representation level. 
Therefore, we train the watermark generation/detection to be very robust to the specific EnCodec compression model later used for audio tokenization (see Sec. <ref>).
This is done by oversampling this EnCodec augmentation so that 50% of batches go through EnCodec before the detection phase. 




 §.§ Audio language model


We select MusicGen <cit.> as the audio LM to watermark. 


Watermarking.
The first step is to watermark the audios with the model of Sec. <ref>.
This is done on the fly at loading time (this takes around 10 ms for a 10-second audio).



Tokenization.
We use the EnCodec compression model to transform the audio signal into a discrete sequence of tokens suitable for language modeling.
It uses residual vector quantization (RVQ) <cit.> which compresses an audio signal s ∈ℝ^T into K streams of tokens (u^(j)_i ) _j ∈ [1,K]   ;  i ∈ [0, T / f_r] (f_r being the frame rate).
This model is trained on audio segments sampled at a rate of 32 kHz and f_r = 50 Hz, the number of codebooks is K=4 and the codebook size is 2048 (u^(j)_i ∈ [1,2048]). Overall, this results in a overall bit rate of 2.2 kbit per second


Language modeling aims to build a probabilistic model of sequences of discrete tokens.
MusicGen implements a delay pattern <cit.> that adds a delay k to the k-th residual. 
It allows the model to generate the tokens in a coarse to fine order. 
This way all the streams can be sampled in parallel while assuring that all the previous residuals are fixed when sampling a given token.
Put differently, the transformer is fed with a sequence of embeddings created from K tokens: s_i = {u^(4)_i-4, u^(3)_i-2, u^(2)_i-1, u^(1)_i}.
The embedding of s_i is then the sum of the embedding of each of its constitutive tokens, with additional sinusoidal embeddings. 
As most current language models <cit.>, training is done with next token prediction and the cross-entropy is computed per codebook.




§ EXPERIMENTS



 §.§ Experimental details


Watermarking models. 
We train on an internal music dataset containing 1.5k songs at 32 kHz sample rate, with 1-second audio excerpts.
The model is trained for 400k steps with batch size 64. Hyper-parameters (network architectures, optimizer...) are kept the same as in the original work <cit.>.


MusicGen models. 
We use 20k hours of licensed music to train the models with two different model sizes: small (300M parameters) and medium (1.5B parameters).
They are similar in quality and diversity to the ones used by Copet et al. <cit.>. 
We use public implementation and default parameters available on the https://github.com/facebookresearch/audiocraftAudioCraft GitHub page.
Each model is trained for 200 epochs with a batch size of 128, with the default optimizer.
We use 64 GPUs for the medium model and 32 for the small.


Inference. 
For music generation sampling, we use top-k sampling with k=250 tokens and a temperature of 1.0.




 §.§ Quality of the audio generative model


We first subjectively evaluate how watermarking influences the quality of the generative models.

We adhere to the original paper's protocol <cit.>, using (OVL) to assess sample quality and (REL) to evaluate relevance to text prompts.
Models are tested on 15-second generation using 40 text prompts from the test set of MusicCaps <cit.>. 
Every sample is rated by 20 listeners that rate them on a scale from 1 to 100. 
For every study, we report both mean score and CI95. 
<ref> shows that the performance difference between a model trained on watermarked data and one trained on normal data is negligible.
This holds true for both sizes, with the rating difference falling within the confidence interval.






 §.§ Detection and localization results


Detection.
To evaluate the detection performance, we generate 10k positive 15-second samples with the watermarked model and use 10k negative samples from our test set that we compress using the codec model. 
The watermark detector gives a score per time-step of the audio, which we average to get a score for the whole audio.
Audio is flagged as watermarked if this score is higher than a threshold τ.
We report in Tab. <ref> the area under the ROC curve, as well as the accuracy for the best τ (and true positive and false positive rates at this τ, TPR, and FPR).
The generated output is indeed watermarked as indicated by the detection metrics: the AUC is close to 1 and TPR is higher than 0.95 at FPR around 10^-4.





Localization.
We then evaluate if the detector still has the property to locally detect watermarked segments.
To do so, we generate 15-second samples and replace parts with other non-watermarked audio from our test set. The proportion of signal that is watermarked is 50% on average.  We then measure the precision of the detection using the detection accuracy at the sample level together with the intersection over union (IoU) metric. For localization, we use a fixed detection threshold set at 0.5.
As shown in Tab. <ref>, results are on-par (although a bit lower) to AudioSeal, showing that the detector keeps a good-enough performance on the localization of generated outputs. 

    

Robustness. We evaluate the robustness to different audio edits, and compare the performance of our in-model watermarking and of post-hoc watermarking that directly applies the watermark to generated outputs with AudioSeal. 
The evaluation is made on 10k samples.




<ref> shows that while in-model watermarking keeps a decent robustness to common audio edits there is a slight performance decrease compared to the original watermarking model. 
Therefore, when post-hoc watermarking is feasible, it might be preferable to in-model watermarking. 
The latter is better suited for scenarios where post-hoc watermarking is not possible, such as when open-sourcing a model.








§ ATTACKS ON THE MODEL'S WATERMARK


We now focus on model-level attacks, i.e., modifications of the model attempting to make its outputs undetectable.


 §.§ Switching decoder


Previous works alter the latent decoder to embed the watermark <cit.>. 
However, audio vocoders are relatively easy to train and interchange <cit.>.
They do not necessitate extensive data or computational power compared to those required for training an audio LM. 
Therefore, replacing the decoder to use the watermark-free generative model is rather straightforward. 
In contrast, our work embeds the watermark at the latent stage for robustness against decoder changes.

We now evaluate how the change of the decoder influences detection performance.
In previous experiments, the default decoder was the codec model from MusicGen. 
We replace it with Multi-Band Diffusion <cit.> which uses diffusion to map discrete EnCodec tokens into the waveform domain, and a discrete version HiFi-GAN <cit.>, which we trained on tokens-waveform pairs.
We use the same experimental setup as in Sec. <ref>, but with different algorithms to decode the tokens. 
<ref> shows that changing the decoder has very little impact on the detection metrics.
Notably, using the diffusion-based decoder reduces the AUC only by around 0.01.






 §.§ Model fine-tuning


One potential attack could be to remove the watermark through “model purification”, which involves fine-tuning the language model on a non-watermarked dataset. 
To test this we fine-tune with different learning rates the small version of the model using a different internal music dataset D_FT of similar size without watermarks.
For each setting, the model is trained for 20 epochs (10% of the total pre-training steps). 
We then generate 10k samples with the purified models and obtain scores through the watermark detector.

For each experiment, we report the accuracy obtained for the best threshold on the detection score as well as the TPR when the threshold is chosen such that the FPR is at 10^-3. 
We also report the Fréchet Audio Distance (FAD) <cit.> that evaluates the quality of the generative model.
We include as a reference the performance of the model before fine-tuning and of a model trained from scratch on D_FT.




<ref> suggests that a higher learning rate during fine-tuning makes watermarks more difficult to detect, but it also causes the distribution of generated data to deviate further from the protected model's dataset; at larger learning rates, the distribution has a similar FAD to a model trained from scratch on different data.
In other words, since the FAD is almost the same as a model trained from scratch, it may not be worthwhile to start from the watermarked model.




§ CONCLUSION & DISCUSSION
 

This work introduced a straightforward yet effective approach to watermarking audio language models.
This is done via watermarking their training data in a way that is robust to the compression algorithm used to create tokens.
It does not require modifications to the model architecture or the training process.
Our method is the first to watermark at the latent level and is robust to changes in the decoding process.
The main drawback of the current approach is that it requires training the model from scratch, which may be difficult for versioning large models or for adapting already-trained models.
While there is a slight decrease in robustness to audio edits compared to post-hoc watermarking, this method allows to keep the watermark in situations for which post-hoc watermarking is not suitable (open sourcing...). 
In conclusion, watermarking can help trace content origin and support regulatory efforts. 
It is not a standalone solution and should be complemented with measures like policies, education, or monitoring.









IEEEbib

