waveforms.**

$l_1$ loss often leads to blurry results [@yu2018free; @chang2019free], so we adopt the perceptual loss [@gatys2015neural] originally used for style transfer to enhance the audio quality. It is also used for image inpainting [@liu2018image; @yu2018free], video inpainting [@chang2019free] and super-resolution [@johnson2016perceptual; @ledig2017photo].

Similar as pre-trained VGG [@simonyan2014very] on ImageNet [@russakovsky2015imagenet] for image perceptual loss, we use pre-trained SoundNet and fine-tune it on our benchmark dataset with classification task for audio perceptual loss: $$\label{percputal_loss}
L_{perc} = \sum_{t=1}^{n} \frac{|\Psi^{O_{t}} - \Psi^{V_{t}}|}{N_{\Psi^{V_{t}}}}$$ where $\Psi$ is the features extracted from last layer before fully-connected of the fine-tuned SoundNet. Note that we follow a similar fashion as how [@chen2018visually] uses a pretrained SoundNet to compute audio perceptual loss.

#### **Perceptual loss on spectrograms.**

Aside from waveforms, we propose to consider perceptual loss on spectrograms, as image classification is relatively easier to learn. We transform waveforms to spectrograms with STFT, which are then used to fine-tune a ResNet50 [@he2016deep] pretrained on ImageNet [@russakovsky2015imagenet] for audio classification. The fine-tuned ResNet50 then serves as a feature extractor for perceptual loss on spectrograms, as shown in Equation \[percputal\_loss\].

Experimental Results
====================

Datasets
--------

#### **SC09.**

SC09 dataset is a subset of the Speech Commands Dataset [@warden2018speech] that contains 