<p>I am not sure if this question is suitable in its current form in this forum (stack exchange might perhaps be a better fit), but as it is quite relevant for DNN based speech synthesis pipelines, I think it is a good idea to expand on it a bit.</p>
<p>We cannot reconstruct exactly the STFT from a Mel spectrogram. The reason is that we the Mel is a 'compressed' version of the STFT with the frequencies coming from the Mel scale and then applying (to the STFT) triangular filters at these frequencies. Generally, we lose information going from STFT to mel. See this excellent article for detailed explanation.</p>
<p><a href="https://haythamfayek.com/2016/04/21/speech-processing-for-machine-learning.html" rel="nofollow noreferrer">https://haythamfayek.com/2016/04/21/speech-processing-for-machine-learning.html</a></p>
<p>Now, to get back to your question - I assume that you are working on speech synthesis in the manner of the Tacotron [1] work - in order to apply Griffin Lim, as you correctly note, we need the linear spectrogram. The way it is done in the paper is to use a neural network to transform Mel to STFT. They call this a postnet, as it serves as a post processor after the Mels are predicted.</p>
<p>To set up this network, convert ground truth (target) audio into Mels, and create a recurrent network (CBHG or anything else) to transform it into STFT equivalents. Minimize loss between these STFT predictions and the actual STFT which we can create from target audio.</p>
<p>[1] <a href="https://arxiv.org/abs/1703.10135" rel="nofollow noreferrer">https://arxiv.org/abs/1703.10135</a></p>
How can I reconstruct the STFT matrix from a mel spectrogram in librosa so I reconstruct the original audio?
<p>I have generated a melspectrogram in librosa using the following code</p>
<pre><code>import os
from matplotlib import pyplot as plt
import librosa
import librosa.display
import pylab
import numpy as np


x, sr = librosa.load('audio/example.wav')
mel = librosa.feature.melspectrogram(x,sr)
P = librosa.power_to_db(mel, ref=np.max)
librosa.display.specshow(P)
pylab.savefig(&quot;example.png&quot;, bbox_inches=None, pad_inches=0)
</code></pre>
<p>As I understand, the spectrogram is simply a visual representation of the STFT matrix for an audio signal. I'm trying to reconstruct the STFT matrix used to generate the spectrogram in order to pass it through the griffin lim function. How should I do this?</p>
<p>Generating Spectrogram using STFT data</p>
<pre><code>def generate_spectrogram(x, sr):
    X = librosa.stft(x)
    Xdb = librosa.amplitude_to_db(abs(X))
    fig = plt.figure(figsize=(10, 10), dpi=100, frameon=False)
    ax = fig.add_axes([0, 0, 1, 1], frameon=False)
    ax.axis('off')
    librosa.display.specshow(Xdb, sr=sr, cmap='gray', x_axis='time', y_axis='hz')
    plt.savefig('example.png', quality=100, bbox_inches=0, pad_inches=0)
    librosa.cache.clear()
</code></pre>