NeMo
rlangman commited on
Commit
5e48511
·
verified ·
1 Parent(s): 22160a7

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +8 -7
README.md CHANGED
@@ -59,11 +59,11 @@ import torch
59
  import soundfile as sf
60
  from nemo.collections.tts.models import AudioCodecModel
61
 
62
- codec_path = ??? # set here the model .nemo checkpoint path
63
  path_to_input_audio = ??? # path of the input audio
64
  path_to_output_audio = ??? # path of the reconstructed output audio
65
 
66
- nemo_codec_model = AudioCodecModel.restore_from(restore_path=codec_path, map_location="cpu").eval()
67
 
68
  # get discrete tokens from audio
69
  audio, _ = librosa.load(path_to_input_audio, sr=nemo_codec_model.sample_rate)
@@ -72,10 +72,11 @@ device = 'cuda' if torch.cuda.is_available() else 'cpu'
72
  audio_tensor = torch.from_numpy(audio).unsqueeze(dim=0).to(device)
73
  audio_len = torch.tensor([audio_tensor[0].shape[0]]).to(device)
74
 
75
- encoded_tokens, encoded_len = nemo_codec_model.encode(audio=audio_tensor, audio_len=audio_len)
76
-
77
- # Reconstruct audio from tokens
78
- reconstructed_audio, _ = nemo_codec_model.decode(tokens=encoded_tokens, tokens_len=encoded_len)
 
79
 
80
  # save reconstructed audio
81
  output_audio = reconstructed_audio.cpu().numpy().squeeze()
@@ -106,7 +107,7 @@ The NeMo Audio Codec is trained on a total of 28.7k hrs of speech data from 105
106
 
107
  We evaluate our codec using several objective audio quality metrics. We evaluate [ViSQOL](https://github.com/google/visqol) and [PESQ](https://lightning.ai/docs/torchmetrics/stable/audio/perceptual_evaluation_speech_quality.html) for perception quality, [ESTOI](https://ieeexplore.ieee.org/document/7539284) for intelligbility, mel spectrogram and STFT distances for spectral reconstruction accuracy, and [SI-SDR](https://arxiv.org/abs/1811.02508) for phase reconstruction accuracy. Metrics are reported on the test set for both the MLS English and CommonVoice data. The model has not been trained or evaluated on non-speech audio.
108
 
109
- | Dataset | ViSQOL |PESQ |ESTOI |Mel Distance |STFT Distance|SI-SDR|
110
  |:-----------:|:----------:|:----------:|:----------:|:-----------:|:-----------:|:-----------:|
111
  | MLS English | 4.50 | 3.69 | 0.94 | 0.066 | 0.033 | 8.33 |
112
  | CommonVoice | 4.53 | 3.55 | 0.93 | 0.100 | 0.057 | 7.63 |
 
59
  import soundfile as sf
60
  from nemo.collections.tts.models import AudioCodecModel
61
 
62
+ model_name = "nvidia/audio-codec-22khz"
63
  path_to_input_audio = ??? # path of the input audio
64
  path_to_output_audio = ??? # path of the reconstructed output audio
65
 
66
+ nemo_codec_model = AudioCodecModel.from_pretrained(model_name).eval()
67
 
68
  # get discrete tokens from audio
69
  audio, _ = librosa.load(path_to_input_audio, sr=nemo_codec_model.sample_rate)
 
72
  audio_tensor = torch.from_numpy(audio).unsqueeze(dim=0).to(device)
73
  audio_len = torch.tensor([audio_tensor[0].shape[0]]).to(device)
74
 
75
+ with torch.no_grad():
76
+ encoded_tokens, encoded_len = nemo_codec_model.encode(audio=audio_tensor, audio_len=audio_len)
77
+
78
+ # Reconstruct audio from tokens
79
+ reconstructed_audio, _ = nemo_codec_model.decode(tokens=encoded_tokens, tokens_len=encoded_len)
80
 
81
  # save reconstructed audio
82
  output_audio = reconstructed_audio.cpu().numpy().squeeze()
 
107
 
108
  We evaluate our codec using several objective audio quality metrics. We evaluate [ViSQOL](https://github.com/google/visqol) and [PESQ](https://lightning.ai/docs/torchmetrics/stable/audio/perceptual_evaluation_speech_quality.html) for perception quality, [ESTOI](https://ieeexplore.ieee.org/document/7539284) for intelligbility, mel spectrogram and STFT distances for spectral reconstruction accuracy, and [SI-SDR](https://arxiv.org/abs/1811.02508) for phase reconstruction accuracy. Metrics are reported on the test set for both the MLS English and CommonVoice data. The model has not been trained or evaluated on non-speech audio.
109
 
110
+ | Dataset | ViSQOL |PESQ |ESTOI |Mel Distance |STFT Distance|SI-SDR |
111
  |:-----------:|:----------:|:----------:|:----------:|:-----------:|:-----------:|:-----------:|
112
  | MLS English | 4.50 | 3.69 | 0.94 | 0.066 | 0.033 | 8.33 |
113
  | CommonVoice | 4.53 | 3.55 | 0.93 | 0.100 | 0.057 | 7.63 |