title: How to reduce noise in signal overlap add in Java?
body: <p>I have been working with a java program (developed by other people) for text-to-speech synthesis. The synthesis is done by concatenation of "di-phones". In the oroginal version, there was no signal processing. The diphones were just collected and concatenated together to produce the output. In order to improve the output, I tried to perform "phase matching" of the concatenating speech signals. The modification I've made is summarized here:  </p>
  
  <ul>
  <li>Audio data is collected from the AudioInputStream into a byte array.
  Since the audio data is 16 bit, I converted the byte array to a short
  array.</li>
  <li>The "signal processing" is done on the short array.</li>
  <li>To output the audio data, short array is again converted to byte array.</li>
  </ul>
  
  <p>Here's the part of the code that I've changed in the existing program:</p>
  
  <p><strong>Audio Input</strong><br>
  This segment is called for every diphone.  </p>
  
  <p><em>Original Version</em>  </p>
  
  <pre><code>audioInputStream = AudioSystem.getAudioInputStream(sound);
  while ((cnt = audioInputStream.read(byteBuffer, 0, byteBuffer.length)) != -1) {
      if (cnt &gt; 0) {
          byteArrayPlayStream.write(byteBuffer, 0, cnt);
      }
  }
  </code></pre>
  
  <p><em>My Version</em></p>
  
  <pre><code>// public varialbe declarations 
  byte    byteSoundFile[];                             // byteSoundFile will contain a whole word or the diphones of a whole word
  short   shortSoundFile[]    = new short[5000000];    // sound contents are taken in a short[] array for signal processing
  short   shortBuffer[];
  int     pos                 = 0;
  int     previousPM          = 0;
  boolean isWord              = false;
  public static HashMap&lt;String, Integer&gt; peakMap1 = new HashMap&lt;String, Integer&gt;(); 
  public static HashMap&lt;String, Integer&gt; peakMap2 = new HashMap&lt;String, Integer&gt;();
  
  // code for receiving and processing audio data
  if(pos == 0) {
      // a new word is going to be processed.
      // so reset the shortSoundFile array
      Arrays.fill(shortSoundFile, (short)0);
  }
  
  audioInputStream = AudioSystem.getAudioInputStream(sound);
  while ((cnt = audioInputStream.read(byteBuffer, 0, byteBuffer.length)) != -1) {
      if (cnt &gt; 0) {
          byteArrayPlayStream.write(byteBuffer, 0, cnt);
      }
  }
  
  byteSoundFile = byteArrayPlayStream.toByteArray();
  int nSamples = byteSoundFile.length;
  byteArrayPlayStream.reset();
  
  if(nSamples &gt; 80000) {   // it is a word
      pos     = nSamples;
      isWord  = true;
  }
  else {              // it is a diphone
      // audio data is converted from byte to short, so nSamples is halved
      nSamples /= 2;
  
      // transfer byteSoundFile contents to shortBuffer using byte-to-short conversion
      shortBuffer = new short[nSamples];
      for(int i=0; i&lt;nSamples; i++) {
          shortBuffer[i] = (short)((short)(byteSoundFile[i&lt;&lt;1]) &lt;&lt; 8 | (short)byteSoundFile[(i&lt;&lt;1)+1]);
      }
  
      /************************************/
      /**** phase-matching starts here ****/
      /************************************/
      int pm1 = 0;
      int pm2 = 0;
      String soundStr = sound.toString();
      if(soundStr.contains("\\") &amp;&amp; soundStr.contains(".")) {
          soundStr = soundStr.substring(soundStr.indexOf("\\")+1, soundStr.indexOf("."));
      }                    
      if(peakMap1.containsKey(soundStr)) {
          // perform overlap and add
          System.out.println("we are here");
          pm1 = peakMap1.get(soundStr);
          pm2 = peakMap2.get(soundStr);
  
          /*
          Idea:
          If pm1 is located after more than one third of the samples,
          then threre will be too much overlapping.
          If pm2 is located before the two third of the samples, 
          then where will also be extra overlapping for the next diphone.
          In both of the cases, we will not perform the peak-matching operation.
          */
          int idx1 = (previousPM == 0) ? pos : previousPM - pm1;
          if((idx1 &lt; 0) || (pm1 &gt; (nSamples/3))) {
              idx1 = pos;
          }
          int idx2 = idx1 + nSamples - 1;
          for(int i=idx1, j=0; i&lt;=idx2; i++, j++) {
              if(i &lt; pos) {
                  shortSoundFile[i] = (short) ((shortSoundFile[i] &gt;&gt; 1) + (shortBuffer[j] &gt;&gt; 1));
              }
              else {
                  shortSoundFile[i] = shortBuffer[j];
              }
          }
          previousPM = (pm2 &lt; (nSamples/3)*2) ? 0 : idx1 + pm2;
          pos = idx2 + 1;
      }
      else {
          // no peak found. simply concatenate the audio data
          for(int i=0; i&lt;nSamples; i++) {
              shortSoundFile[pos++] = shortBuffer[i];
      }
      previousPM = 0;
  }
  </code></pre>
  
  <p><strong>Audio Output</strong><br>
  After collecting all the diphones of a word, this segment is called to play the audio output.<br>
  <em>Original Version</em>  </p>
  
  <pre><code>byte audioData[] = byteArrayPlayStream.toByteArray();
  ... code for writing audioData to output steam
  </code></pre>
  
  <p><em>My Version</em>  </p>
  
  <pre><code>byte audioData[];
  if(isWord) {
      audioData = Arrays.copyOf(byteSoundFile, pos);
      isWord = false;
  }
  else {
      audioData = new byte[pos*2];
      for(int i=0; i&lt;pos; i++) {
          audioData[(i&lt;&lt;1)]   = (byte) (shortSoundFile[i] &gt;&gt;&gt; 8);
          audioData[(i&lt;&lt;1)+1] = (byte) (shortSoundFile[i]);
      }
  }
  pos = 0;
  ... code for writing audioData to output steam
  </code></pre>
  
  <p>But after the modification has done, the output has become worse. There is a lot of noise in the output. </p>
  
  <p>Here is a sample audio with modification: <a href="https://app.box.com/s/80ivz92x55jzd659zz77s4awe5g2fnih" rel="nofollow noreferrer">modified output</a></p>
  
  <p>Here is a sample audio from the original version: <a href="https://app.box.com/s/72pxqf1yzc756yejw5rst37gmd9ar1cd" rel="nofollow noreferrer">original output</a></p>
  
  <p>Now I'd appreciate it if anyone can point out the reason that generates the noise and how to remove it. Am I doing anything wrong in the code? I have tested my algorithm in <strong>Mablab</strong> and it worked fine.</p>