title: Audio streaming via TCP socket on Android
body: <p>I am streaming mic input from a C Server via socket. I know the stream works because it does with a C client and I am getting the right values on my Android client.</p>
  
  <p>I am streaming a 1024 floatarray. One float are 4 bytes. So I got a incoming stream with 4096 bytes per frame. I am getting the floats out of this bytes and I know this floats are the ones I sent, so that part should work.  </p>
  
  <p>Now I want to get that stream directly to the phones speakers by using AudioTrack. I tried to input the bytes I received directly: just noise. I tried to cast it back to a byte array, still the same. I tried to cast that float into short (because AudioTrack takes bytes or short). I could get something that could have been my mic input (knocking), but very scratchy and and extremely laggy. I would understand if there was a lag between the frames, but I can't even get one clear sound.
  I can, however, output a sin sound clearly that I produce locally and put into that shortarray.
  Now I wonder if I got some issues in my code anyone of you can see, because I don't see them.</p>
  
  <p>What I am doing is: I put 4 bytes in a byte array. I get the float out of it. As soon as I got one Frame in my float array (I am controlling that with a bool, not nice, but it should work) I put it in my shortarray and let audiotrack play it. This double casting might be slow, but I do it because its the closest I got to playing the actual input.</p>
  
  <p><strong>Edit:</strong>
  I checked the endianess by comparing the floats, they have the proper values between -1 and 1 and are the same ones I send. Since I don't change the endianess when casting to float, I don't get why forwarding a 4096 byte array to AudioTrack directly doesn't work neither. There might be something wrong with the multithreading, but I don't see what it could be. </p>
  
  <p><strong>Edit 2:</strong>  I discovered a minor problem - I reset j at 1023. But that missing float should not have been the problem. What I did other than that was to put the method that took the stream from the socket in another thread instead of calling it in a async task. That made it work, I now am able to understand the mic sounds. Still the quality is very poor - might there be a reason for that in the code? Also I got a delay of about 10 seconds. Only about half a second is caused by WLAN, so I wonder if it might be the codes fault. Any further thoughts are appreciated.</p>
  
  <p><strong>Edit 3:</strong> I played around with the code and implemented a few of greenapps ideas in the comments. With the new thread structure I was facing the problem of not getting any sound. Like at all. I don't get how that is even possible, so I switched back. Other things I tried to make the threads more lightweight didn't have any effect. I got a delay and I got a very poor quality (I can identify knocks, but I can't understand voices). I figured something might be wrong with my convertions, so I put the bytes I receive from the socket directly in AudioTrack - nothing but ugly pulsing static noise. Now I am even more confused, since this exact stream still works with the C client. I will report back if I find a solution, but still any help is welcome.</p>
  
  <p><strong>Edit 4</strong>  I should add, that I can play mic inputs from another android app where I send that input directly as bytes (I would exclude the float casting stuff and put the bytes I receive directly to audioTrack in my player code).<br>
  Also it occured to me, that it could be a problem, that the said floatarray that is streamed by the C Server comes from a 64bit machine while the phone is 32bit. Could that be a problem somehow, even though I am just streaming floats as 4 bytes?
  Or, another thought of mine: The underlying number format of the bytes I receive is float. What format does AudioTrack expect? Even if put in just bytes - would I need to cast that float to a int and cast that back to bytes or something?</p>
  
  <p><strong>new code:</strong>  </p>
  
  <pre><code>public class PCMSocket {
  
  AudioTrack audioTrack;
  boolean doStop = false;
  int musicLength = 4096;
  byte[] music;
  Socket socket;
  short[] buffer = new short[4096];
  float[] fmusic = new float[1024];
  WriteToAudio writeThread;
  ReadFromSocket readThread;
  
  
  public PCMSocket()
  {
  
  }
  
  public void start()
  {
      doStop = false;
      readThread = new ReadFromSocket();
      readThread.start();
  }
  
  public class ReadFromSocket extends Thread
  {       
      public void run()
      {
      doStop=true;
  
      InetSocketAddress address = new InetSocketAddress("xxx.xxx.xxx.x", 8000);
  
      socket = new Socket();
      int timeout = 6000;   
      try {
          socket.connect(address, timeout);
      } catch (IOException e2) {
          e2.printStackTrace();
      }
  
       musicLength = 1024;
  
      InputStream is = null;
  
      try {
          is = socket.getInputStream();
      } catch (IOException e) {
          e.printStackTrace();
      }
  
      BufferedInputStream bis = new BufferedInputStream(is);
      DataInputStream dis = new DataInputStream(bis);     
  
      try{
  
      int minSize =AudioTrack.getMinBufferSize( 44100, AudioFormat.CHANNEL_CONFIGURATION_STEREO, AudioFormat.ENCODING_PCM_16BIT ); 
  
      audioTrack = new AudioTrack(AudioManager.STREAM_MUSIC, 44100,
              AudioFormat.CHANNEL_OUT_STEREO, 
              AudioFormat.ENCODING_PCM_16BIT, minSize,
              AudioTrack.MODE_STREAM);
          audioTrack.play();
  
        } catch (Throwable t)
        {
            t.printStackTrace();
          doStop = true;
        }
  
      writeThread = new WriteToAudio();
      readThread.start();
  
      int i = 0;   
      int j=0;
  
      try {
          if(dis.available()&gt;0)Log.d("PCMSocket", "receiving");
          music = new byte[4];
          while (dis.available() &gt; 0)
          {
              music[i]=0;
              music[i] = dis.readByte(); 
  
              if(i==3)
              {
                  int asInt = 0;
                  asInt = ((music[0] &amp; 0xFF) &lt;&lt; 0) 
                          | ((music[1] &amp; 0xFF) &lt;&lt; 8) 
                          | ((music[2] &amp; 0xFF) &lt;&lt; 16) 
                          | ((music[3] &amp; 0xFF) &lt;&lt; 24);
                  float asFloat = 0;
                  asFloat = Float.intBitsToFloat(asInt);
                  fmusic[j]=asFloat;
              }
  
              i++;
              j++;
              if(i==4)
              {
                  music = new byte[4]; 
                  i=0;
              }
              if(j==1024)
              {
                  j=0;
                  if(doStop)doStop=false;
              }
          }
      } catch (IOException e) {
          e.printStackTrace();
      }
  
      try {
          dis.close();
      } catch (IOException e) {
          e.printStackTrace();
      }  
  
      }
  };
  
  
  public class WriteToAudio extends Thread
  {       
      public void run()
      {
          while(true){
          while(!doStop)
          {           
              try{
                  writeSamples(fmusic);
  
              }catch(Exception e)
              {
                  e.printStackTrace();
              }    
              doStop = true;
          }
          }
      }
  };
  
  
  public void writeSamples(float[] samples) 
  {   
     fillBuffer( samples );
     audioTrack.write( buffer, 0, samples.length );
  }
  
  private void fillBuffer( float[] samples )
  { 
     if( buffer.length &lt; samples.length )
        buffer = new short[samples.length];
  
     for( int i = 0; i &lt; samples.length; i++ )
     {
        buffer[i] = (short)(samples[i] * Short.MAX_VALUE);
     }
  }   
  
  
  }
  </code></pre>
  
  <p><strong>old code:</strong>  </p>
  
  <pre><code>public class PCMSocket {
  AudioTrack audioTrack;
  WriteToAudio thread;
  boolean doStop = false;
  int musicLength = 4096;
  byte[] music;
  Socket socket;
  short[] buffer = new short[4096];
  float[] fmusic = new float[1024];
  
  
  public PCMSocket()
  {
  
  }
  
  public void start()
  {
      doStop = false;
      new GetStream().executeOnExecutor(AsyncTask.THREAD_POOL_EXECUTOR);
  }
  
  private class GetStream extends AsyncTask&lt;Void, Void, Void&gt; {
  
      @Override
      protected Void doInBackground(Void... values) { 
          PCMSocket.this.getSocket();
          return null;
  
      }
  
      @Override
      protected void onPreExecute() {
      }
  
  
  
      @Override
      protected void onPostExecute(Void result)
      {
          return;
      }
  
      @Override
      protected void onProgressUpdate(Void... values) {
      }
  }
  
  private void getSocket()
  {
      doStop=true;
  
      InetSocketAddress address = new InetSocketAddress("xxx.xxx.xxx.x", 8000);
  
      socket = new Socket();
      int timeout = 6000;   
      try {
          socket.connect(address, timeout);
      } catch (IOException e2) {
          e2.printStackTrace();
      }
  
       musicLength = 1024;
  
      InputStream is = null;
  
      try {
          is = socket.getInputStream();
      } catch (IOException e) {
          e.printStackTrace();
      }
  
      BufferedInputStream bis = new BufferedInputStream(is);
      DataInputStream dis = new DataInputStream(bis);     
  
      try{
  
      int minSize =AudioTrack.getMinBufferSize( 44100, AudioFormat.CHANNEL_CONFIGURATION_STEREO, AudioFormat.ENCODING_PCM_16BIT ); 
  
      audioTrack = new AudioTrack(AudioManager.STREAM_MUSIC, 44100,
              AudioFormat.CHANNEL_OUT_STEREO, 
              AudioFormat.ENCODING_PCM_16BIT, minSize,
              AudioTrack.MODE_STREAM);
          audioTrack.play();
  
        } catch (Throwable t)
        {
            t.printStackTrace();
          doStop = true;
        }
  
      thread = new WriteToAudio();
      thread.start();
  
      int i = 0;   
      int j=0;
  
      try {
          if(dis.available()&gt;0)Log.d("PCMSocket", "receiving");
          music = new byte[4];
          while (dis.available() &gt; 0)
          {
              music[i]=0;
              music[i] = dis.readByte(); 
  
              if(i==3)
              {
                  int asInt = 0;
                  asInt = ((music[0] &amp; 0xFF) &lt;&lt; 0) 
                          | ((music[1] &amp; 0xFF) &lt;&lt; 8) 
                          | ((music[2] &amp; 0xFF) &lt;&lt; 16) 
                          | ((music[3] &amp; 0xFF) &lt;&lt; 24);
                  float asFloat = 0;
                  asFloat = Float.intBitsToFloat(asInt);
                  fmusic[j]=asFloat;
              }
  
              i++;
              j++;
              if(i==4)
              {
                  music = new byte[4]; 
                  i=0;
              }
              if(j==1023)
              {
                  j=0;
                  if(doStop)doStop=false;
              }
          }
      } catch (IOException e) {
          e.printStackTrace();
      }
  
      try {
          dis.close();
      } catch (IOException e) {
          e.printStackTrace();
      }  
  
  }
  
  
  public class WriteToAudio extends Thread
  {       
      public void run()
      {
          while(true){
          while(!doStop)
          {           
              try{
                  writeSamples(fmusic);
  
              }catch(Exception e)
              {
                  e.printStackTrace();
              }    
              doStop = true;
          }
          }
      }
  };
  
  
  public void writeSamples(float[] samples) 
  {   
     fillBuffer( samples );
     audioTrack.write( buffer, 0, samples.length );
  }
  
  private void fillBuffer( float[] samples )
  { 
     if( buffer.length &lt; samples.length )
        buffer = new short[samples.length*4];
  
     for( int i = 0; i &lt; samples.length; i++ )
     {
        buffer[i] = (short)(samples[i] * Short.MAX_VALUE);
     }
  }   
  
  
  }
  </code></pre>