title: Reading big chunk of xml data from socket and parse on the fly
body: <p>I am working on an android client which reads continues stream of xml data from my java server via a TCP socket.  The server sends a '\n' character as delimiter between consecutive responses. Below given is a model implementation..</p>
  
  <pre><code>&lt;response1&gt;
     &lt;datas&gt;
        &lt;data&gt;
             .....
             .....
        &lt;/data&gt;
        &lt;data&gt;
             .....
             .....
        &lt;/data&gt;
        ........
        ........
     &lt;/datas&gt;
  &lt;/response1&gt;\n    &lt;--- \n acts as delimiter ---/&gt; 
  &lt;response2&gt;
  
     &lt;datas&gt;
        &lt;data&gt;
             .....
             .....
        &lt;/data&gt;
        &lt;data&gt;
             .....
             .....
        &lt;/data&gt;
        ........
        ........
     &lt;/datas&gt;
  &lt;/response2&gt;\n
  </code></pre>
  
  <p>Well I hope the structure is clear now. This response is transmitted from server zlib compressed. So I have to first inflate whatever I am reading from the server, separate on response using delimiter and parse. And <strong>I am using SAX to parse my XML</strong></p>
  
  <p>Now my main problem is the xml response coming from server can be very large (can be in the range of 3 to 4 MB). So</p>
  
  <ul>
  <li><p>to separate responses based on delimiter (\n) I have to use a
  <strong>stringBuilder</strong> to store response blocks as it reads from socket
  and on some phones StringBuilder cannot store strings in the
  MegaBytes range. It is giving <strong>OutOfMemory</strong> exception, and from
  threads  like <a href="https://stackoverflow.com/questions/4729188/android-convert-stream-to-string-without-running-out-of-memory">this</a> I got to know keeping large strings (even on a
  temporary basis) is not such a  good idea.</p></li>
  <li><p>Next I tried to pass the inflatorReadStream (which in turn takes data
  from socket input stream) as the input stream of SAX parser (without
  bothering to separate xml myself and relying on SAX's ability to find
  the end of document based on tags). This time one response gets
  parsed successfully, but then on finding the '\n' delimiter SAX
  throws <strong>ExpatParserParseException</strong> saying  <strong>junk after document
  element</strong> .</p></li>
  <li>After catching  that <strong>ExpatParserParseException</strong> I tried to read
  again, but after throwing exception SAX Parser closes the stream, so
  when I try to read/parse again, it is giving <strong>IOException</strong> saying
  input stream is closed.</li>
  </ul>
  
  <p>A code snippet of what I have done is given below (removed all unrelated try catch blocks for clarity). </p>
  
  <pre><code>private Socket clientSocket     =   null;
  DataInputStream readStream      =   null;
  DataOutputStream writeStream        =   null;
  private StringBuilder incompleteResponse    =   null;
  private AppContext  context     =   null;
  
  
  public boolean connectToHost(String ipAddress, int port,AppContext myContext){
          context                     =   myContext;
          website                     =   site;
          InetAddress serverAddr          =   null;
  
      serverAddr                      =   InetAddress.getByName(website.mIpAddress);
  
      clientSocket                    =   new Socket(serverAddr, port);
  
      //If connected create a read and write Stream objects..
      readStream   =  new DataInputStream(new InflaterInputStream(clientSocket.getInputStream()));
      writeStream             =   new DataOutputStream(clientSocket.getOutputStream());
  
      Thread readThread = new Thread(){
              @Override
              public void run(){                              
              ReadFromSocket();                   
          }
      };
      readThread.start();     
      return true;
  }
  
  
  public void ReadFromSocket(){
     while(true){
         InputSource xmlInputSource = new InputSource(readStream);
         SAXParserFactory spf =   SAXParserFactory.newInstance();
         SAXParser sp =   null;
         XMLReader xr =   null;
         try{
             sp   = spf.newSAXParser();
         xr   = sp.getXMLReader();
         ParseHandler xmlHandler =    new ParseHandler(context.getSiteListArray().indexOf(website), context);
         xr.setContentHandler(xmlHandler);
         xr.parse(xmlInputSource);
     //  postSuccessfullParsingNotification();
         }catch(SAXException e){
             e.printStackTrace();
             postSuccessfullParsingNotification();
         }catch(ParserConfigurationException e){
             e.printStackTrace();
             postSocketDisconnectionBroadcast();
             break;
         }catch (IOException e){
             postSocketDisconnectionBroadcast();
             e.printStackTrace();
             e.toString();
             break;
         }catch (Exception e){
             postSocketDisconnectionBroadcast();
             e.printStackTrace();
             break;
         }
      }
  }
  </code></pre>
  
  <p>And now my questions are</p>
  
  <ol>
  <li>Is there any way to make SAX Parser ignore junk characters after on
  xml response, and not throw exception and close the stream..</li>
  <li>If not is there any way to avoid out of memory error on
  stringBuilder. To be frank,I am not excepting a positive answer on
  this. Any workaround?</li>
  </ol>