title: Heap Overload JVM
body: <p>EDIT***: As it stands right now, I have a program that is getting caught in an infinite loop somehow, but im not sure what is causing it. My program reads in data from a text file and essentially "saws off" the first section of data on each line until the first occurrence of a ',' character, and then reads each subsequent string (terminated by a ' ' in the general case) until the next ',' (special case, indicating that the following data is also junk). From there, it skips down to the next line and repeats. My program was working fine a few weeks ago but I tinkered with it and now it is officially broken. Below this edit are some of the other errors I ran into (that I may still need to address) but at the moment, I am being killed by this loop somewhere. I admittedly have a knack for screwing myself with nested FOR loops, and for using funky iterators and updates, but this one I just can't crack. Here is the code, with some explanation. This entire snippet is just thrown into the constructor for some arbitrary class taking a string "synsets" and an instance is being created in my main. The rest I have commented as best possible:</p>
  
  <pre><code>    In in = new In(synsets); // Custom input stream class, courtesy 
      // of Princeton U
      Out fout = new Out("log.txt"); // ostream analogue
  
      int linecount=0; // Marker used to keep track of line # in input file
      int nouncount=0; // Marker for keeping track of the number of 
      // "important" data items
  
      // Data comes in the format: 
      // "junk,important important important,junk
      // junk,important,junk
      // junk,important important,junk" etc.
  
      {
          int i=0;
          for ( String str=in.readLine();str!=null;str=in.readLine() )
          {
              i=0; // reset iterator for a new line
              for ( char next=str.charAt(i); next!=',';next=str.charAt(i) )
                  i++; // This FOR loop cuts out the junk at the start of 
                       // a line
              i++; // increment to after first comma
              for ( char next=str.charAt(i);next!=',';next=str.charAt(i))
              {
                  for (; next!=',' &amp;&amp; next!=' ' ;next=str.charAt(i))
                  {
                      i++;
                      fout.print(next);
                      // count the "nouns" (important data) on a line
                  }
                  nouncount++; 
                  // count the last noun on line, and subsequently fall 
                  // through loops to skip the rest of the junk at the 
                  // current line
                  fout.print('\n');
              }
  
          }
          fout.print(nouncount);
          in.close();
  </code></pre>
  
  <p>My output file is fine, until line 18 of the input, at which point it just begins print new line characters (like its caught on the loop which appends them but cant fall through to the next loop). Here are the input lines in question. It reads 24-karat_gold just fine, but isn't making it to pure_gold for some reason. Any thoughts? Thanks yall!</p>
  
  <p>"17,24-karat_gold pure_gold,100 per cent gold<br>
  18,24/7,uptime is 24 hours a day 7 days a week"</p>
  
  <p>EDIT*: I was coding a program to read in some data from a text file and got a new (to me) error message that I had run out of heap space. I tried tinkering to fix the bug, but only succeeded in getting rid of the error message. The text file I was reading from is about 90K lines, but my program threw the message at line 18. My tinkering did reveal some things, particularly, that my program was at least able to do SOME processing on the file as a whole, and that the issue may have involved some data corruption. The reason I deduced this was because of an array index I was using "j" which was firing off an IndexOutOfBoundsException. The problem, j only scaled by 1 each time it was incremented, and somehow, it was (is) going out of bounds over 150,000 higher than the array's bounds. Thus, it is my assumption that "j" is somehow being filled with data (possibly because the stack is overwriting into the heap?) which has nothing to do with "j" or the array in question. Below is my original post, though I was admittedly pretty tired, so it may not make a whole lot of sense. THANKS YALL! :)</p>
  
  <p>EDIT**: I did a check of nouncount, and it is 399850, so I wasn't experiencing data corruption, as I had suspected. One of my loops is just executing ad infinitum. I'll try to figure out which one and post back if I can correct the problem. I'll at least be back to clean up this post a bit for anyone in need in the future.</p>
  
  <p>Im writing a program that reads in a text file and it has to do some counting (the data in the file is divided up in a very syntactic way) but my code isn't able to get very far. While the file is pretty substantial (around 90K lines) my code is only getting through 17 before I get heap overflow. </p>
  
  <p>Some background, the "In" class is just a specialized input stream, and what is actually killing me is not the first FOR loop, where I count the lines (the code executes this portion just fine, reading in every line individually, each once). Rather, it is the second part, but I don't understand, because I am not using any additional heap space (I thought?) in that portion. I previously had the blocks together, doing the major functions together on a line-by-line basis, but again, the program would stop at line 17. I tried allocating up to a GB of heap space in Dr Java, but to no avail. Is there any section in the lower half in which I could set aside additional heap space?</p>
  
  <pre><code>        In in = new In(synsets);
  
      StringBuilder nounData = new StringBuilder();
      int linecount=0;
      int nouncount=0;
  
      {
          String str;
          int i=0;
          char next='\0';
          for ( str=in.readLine();str!=null;str=in.readLine() )
          {
              linecount++;
          }
          in.close();
  
          in = new In(synsets);
          for ( str=in.readLine();str!=null;str=in.readLine() )
          {
              i=0;
  // The first portion of each line is "trash" until the first comma
              for ( next=str.charAt(i) ; next!=',' ; next=str.charAt(++i) ){}
              i++;
  
  // This actually reads/processes the data until the next comma, then
  // jumps to the next line. "What" i need done is really secondary, I 
  // just need to figure out what is eating so much space so I can 
  // trim it
              for ( next=str.charAt(i);next!=',';next=str.charAt(i))
                  for (;next!=','&amp;&amp;next!=' ';next=str.charAt(++i))
                  nouncount++;
          }
      }
  </code></pre>
  
  <p>Here is the updated code, wherein I tried to recreate the Heap message from Dr Java. While I couldnt do that, I did get an interesting error message (at line 18 of course) about an array out of bounds. Still stumped though, as I dont see how the variable in question could be more than 17 at the time it gets hit.</p>
  
  <p>Here is the code:</p>
  
  <pre><code>In in = new In(synsets);
  
      StringBuilder nounData = new StringBuilder();
      int linecount=0;
      int nouncount=0;
  
      {
          int i=0;
          for ( String str=in.readLine();str!=null;str=in.readLine() )
          {
              i=0;
              for ( char next=str.charAt(i);next!=',';next=str.charAt(i))
                  for (;next!=','&amp;&amp;next!=' ';next=str.charAt(++i))
                  nouncount++;
          }
          in.close();
          in=new In(synsets);
  
          String[] nouns = new String[nouncount];
          int j=0;
          for ( String str=in.readLine();str!=null;str=in.readLine() )
          {
              linecount++;
              i=0;
              for ( char next=str.charAt(i) ; next!=',' ; next=str.charAt(++i) ){}
              i++;
              for ( char next=str.charAt(i);next!=',';next=str.charAt(i))
              {
                  for (;next!=','&amp;&amp;next!=' ';next=str.charAt(++i))
                      nounData.append(next);
                  nouns[j++]=nounData.toString();
                  nounData.delete(0,nounData.capacity()-1);
              }
              System.out.println("Current line count is: " + linecount);
          }
      }
      in.close();
      System.out.println("line count = "+linecount);
      System.out.println("noun count = "+nouncount);
      String[] nouns = new String[nouncount];
  </code></pre>
  
  <p>and here is the error message:</p>
  
  <pre><code>Current line count is: 1
  Current line count is: 2
  Current line count is: 3
  Current line count is: 4
  Current line count is: 5
  Current line count is: 6
  Current line count is: 7
  Current line count is: 8
  Current line count is: 9
  Current line count is: 10
  Current line count is: 11
  Current line count is: 12
  Current line count is: 13
  Current line count is: 14
  Current line count is: 15
  Current line count is: 16
  Current line count is: 17
  java.lang.ArrayIndexOutOfBoundsException: 399850
      at WordNet.&lt;init&gt;(WordNet.java:39)
      at WordNet.main(WordNet.java:212)
  </code></pre>
  
  <p>For the record, line "39" in my code is:</p>
  
  <pre><code>nouns[j++]=nounData.toString();
  </code></pre>
  
  <p>Character count in the file up until line 18 is 917, at line 19 it is 966, so i dont think I misplaced that line.</p>
  
  <p>EDIT: Furthermore, I did a test, and there are only about 147K "nouns" in the file, so my guess is that "j" is being corrupted somehow, seeing as how it must have had to have "jumped" from something between 0 and 147K to 399K+. Unfortunately, its past my bedtime, so I shan't be able to continue updating tonight but feel free to post any ideas, Ill be checking via email tomorrow morning :) Thanks everyone!</p>