title: Inefficient memory usage while dealing with strings read from a text file
body: <p>I have a large file - about 1.7 GB - consisting of 25 million lines with each line in the following format : text, number.</p>
  
  <p>The thing is the text could be same in multiple lines i.e.</p>
  
  <p>I am a boy, 34</p>
  
  <p>.. </p>
  
  <p>..</p>
  
  <p>I am a boy, 56</p>
  
  <p>I am reading each line of the file in a while loop and putting the string and number in a hashmap.</p>
  
  <p>Code : </p>
  
  <pre><code>InputStream is = Thread.currentThread().getContextClassLoader().getResourceAsStream("filename.txt");
  
  InputStreamReader stream = new InputStreamReader(is, StandardCharsets.UTF_8);
  
  BufferedReader reader = new BufferedReader(stream);
  
  while(true)
  {
      line =reader.readLine();
      if (line == null) 
      {
          break;
      }
      String text= line.substring(0, line.lastIndexOf(",")).trim();
  
      String id = line.substring(line.lastIndexOf(",") + 1).trim();
  
      hm.put(text,id);
  }
  </code></pre>
  
  <p>where hm is a hashmap. </p>
  
  <p>The reason I am using <strong>substring</strong> and <strong>lastIndexOf</strong> is because the text might have comma itself. </p>
  
  <p>My question is : am I using too many Strings? Are the duplicate strings extracted from the text file going to remain in memory as copies or only one of them will be present. Is there a way (using intern method of strings or something ) to reduce the number of strings used.When I use Eclipse MAT for analysing memory usage, it shows about 9 GB of Heap Space used. </p>