title: Stop words nltk/python problem
body: <p>I have some code that processes a dataset for later use, the code i'm using for the stop words seems to be ok, however I think the problem lies within the rest of my code as it seems to only remove some of the stop words.</p>
  
  <pre><code>import re
  import nltk
  
  # Quran subset
  filename = 'subsetQuran.txt'
  
  # create list of lower case words
  word_list = re.split('\s+', file(filename).read().lower())
  print 'Words in text:', len(word_list)
  
  word_list2 = [w for w in word_list if not w in nltk.corpus.stopwords.words('english')]
  
  
  
  # create dictionary of word:frequency pairs
  freq_dic = {}
  # punctuation and numbers to be removed
  punctuation = re.compile(r'[-.?!,":;()|0-9]') 
  for word in word_list2:
      # remove punctuation marks
      word = punctuation.sub("", word)
      # form dictionary
      try: 
          freq_dic[word] += 1
      except: 
          freq_dic[word] = 1
  
  
  print '-'*30
  
  print "sorted by highest frequency first:"
  # create list of (val, key) tuple pairs
  freq_list2 = [(val, key) for key, val in freq_dic.items()]
  # sort by val or frequency
  freq_list2.sort(reverse=True)
  freq_list3 = list(freq_list2)
  # display result
  for freq, word in freq_list2:
      print word, freq
  f = open("wordfreq.txt", "w")
  f.write( str(freq_list3) )
  f.close()
  </code></pre>
  
  <p>The output is looking like this</p>
  
  <pre><code>[(71, 'allah'), (65, 'ye'), (46, 'day'), (21, 'lord'), (20, 'truth'), (20, 'say'), (20, 'and')
  </code></pre>
  
  <p>This is just a small sample, there are others that should have been removed.
  Any help is appreciated.</p>