title: How to use pos_tag in NLTK?
body: <p>So I was trying to tag a bunch of words in a list (POS tagging to be exact) like so:</p>
  
  <pre><code>pos = [nltk.pos_tag(i,tagset='universal') for i in lw]
  </code></pre>
  
  <p>where <code>lw</code> is a list of words (it's really long or I would have posted it but it's like <code>[['hello'],['world']]</code> (aka a list of lists which each list containing one word) but when I try and run it I get:</p>
  
  <pre><code>Traceback (most recent call last):
    File "&lt;pyshell#183&gt;", line 1, in &lt;module&gt;
      pos = [nltk.pos_tag(i,tagset='universal') for i in lw]
    File "&lt;pyshell#183&gt;", line 1, in &lt;listcomp&gt;
      pos = [nltk.pos_tag(i,tagset='universal') for i in lw]
    File "C:\Users\my system\AppData\Local\Programs\Python\Python35\lib\site-packages\nltk\tag\__init__.py", line 134, in pos_tag
      return _pos_tag(tokens, tagset, tagger)
    File "C:\Users\my system\AppData\Local\Programs\Python\Python35\lib\site-packages\nltk\tag\__init__.py", line 102, in _pos_tag
      tagged_tokens = tagger.tag(tokens)
    File "C:\Users\my system\AppData\Local\Programs\Python\Python35\lib\site-packages\nltk\tag\perceptron.py", line 152, in tag
      context = self.START + [self.normalize(w) for w in tokens] + self.END
    File "C:\Users\my system\AppData\Local\Programs\Python\Python35\lib\site-packages\nltk\tag\perceptron.py", line 152, in &lt;listcomp&gt;
      context = self.START + [self.normalize(w) for w in tokens] + self.END
    File "C:\Users\my system\AppData\Local\Programs\Python\Python35\lib\site-packages\nltk\tag\perceptron.py", line 240, in normalize
      elif word[0].isdigit():
  IndexError: string index out of range
  </code></pre>
  
  <p>Can someone tell me why and how I get this error and how to fix it? Many thanks.</p>