title: Python (NLTK) - more efficient way to extract noun phrases?
body: <p>I've got a machine learning task involving a large amount of text data. I want to identify, and extract, noun-phrases in the training text so I can use them for feature construction later on in the pipeline. 
  I've extracted the type of noun-phrases I wanted from text but I'm fairly new to NLTK, so I approached this problem in a way where I can break down each step in list comprehensions like you can see below. </p>
  
  <p>But my real question is, am I reinventing the wheel here? Is there a faster way to do this that I'm not seeing?</p>
  
  <pre><code>import nltk
  import pandas as pd
  
  myData = pd.read_excel("\User\train_.xlsx")
  texts = myData['message']
  
  # Defining a grammar &amp; Parser
  NP = "NP: {(&lt;V\w+&gt;|&lt;NN\w?&gt;)+.*&lt;NN\w?&gt;}"
  chunkr = nltk.RegexpParser(NP)
  
  tokens = [nltk.word_tokenize(i) for i in texts]
  
  tag_list = [nltk.pos_tag(w) for w in tokens]
  
  phrases = [chunkr.parse(sublist) for sublist in tag_list]
  
  leaves = [[subtree.leaves() for subtree in tree.subtrees(filter = lambda t: t.label == 'NP')] for tree in phrases]
  </code></pre>
  
  <p>flatten the list of lists of lists of tuples that we've ended up with, into
  just a list of lists of tuples</p>
  
  <pre><code>leaves = [tupls for sublists in leaves for tupls in sublists]
  </code></pre>
  
  <p>Join the extracted terms into one bigram</p>
  
  <pre><code>nounphrases = [unigram[0][1]+' '+unigram[1][0] in leaves]
  </code></pre>