title: NLTK PunktSequenceTokenizer return type or a way to use it faster in an iterative function?
body: <p>Within my PHP function, I am calling a Python script like this:</p>
  
  <pre><code> $foo = exec("python tokenize.py $bar");
  </code></pre>
  
  <p>The problem is, now I have built a function that executes the command above iteratively and it takes more than five minutes to finish, because of the code I use below:</p>
  
  <pre><code>train_text = state_union.raw("1963-Johnson.txt") 
  custom_sent_tokenizer = PunktSentenceTokenizer(train_text)
  </code></pre>
  
  <p>The operation of training my PST takes some time even for one of the shortest corpora in the state_union package. </p>
  
  <p>I tried to store the output in a plain txt file but I cannot find the return type in the documentation <a href="http://www.nltk.org/api/nltk.tokenize.html" rel="nofollow">here</a>. I guess it is an iterator like everything else in the package, but I've tried to convert the iterator to the list and failed miserably. </p>
  
  <p><strong>The questions are:</strong></p>
  
  <p><strong>1. What is the return type of the PunktSentenceTokenizer and can I store it?</strong></p>
  
  <p><strong>2. Will reading it from the .txt file or any other source be faster than training it over and over when executing my PHP program?</strong> </p>
  
  <p><strong>3. Do you have any other idea how to use PST so it remains trained over the same portion of text so I can use it with my script faster?</strong></p>