title: How to retrieve all historical public tweets with Twitter Premium Search API in Sandbox version (using next token)
body: <p>I want to download all historical tweets with certain hashtags and/or keywords for a research project. I got the Premium Twitter API for that. I'm using the amazing <code>TwitterAPI</code> to take care of auth and so on.</p>
  
  <p>My problem now is that I'm not an expert developer and I have some issues understanding how the <code>next</code> token works, and how to get all the tweets in a csv. </p>
  
  <p>What I want to achieve is to have all the tweets in one single csv, without having to manually change the dates of the <code>fromDate</code> and <code>toDate</code> values. Right now I don't know how to get the <code>next</code> token and how to use it to concatenate requests.</p>
  
  <p>So far I got here:</p>
  
  <pre><code>from TwitterAPI import TwitterAPI
  import csv
  
  SEARCH_TERM = 'my-search-term-here'
  PRODUCT = 'fullarchive'
  LABEL = 'here-goes-my-dev-env'
  
  api = TwitterAPI("consumer_key", 
               "consumer_secret", 
               "access_token_key", 
               "access_token_secret")
  
  r = api.request('tweets/search/%s/:%s' % (PRODUCT, LABEL), 
              {'query':SEARCH_TERM, 
              'fromDate':'200603220000',
              'toDate':'201806020000'
              }
              )
  csvFile = open('2006-2018.csv', 'a')
  csvWriter = csv.writer(csvFile)
  
  for item in r:
      csvWriter.writerow([item['created_at'],item['user']['screen_name'], item['text'] if 'text' in item else item])
  </code></pre>
  
  <p>I would be really thankful for any help! 
  Cheers!</p>