title: I have many tweets, but only 20 can be brought
body: <p>I am crawling Twitter.</p>
  
  <p>But I can not collect all the data.</p>
  
  <p>I am typing the keyword and crawling the url that specified the date.</p>
  
  <p>But there are certainly a number of tweets that can only bring twenty.</p>
  
  <p>(For example, tweets that include the keyword "money" from July 18, 1917 to July 20,)</p>
  
  <p>What is the problem?
  How should I fix it?</p>
  
  <pre><code># -*- coding:utf-8 -*-
  #python3
  import urllib.request
  from bs4 import BeautifulSoup
  
  
  # save txt file
  OUTPUT_FILE_NAME = 'output_sample.txt'
  
  # scraping URL
  URL = 'https://twitter.com/search?l=&amp;q=money%20since%3A2017-07-18%20until%3A2017-07-20&amp;src=typd&amp;lang=ko'
  
  #crawler
  def spider(URL):
      source_code_from_url = urllib.request.urlopen(URL)
      soup = BeautifulSoup(source_code_from_url, 'lxml', from_encoding='utf-8')
      text = ''
      for item in soup.find_all('p', {'class': 'tweet-text'}):
  
          text = text + str(item.find_all(text=True)) + "\n"
      return text
  
  def main():
      open_output_file = open(OUTPUT_FILE_NAME, 'w', encoding='utf-8')
      result_text = spider(URL)
      open_output_file.write(result_text)
      open_output_file.close()
  
  if __name__ == '__main__':
      main()
  </code></pre>