title: Scraping Instagram with API ?__a=1
body: <p>I've been trying scraping Instagram posts for a certain hashtag for the keys: display_url, taken_at_timestamp, text, edge_liked_by. This goes perfect for some hundreds in the start, but then stops fetching 'text' keyword only. Other three fields are successfully fetched though. I am not sure why it's happening. </p>
  
  <p>I am parsing the JSON <a href="https://www.instagram.com/explore/tags/something/?__a=1" rel="nofollow noreferrer">https://www.instagram.com/explore/tags/something/?__a=1</a>. </p>
  
  <pre><code>base_url = "https://www.instagram.com/explore/tags/salonedelmobile/?__a=1"
  url = "https://www.instagram.com/explore/tags/salonedelmobile/?__a=1"
  while True:
      response = url_req.urlopen(url)
      json_file = json.load(response)
      for i in json_file['graphql']['hashtag']['edge_hashtag_to_media']['edges']:
          try:
              post_text = i['node']['edge_media_to_caption']['edges'][0]['node']['text']
          except IndexError as e:
              post_text = e
          try:
              display_url = i['node']['display_url']
          except:
              display_url = e
          try:
              like_count = i['node']['edge_liked_by']['count']
          except:
              like_count = e
          try:
              time_stamp = i['node']['taken_at_timestamp']
          except:
              time_stamp = e
          output.append([display_url, like_count, time_stamp, post_text])
          df = pd.DataFrame(output,columns=['URL', 'Like Count', 'Time', 'Text'])
          try:
              df.to_excel('instagram.xlsx')
          except:
              pass
          if json_file['graphql']['hashtag']['edge_hashtag_to_media']['page_info']['has_next_page'] == True:
              end_cursor = json_file['graphql']['hashtag']['edge_hashtag_to_media']['page_info']['end_cursor']
              url = base_url + '&amp;max_id=' + end_cursor
          else:
              break
  </code></pre>