title: How to retrieve text at url which is hidden behind "Click to Expand" link?
body: <p>I'm trying to make a Python script that will retrieve PhysRevPER articles for me in html format so that I can put them on my Kindle and read them away from my desk (and internet connection).  However, while the full text of the articles can be seen by going to the website, one has to click on several "Click to Expand" (or simply "+") "links" in order to see it all.  These "links" don't point to new urls, so I assume they are controlling some sort of script which determines the visibility of each section of the article.  Is there some way to instruct urllib to send the appropriate script instructions that will cause the "links" to expand and download the page after everything is expanded?  When I simply try to retrieve the page I don't get any of the hidden text.</p>
  
  <p>Here's a link to a recent article:
  <a href="https://journals.aps.org/prper/abstract/10.1103/PhysRevPhysEducRes.15.010134" rel="nofollow noreferrer">https://journals.aps.org/prper/abstract/10.1103/PhysRevPhysEducRes.15.010134</a></p>
  
  <p>My stub script (takes the DOI information, 10.1103/PhysRevPhysEducRes.15.010134 for the above example, as first argument and, optionally, the name of the output file as the second):</p>
  
  <pre><code>#! /usr/bin/env python3
  
  import urllib.request
  import sys
  import io.open
  
  print('retreive url and convert to text')
  url = "https://journals.aps.org/prper/abstract/" + sys.argv[1]    
  codec = "utf_8"
  data = urllib.request.urlopen(url)
  doc = data.read().decode(codec)
  
  if len(sys.argv) == 3:
      filename = sys.argv[2]
  else
      filename = "default.html"
  
  a = io.open(filename, mode='wt', encoding='utf-8')
  for i in range(len(information)):
      information[i] = '%s\n' % information[i]
  a.writelines(information)
  a.close()
  </code></pre>