title: Querying JSON data from CMS NPI data
body: <p>Banging head on this for a couple of days, could use a wake up call !
  CMS (Centers for Medicare &amp; Medicaid Services) offers an API for accessing medical practitioner information based on the person's NPI (National Provider Identifier).</p>
  
  <p>There is a wealth of information here including megafile monthly downloads, etc. but I don't need any of that.  I simply need to issue a query (low volume) for a single NPI that I have prequalified and return a few values from the record that is retrieved.</p>
  
  <p>Here is a sample query for a randomly-selected NPI --
  <a href="https://npiregistry.cms.hhs.gov/api/resultsDemo2/?number=1881761864&amp;pretty=on" rel="nofollow noreferrer">https://npiregistry.cms.hhs.gov/api/resultsDemo2/?number=1881761864&amp;pretty=on</a></p>
  
  <p>If you run this in a browser window you see the resulting JSON data encapsulated in some header/footer HTML.  </p>
  
  <p>I can dump the entire query result and print it a few different ways but haven't been able to pick out and print specific data elements such as name, address or telephone number.  If you run the query in a browser you can see the raw output and the snippet below prints the sanitized version of the results.  See below.  Ideas?</p>
  
  <pre><code>import urllib
  from bs4 import BeautifulSoup
  import json
  
  def main():
  
  url = "https://npiregistry.cms.hhs.gov/api/resultsDemo2/?number=1881761864&amp;pretty=on"
  html = urllib.urlopen(url).read()
  soup = BeautifulSoup(html,"lxml")
  
  for script in soup(["script", "style"]):
      script.extract()
  
  practitioner_rec = soup.get_text()
  
  # strip out the html to retain the JSON record
  lines = (line.strip() for line in practitioner_rec.splitlines())
  chunks = (phrase.strip() for line in lines for phrase in line.split("  "))
  practitioner_rec = '\n'.join(chunk for chunk in chunks if chunk)
  
  # get a count of lines generated by the query, valid queries are greater than 3 lines long
  linect = practitioner_rec.count('\n') +1
  
  if linect == 3:
      VALID_NPI="FALSE"
      VALID_MD="FALSE"
  else: # approx. 69 lines of output here
      #   possible issue with JSON formmatting here
      #   In particular, the line   
      #   "result_count":1, "results":[
      #   since result count will always be 1, discard it
      practitioner_rec = practitioner_rec.replace('"result_count":1, ', '')
      print(practitioner_rec)
  
      practitioner_data = json.loads(practitioner_rec)
      VALID_NPI="TRUE"
      VALID_MD="TRUE"
  
      '''
      none of these constructs works to print the provider name
      print ['result_count']['results']['basic']['name'],"name"
      print result_count['results']['basic']['name'],"name"
      print practitioner_data['results']['basic']['name'],"name"
      print results['basic']['name'],"name"
      print ['basic']['name'],"name"
      print basic['name'],"name"
      print results[2]['basic']['name'],"name"
      print results['basic']['name'],"name"
  
      this works, but not useful if I can't pick values out
      print(json.dumps(practitioner_data))      
  
      print "VALID_NPI is ",VALID_NPI
      print "VALID_MD is  ",VALID_MD
      return [VALID_NPI,VALID_MD]    
      '''
  
  
  if __name__ == '__main__':
      main()
  </code></pre>