title: How to get a JSON response from a Google Chrome Selenium Webdriver client?
body: <p>Currently I have Selenium hooked up to python to scrape a webpage. I found out that the page actually pulls data from a JSON API, and I can get a JSON response as long as I'm logged in to the page.</p>
  
  <p>However, my approach of getting that response into python seems a bit junky; I select text enclosed in <code>&lt;pre&gt;</code> tags and use python's <code>json</code> package to parse the data like so:</p>
  
  <pre><code>import json
  from selenium import webdriver
  
  url = 'http://jsonplaceholder.typicode.com/posts/1'
  driver = webdriver.Chrome()
  driver.get(url)
  json_text = driver.find_element_by_css_selector('pre').get_attribute('innerText')
  json_response = json.loads(json_text)
  </code></pre>
  
  <p>The only reason I need to select within <code>&lt;pre&gt;</code> tags at all is because when JSON appears in Chrome, it comes formatted like this:</p>
  
  <pre><code>&lt;html&gt;
  &lt;head&gt;&lt;/head&gt;
  &lt;body&gt;
  &lt;pre style="word-wrap: break-word; white-space: pre-wrap;"&gt;{
    "userId": 1,
    "id": 1,
    "title": "sunt aut facere repellat provident occaecati excepturi optio reprehenderit",
    "body": "quia et suscipit\nsuscipit recusandae consequuntur expedita et cum\nreprehenderit molestiae ut ut quas totam\nnostrum rerum est autem sunt rem eveniet architecto"
  }&lt;/pre&gt;
  &lt;/body&gt;
  &lt;/html&gt;
  </code></pre>
  
  <p>And the only reason I need to do this inside selenium at all is because I need to be logged into the website in order to get a response. Otherwise I get a 401 and no data.</p>