title: API and web scraping
body: <p>I am trying to access the contents on text files on the <a href="https://usda.library.cornell.edu/concern/publications/c821gj76b?locale=en&amp;page=2#release-items" rel="nofollow noreferrer">page</a>. Since each text file has a different url, I am unable to generate urls in python and <strong>scrape</strong> the content using Pandas. Therefore, I was trying to use the <a href="https://usda.library.cornell.edu/apidoc/index.html#/account/getUserToken" rel="nofollow noreferrer">API</a> for this. When I execute for user token, I get something like this:</p>
  <pre class="lang-py prettyprint-override"><code>{
    &quot;jwt&quot;: &quot;eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOjU5MDR9.b9elxkmNj0kmWxDPjal0_mLY9UPg7enoT7Cdg7gN1d0&quot;
  }
  </code></pre>
  <p>Now, I am not sure how to use this to get access to all the text files on the first page I mentioned above. Can someone guide me on how to progress?</p>