<p>AWS announces "frequency of interruption" on the page without API in here: <code>https://aws.amazon.com/ec2/spot/instance-advisor/</code></p>

<p>I need to get all types(255 pieces) on Virginia in a scheduled way. How can I do that?</p>

<p>I solved it by crawling the page with selenium and Python. You can run the following script in docker, it will create <code>aws-spot-instance.txt</code> the pwd. </p>

<p>You can change the region, it is <code>region = 'US East (N. Virginia)'</code> in the script.</p>

<pre><code>from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

options = Options()
options.add_argument("--headless") # Runs Chrome in headless mode.
options.add_argument('--no-sandbox') # # Bypass OS security model
options.add_argument('start-maximized')
options.add_argument('disable-infobars')
options.add_argument("--disable-extensions")
options.add_argument('--disable-gpu')

region = 'US East (N. Virginia)'

driver = webdriver.Chrome(chrome_options=options)
driver.get("https://aws.amazon.com/ec2/spot/instance-advisor")
element = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.CSS_SELECTOR, ".dropdown-container.aws-dropdown-region.dropdown-built")))
element.click()
driver.find_element_by_xpath("//span[text()='%s']" %region).click()
driver.find_element_by_css_selector(".aws-spot-advisor-button-expand.button").click()

table_data = driver.find_element_by_css_selector(".table.table-striped").text
fw = open('aws-spot-instance.txt', "w")
fw.write(table_data)
fw.close()
</code></pre>

<p>save this script a <code>aws.py</code> and run the following command:</p>

<pre><code>docker pull gunesmes/python-selenium-behave-page-object-docker
docker run --rm --name aws -v $PWD:/project gunesmes/python-selenium-behave-page-object-docker bash -c "python3 aws.py"
</code></pre>

<p><a href="https://i.stack.imgur.com/zXGBT.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/zXGBT.png" alt="enter image description here"></a></p>
