Web scraping - selection of a table <p>I'm trying to extract <a href="http://finance.yahoo.com/quote/%5EFTSE/history?period1=946684800&amp;period2=1470441600&amp;interval=1mo&amp;filter=history&amp;frequency=1mo" rel="nofollow">the table of historical data</a> from Yahoo Finance website. </p> <p>First, by inspecting the source code I've found that it's actually a table, so I suspect that <code>html_table()</code> from <code>rvest</code> should be able to work with it, however, I can't find a way to reach it from R. I've tried providing the function with just the full page, however, it did not fetch the right table:</p> <pre><code>url &lt;- https://finance.yahoo.com/quote/^FTSE/history?period1=946684800&amp;period2=1470441600&amp;interval=1mo&amp;filter=history&amp;frequency=1mo read_html(url) %&gt;% html_table(fill = TRUE) # Returns only: # [[1]] # X1 X2 # 1 Show all results for Tip: Use comma to separate multiple quotes Search </code></pre> <p>Second, I've found an xpath selector for the particular table, but I am still unsuccessful in fetching the data:</p> <pre><code>xpath1 &lt;- '//*[@id="main-0-Quote-Proxy"]/section/div[2]/section/div/section/div[3]/table' read_html(url) %&gt;% html_node(xpath = xpath1) # Returns an empty nodeset: # {xml_nodeset (0)} </code></pre> <p>By removing the last term from the selector I get a non-empty nodeset, however, still no table:</p> <pre><code>xpath2 &lt;- '//*[@id="main-0-Quote-Proxy"]/section/div[2]/section/div/section/div[3]' read_html(url) %&gt;% html_node(xpath = xpath2) %&gt;% html_table(fill = TRUE) # Error: html_name(x) == "table" is not TRUE </code></pre> <p>What am I doing wrong? Any help would be appreciated!</p> <p>EDIT: I've found that <code>html_text()</code> with the last xpath returns</p> <pre><code>read_html(url) %&gt;% html_node(xpath = xpath2) %&gt;% html_text() [1] "Loading..." </code></pre> <p>which suggests that the table is not yet loaded when R did the read. This would explain why it failed to see the table. Question: any ways of bypassing that loading text?</p>