title: Pulling historic analyst opinions from yahoo finance in R
body: <p>Yahoo Finance has <a href="http://finance.yahoo.com/q/ud?s=GOOG" rel="nofollow noreferrer">data on historic analyst opinions</a> for stocks.  I'm interested in pulling this data into R for analysis, and here is what I have so far:</p>
  
  <pre><code>getOpinions &lt;- function(symbol) {
      require(XML)
      require(xts)
      yahoo.URL &lt;- "http://finance.yahoo.com/q/ud?"
      tables &lt;- readHTMLTable(paste(yahoo.URL, "s=", symbol, sep = ""), stringsAsFactors=FALSE)
      Data &lt;- tables[[11]]
      Data$Date &lt;- as.Date(Data$Date,'%d-%b-%y')
      Data &lt;- xts(Data[,-1],order.by=Data[,1])
      Data
  }
  
  getOpinions('AAPL')
  </code></pre>
  
  <p>I'm worried that this code will break if the position of the table (currently 11) changes, but I can't think of an elegant way to detect which table has the data I want.  I tried <a href="https://stackoverflow.com/questions/1395528/scraping-html-tables-into-r-data-frames-using-the-xml-package/1849388#1849388">the solution posted here</a>, but it doesn't seem to work for this problem.</p>
  
  <p>Is there a better way to scrape this data that is less likely to break if yahoo re-arranges their site?</p>
  
  <p>edit: it looks like there's already a package (<a href="http://cran.r-project.org/web/packages/fImport/index.html" rel="nofollow noreferrer">fImport</a>) out there to do this.</p>
  
  <pre><code>library(fImport)
  yahooBriefing("AAPL")
  </code></pre>
  
  <p>Here is their solution, which doesn't return an xts object, and will probably break if the page layout changes (the yahooKeystats function in fImport is already broken):</p>
  
  <pre><code>function (query, file = "tempfile", source = NULL, save = FALSE, 
      try = TRUE) 
  {
      if (is.null(source)) 
          source = "http://finance.yahoo.com/q/ud?s="
      if (try) {
          z = try(yahooBriefing(query, file, source, save, try = FALSE))
          if (class(z) == "try-error" || class(z) == "Error") {
              return("No Internet Access")
          }
          else {
              return(z)
          }
      }
      else {
          url = paste(source, query, sep = "")
          download.file(url = url, destfile = file)
          x = scan(file, what = "", sep = "\n")
          x = x[grep("Briefing.com", x)]
          x = gsub("&lt;/", "&lt;", x, perl = TRUE)
          x = gsub("/", " / ", x, perl = TRUE)
          x = gsub(" class=.yfnc_tabledata1.", "", x, perl = TRUE)
          x = gsub(" align=.center.", "", x, perl = TRUE)
          x = gsub(" cell.......=...", "", x, perl = TRUE)
          x = gsub(" border=...", "", x, perl = TRUE)
          x = gsub(" color=.red.", "", x, perl = TRUE)
          x = gsub(" color=.green.", "", x, perl = TRUE)
          x = gsub("&lt;.&gt;", "", x, perl = TRUE)
          x = gsub("&lt;td&gt;", "@", x, perl = TRUE)
          x = gsub("&lt;..&gt;", "", x, perl = TRUE)
          x = gsub("&lt;...&gt;", "", x, perl = TRUE)
          x = gsub("&lt;....&gt;", "", x, perl = TRUE)
          x = gsub("&lt;table&gt;", "", x, perl = TRUE)
          x = gsub("&lt;td nowrap", "", x, perl = TRUE)
          x = gsub("&lt;td height=....", "", x, perl = TRUE)
          x = gsub("&amp;amp;", "&amp;", x, perl = TRUE)
          x = unlist(strsplit(x, "&gt;"))
          x = x[grep("-...-[90]", x, perl = TRUE)]
          nX = length(x)
          x[nX] = gsub("@$", "", x[nX], perl = TRUE)
          x = unlist(strsplit(x, "@"))
          x[x == ""] = "NA"
          x = matrix(x, byrow = TRUE, ncol = 9)[, -c(2, 4, 6, 8)]
          x[, 1] = as.character(strptime(x[, 1], format = "%d-%b-%y"))
          colnames(x) = c("Date", "ResearchFirm", "Action", "From", 
              "To")
          x = x[nrow(x):1, ]
          X = as.data.frame(x)
      }
      X
  }
  </code></pre>