title: Web scraping with R (Scrape Hidden Number "Click here to show number" )
body: <p>As an intern in a data science team, I was given the task to find a way to automatically collect specific data on a real estate ad website, using R.</p>
  
  <p>Thanks to the answer given on this post (<a href="https://stackoverflow.com/questions/5990525/web-scraping-with-r-over-real-estate-ads">Web scraping with R over real estate ads</a>) and some changes in the code, I managed to perform the task I want. But my problem is that I can't scrape the phone number. I tried several things but without any success.</p>
  
  <p>I want to do exactly the same as in the previous post but with the phone number as a new variable.</p>
  
  <p>Here is the detail of an ad: <a href="https://www.leboncoin.fr/ventes_immobilieres/1074663461.htm?ca=13_s" rel="nofollow noreferrer">https://www.leboncoin.fr/ventes_immobilieres/1074663461.htm?ca=13_s</a> My variables are: the price ("Prix"), the city ("Ville"), the surface ("surface"), the "GES, the "Classe énergie" , the number of room ("Pièces") and the phone number , as well as the number of pictures shown in the ad. </p>
  
  <p>I noticed that the code given on the answer is no longer working since at the time the question was asked, the website was not secured (http). Today it includes the beginning 'https'. That is why I made some changes in the code.</p>
  
  <p>I am a beginner with R and any help would be really appreciated  (sorry for my bad english). </p>
  
  <pre><code>get_ad_links = function(page){
  require(rvest)
  # construct url to page (!when running the code put the url in 1 line!)
  url_base = "https://www.leboncoin.fr/ventes_immobilieres/offres/
  languedoc_roussillon/pyrenees_orientales"
  url      = paste(url_base, "?o=", page,"&amp;ret=1&amp;ret=2&amp;f=p", sep = "")
  page     = read_html(url)
  
  # extract links to ads on page
  a="//*/section/section/ul/li["
  b="]/a/@href"
  t =lapply(1:30, function(i)  paste(a,i,b, sep = ""))
  ad_links = sapply(1:30,function(i) { page %&gt;% 
  html_node(xpath=as.character(t[i])) %&gt;% html_text()})
  return(ad_links)  
  }
  
  # Function to Get Ad Details by Ad URL
  get_ad_details = function(ad_url){
  require(rvest)
  # parse ad url to html tree
  doc = read_html(paste("https:",ad_url,sep=""))
  
  # extract labels and values using xpath expression
  pattern&lt;- "&lt;/?\\w+((\\s+\\w+(\\s*m\\s*(?:\".*?
  \"|'.*?'[^'\"&gt;\\s]+))?)+\\s*|\\s*)/?&gt;"
  
  prix = doc %&gt;% 
  html_node(xpath="//section/section/section[2]/div[4]/h2/span[2]") %&gt;% 
  html_text()
  PRIX = stringr::str_replace_all(prix,pattern,"")
  PRIX =stringr::str_wrap(PRIX)
  ville = doc %&gt;% 
  html_node(xpath="//section/section/section[2]/div[5]/h2/span[2]") %&gt;% 
  html_text()
  VILLE = stringr::str_replace_all(ville,pattern,"")
  VILLE = stringr::str_wrap(VILLE)
  surface = doc %&gt;% 
  html_node(xpath="//section/section/section[2]/div[8]/h2/span[2]") %&gt;% 
  html_text()
  SURFACE = stringr::str_replace_all(surface,pattern,"")
  SURFACE = stringr::str_wrap(SURFACE)
  pieces = doc %&gt;% 
  html_node(xpath="//section/section/section[2]/div[7]/h2/span[2]") %&gt;% 
  html_text()
  PIECES = stringr::str_replace_all(pieces,pattern,"")
  PIECES = stringr::str_wrap(PIECES)
  type = doc %&gt;% 
  html_node(xpath="//section/section/section[2]/div[6]/h2/span[2]") %&gt;% 
  html_text()
  TYPE_BIEN = stringr::str_replace_all(type,pattern,"")
  TYPE_BIEN = stringr::str_wrap(TYPE_BIEN)
  ges = doc %&gt;% 
  html_node(xpath="//section/section/section[2]/div[9]/h2/span[2]") %&gt;% 
  html_text()
  GES = stringr::str_replace_all(ges,pattern,"")
  GES = stringr::str_wrap(GES)
  values  = c(PRIX, VILLE,SURFACE,PIECES,TYPE_BIEN,GES)
  
  # convert to data frame and add labels
  mydf  = as.data.frame(t(values))
  names(mydf)= c("PRIX", "VILLE","SURFACE","PIECES" ,"TYPE_BIEN","GES")
  return(mydf)
  }
  
  
  ad_links = get_ad_links(page = 1)
  
  # grab ad details for first 30 links from page 1
  require(plyr)
  ad_details = ldply(ad_links[1:30], get_ad_details, .progress = 'text')
  </code></pre>