title: Loop Scraping in R with rvest - Football Stats
body: <p>I am trying to get R to loop through player profiles on transfermarket.com, I first grab the roster URLs with the following. </p>
  
  <pre><code>#/ Add the Team’s URL to scrape
  
  TeamScrape &lt;- read_html("http://www.transfermarkt.com/jumplist/startseite/verein/2778")
  
  
  #// Get Club Name
  
  ClubName &lt;- TeamScrape %&gt;%
  html_nodes(".spielername-profil") %&gt;%
  html_text()
  
  #// Get All Player URLs 
  
  PlayerURLs &lt;- TeamScrape %&gt;%
  html_nodes(".spielprofil_tooltip") %&gt;%
  html_attr("href")
  
  PlayerURLs &lt;- unique(PlayerURLs)
  PlayerURLs &lt;- na.omit(PlayerURLs)
  
  PlayerURLs &lt;- paste0("http://www.transfermarkt.com", PlayerURLs)
  
  PlayerLinks = data.frame(ClubName, PlayerURLs)
  </code></pre>
  
  <p>This gives me a data.frame including the URLs that I want to loop through my next scraper - the 'Player Profile Scraper'. </p>
  
  <pre><code>#/ Add the Player’s URL that you want to scrape
  URLLink &lt;- PlayerURLs[13]
  PlayerTest &lt;- read_html(URLLink)
  
  
  #// Squad No 
  
  SquadNo &lt;- PlayerTest %&gt;%
  html_nodes(".rueckennummer-profil") %&gt;%
  html_text()
  
  
  #// Name 
  
  Name &lt;- PlayerTest %&gt;%
  html_nodes(".spielername-profil") %&gt;%
  html_text() 
  
  #// Nationality 
  
  Nationality &lt;- PlayerTest %&gt;%
  html_nodes(".flaggenrahmen+ span") %&gt;%
  html_text() 
  
  #// Club 
  
  Club &lt;- PlayerTest %&gt;%
  html_nodes(".vereinprofil_tooltip+ .vereinprofil_tooltip") %&gt;%
  html_text() 
  
  #// Position 
  
  Position &lt;- PlayerTest %&gt;%
  html_nodes(".list+ .list tr:nth-child(3) td") %&gt;%
  html_text()
  
  #// DOB
  
  DOB &lt;- PlayerTest %&gt;%
  html_nodes(".wsnw") %&gt;%
  html_text()
  
  #// Age 
  
  Age &lt;- PlayerTest %&gt;%
  html_nodes(".profilheader .hide-for-small td") %&gt;%
  html_text() %&gt;%
  as.numeric()
  
  #// Value 
  
  Value &lt;- PlayerTest %&gt;%
  html_nodes(".marktwert a") %&gt;%
  html_text()
  
  #// Matches Played this Season
  
  Matches &lt;- PlayerTest %&gt;%
  html_nodes(".hide.hide-for-small+ .zentriert") %&gt;%
  html_text() %&gt;%
  as.numeric()
  
  #// Goals Scored this Season
  
  Goals &lt;- PlayerTest %&gt;%
  html_nodes("#yw1 tfoot .zentriert:nth-child(4)") %&gt;%
  html_text() %&gt;%
  as.numeric()
  
  #// Assists Made this Season
  
  Assists &lt;- PlayerTest %&gt;%
  html_nodes("tfoot .zentriert:nth-child(5)") %&gt;%
  html_text() %&gt;%
  as.numeric()
  
  #// Mins Played this Season
  
  Minutes &lt;- PlayerTest %&gt;%
  html_nodes("tfoot .zentriert:nth-child(7)") %&gt;%
  html_text() %&gt;%
  as.numeric()
  
  #// Some Cleaning Up of the Data 
  
  # to_remove_SquadNo &lt;- paste(c("#"))
  # SquadNo &lt;- gsub(to_remove_SquadNo, "", SquadNo)
  
  # Minutes &lt;- regmatches(Minutes, gregexpr("[[:digit:]]+", Minutes))
  # as.numeric(unlist(Minutes))
  
  #// Create the Data Frame 
  
  output = data.frame(SquadNo, Name, Nationality, Club, Position, DOB, Age, Value, Matches, Goals, Assists, Minutes)
  </code></pre>
  
  <p>My aim is to loop the player profile scraper based on the URLs that come out of the Team Scraper. I have tried numerous different loop attempts and I am lost! Would really appreciate some advice! </p>