title: Scraping data from tables on multiple web pages in R (football players)
body: <p>I'm working on a project for school where I need to collect the career statistics for individual NCAA football players. The data for each player is in this format.</p>
  
  <p><a href="http://www.sports-reference.com/cfb/players/ryan-aplin-1.html" rel="nofollow noreferrer">http://www.sports-reference.com/cfb/players/ryan-aplin-1.html</a></p>
  
  <p>I cannot find an aggregate of all players so I need to go page by page and pull out the bottom row of each passing scoring Rushing &amp; receiving etc. html table</p>
  
  <p>Each player is catagorized by their last name with links to each alphabet going here.</p>
  
  <p><a href="http://www.sports-reference.com/cfb/players/" rel="nofollow noreferrer">http://www.sports-reference.com/cfb/players/</a></p>
  
  <p>For instance, each player with the last name A is found here.</p>
  
  <p><a href="http://www.sports-reference.com/cfb/players/a-index.html" rel="nofollow noreferrer">http://www.sports-reference.com/cfb/players/a-index.html</a></p>
  
  <p>This is my first time really getting into data scraping so I tried to find similar questions with answers. The closest answer I found was <a href="https://stackoverflow.com/questions/7774991/how-do-i-scrape-multiple-pages-with-xml-and-readhtmltable">this question</a></p>
  
  <p>I believe I could use something very similar where I switch page number with the collected player's name. However, I'm not sure how to change it to look for player name instead of page number. </p>
  
  <p>Samuel L. Ventura also gave a talk about data scraping for NFL data recently that can be found <a href="http://www.stat.cmu.edu/~sventura/?page_id=22" rel="nofollow noreferrer">here</a>. </p>
  
  <p>EDIT:</p>
  
  <p>Ben was really helpful and provided some great code. The first part works really well, however when I attempt to run the second part I run into this.</p>
  
  <pre><code>&gt; # unlist into a single character vector
  &gt; links &lt;- unlist(links)
  &gt; # Go to each URL in the list and scrape all the data from the tables
  &gt; # this will take some time... don't interrupt it! 
  &gt; all_tables &lt;- lapply(links, readHTMLTable, stringsAsFactors = FALSE)
  Error in UseMethod("xmlNamespaceDefinitions") : 
   no applicable method for 'xmlNamespaceDefinitions' applied to an object of class "NULL"
  &gt; # Put player names in the list so we know who the data belong to
  &gt; # extract names from the URLs to their stats page...
  &gt; toMatch &lt;- c("http://www.sports-reference.com/cfb/players/", "-1.html")
  &gt; player_names &lt;- unique (gsub(paste(toMatch,collapse="|"), "", links))
  Error: cannot allocate vector of size 512 Kb
  &gt; # assign player names to list of tables
  &gt; names(all_tables) &lt;- player_names
  Error: object 'player_names' not found
  &gt; fix(inx_page)
  Error in edit(name, file, title, editor) : 
    unexpected '&lt;' occurred on line 1
   use a command like
   x &lt;- edit()
   to recover
  In addition: Warning message:
  In edit.default(name, file, title, editor = defaultEditor) :
    deparse may be incomplete
  </code></pre>
  
  <p>This could be an error due to not having sufficient memory (only 4gb on computer I am currently using). Although I do not understand the error </p>
  
  <pre><code>    &gt; all_tables &lt;- lapply(links, readHTMLTable, stringsAsFactors = FALSE)
  Error in UseMethod("xmlNamespaceDefinitions") : 
   no applicable method for 'xmlNamespaceDefinitions' applied to an object of class "NULL"
  </code></pre>
  
  <p>Looking through my other datasets my players really only go back to 2007. If there would be some way to pull just people from 2007 onwards that may help shrink the data. If I had a list of people whose names I wanted to pull could I just replace the lnk in</p>
  
  <pre><code> links[[i]] &lt;- paste0("http://www.sports-reference.com", lnk)
  </code></pre>
  
  <p>with only the players that I need?</p>