I'm trying to parse the HTML of a page with infinite scrolling.  I want to load all of the content so that I can parse it all.  I'm using Python.  Any hints?
Those pages update their html with AJAX. Usually you just need to find the new AJAX requests send by browser, guess the meaning of the AJAX url parameters and fetch the data from the API.
API servers may validate the user agent, referer, cookie, oauth_token ... of the AJAX request, keep an eye on them.
Parse HTML Infinite Scroll