title: Scraping More than rendered Data with Beautiful Soup
body: <p>I'm scraping apps names from Google Play Store and for each URL as input I get only 60apps (because the website rendered 60apps if the user doesn't scroll down). How is it working and how can I scrape all the apps from a page using BeautifulSoup and/or Selenium ?</p>
  
  <p>Thank you</p>
  
  <p>Here is my code :</p>
  
  <pre><code>urls = []
  
  urls.extend(["https://play.google.com/store/apps/category/NEWS_AND_MAGAZINES/collection/topselling_paid"])
  
  for i in urls:
      response = get(i)
      html_soup = BeautifulSoup(response.text, 'html.parser')
      app_container = html_soup.find_all('div', class_="card no-rationale square-cover apps small")
      file = open("./InputFiles/applications.txt","w+")
      for i in range(0, len(app_container)):
          #print(app_container[i].div['data-docid'])
          file.write(app_container[i].div['data-docid'] + "\n")
  
      file.close()
  num_lines = sum(1 for line in open('./InputFiles/applications.txt'))
  print("Applications : " + str(num_lines) )
  </code></pre>