title: How to scrape a javascript website in Python?
body: <p>I am trying to scrape a website. I have tried using two methods but both do not provide me with the full website source code that I am looking for. I am trying to scrape the news titles from the website URL provided below.</p>
  <p>URL: &quot;https://www.todayonline.com/&quot;</p>
  <p>These are the two methods I have tried but failed.</p>
  <h1>Method 1: Beautiful Soup</h1>
  <pre><code>tdy_url = &quot;https://www.todayonline.com/&quot;
  page = requests.get(tdy_url).text
  soup = BeautifulSoup(page)
  soup  # Returns me a HTML with javascript text
  soup.find_all('h3')
  
  ### Returns me empty list []
  </code></pre>
  <h1>Method 2: Selenium + BeautifulSoup</h1>
  <pre><code>tdy_url = &quot;https://www.todayonline.com/&quot;
  
  options = Options()
  options.headless = True
  
  driver = webdriver.Chrome(&quot;chromedriver&quot;,options=options)
  
  driver.get(tdy_url)
  time.sleep(10)
  html = driver.page_source
  
  soup = BeautifulSoup(html)
  soup.find_all('h3')
  
  ### Returns me only less than 1/4 of the 'h3' tags found in the original page source 
  </code></pre>
  <p>Please help. I have tried scraping other news websites and it is so much easier. Thank you.</p>