title: Selenium find all elements by xpath
body: <p>I used selenium to scrap a scrolling website and conducted the code below   </p>
  
  <pre><code>import requests
  from bs4 import BeautifulSoup
  import csv
  from selenium import webdriver
  from selenium.webdriver.support.ui import WebDriverWait
  import unittest
  from selenium.webdriver.common.by import By
  from selenium.webdriver.support import expected_conditions as EC
  import time
  import unittest
  import re
  
  output_file = open("Kijubi.csv", "w", newline='')  
  
  class Crawling(unittest.TestCase):
      def setUp(self):
          self.driver = webdriver.Firefox()
          self.driver.set_window_size(1024, 768)
          self.base_url = "http://www.viatorcom.de/"
          self.accept_next_alert = True
  
      def test_sel(self):
          driver = self.driver
          delay = 3
          driver.get(self.base_url + "de/7132/Seoul/d973-allthingstodo")
          for i in range(1,1):
              driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
              time.sleep(2)
      html_source = driver.page_source
      data = html_source.encode("utf-8")
  </code></pre>
  
  <p>My next step was to crawl specific information from the website like the price.</p>
  
  <p>Hence, I added the following code:</p>
  
  <pre><code> all_spans = driver.find_elements_by_xpath("/html/body/div[5]/div/div[3]/div[2]/div[2]/div[1]/div[1]/div")
      print(all_spans)
      for price in all_spans:
          Header = driver.find_elements_by_xpath("/html/body/div[5]/div/div[3]/div[2]/div[2]/div[1]/div[1]/div/div[2]/div[2]/span[2]")
          for span in Header:
              print(span.text)
  </code></pre>
  
  <p>But I get just one price instead all of them. Could you provide me feedback on what I could improve my code? Thanks:)</p>
  
  <p><strong>EDIT</strong></p>
  
  <p>Thanks to your guys I managed to get it running. Here is the additional code:</p>
  
  <pre><code>    elements = driver.find_elements_by_xpath("//div[@id='productList']/div/div")
  
      innerElements = 15
  
      outerElements = len(elements)/innerElements
  
      print(innerElements,  "\t", outerElements, "\t", len(elements))
  
      for j in range(1, int(outerElements)):
  
          for i in range(1, int(innerElements)):
  
  
              headline = driver.find_element_by_xpath("//div[@id='productList']/div["+str(j)+"]/div["+str(i)+"]/div/div[2]/h2/a").text
  
              price = driver.find_element_by_xpath("//div[@id='productList']/div["+str(j)+"]/div["+str(i)+"]/div/div[2]/div[2]/span[2]").text
              deeplink = driver.find_element_by_xpath("//div[@id='productList']/div["+str(j)+"]/div["+str(i)+"]/div/div[2]/h2/a").get_attribute("href")
  
              print("Header: " + headline + " | " + "Price: " + price + " | " + "Deeplink: " + deeplink)
  </code></pre>
  
  <p>Now my last issue is that I still do not get the last 20  prices back, which have a English description. I only get back the prices which have German description. For English ones, they do not get fetched although they share the same html structure.</p>
  
  <p>E.g. html structure for the English items </p>
  
  <pre><code>     headline =   driver.find_element_by_xpath("//div[@id='productList']/div[6]/div[1]/div/div[2]/h2/a")
  </code></pre>
  
  <p>Do you guys know what I have to modify? Any feedback is appreciated:) </p>