title: Selecting a value from Dynamic List of Values using Selenium and Scrapy
body: <p>I am trying to scrape a website which displays a list of values. I need to select one value and scrape, followed by another selection. I am just unable to do this, after I save the page, I see the following code for the list of value:</p>
  
  <pre><code>&lt;div class="dropdown-component spacing-left spacing-top"&gt;
  &lt;!-- ko component: { name: "dropdown-component", params: { value: categoryDropdown } } --&gt;&lt;div class="btn-group" data-bind="attr: { &amp;#39;data-id&amp;#39;: id }" data-id="dropdown-component-118db170-aa45-43ac-94ee-8d78c72bda58"&gt;
      &lt;button type="button" class="dropdown-toggle dropdown-component clear" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false" data-bind="event: { keydown: onKeyDown }"&gt;
          &lt;div class="table-dropdown"&gt;
              &lt;div class="text"&gt;
                  &lt;span data-bind="i18n: labelKey, visible: labelKey"&gt;Category&lt;/span&gt;
                  &lt;span data-bind="text: seperator, visible: seperator &amp;amp;&amp;amp; selectedOptionHasText()"&gt;:&lt;/span&gt;
                  &lt;!-- ko if: selectedText() --&gt;
                  &lt;span data-bind="text: selectedText"&gt;All&lt;/span&gt;
                  &lt;!-- /ko --&gt;
                  &lt;!-- ko if: selectedTextKey() --&gt;&lt;!-- /ko --&gt;
              &lt;/div&gt;
              &lt;div class="icon caret"&gt;&lt;/div&gt;
          &lt;/div&gt;
      &lt;/button&gt;
      &lt;ul class="dropdown-menu scrollable-dropdown" data-bind="foreach: options, css: css"&gt;
  
          &lt;li data-bind="css: { highlighted: !isCategory &amp;amp;&amp;amp; id === $parent.highlightedOption() }, event: { mouseenter: $parent.onMouseEnter, mouseout: $parent.onMouseOut }" class=""&gt;
              &lt;!-- ko if: isNoType --&gt;
              &lt;a data-bind="text: text, click: $parent.action.bind($index), i18n: textKey, &amp;#39;i18n-options&amp;#39;: textKeyReplacements"&gt;All&lt;/a&gt;
              &lt;!-- /ko --&gt;
              &lt;!-- ko if: isCategory --&gt;&lt;!-- /ko --&gt;
              &lt;!-- ko if: isCategoryItem --&gt;&lt;!-- /ko --&gt;
          &lt;/li&gt;
  
          &lt;li data-bind="css: { highlighted: !isCategory &amp;amp;&amp;amp; id === $parent.highlightedOption() }, event: { mouseenter: $parent.onMouseEnter, mouseout: $parent.onMouseOut }" class=""&gt;
              &lt;!-- ko if: isNoType --&gt;
              &lt;a data-bind="text: text, click: $parent.action.bind($index), i18n: textKey, &amp;#39;i18n-options&amp;#39;: textKeyReplacements"&gt;Not Classified&lt;/a&gt;
              &lt;!-- /ko --&gt;
              &lt;!-- ko if: isCategory --&gt;&lt;!-- /ko --&gt;
              &lt;!-- ko if: isCategoryItem --&gt;&lt;!-- /ko --&gt;
          &lt;/li&gt;
  </code></pre>
  
  <p>It shows that the lov has a dynamic id assigned to it. Any idea how to select values from this lov? It has values like "All", "Not Classified" etc.</p>
  
  <p>I have been able to resolve this by below </p>
  
  <pre><code>driver.find_element_by_xpath("//span[contains(.,'Category')]").click()
  driver.execute(driver.find_element_by_xpath("//a[contains(text(),'Not Classofied')]").click())
  </code></pre>
  
  <p>This solves my issue, but I receive an error in python as: "KeyError: None".</p>
  
  <p>I am not sure why this error comes, but for time being I am ignoring it by a try-except block.</p>