<h3 dir="auto">Description</h3>
<p dir="auto">Incompatible method call to SSL</p>
<h3 dir="auto">Steps to Reproduce</h3>
<ol dir="auto">
<li>Run scrapy</li>
</ol>
<p dir="auto"><strong>Expected behavior:</strong> Loading the tls.py file doesn't crash.</p>
<p dir="auto"><strong>Actual behavior:</strong> Loading the tls.py file crashes.</p>
<p dir="auto"><strong>Reproduces how often:</strong> 100%</p>
<h3 dir="auto">Versions</h3>
<p dir="auto">Scrapy       : 2.5.1<br>
lxml         : 4.9.1.0<br>
libxml2      : 2.9.14<br>
cssselect    : 1.1.0<br>
parsel       : 1.6.0<br>
w3lib        : 2.0.1<br>
Twisted      : 22.8.0<br>
Python       : 3.9.14 (main, Sep  6 2022, 23:29:09) - [Clang 13.1.6 (clang-1316.0.21.2.5)]<br>
pyOpenSSL    : 22.1.0 (OpenSSL 3.0.5 5 Jul 2022)<br>
cryptography : 38.0.1<br>
Platform     : macOS-12.6-x86_64-i386-64bit</p>
<h3 dir="auto">Additional context</h3>
<p dir="auto">File "/Users/nroose/Library/Python/3.9/lib/python/site-packages/scrapy/core/downloader/tls.py", line 23, in <br>
METHOD_SSLv3: SSL.SSLv3_METHOD,                     # SSL 3 (NOT recommended)<br>
AttributeError: module 'OpenSSL.SSL' has no attribute 'SSLv3_METHOD'</p>
<p dir="auto">I am scraping a web page which has fragment identifiers (on the home page) for same page navigation. To make it clear I have this anchor tag <code class="notranslate">&lt;a href="#idx1"&gt;1. Histoire&lt;/a&gt;</code> and when you click on it you are taken to the 'Histoire' header on the same page.  <code class="notranslate">&lt;h2 id="idx1"&gt;1. Histoire&lt;/h2&gt;</code>.</p>
<p dir="auto">Scrapy is following this fragment as <code class="notranslate">https://www.mysite.com/#idx1</code> (or <code class="notranslate">https://www.mysite.com#idx1</code>) and scraping the whole page again. Since this is the home page, it is yielding the same results as if it scraped the home page.</p>
<p dir="auto">This behavior is not repeated for the rest of the fragment identifiers (I have many more). So for this anchor tag (on the same page) the above scenario is not manifested. <code class="notranslate">&lt;a href="#idx2"&gt;2. Idées reçues&lt;/a&gt;</code> -&gt; Do not repeat itself for this.</p>
<p dir="auto">Is this intentional or am I missing something?</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="import scrapy
from datetime import datetime
from netlinking.items import NetlinkItem
from urllib.parse import urlparse, urlunparse


class NetlinkingSpider(scrapy.Spider):
    name = &quot;netlinking&quot;
    referent_netloc = ''

    def __init__(self, url=None, *args, **kwargs):
        super(NetlinkingSpider, self).__init__(*args, **kwargs)
        parsed_uri = urlparse(url)
        self.referent_netloc = parsed_uri.netloc
        self.referent_netloc = self.referent_netloc.replace('www.', '')
        self.allowed_domains = [('%s' % self.referent_netloc)]
        self.start_urls = [('%s' % url)]

    def parse(self, response):
        for anchor in response.css('a'):
            netlink = NetlinkItem()
            netlink['anchor_text'] = anchor.css('a::text').get()
            netlink['referent_website'] = response.request.url
            netlink['referent_netloc'] = self.referent_netloc
            netlink['target_url'] = anchor.css('a::attr(href)').get()

            parsed_target_uri = urlparse(netlink['target_url'])
            if parsed_target_uri.netloc.strip() \
                    and (self.referent_netloc !=
                         parsed_target_uri.netloc.lower().replace('www.', '')):

                netlink['target_website'] = \
                    '{uri.scheme}://{uri.netloc}/'.format(
                    uri=parsed_target_uri)

                netlink['target_netloc'] = '{uri.netloc}'.format(
                    uri=parsed_target_uri).replace('www.', '')

                netlink['timestamp'] = datetime.now().strftime(
                    &quot;%Y-%m-%d %H:%M:%S&quot;)

                yield netlink

       #Here I am aware that I am following all links but this problem does not repeat for #idx2 
       for a in response.css('a'):
           yield response.follow(a, callback=self.parse)"><pre class="notranslate"><code class="notranslate">import scrapy
from datetime import datetime
from netlinking.items import NetlinkItem
from urllib.parse import urlparse, urlunparse


class NetlinkingSpider(scrapy.Spider):
    name = "netlinking"
    referent_netloc = ''

    def __init__(self, url=None, *args, **kwargs):
        super(NetlinkingSpider, self).__init__(*args, **kwargs)
        parsed_uri = urlparse(url)
        self.referent_netloc = parsed_uri.netloc
        self.referent_netloc = self.referent_netloc.replace('www.', '')
        self.allowed_domains = [('%s' % self.referent_netloc)]
        self.start_urls = [('%s' % url)]

    def parse(self, response):
        for anchor in response.css('a'):
            netlink = NetlinkItem()
            netlink['anchor_text'] = anchor.css('a::text').get()
            netlink['referent_website'] = response.request.url
            netlink['referent_netloc'] = self.referent_netloc
            netlink['target_url'] = anchor.css('a::attr(href)').get()

            parsed_target_uri = urlparse(netlink['target_url'])
            if parsed_target_uri.netloc.strip() \
                    and (self.referent_netloc !=
                         parsed_target_uri.netloc.lower().replace('www.', '')):

                netlink['target_website'] = \
                    '{uri.scheme}://{uri.netloc}/'.format(
                    uri=parsed_target_uri)

                netlink['target_netloc'] = '{uri.netloc}'.format(
                    uri=parsed_target_uri).replace('www.', '')

                netlink['timestamp'] = datetime.now().strftime(
                    "%Y-%m-%d %H:%M:%S")

                yield netlink

       #Here I am aware that I am following all links but this problem does not repeat for #idx2 
       for a in response.css('a'):
           yield response.follow(a, callback=self.parse)
</code></pre></div>