text: Title: In certain situations, it is impossible to enter the method under @crawler.router.default_handler, and the execution ends directly.
  Body: code ：
  ```py
  async def crawling(urls: List, output_path: str, unique_filter:set) -> None:
      # initialize crawler configuration
      concurrency_settings = ConcurrencySettings(
          max_concurrency=50,
          max_tasks_per_minute=200,
      )
      session_pool = SessionPool(max_pool_size=100)
      crawler = BeautifulSoupCrawler(
          max_request_retries=3,
          request_handler_timeout=timedelta(
              seconds=30,
          ),
          max_requests_per_crawl=100,
          max_crawl_depth=1,
          concurrency_settings=concurrency_settings,
          session_pool=session_pool,
      )
  
      # Define the default request handler, which will be called for every request
      @crawler.router.default_handler
      async def request_handler(context: BeautifulSoupCrawlingContext) -> None:
          url = context.request.url
          logger.info(f'Processing {url} ...')
          depth = context.request.crawl_depth
          logger.info(f'The depth of {url} is:{depth}.')
          await context.enqueue_links(
              strategy='same-hostname',
              include=init_filters(),
              transform_request_function=transform_request,
          )
  
      await crawler.run(urls)
  ```
  
  Issue 1: In the above code, debugging revealed that after executing the crawling method, when it reaches await crawler.run(urls), it does not enter the method under @crawler.router.default_handler (sometimes it works, sometimes it doesn't).
  
  Issue 2: If crawling a URL fails due to network issues, attempting to crawl the same URL again also does not enter the @crawler.router.default_handler.