title: Swift scraping a webpage using regex or alternative
body: <p>See updates below first.</p>
  
  <p>I am trying to scrape all the moderators for a specified sub-reddit on reddit.
  The API only lets you get all the moderators usernames for a sub-reddit, so initially I had gotten all these and then performed an additional request for each of these profiles to get the avatar url. This ended up going past the API limit.</p>
  
  <p>So instead I want to just get the source of the following page and paginate through while collecting the 10 usernames and avatar url's on each page. This will end up polling the website with less requests. I understand how to do the pagination part but for now I am trying to understand how to gather the usernames and adjoining avatar URLs.</p>
  
  <p>So take the following url:</p>
  
  <pre><code>https://www.reddit.com/r/videos/about/moderators/
  </code></pre>
  
  <p>So I will pull the entire page source,</p>
  
  <p>Add all the mods usernames &amp; urls into a mod object, then into an array. </p>
  
  <p>Would using regex on the string I get back be a good idea?</p>
  
  <p>This is my code so far, any help would be great:</p>
  
  <pre><code>    func tester() {
         let url = URL(string: "https://www.reddit.com/r/videos/about/moderators")!
  
         let task = URLSession.shared.dataTask(with: url) { data, response, error in
             guard let data = data, error == nil else {
                 print("\(error)")
                 return
             }
  
          let string = String(data: data, encoding: .utf8)
  
              let regexUsernames = try? NSRegularExpression(pattern: "href=\"/user/[a-z0-9]\"", options: .caseInsensitive)
  
              var results = regexUsernames?.matches(in: string as String, options: [], range: NSRange(location: 0, length: string.length))
  
              let regexProfileURLs = try? NSRegularExpression(pattern: "&gt;&lt;img src=\"[a-z0-9]\" style", options: .caseInsensitive)
  
             print("\(results)") // This shows as empty array
         }
  
         task.resume()
     }
  </code></pre>
  
  <p>I have also tried the following but get this error:</p>
  
  <pre><code>Can't form Range with upperBound &lt; lowerBound
  </code></pre>
  
  <p>Code:</p>
  
  <pre><code>    func tester() {
         let url = URL(string: "https://www.reddit.com/r/videos/about/moderators")!
  
         let task = URLSession.shared.dataTask(with: url) { data, response, error in
             guard let data = data, error == nil else {
              print("data was nil")
                 return
             }
  
          guard let htmlString = String(data: data, encoding: .utf8) else {
              print("cannot cast data into string")
              return
          }
  
          let leftSideOfValue = "href=\"/user/"
          let rightSideOfValue = "\""
  
          guard let leftRange = htmlString.range(of: leftSideOfValue) else {
              print("cannot find range left")
              return
          }
  
          guard let rightRange = htmlString.range(of: rightSideOfValue) else {
              print("cannot find range right")
              return
          }
  
          let rangeOfTheValue = leftRange.upperBound..&lt;rightRange.lowerBound
  
          print(htmlString[rangeOfTheValue])
  }
  </code></pre>
  
  <p>UPDATE:</p>
  
  <p>So I have gotten to a point where it will give me the first username, however I am looping and just getting the same one, over and over. What would be the best way to move on each incremental step? Is there a way to do something like let newHTMLString = htmlString.dropFirst(k: ?) to replace the htmlString with a substring that is after the elements we just got?</p>
  
  <pre><code>func tester() {
         let url = URL(string: "https://www.reddit.com/r/pics/about/moderators")!
  
         let task = URLSession.shared.dataTask(with: url) { data, response, error in
             guard let data = data, error == nil else {
              print("data was nil")
                 return
             }
  
          guard let htmlString = String(data: data, encoding: .utf8) else {
              print("cannot cast data into string")
              return
          }
  
  
          let counter =  htmlString.components(separatedBy:"href=\"/user/")
          let count = counter.count
  
          for  i in 0...count {
  
              let leftSideOfUsernameValue = "href=\"/user/"
              let rightSideOfUsernameValue = "\""
  
              let leftSideOfAvatarURLValue = "&gt;&lt;img src=\""
              let rightSideOfAvatarURLValue = "\"&gt;"
  
  
            guard let leftRange = htmlString.range(of: leftSideOfUsernameValue) else {
                  print("cannot find range left")
                  return
              }
  
              guard let rightRange = htmlString.range(of: rightSideOfUsernameValue) else {
                  print("cannot find range right")
                  return
              }
  
              let username = htmlString.slice(from: leftSideOfUsernameValue, to: rightSideOfUsernameValue)
              print(username)
              guard let avatarURL = htmlString.slice(from: leftSideOfAvatarURLValue, to: rightSideOfAvatarURLValue) else {
                  print("Error")
                  return
              }
              print(avatarURL)
  
          }
  
         }
  
         task.resume()
     }
  </code></pre>
  
  <p>I have also tried:</p>
  
  <pre><code>           let endString = String(avatarURL + rightSideOfAvatarURLValue)
              let endIndex = htmlString.index(endString.endIndex, offsetBy: 0)
              let substringer = htmlString[endIndex...]
              htmlString = String(substringer)
  </code></pre>