title: How does recursive download work in wget?
body: <p>wget is used to mirror sites, but i wanted to know how does the utility download all the URLs for the domain.</p>
  
  <p>wget -r www.xyz.com</p>
  
  <p>How does wget download all the URLs of the domain xyz? Does it visit the index page and parse it and extract links like a crawler ?</p>