title: How to create a whitelist of URLs using a sitemap in robots.txt?
body: <p>I'm trying to create a whitelist of URLs for crawlers to visit (in other words,  I want a list of URLs that crawlers visit, and I want the crawlers to ONLY visit those files).</p>
  
  <p>I've got a sitemap with all the URLs on the whitelist in it.</p>
  
  <p>How should I format the robots.txt?</p>
  
  <p>Does this work?</p>
  
  <pre><code>User-agent: *
  Disallow: /
  Sitemap: sitemap.txt
  </code></pre>
  
  <p>Or do I have to do it like this?</p>
  
  <pre><code>User-agent: *
  Disallow: /
  Allow: whitelist/*
  Sitemap: sitemap.txt
  </code></pre>