Fair use allows for this kind of thing, but is fuzzy: <a href="https:&#x2F;&#x2F;en.m.wikipedia.org&#x2F;wiki&#x2F;Fair_use" rel="nofollow">https:&#x2F;&#x2F;en.m.wikipedia.org&#x2F;wiki&#x2F;Fair_use</a><p>The web is a little different. One factor is that robots.txt is effectively an opt-out. But no, in general indexing a site doesn&#x27;t require permission.