An index is a search tool that uses automated programs that are
capable of following the hyperlinks found on a web page to other
web pages and in that way move through a large number of web
sites with no human intervention. While doing their wandering,
these "web crawlers" can also create a database of keywords from
each web page they encounter. A search engine can then be
designed to take advantage of the specific ways that the
database is indexed making searching the index for all
occurrences of your keywords extremely fast. Because crawlers
can process enormous numbers of web pages each day, older links
are constantly being dropped or updated as the new URLs are
added making the maintenance of the database automated as well.
One of the most advanced index search engines is Alta Vista,
created and run by Digital Equipment Corporation. Within two
months of its public debut on December 15th, 1995, Alta Vista
had grown to include 21 million web pages, 10 billion words, and
was handling over 4 million request p