Spider

A spider, sometimes called a spiderbot or web crawlerA program designed to browse and read the web by following hyperlinks. Search engines use the inform... and often shortened to crawlerA program designed to browse and read the web by following hyperlinks. Search engines use the inform..., is an Internet bot that systematically browses the World Wide Web, typically operated by search engines for the purpose of Web indexingA method of collecting, arranging and categorizing the information that crawlers, bots, or spiders f....