Definition of Spider

Spiders are software processes, technically known as robots, employed by search engines to index web pages of registered sites on a regular basis. They follow links between pages and record the reference URL of a page for future analysis.

Missing a definition?

Please let us know