web hosting reviews


Crawler

Definition by BestHost Ratings

Also referred to as a spider. It is an automated program used by search engines to collect data for web sites. A crawler will download your web page and will analyze its content and links. Then it will usually visit each link and so on. The data collected is used for building the search engine index which is used for providing search results to the users.




Previous: CPU Next: D Channel
Go back to Webhosting Glossary


About - Terms of Use - Privacy Policy -
Web Hosting Reviews