Crawler

Also known as a bot is the software used by search engines to discover, crawl, and index the web page to report back to be put into an algorithm to determine its ranking.