Google Digging Deep into the Internet with Form Crawling
April 14th, 2008 by Priyam
Google is digging deeper into internet search by crawling forms. Only a small number of usefull websites are being crawled and that to following the rules like robots.txt, nofollow etc.
Here is an excerpt:
In the past few months we have been exploring some HTML forms to try to discover new web pages and URLs that we […]
Tags:
