December 27, 2008 Leave a comment
In order to allow google crawlers to access your website, you need to upload the robots.txt file on your site. The file is composed of the following two lines:
User agent: *
Save these two lines in any text file using notepad and name the file as robots.txt and make sure the file is uploaded with your website files. This file is your invitation for the google crawlers to access your pages.