Bloggers who have SEO Knowledge, should know about robots.txt and its use. But it is quite unfamiliar to the new bloggers. But some of them might be interested in Blogger's search preferences and they must ask about what is Custom robots.txt? Well this is not so easy to explain. To the new bloggers, the explanation might be confusing. In order to explain the term robots.txt , I've to tell about search bots or search crawlers. But I will try to explain the term clearly. Suppose, you've opened a blog in blogger. continuously you're publishing new posts. If everything is okay, your post will appear in Google Search within few hours. If your post is matched with someone's search topic, your site will appear in the search results. How it's possible? Blogger's Default robots.txt: Search engines scan your site for new posts continuously. This is known as crawling. Search bots crawl your site regularly and index your posts in their directory...
Your ultimate choice for Computer, Internet, Blogging, & PC Troubleshooting Tips!