Skip to main content

AdSense Friendly Custom Robots.txt for Blogger


Custom Robots.txt for Blogger


Bloggers who have SEO Knowledge, should know about robots.txt and its use. But it is quite unfamiliar to the new bloggers. But some of them might be interested in Blogger's search preferences and they must ask about what is Custom robots.txt? Well this is not so easy to explain. To the new bloggers, the explanation might be confusing. In order to explain the term robots.txt, I've to tell about search bots or search crawlers. But I will try to explain the term clearly. 


Suppose, you've opened a blog in blogger. continuously you're publishing new posts. If everything is okay, your post will appear in Google Search within few hours. If your post is matched with someone's search topic, your site will appear in the search results. How it's possible? 


Blogger's Default robots.txt:

Search engines scan your site for new posts continuously. This is known as crawling. Search bots crawl your site regularly and index your posts in their directory. By default, blogger comes with a robots.txt for your site that helps search engines crawl your site.

But as we know, the default or general settings never guarantee the accurate result. So we need to customize the settings as our requirement. Same rule is applicable for blogger's robots.txt.


Custom Robots.txt for Blogger Blog:

Today you will learn how to customize the robots.txt of blogger. There is a rumor that the default robots.txt can't index the old posts properly. If you have a proper custom robots.txt, your posts will be indexed properly. In my last post, I discussed about creating XML sitemap for blogger blog. There I gave example of XML sitemap. You can use that sitemap as your custom robots.txt content.



Custom Robots.txt Settings for Blogger


Look at the above image. This is how you can customize your robots.txt element. Or use the instruction below:

Sign in to Blogger Dashboard > Settings > Search Preferences > Crawler and Indexing > Custom robots.txt > Enable? > Yes > Now add the code below > Save Changes.


# Blogger Sitemap generated on 2013.11.06
User-agent: *
Disallow: /search
Allow: /
Sitemap: http://www.markspcsolution.com/atom.xml?redirect=false&start-index=1&max-results=500


Here you just need to change the site URL. Replace the word munnamark with yours. And for sure, this will help search engines for better crawling of your site.



AdSense Friendly Custom Robots.txt for Blogger:

This custom robots.txt might be less useful for AdSense enabled blog. If you use this type of custom robots.txt in your blog, AdSense will not be able to show relevant ads on your site. Consequently your ads impression you will affected seriously. And ultimately, your revenue will fall.

But a little change in that could save you all the day. If your site is AdSense enabled, then use the following code:


User-agent: Mediapartners-Google
Disallow:
# Blogger Sitemap generated on 2013.11.04
User-agent: *
Disallow: /search
Allow: /
Sitemap: http://
www.markspcsolution.com/atom.xml?redirect=false&start-index=1&max-results=500


Look, only the blue part has been added with the previous code. If you're working with AdSense then consider using this code as your robots.txt element. Actually the blue part is used by Google AdSense only. 

If you use this code (User-agent: Mediapartners-Google Disallow:), AdSense will be able to crawl your posts properly. Then it will show relevant ads on your site. And the other part of the code will be used by the search engines for indexing your contents. 



Stay with Marks PC Solution to get more interesting IT topics!



Comments

  1. Rahaman sir I am Bis followers of your blog. Sir I am suffering from 55 Duplicate meta description problem in my blog http://livetechnologybd.blogspot.com/. Sir I am using free downloaded template in my blog. Sir if you have any effective solution please help me. Sir my humble request please help me.

    ReplyDelete
    Replies
    1. :)
      I will never visit your site if you don't change your template - I promise. Cause I use only 1 GB monthly pack.

      Delete
    2. sir i have change my blog template, please now tell me how to fix duplicate meta description.

      Delete
    3. Actually you didn't get me. You were supposed to make your site faster. Use a simple template. Or simply use one of the default blogger templates.

      By the way - where do you get that you have duplicate meta? Have you changed your template manually? Maybe your current template has some problems. Use simple blogger template.

      Forget about styles, social gadgets. Visitors hate these. People hardly like or plus one your site. So remove them if possible. Speed is necessary.

      Delete
    4. Sir now I am using Simple Template of blogger. I get duplicate meta description in webmaster tool.

      Delete
  2. thank you very helpful information from me to increase adsense earnings on this blog. belajarkreatif [dot] net

    ReplyDelete
  3. I've used your given code in my blog but what about the :generated on 2013.11.04?? in code.Is it important to edit the date?

    ReplyDelete
    Replies
    1. Maybe not. But you can use recent date if you wish.
      You can use 2014.01.06 . . .

      Delete
  4. Just replace the URL in sitemap. Maybe date is not much inportant. But you can use the current date.

    And to reduce the fall in pageviews, you have to add the new domain in Google Webmaster. And only then you can change your address. I mean .blogspot to .com.

    ReplyDelete
  5. :)
    You know this is simple template. But why do you want this one? I've manually edited it. And everywhere you will get my site's link. You have to change them again.

    Better you can choose the simple template (your site is too heavy). Then I can help you to customize it. Even I've tips to customize the simple template as you want. You can chat me: munnamark at Google Talk or Gmail.

    ReplyDelete
  6. This comment has been removed by the author.

    ReplyDelete
    Replies
    1. How old your site is? It may take some times to be indexed in Google. And I've checked your site. There is no content.

      Try to write something. Trying to index in the search engines without writing is not a good idea at all !!!

      Delete
  7. JazakAllah Brother...
    you are doing eternal blessed work..... just about learning..... very useful.... and I also get some out of above..... Stay Blessed!

    https://scoutshr.blogspot.com

    ReplyDelete
  8. Hello sir, can you please tell me how can i get my blog posts in google search some of my stuffs are unique some are indexed by google and some are not my blog link is www.allinone1000.blogspot.com

    ReplyDelete
    Replies
    1. Keep Writing. Some words are highly competitive. In those keywords your site will perform poor in search resutls.

      But as soon as you rank higher, your posts will start appearing in Goolge.

      Delete
  9. Asalam alaekum,

    i got 86euro since 5month ago and everything stopped after then, i hardly see 2cent per day now.
    i serious need help! my blog is http://betonmarketsecret.blogspot.com/
    let me highlight my problem more, so that You could help to the last.
    1st : i need to know how get the Right Keyword to that blog.
    2nd : i need to change it to Dot.com
    3rd : i need to know how to get Pure traffic to my blog.

    Thanks
    Abdul Akeem

    ReplyDelete
    Replies
    1. Contact me via Google Talk. munnamark@gmail.com

      Delete

Post a Comment

Popular posts from this blog

GTA San Andreas Cheat Code (PDF)

After Vice City, San Andreas is the most popular game released by Grand Theft Auto (GTA). Usually GTA releases role playing game. San Andreas is such a game where you can act like a person. You can do everything. You have to take part in missions and complete them.  To make your missions easier, there are lots of cheat codes. If you can properly apply these codes, the tough parts of the game will be easier. Cheat codes are somewhat like keyboard shortcuts of computer programs.  I've shared 63 cheat codes below. There were more cheat codes in my collection, but I've omitted the less important cheats. Effects or result of the cheats are on the left side, and the codes are on the right side.  You can copy the following codes in your PC. And for your convenience, I've also uploaded a PDF file of these codes. You will find the download link at the bottom.  Effect Cheat Code Adrenaline Mode MUNASEF All Cars Explode ALLCARSGOB

Download Puzzle Flash Game - Save Them

Hey Flash Game Lovers! Today you're getting an amazing puzzle game - Save Them! It's a flash game and the download size is only 382 KB. In this game, you have to move 3 missionaries and 3 cannibals from one side to another through a boat. In any side, if the number of cannibals is more than the number of missionaries, then cannibals will eat the missionaries!  Download Save Them Download the game from the above link. It's a zipped file. Download size is only 382 KB! After downloading the game, you have to unzip it first. Then double click on the SaveThem to start the game.  How to Play? After starting the game, you will get the instruction. You have to make sure that in no side, the number of cannibals is not more than the number of missionaries. This is your only challenge. You have to take them from left side to the right side of the lake through a boat.  Press on a character to jump to the boat. Thus you can take any two of them a

Rules for Bengali Typing in PDF (Bijoy Layout)

Bijoy is the most popular layout for Bangla typing. The layout was developed by Mostafa Jabbar. He is the pioneer of Bangla typing in computer. More than 90% people use his Bijoy layout for Bangla typing. Today, I'm gonna share a PDF file with you that contains the Bijoy Keyboard layout with English fonts! If your keyboard doesn't have bengali fonts then you can print this PDF file to know where the bengali fonts are located in your keyboard. Additionally this file contains all the necessary rules and tips which are essential for Bangla Typing.  Now there are many other software available for Bangla Typing Layout. Such as Avro Easy, Unicode, Unibijoy etc. And there rules are also somewhat different from Bijoy. And some of their layouts follow the same rules like Bijoy. Hopefully this PDF file will support most of the rules when a layout is similar to Bijoy Layout.  Click to enlarge the image Download the PDF file from the button below. Follow