Why Are the ReviewAZON Demo Sites not Indexed In Google??

Posted:  June 2nd, 2009 by:  Brad comments:  3

I have had lots of questions over the past couple of days about my demo sites not being indexed in Google and the other search engines. There is a very simple answer and it it is because I don’t want them to be indexed. I use these sites for PPC (Pay Per Click) traffic and I will eventually set these sites up for normal SEO traffic at some point, but right now I don’t have the bandwidth to do so and I don’t want these sites to be indexed until I can set them up the way that I want them.

There is a setting in WordPress in settings -> privacy section that gives you the option to have your site indexed or not. The option that I chose was I would like to block search engines, but allow normal visitors . This creates a robots.txt file with Disallow / turned on. No search engine will index the site with this present.

When you are setting up a site from scratch, this is an excellent way for you to get it the way you want before you open it up for indexing. It is a pain to have to go back and put in redirects to indexed pages after the fact, especially if you change the permalinks to your posts.

I hope this clarifies things for everyone and please shoot me an email if you have any questions!

-Brad

    • 3 Comments

    Posted By: Dirk On: July 16, 2009 At: 7:49 am

    Hey Brad,

    I agree, because I do the same on my regular blogs I was setting up prior to finding your WP tool. I would disable my ping list as well. Plus, when you first initially set up your blog, with Title and Subheading it gives you a check box at the bottom to check if you want Google to index the site. Just leave it unchecked and it will default to what Brad was mentioning earlier. This way when you’re ready you just change the Privacy setting, Blog visibility to “I would like my blog to be visible to everyone, including search engines (like Google, Sphere, Technorati) and archivers”. Because as Brad mentioned earlier, it is a lot harder to revert, than getting it done right the first time.

    Happy ReviewAzoning Everyone,
    Dirk

    Posted By: Binh On: October 28, 2009 At: 11:45 am

    Is it possible to still block some of the pages of the website but not the whole thing? Let’s say to not index the squeeze pages, etc.?

    Thanks
    Binh

    Posted By: Brad On: November 18, 2009 At: 10:13 am

    You can block certain pages using robots.txt, but not all spiders and search engines will adhere to the rules.

    -Brad

    Leave a reply