Q: How do I stop the search engines from indexing a sub domain?

A: So you have your main domain (www.yourdomain.com) and now you have created a sub domain (sales.yourdomain.com)

If you do not want the search engines to index your sub domain - here is what you have to keep in mind.

1. Search engines see sub domains as a different site completely.

So www.yourdomain.com and sales.yourdomain.com are two completely different sites in a search engines mind - and therefore you would require a separate robots.txt file.

NOTE: A robots.txt file is basically a text file that tells the search engines
a) what files/folders to index
b) what files/folders not to index
c) where your sitemap is located

Why would you want to block search engines from certain pages of your site?

- if the page contains little or no content (eg. such as a login page)
- if the page contains just images (images that you don't want indexed)
- privacy pages
- when your site is still in development (eg. under construction page)

All these pointed mentioned above can impact your search engine rankings if you do not set up a robots.txt file to block these type of pages.

If Google (spider) comes to your site and all it sees is an under construction page - your site is already hit negatively.

A robots.txt file saves the search engines time - and lets them know what pages not to waste their time on - so they can spend more time on going to the higher quality pages of your website more frequently.

OK I mentioned before that Google will see sub domains as different sites.

However, it really depends on how the sub domain was set up.

If sales.yourdomain.com goes to the main web directory (as www.yourdomain.com) – you would then need a script to redirect it – and this method only requires one robots.txt file

If the sub domain was created from the DNS – then you would need two separate robots txt file.

If you are not sure - you can always log into your Google Webmaster Tools account - and under the Tools Section – there is a section called ‘Analyze Robots.txt’ – which will verify if you have blocked the sub domain accordingly.