Robots noindex meta tag

Published on: 22 June 2009 and updated: 25 January 2011

Matt Cutts did some great research on how the index meta tag is handled between search engines. I summarise his results in this article.

What am I talking about? The Robots meta tag looks like this:

meta name="robots" content="noindex, nofollow"

When the content is set to noindex, nofollow the search crawler should not index or follow any links on the page.

Matt's results found the following for a page that contained the above tag.

Google does not index the page in any way.
Ask does not index the page in any way.
MSN displays a url reference and Cached link, but no snippet. Clicking the cached link doesnâ€™t return anything.
Yahoo! returns a url reference and Cached link, but no snippet. Clicking on the cached link returns the cached page.

The above results of Matt's test imply that you may need to use more than just a meta tag to remove pages from Yahoo and MSN.

Reference