Robots plays important role in the field of SEO . We have two ways to control pages and folders one is using Robots META tag and other is through robots.txt A web page creator can specify which page should be indexed and which page should not be indexed by search engines by placing Robots META tag in the HTML section Here are some Robots tags that are common < content="NOINDEX" name="ROBOTS"> - Ignore content and follow links < content="NOFOLLOW, INDEX " name="ROBOTS"> - Include content and do not follow links < content="NOINDEX,NOFOLLOW" name="ROBOTS"> - Ignore content and do not follow links < content="INDEX,FOLLOW" name="ROBOTS"> - Include content and follow links < content="NOARCHIVE" name="ROBOTS"> - Cache link should not show Search results pages < content="NOODP" name="ROBOTS"> - The Open Directory Project (ODP) t...