You may not want certain pages of your site crawled because they might not be useful to users if found in a search engine's search results. If you do want to prevent search engines from crawling your pages, Google Search Console has a friendly robots.txt generator to help you create this file. Note that if your site uses subdomains and you wish to have certain pages not crawled on a particular subdomain, you'll have to create a separate robots.txt file for that subdomain. For more information on robots.txt, we suggest this Webmaster Help Center guide on using robots.txt files13.
On the other hand, all of the results for the PageRank engine (aside from a single secondary listing) link to the homepage of major American universities. The results are much more logical and useful in nature. If you search for “university,” are you going to want the homepages for popular universities, or random subpages from a sprinkling of colleges all over the world?

Great post. I’m posting a link back to this article from our blog along with some comments. I do have a question. In your article, you post “The only place I deliberately add a nofollow is on the link to my feed, because it’s not super-helpful to have RSS/Atom feeds in web search results.” Yet when I look at this article, I noticed that the comment links are “external, nofollow”. Is there a reason for that?

Private corporations use Internet marketing techniques to reach new customers by providing easy-to-access information about their products. The most important element is a website that informs the audience about the company and its products, but many corporations also integrate interactive elements like social networking sites and email newsletters.


Understand that whatever you're going to do, you'll need traffic. If you don't have any money at the outset, your hands will be tied no matter what anyone tells you. The truth is that you need to drive traffic to your offers if you want them to convert. These are what we call landing pages or squeeze pages. This is where you're coming into contact with the customers, either for the first time or after they get to know you a little bit better.
While ordinary users were not that interested in pages' scores, SEOs of a different caliber felt that this was a great opportunity to make a difference for their customers. This obsession of SEOs with PageRank made everyone feel that this ranking signal is more or less the only important one. In spite of the fact that pages with a lower PR score can beat those with a higher score! What did we receive then, as a result?

Page Rank

×