Sometimes I check my old ‘link exchange’ e-mail account I used to build link popularity for my very first website. There are lots of people contacting me daily with exchange proposals. Well, not actually people – they are mostly bots.

Probably one of the reasons I still maintain that e-mail is that those requests are a source of a persistent amusement for me. One example: a request in pink letters with images of dancing puppies and bouncing hearts written by a ‘blond chick’ (picture attached) asking me to link to her pharmacy site! Or maybe I just enjoy reading the admiring comments on the outlook and content of my site that precede every exchange proposal?

Link exchange scam is an interesting theme for a study per se and still awaits its researchers. But in the meanwhile the SEO community is being successful in summarizing the guidelines for the most perfect link exchange scam.

Filing an Exchange Request

• Send an automated e-mail request or use a bot to submit it via an online contact form. Combining the both methods is preferred whenever possible.
• Use a free e-mail account such as Gmail, or better yet, some foreign free e-mail service to send your message. Sending a duplicate request from your company account is also beneficial.
• Do send follow up e-mails. Sooner or later your victim will give up and read one of them.
• Send minimum 100-300 automated requests every day. Push your mail server’s spam detection to the limits.
• Make sure that the website you are trying to contact is absolutely unrelated to your field.
• Send your request to every e-mail address you can find on the target site. Let the sales or customer support guys forward them to the webmaster.

Writing Your Request

• Address properly. No names required. Best thing is to use the website’s title or at least the URL: “Dear Blue Cheap Online Widgets”, or “Hello www.bluewidgets.com”
• Kiss ass. Tell your victims how much you adore their websites. Do use superlatives.
• Inform. Let your recipients know how important PageRank and incoming links are. Go in depth with the mysteries and magnificence of the PageRank and how the high PageRank will ensure them the first positions in Google.
• Scare. Notify them that their link popularity is low, and their positions in search engines are threatened.
• Share a secret. Tell them that the three-way linking is more effective, since search engines detect and ignore two-way links.
• Threaten. Notify them that their link will be removed from your high quality directory if they do not provide a link back in the specified number of days.
• Show your scale. Make your message easily detectable as a bulk sending by setting a different font size and color for the recipient’s address and site name.
• Be unofficial. Use the Internet argot in your e-mail. Like ‘u r’ instead of ‘you are’. This is the Internet – formalism is unacceptable.
• Threaten them again. With hundreds of reminder e-mails.
• Use a girl’s name. Most webmasters are male and should not resist a lady asking for a favor.

Prepare a Sound Links Page

• Your link page must have at least 100 outgoing links, preferably uncategorized. Make sure that minimum 50% point to pharmacy and gambling websites.
• Your proposed links page has to be deeply buried in a keyword-rich URL like: http://www.yoursite.com/widgets/cheap-widgets/amazingly-cheap-widgets/widgets-links/
• Make sure the links page URL contains at least one poison keyword like ‘links’, ‘partners’, ‘directory’, or ‘exchanges’.
• Alternatively provide a dynamic URL with a minimum of 100 characters of meaningless parameter values.
• Choose links pages that are in Google’s supplementary index.
• The PageRank for your page has to be between 0 and 3 with 0 being the best.
• Make your page look more credible by putting AdSense ads on it. “Well, if Google approves this page, then it is worth having a link from it”.
• Disguise your low PR links pages by opening them in a high PR frame.
• Orphan pages are the best.

Use SEO Tricks

• Link to your partners using one of the following options:
  • ‘nofollow’ attribute
  • javascript links
  • 302 ‘Found’ redirects
• Edit robots.txt to restrict spiders from indexing your links pages.
• Double protect your links pages from indexing by adding meta ‘noindex,nofollow’ tags.

The above guidelines are compiled from my own experience and the hilarious thread ‘SEO Link Exchange’ from the WebMasterWorld forum.

The list can be continued. Any suggestions?

Link building has always been a hot topic. In the beginning of the web hyperlinks were virtually the only way to get visitors to a site, because search engines were in their infancy. When search engines grew to be the major source of the web traffic, links didn’t lose their weight, as search algorithms started to rank sites according to the quantity and quality of their incoming links. And today links become increasingly important with the growing significance of the new Web 2.0 social networks.

Link Popularuty Building Strategies

Thus, links rule the Internet. Once a routine task of a webmaster, link building has emerged itself into a full scale industry with millions of dollars in turnover. Ranking algorithms perceive links as a proxy for a human judgment, or a user’s positive endorsement of a page. The idea is as follows: a user discovers a page, likes its content, links to the page, and the page gets higher ranking. This is the so-called ‘natural way’ of acquiring links. The natural way of acquiring link works too slow and can be pretty unfair. New pages on big and established websites are far more likely to be discovered by web users, and these pages will get the biggest part of the new links (like 90%); while new pages on fresh sites will get trinkets. This is a serious defect of the link ranking system which is discussed more in details in my article Popularity Ranking Faults.

Since the natural way of getting links for a new website can take forever, some additional boost is required. There are many strategies of link building able to ensure you some initial ranking and exposure, which are necessary to make the ‘natural way’ work. Some of these strategies can be very tricky and do more harm than use. So it is critically important to keep in mind the following tips of link building.

Link Building Tips

Be a user when building links. The point is to make your link exchanges look like they are acquired the natural way. Make sure that your links appear in places where search engine expect them to be. This should be pages relevant to your content. Link must be in the page copy or in a sidebar possibly among the other links pointing to pages also relevant to your topic. The anchor text must look naturally – so no keyword stuffing.

Analyze your own motives of linking to the sites you like. What motivates you to cite a web resource? Is it a collection of online tools or handy tutorials? Or may be it is a provoking title? Apply this ‘reverse engineering’ to your pages, and use unique interesting content to attract links.

Avoid things that can damage your reputation in the eyes of search engines. No link farms, suspicious looking websites or poor quality link exchanges. Forget the reciprocal links – they no longer have any significant weight. Do not participate in three-way or similar linking schemes – these attempts to disguise reciprocal linking are easy to detect. NASA managed to get a man on the Moon with computers less powerful than a GameBoy, so why do you think Google can’t discover link triangles with all the computing resources at its disposal?

Buying links. This practice is pretty much discouraged by Google, because it undermines the idea of the proxy for human judgment. This means you have to be especially savvy when buying links. Avoid link trading sites or any site publicly announcing that is sells links. Don’t mix buying links with paid advertising. You pay for an advertisement on a high traffic page expecting visitors referred by your ad. Buying links has a different purpose – increasing your link popularity.

Do not be obsessed with backlinks. There is an intense focus on link building but not enough focus on content creation. Links must reflect the quality of content. If you think your site has not enough incoming links, you should concentrate on how to improve the quality of content and make it more appealing, not on more link exchanges.

Link penalties. Many people are afraid to get penalized for linking or being linked by fishy websites. If there is a need to put a link to a site which you do not want to be related with, use rel ‘nofollow’. Google confirms that this attribute is critical in link analysis, so you should be fine. Links from dubious sources to your site are out of your control and all the major search engines assure that they don’t punish people for that. However too many links from such sites (like tens of thousands) can bring an unwanted attention of search engine quality teams. They can ban your site if they find you responsible for boosting your rankings, but you can always submit reinclusion.

To sum up:

Make your linking strategy look natural. Avoid the known patterns of artificial link building and do not obsess with links at the expense of content creation.

The idea behind AT(k) Algorithm is using only k highest authority weights instead of calculating average weight from every authority pointed by a hub. The parameter k is called authority threshold. A variant of an AT algorithm is MAX algorithm, where k=1, i.e. a hub is as good as the best authority it links to.

In general AT(k) algorithm uses the same formula as HITS. The difference is that when calculating the weight of a hub we consider top k authorities only, i.e. Fk(i) is a subset of outgoing links F(i). If the number of outgoing links |F(i)| is less or equal k than the AT(k) algorithm works exactly the same as HITS.

To overcome the shortcoming of the HITS algorithm of a hub getting a high weight when it points to numerous low-quality authorities, the following refinement was suggested. While using the same formula to calculate authority weights, the hub score h is now averaged by a number of outgoing links |F(i)|:

So in order to achieve a high weight a hub should link good authorities. Unfortunately this approach has its own flaw. Consider two hubs pointing to an equal number of equally good authorities. The two hubs are identical until one puts one more link to a low quality authority. The average sum of the authorities it points to sinks, and it gets penalized in weight. This is quite illogical but can be fixed by using so-called Authority Threshold Algorithm.

This algorithm was first described by Jon Kleinberg in his work “Authoritative Sources in a Hyperlinked Environment” (1998). The idea behind the HITS (Hyperlink Induced Topic Distillation) algorithm is that the authorities and hubs mutually reinforce each other. Authority weight of a page is calculated as a sum of hub weights pointing to it, and weight of a hub – as a sum of weights of authorities pointed to by it. In other words a hub is as good as the authorities linked by it, and vice versa.

The notation of the algorithm is as follows. Let S be a set of pages for which hub and authority weights are being calculated, n – the number of pages in the set. Then H is a subset of S containing pages acting as hubs, and A is a subset of S containing authorities. Since each page can be an authority and a hub, A and H overlap. For every page i in its hub role F(i) is the number of outgoing links. For every page i in its authority role B(i) is the number of incoming links. The n-dimensional vector of authority weights is denoted as a, and vector of hub weight – as h. Then hub and authority weights are calculated by the following formula:

The process is iterative. First all the weights receive value of 1. Then hubs and authority weights are calculated and the vectors are normalized. This stage is repeated until vectors a and h converge.

The algorithm however has a number of flaws. For example the nature of mutual reinforcement creates the following situation. Consider a hub that points to many authorities (hub B on the picture below), and a number of hubs pointing to a single authority (authority A on the picture). If the number of authorities pointed to by B is larger then the number of hubs pointing to A, then the HITS algorithm will allocate all the weight to the authorities in the right part of the picture and the authority A will get a weight near zero.

The reason is that hub B will initially get a very high score and propagate it to the authorities it links to. In the same time hubs on the left side will get very low score, and consequently A will get low weight too, although obviously it deserves more.

Cited Resources

• Kleinberg, J. May 1997, ‘Authoritative sources in a hyperlinked environment’. Technical Report RJ 10076, IBM,. Available at http://citeseer.ist.psu.edu/article/kleinberg98authoritative.html

HITS algorithm calculates hubs and authorities in query-time but relies on a relatively small subset of the Web – the immediate neighborhood of a page, since otherwise computation time would be unacceptably long. Hilltop algorithm analyses a query and calculates score values by finding pages that seem to be experts in the query-specific topic. This algorithm restricts itself to popular queries, since it can’t produce score values when no experts for an uncommon search term are found.

Topic-Sensitive PageRank extends the original PageRank idea by adding a query-time topic-sensitive adjustment. Instead of a single vector of PageRank values, multiple topic-specific PageRank vectors are calculated. Creation of a PageRank vectors for every possible topic would require extensive resources, so in practice the algorithm uses only 16 topic-specific ranking vectors representing the top categories of the ODP project. Other sources of topics can be used for this purpose as well, but since ODP project is created and edited by a large number of independent volunteers, it is the least likely to be influenced by any one party. For each page in the Web a set of importance scores with respect to various topics is precomputed and stored offline. In query time the topic-specific score is combined with other scores (e.g. content analysis) to form the final ranking for a page.

Let c_j be one of the 16 top-level ODP categories. For each topic c_j it is necessary to calculate a biased PageRank vector. Let M be a modified adjacency matrix. Each element m_ji has value 1/N_j, if there is a link from j to i, and where N_j is the number of outgoing links on page j. Otherwise element value is 0. Then the original PageRank formula in matrix notation looks as following:

Parameter α here is the dumping factor that equals 1-d in the original PageRank formula. The resulting vector of PageRank values is denoted as PR(α, p).

With minor modifications the same formula is used to calculate the topic-sensitive ranking vectors. Let T_j be the set of pages under a topic c_j. Then instead of the uniform distribution damping factor p, a non-uniform vector v_j is used, where:

Resulting PageRank vector is denoted as PR(a, v_ij). Additionally using all the documents under each topic c_j a term vector D_j is constructed. Term vector elements D_jt are the numbers of occurrences of every term under topic c_j. In order to detect a topic, to which a search query term relates to, two scenarios are considered. In the first scenario a user highlights a keyword in a page and initiates a search. In this case the search topic is defined by the page content. For example if word ‘architecture’ is highlighted in a page about famous buildings, the pages on CPU architecture should not appear among search results. So if a term q is highlighted in some page u, its context q’ would be the words in u. In the second scenario a user enters a keyword into a search form in the conventional way. In this case the context of the query q is the search term itself: q’ = q. When a history of search terms is kept, it is also possible to use it as the context q’. In query time the proximity of query context q’ to one of the topics c_j is calculated:

Proximity value P(q’|c_j) is calculated using the term vectors D_j. Then a query-sensitive score values are computed for every page d from the index that contains the original search term q. For each document d we sum up the products of topic proximity values P(c_j|q’) and the page rank r_d. The rank r_d is an element of the page rank vector PR(a, v_ji) of a topic to which the document d belongs to:

The final search results are sorted by the values of s_d. Since the PageRank calculations are performed in advance, the algorithm is able to quickly perform topic adjustments in the query time.

Cited resources

• Haveliwala, T.H. ‘Topic-sensitive PageRank’. In Proceedings of the Eleventh International World Wide Web Conference, Honolulu, Hawaii, May 2002. Available at http://citeseer.ist.psu.edu/haveliwala02topicsensitive.html
• Haveliwala, T.H. ‘Topic-sensitive pagerank: A context-sensitive ranking algorithm for web search’. IEEE Trans. Knowl. Data Eng., 15(4):784–796, 2003. Available at http://citeseer.ist.psu.edu/article/haveliwala03topicsensitive.html

PageRank extends the idea behind the InDegree algorithm by assigning different weights to the links. Links from high quality pages should make a stronger impact on the rank of a page. Therefore it is not only important how many incoming links a page has, but also how important the pointing pages are.

To determine the authority of internet pages PageRank simulates the behavior of a random web-surfer, by starting its walk from some random page and then following the outgoing links. The starting point is usually drawn from a uniform distribution, but other distributions can be used as well. The process of the random walk is as follows: at the starting point an outgoing link is randomly chosen and the surfer follows it with a probability 1-d. With probability d the surfer jumps to another random page without following any link (“gets bored clicking links”). The parameter d is called “the dumping factor”, its value can vary but generally it is assumed that equals 0.85.

The PageRank of a page is calculated by a formula given below:

The algorithm considers a set of interlinked pages M(p_i) with the number of elements N. PR(p_j) is the PageRank of a page p_j in the set, and L(p_j) is the number of outgoing links on that page. Each page that passes to p_i a fraction of its own PageRank depending on how many other pages it links to.

The incoming links in the set M(p_i) can be presented in a form of adjacency matrix:

Where l(p₁,p₂) equals 1 if there is a link between pages p₁ and p₂ and zero otherwise. PageRanks of all the pages in the set M(p_i) form a vector R, which is the dominant eigenvector to the adjacency matrix.

Then the PageRank formula can be written as:

PageRank values are precomputed and do not depend on search queries. When it is necessary to rank the results for some search term PageRank is used in conjunction with query specific scores. Having the PageRank scores precomputed allows faster results sorting.

References

• Borodin, A, Roberts, G.O., Rosenthal, J.S. and Tsaparas, P. ‘Finding authorities and hubs from link structures on the World Wide Web’. In Proceedings of the 10 th International World Wide Web Conference, Hong Kong, May 2001. Available at http://citeseer.ist.psu.edu/borodin01finding.html
• Wikipedia: PageRank

This simplest algorithm uses page popularity as a ranking factor. Page popularity is measured as a number of incoming links – similar to document citation in the academic world. In the early days of the Web this algorithm was widely used by search engines. The InDegree algorithm is not very effective – we would need to consider links not just from any page but from those which are relevant to the query. Otherwise the algorithm can be easily manipulated by obtaining thousands of links from anywhere in the Web, thus artificially inflating the link popularity (link farms). The popularity a_i of a page i is calculated by a simple formula:

a_i = |B(i)|,

where B(i) is a set of pages pointing to page i, and |B(i)| is a number of elements in the set.

Reference

• Borodin, A, Roberts, G.O., Rosenthal, J.S. and Tsaparas, P. ‘Finding authorities and hubs from link structures on the World Wide Web’. In Proceedings of the 10 th International World Wide Web Conference, Hong Kong, May 2001. Available at http://citeseer.ist.psu.edu/borodin01finding.html

When a user queries a search engine with a keyword, he expects more than just relevant results. For example if someone searches for ‘Bali vacations’ he would be disappointed to get a page of a personal blog with a story about John Doe’s awesome Bali vacation last summer. Obviously, what the user was looking for was a travel agent like Expedia. Thus it is critical that users get not just relevant but also authoritative results. And the more pages appear daily in the Internet the bigger is a shift from the relevance to the authoritativeness in search algorithms.

Nowadays the relevance of a page is defined differently. It is not just about the keyword saturation, or copy structure. Currently the context where the page exists defines its relevance. The context is a set of pages linking to or linked by the given page. If these pages are about Bali vacations, then it is naturally to expect a page linked by them to be about Bali vacations as well. The page content would be used to adjust the algorithm’s results in cases when links point to an irrelevant page, for example a widely used free web statistics system.

Link Analysis Ranking Algorithms

So how come the page content analysis is no longer enough to get relevant search results? There is a problem of abundance: the number of pages considered to be relevant basing on the page content analysis is too big for a human to digest. And this is where searching for authoritative pages helps to narrow down the results. But authority is even a vaguer notion than relevance. Authority has to express the importance and the weight of a web document. The nature of the Web as an interlinked hypertext environment suggests that links can be used to measure the degree of “public recognition” of web pages.

This idea has been existing since the creation of the Internet, and Jon Kleinberg was one of the first to create a workable approach, which he described in his seminal work “Authoritative Sources in a Hyperlinked Environment” (1998). He suggested that web pages can be either “hubs” or “authorities”. Authority is a page that has many incoming links or high in-degree. Authorities returned as relevant to some query should demonstrate an overlap in pages pointing to them. Those pages containing links to the relevant resources are called hubs. Hubs determine the relevance of authorities on a given topic and allow discarding other non-relevant pages with high in-degree.

Link analysis ranking algorithms use hyperlink graphs similar to the shown above. The nodes of the hyperlink graph are web pages, and links are the directed edges. The graph is simple: if there is more than one link between two nodes, only one is considered and neither are the links from a page to itself. A different weight can be assigned to edges (links) according to the web page analysis or other factors, which search engines consider important, e.g. link or domain age.

In my further posts I will describe the most widely used link analysis algorithms.

References

• Kleinberg, J. May 1997, ‘Authoritative sources in a hyperlinked environment’. Technical Report RJ 10076, IBM,. Available at http://citeseer.ist.psu.edu/article/kleinberg98authoritative.html
• Borodin, A, Roberts, G.O., Rosenthal, J.S. and Tsaparas, P. ‘Finding authorities and hubs from link structures on the World Wide Web’. In Proceedings of the 10 th International World Wide Web Conference, Hong Kong, May 2001. Available at http://citeseer.ist.psu.edu/borodin01finding.html

The earliest web-like development by Tim Berners-Lee – the Enquire program – allowed the usage of various types of relationships between documents, such as similar-to, part-of, or made-by. However in his later work on the World Wide Web and HTML Tim Berners-Lee has abandoned multiple-types of relationships for simplicity. Flat links with no relationship information have been used ever since.

Link Abuse

The absence of this feature has been widely exploited by people trying to manipulate search engines results since the introduction of the link-based popularity ranking. The most notable examples are link farms, which used to flourish just a couple years ago. Search engines had realized the problem of abuse and started penalizing link farms and sites that link to them. But still with the anchor text and the host page’s own authority being the only parameters used to evaluate the link impact, there exist many other ways to manipulate rankings. By using anchor text large web-users communities can perform the infamous Google-bombing: see ‘miserable failure’ search results in Google. And this can be employed against other search engines as well, e.g. ‘друг народа’ (‘people’s friend’) query in the Russian major search engine Yandex.ru returns V.Putin’s bio on the top of the results

Adding Semantics Info into Link Markup

The problem lies in the very nature of HTML. Designed as a document structure markup language it is ineffective for describing the semantics of a document. Some extensions had to be introduced into the link markup to make it able to reflect the nature of the relationship to the linked document. One example is the rel attribute. This attribute is used in the VoteLinks microformat proposed by the Technorati.com to allow users to express their opinion about linked blogs. The rev attribute in the VoteLinks format can have the following values: ‘vote-for’, ‘vote-abstain’ and ‘vote-against’, thus reflecting positive, neutral and negative opinions. This makes it possible to augment or reduce the authority of the linked document respectively.

The rel attributes values such as ‘friend’, ‘colleague’, ‘sibling’, ‘neighbor’ or ‘spouse’ allow a different approach, reflecting the nature of the relationships between two documents, rather than the opinion. This approach is also used by Technorati.com and might have certain value within a community of bloggers.

None of these approaches is currently used by any major search engine. The rel attribute value ‘nofollow’ is the only one to be taken into account by Google, MSN and Yahoo. This value is intended to show that the linking person is not certain about his approval of the linked document. The search engine crawlers are supposed not to follow such links, but many webmasters report that actually they still do. However it is possible that such links do not propagate authority.

Possible Impact of Trust Links

What if search engines were able to determine the trust or distrust nature of a link? Would the existing ranking be significantly different? Currently inbound links show the measure of attention to a certain page. Therefore even those pages which are linked to express distrust or even disgust might be ranked higher than positively endorsed pages for the same query.There is another approach that would have to be reconsidered. Now if page A links to page B, it propagates a fraction of its authority value to B. If there is a reciprocal link, some authority can be returned to A, so that both pages benefit from the mutual trust. But consider now that a highly ranked page A distrusts page B. Page B also adds a distrust link to A in revenge. Should we decrease the ranking of A because a distrusted page B also negatively links to it? Or should we stop considering any opinion from B?

To determine the effect of the search engines’ ability to distinguish between positive and negative opinions in web links Paolo Massa and Conor Hayes have conducted an experiment using the linkage data from the Epinions.com website. Epinions.com is a web community where users can write their opinions on movies, books, goods and services. It is also possible to rate to use other users’ reviews, thus expressing trust or distrust. In the experiment’s model, users are considered as web pages, while their ratings are seen as incoming links. The 20 top ranked users were sorted by their total number of ratings (positive and negative) and by the number of the positive rankings only. The total number of rankings shows the attention to the user, which is similar to the link popularity of a web page. Using statistical methods P. Massa and C. Hayes have determined the alignment factor between the attention ranking and the trust ranking. The value is 0.8 and it should be noted that since people tend to assign positive ranking more frequently than negative (often a disappointed user would simply not rate at all) – the actual alignment value can be even less.

However trust linking would be ineffective in the web. The introduction of trust/distrust linking would create wide opportunities for abuse, and not just from gray spammers. Any highly-ranked website would be able to destroy authority of any less ranked competitor with a power of a single link! While in relatively small communities of bloggers (Technorati.com) or reviewers (Epinions.com) such wars can be prevented by human moderation, the existing technology still doesn’t allow search engines to perform a similar task in the Web.

References:

• Massa, P. and Hayes, C. 2005, ‘Page-reRank: using trusted links to re-rank authority’. Istituto Trentino di Cultura. Available at http://sra.itc.it/people/massa/publications/wi05_page_rerank_massa_hayes.pdf. Retrieved on 29.06.06
• Ntoulas, A., Cho, J., Cho, H.K., et al. 2004, ‘A Study On The Evolution Of The Web’. University of California. Available at http://oak.cs.ucla.edu/~cho/papers/ntoulas-evolution.pdf . Retrieved on 15.06.06