The iceberg effect

I was looking at the Sitemeter traffic statistics generated by TruthLaidBear. While the statistics is not comprehensive e.g. like Technorati, the volume of traffic and number of random users is sufficient to regard it as a good representative sample. Based on a snapshot taken about 10 days ago, I found the following. Of the top 5000 Sitemeter sites, the top 60 blogs represent almost 40 percent of the blog readership. The top 600 blogs represent 50% and the rest of the 5000 represent the bottom 50%. The top slot attracts over 1 million views a day and by the time you are around #5000, you have around 70 views per day. Given there are over 37 million blogs out there, it is a very long tail. The A-list in fact represents less than 2% of the blog population. The B-list maybe the next 10-15% and everyone slaves away in their tiny blogs which to all intent and purposes could be private blogs given the amount of traffic they attract. Forget the 80/20 rule it's more like the 99/1 rule.

The Adsense model works because of its simplicity and widespread appeal. It does not pay high traffic sites better rates and at the same time are happy to have the 99% of the long tail as members. In short, they cover the entire tail without paying a premium for the top performers. But the sad truth is that most bloggers do not realize that unless their traffic is in the 1000 or more views per day (only 5% or less of the blogger population), you won't be making any money from Adsense. For focused sites, a click through ratio of 1:100 would be phenomenal. For a generic blog with random topics you could be looking at 1 in 500 or even 1000. So literally you could be making cents per month, a very disappointing outcome for all who believe it is easy making blog money or literally blood money given the time we spent crafting our blogs.

Given the sheer volume of posts and bloggers, even great sites like Technorati and some of the blogger network sites do not help solve this problem because while they aggregate content and provide search, they also survive via popularity rankings. In fact, I suspect the long term effect is that they are reinforcing the long tail effect rather than flattening the steep curve. It is also telling that the system of ranking is about to fail when the top Technorati blog cannot be read by the majority of the North American population where most of the blogger community resides. The blog is in Chinese and has succeeded to take #1 spot because of the sheer number of Chinese links it has attracted.

So the problem of building a single point to aggregate and serve, while attractive, is not going to solve the problem of distributing great content and exposing some terrific blogs that today I only find via some random click event. Unfortunately, the majority of Web 2.0 and other search engine wannabees are precisely attacking the problem this way. Building bigger and bigger warehouses is a typical old fashion industrial approach. What we need is something like a just-in-time dissemination of blogs that do not rely on some central search and storage mechanism. While RSS is a terrific mechanism to keep you updated with the blogs you love, it doesn't address the millions of blogs you could fall in love with but didn't know they existed. Pretty much like sitting on top of the iceberg and thinking that is your castle oblivious of the 90% that is beneath the water.