The Idea Dude


Monday, October 16, 2006

Webstats don't lie

Had an interesting question on webstatistics over the weekend. I was trying to track down the disparity between our statistics and some webpackages that claimed 5 times the traffic. Here's a note why the web statistics from your hoster may not be what you think it is. That is not to say the numbers are wrong but the interpretation of them should be done with care. The most important number for all bloggers is obviously the number of readers that visited your blog. We believe our numbers are an accurate reflection of this since we get called once every time we appear on a blog within a browser page. That would be our definition of a reader visit. Here's where we start to deviate from the webstats that your ISP is showing you.

  • Unique visits. This should be the same as our number right? Not always. Every page usually loads 4-5 (sometimes more) additional files including stylesheets, scripts, images etc. But unique means they should all collapse to one IP. Well, yes and no. If the web hoster had multiple servers and each had it's own web stat package and they present you a number by simply adding the stats from each webserver, unique is not unique anymore. If they performed simple round robin load balancing, your 5 requests from one page and one IP would be spread evenly across multiple servers each registering a unique hit. Unless they have a common web stat database which would be performance bottleneck, each webstat package doesn't know about the others. You final count for each visitor if you had 5 servers? 5 unique visits.
  • Total pages. This can be severely bloated because of web crawlers. It's pretty hard for web hosters to exclude them from their statistics because web crawlers may use more than one server and IPs change all the time. At the same time, web masters do want all the stats because they need to know the total number of pages that are being requested whether it is a human reader or web crawler. So if your blog pages have many links to comments, archives and other blog entries, the crawler will add a significant number of pages to your total. It is not uncommon for large blogs to have as much as 100 links to it's own domain. So each time a crawler comes to your site and follows the links, your web hoster is logging a couple hundred pages. The number of web crawlers vary but not limited to the big 3 or 4 search engines. A popular site may get 10-20 a day if not more.

The best way to get a true reflection of who your readers really are is actually using something external like Sitemeter or TheGoodBlogs because it doesn't get aggregated with the total number of support files and web crawlers that may visit your blog. Again it is important to stress that the numbers that web packages are designed to give you is to help you with performance and bandwidth so they will often include all the stuff that are not related to the number you want, the number of unique blog readers.


Anonymous Anonymous said...

Sadly, there's also a disparity between external logs. I've tried Performancing Metrics, IceTracker, SiteMeter and MyBlogLog and there are enormous differences (up to 50%). The only realistic or useful measures appear to be comparative ones.

I tend to get rid of the ones with bad results ;)

9:33 PM  
Anonymous Anonymous said...

What's interesting about the goodblog stats is to work out what gets conversions - not that I want to obsess on it but clearly if Vern can run some kind of analysis there you'd know what triggers people to seek a blog post. For example what's the range of conversions on different blogs? What does that tell us?

11:23 AM  
Blogger The Idea Dude said...

I think that is the advantage of being in a community because collective stats can tell you more than any one blog. It is part of our goal to dissect the results and share that with the TGB community. I have been doing that in an informal basis over the last two months when I find interesting things that certainly I would never expect.

The best part is that I do have a lot of dialog with many members so I get a really good idea what's happening out there. Many of the changes over the last while has been the input of our members and advisors and I can say without reservation that it has made us slicker, more useful and more usable.

I look forward to sharing more as we learn more.

11:39 AM  

Post a Comment

<< Home