Webstats don't lie

Had an interesting question on webstatistics over the weekend. I was trying to track down the disparity between our statistics and some webpackages that claimed 5 times the traffic. Here's a note why the web statistics from your hoster may not be what you think it is. That is not to say the numbers are wrong but the interpretation of them should be done with care. The most important number for all bloggers is obviously the number of readers that visited your blog. We believe our numbers are an accurate reflection of this since we get called once every time we appear on a blog within a browser page. That would be our definition of a reader visit. Here's where we start to deviate from the webstats that your ISP is showing you.

• Unique visits. This should be the same as our number right? Not always. Every page usually loads 4-5 (sometimes more) additional files including stylesheets, scripts, images etc. But unique means they should all collapse to one IP. Well, yes and no. If the web hoster had multiple servers and each had it's own web stat package and they present you a number by simply adding the stats from each webserver, unique is not unique anymore. If they performed simple round robin load balancing, your 5 requests from one page and one IP would be spread evenly across multiple servers each registering a unique hit. Unless they have a common web stat database which would be performance bottleneck, each webstat package doesn't know about the others. You final count for each visitor if you had 5 servers? 5 unique visits.
  
• Total pages. This can be severely bloated because of web crawlers. It's pretty hard for web hosters to exclude them from their statistics because web crawlers may use more than one server and IPs change all the time. At the same time, web masters do want all the stats because they need to know the total number of pages that are being requested whether it is a human reader or web crawler. So if your blog pages have many links to comments, archives and other blog entries, the crawler will add a significant number of pages to your total. It is not uncommon for large blogs to have as much as 100 links to it's own domain. So each time a crawler comes to your site and follows the links, your web hoster is logging a couple hundred pages. The number of web crawlers vary but not limited to the big 3 or 4 search engines. A popular site may get 10-20 a day if not more.
  

The best way to get a true reflection of who your readers really are is actually using something external like Sitemeter or TheGoodBlogs because it doesn't get aggregated with the total number of support files and web crawlers that may visit your blog. Again it is important to stress that the numbers that web packages are designed to give you is to help you with performance and bandwidth so they will often include all the stuff that are not related to the number you want, the number of unique blog readers.