Netcraft Web Server Stats Challenged

kolchak writes "An article in The Age has an interesting analysis of the Netcraft Web Server Usage Reports. According to Port80 Software, Netcraft's surveys are biased towards domain name parkers and very small web sites, not taking into account how popular a site may be - there's some interesting results in the competing Port80 survey." However, it should be pointed out that Port80 "develops software products to enhance the security, performance and user experience of Microsoft's Internet Information Services (IIS) Web server."

Comments

A bit more than the average MS bias

[SeanTobin]

This is wrong on soooooo many levels. I could understand trying to twist the truth by redefining what a webserver is... but thier sampling method is straight out wrong.

Want proof? Here it is. Go to the linked article, (or click here) and where they have the box to check your server header (about half way down the page) type in www.microsoft.com - you will see its running IIS/6. A nice happy IIS server.

Now, type in my web server - http://www.isthatdamngood.com - its a nice Linux/Apache server. My server will CRASH thier app! Actually, a lot of linux servers will crash it...

Kinda hard to claim your results are more indicitative of the market when your scanning technology is flat out broken.

Re:A bit more than the average MS bias

[_xeno_]

Worked for me. I tried "slashdot.org" and "www.theregister.co.uk" - both of them worked just fine. However, "www.isthatdamngood.com" did indeed cause a scripting error - but I doubt it would effect their actual surveying, it's just an ASP error, not an actual "crash."

Anyway, it's long been known that Netcraft's methods are flawed, since it counts individual web servers multiple times for each virtual domain. It should only count unique sites. (For example, Slashdot counts for something like 13 sites - the individual sections (like apple.slashdot.org - I'm not listing all of them), slashdot.org, www.slashdot.org, images.slashdot.org.)

It's still debatable what the correct survey method is (and whether Port80's method is any better), but Netcraft is biased towards sites with lots of virtual domain names. (I'd imagine SourceForge gets counted many times, too...) Of course, it's also questionable if individual servers in a round-robin load-balancing solution should be counted, so counting by IP instead of domain name is questionable too.

As is often said, "there are lies, damned lies, and statistics" - any counting method has issues.

Blah, I can't preview because Mozilla is f***ing broken and won't display the preview page, so please pardon any typos.

Re:A bit more than the average MS bias

[panaceaa]

The parent poster's point is that their site grabber program can get IIS sites but crashes on some Apache sites. Port80 Software may use the same code to run their surveys since both the grabber and survey programs need the core feature of analyzing a site's HTTP headers.

So if their survey script also returns invalid data for Apache sites, then the IIS numbers would be much higher than they actually are. I would at least like to see some actual numbers rather than pure percents before I believed their data. They surveyed 1000 sites -- how many sites are included in the survey's data?

Another thing that seems odd to me is Netscape iPlanet usage is higher than Apache. Where's the primary data to support that?

Like that's going to work

[BigRedFish]

a product .... to confuse script kiddies

I am running Apache on Linux, and I still get 1000 hits a day trying to crack MSADC with buffer overflows, and FrontPage exploit attempts. It's not like the script kiddies check the server ID or pay any attention to it even if they do.

Free Software Wins again.

[Anonymous Coward]

and what would that one line be?I want my $50 worth on my apache server

• Unpack the Apache distro file (apache_1.x.xx.tar.gz) and run the configure script.
  
  Now do the following commands:
  
  
• cd src/os/unix
  (With Apache 2.x, cd os/unix)
  
• vi os.h
  
• Search for:
  #define PLATFORM "Unix"
  
• Replace "Unix" with whatever you want your OS identification to be. (Some of the more creative ones I've done are 'NachOS,' 'PathOS,' 'StratOS,' 'ZerOS,' and 'WinDos'...anything.)
  
• Save the file.
  
• cd ../../include
  
• vi httpd.h
  (With Apache 2.x, vi ap_release.h)
  
• Search for:
  #define SERVER_BASEVENDOR "Apache Group"
  #define SERVER_BASEPRODUCT "Apache"
  #define SERVER_BASEREVISION "1.x.xx"
  
• Replace "Apache" and "1.x.xx" with whatever you want your Server and version number to be. (I recommend "Port80Software-Is-A-Fucking-Ripoff" and "Holy-Jumping-Jesus-This-Was-Easy", respectively.)
  
• Save the file.
  
• cd ../..
  (With Apache 2.x, cd ..)
  
• make
  

You're done. Congratulations. You just saved yourself $49 dollars!!!

Re:Free Software Wins again.

[ivan.ristic]

If you're using mod_security on your Apache server then you only need to add one line to the configuration file:

SecServerSignature "MyServer/19.5.1"