Log Analysis Tools for Windows?

FunkMonkey asks: "I administer a custom web app for my company (unfortunately, it's a Windows-only company -- servers and all), and I've been asked to find a web log analysis tool that our users can use to look at standard stats (number of visits/hits, etc.), as well as the ability to filter by authenticated users (including putting those users into groups). Additionally, the tool must be free or under $100.00, and be able to generate Excel (or Excel-compatible) reports. My app generates ECLF reports, so just about any web log analyzer should be able to read the logs themselves. I see this as a good opportunity to weasel some Open-Source stuff into the company. Any help you can offer -- suggesting apps, tips, forums to which I could post this question, etc. -- would be most welcome. Thanks in advance!"

Comments

Analog

[henrik]

www.analog.cx

Re:Analog

[xmas2003]

Roger that recommendation on Analog - works good for me - and was helpful in doing an analysis of the Slashdot Effect on Christmas Lights which includes actual output from Analog that you might find interesting.

Re:Analog

[Anonymous Coward]

Wow, this sounds almost like www.goatse.cx

Re:Analog

[Anonymous Coward]

I'm surprised anyone would even consider something other than Analog. But hey, on the other hand, if you already have Crystal Reports running at the company, take a look at what it does.

Re:Analog

[exhilaration]

Yes, Analog, AWStats, and Webalizer (here's Webalizer Win32) are the three packages that my web host installs for all its users.

Re:Analog

[halfnerd]

If he'll already be using perl, why not use SpreadSheet::WriteExcel from CPAN.

Re:Analog

[bjpirt]

I currently use webalizer, but felt like it should be doing a lot more with the information in the log files. Hell, even I could spot more trends from just looking at the log file raw.

So I took a look at AWStats and (although slower - it's written in perl not compiled c) it looks to have a lot more useful features.

Usefully though, I came across this comparison chart comparing these three options plus another. Hope this helps.

bjpirt

Windows?

[Markus Registrada]

Why are the editors posting a Windows-only question? There's nothing nerdy about Windows.

Just go buy something.

Webtrends

[spacecowboy420]

n/t

PHP

[NanoGator]

You can always turn on Windows IIS or install Apache. Then, install PHP. Then write some code to count the statistics you need. Once you're finished, you can give it away to the community, and it'll run on anything that can execute PHP!

Re:PHP

[T-Ranger]

How would php help at all?

First of all; he already has logs. He asked about log analysis, not log creation. If you mean to insert a custom php function into every file dologline() or something, that would be a) slow, b) hard-to-impossible to implement on potentially thousands of files c) incompatable if he is using any other server generated pages. And it would miss static objects, images for example.
If you mean, write your own log parser in PHP, again that would be both slow and painfull. PHP isnt paticularly good at text manipulation (which is ironic, seeing as how thats what html generation basicly is). And even if it was as good (or better then), say Perl, Perl has massive amounts of free modules avialable desigined for log analysis, statistics, report generation. PHP is lacking in the third party lib field.

I dont know the state of Activestate (*cough*) so I dont know how well existing, built for Unix, Perl log analysis packages would work. I suspect that many just would. To which, I point out a freshmeat link.

Re:PHP

[Hungus]

The poster means that one should use PHP to parse the Logs. It is quite possible and very easy to use it in such a manner. Its a pity that BBEdit doesnt run on x86 or I would suggest that but in all honesty anything that will allow you to use grep is essentially ideal for parsing logs. save as csv and any spreadsheet around wil open it fine. Best 3 things about grep? 1) Flexability 2) Speed 3) Price Oh and as to your statement that PHP is lacking in the 3rd party lib field? And you comment that php isn't very good at parsing text, well you might want to stop smoking that crack. or maybe get your head out of Larry's

Re:PHP

[T-Ranger]

If you using grep then your not using PHP. $usefull = system("grep $file"); does not count as using PHP. If you want to use the UNIX tool set, then use the UNIX tool set.

PHP is not paticularly good at manipulating text. It is infinitly better then C, or Modula. But compared to gawk, sed... the unix toolkit, let alone Perl, it sucks. The PHP developers know this, and they have wraped up some of espically usefull features of Perl in to PHP functions... Things that Perl treats as natiave opearators. Compare equivelent search/replace code.
PHP:
$data = preg_replace("/hello/", "goodby", $data);
Perl:
$data =~ s/hello/goodby/;

Minimizing things down to 1 character variable names, 1 character strings/regular expressions, PHP has a character count of 35. Perl is less then half at 14. Because Perl has string manipluation facilities built into the core language. String manipulation is Perl's first, and primary design goal.

I use PHP all the time. And Im probabaly a better PHP programmer then a Perl programmer. But for text analysis, Perl is king.

Re:PHP

[Hungus]

did i say anything about perl vs php? No did i say php is or uses grep? no i didnt say that either.

Oh and for the record .. I develop medical security and legal databases more secure than banking software and a hell of a lot larger than anything you have ever seen. How can I say that? Well deployed my last project was 27TB. Oh yeah and it was built using PHP does that qualify me as a php developer? Sheesh. Learn to read the whole post and don't immediately jump to conclusions. Are there better tools than php? Absolutely are teh ones you mentioned ( as well as my own mentionings) better than php certainly. does php suck at regular expressions? Hardly. Why do I even bother.

Re:PHP

[DAldredge]

Well, you didn't even bother to proof you post before you went off the handle. Perhaps next time you wish to sound like Gods gift to programming you should proofread what you post?

*This post has NOT been proofread (It is late)*

Re:PHP

[DrSkwid]

did you notice the "for free or under $100" part.

I don't know about you but $100 won't buy you much of my time.

Re:PHP

[Hungus]

:) That made me smile. Thank you.

Re:PHP

[E_elven]

>PHP:
>$data = preg_replace("/hello/", "goodby", $data);
>Perl:
>$data =~ s/hello/goodby/;

What the hell does it matter how you *call* the function (aside from the PHP being clearer)? If you want to argue which is better, argue over the *implementation* of the replacing algorithm (which is probably going to be pretty much the same.)

As far as I can recall, the PHP text functionality lies in the compiled C modules, not the top-layer modules written in PHP. I could be wrong.

Re:PHP

[T-Ranger]

=~ isnt a function. Its an opearator. Perl has string manipulation facilities buit into it at a very low level. PHP does not.

PHP may be clearer, but that is because you (obviously) dont know what the =~ opearator does. Which, if you've never seen Perl code before, is quite resonable. Lets say you dont know what + does:

Stupid example language:
$i = add_values($i, 2);

PHP (or Perl, C, Java...):
$i = $i + 2;
or:
$i += 2;

Assuming you dont know what + does, the first example is the most clear. The last example is the least clear, but I suspect is the most common way to add 2 to something. Since addition is a very common thing to do, most languages have + built in as a native opearator. Perl is desigined to do string manipulation, so it has string opearators built into the language.

Re:PHP

[E_elven]

*breaks out the blackboard for drawing a picture*

What I am saying is that it's irrelevant how the function is called. Using =~ instead of rx_replace() does not make a language better or worse suited for string manipulation. The algorithm used for the actual manipulation does. If Perl and PHP use the same algorithms, then they are by definition equally good in string processing.

Excel compatibility? Scripting?

[cuiousyellow]

Try Open Office.

The person forcing you to use MS can be placated knowing that it uses XML somehow and you get an excuse to install perl on the servers!

(are there places on this fancy intarweb that don't have access to google?)

Analog

[embo]

You should look at Analog. It is free, and open source. While it probably doesn't export straight to Excel, you would likely have two choices there:

First, since it's open source, you could add support to export to CSV fairly easily.

Second, Analog can export to what it calls "COMPUTER" output, which is designed for easy parsing. Couple the COMPUTER output with a little Python or PERL, and you'll have a CSV file fairly quickly.

When you're finished looking at Analog, make sure you also consider Report Magic for Analog to make things look pretty in a browser.

err.... um....

[Anonymous Coward]

what are you getting paid for again?

Hi Slashdot!

[reynaert]

Will you please do my job for me?

Excel

[BladeMelbourne]

FYI - many web applications that export to a .xls file, are simply creating an HTML document, (hopefully with tables in it), and changing the header so the browser does not think the file is text/html.

eg:

Response.ContentType = "mime-type application/vnd.ms-excel"
Response.AddHeader "content-disposition", "attachment; filename=auto-gen.xls"

You can also do the same to generate M$ Word files - although they dont need a table inside. These files can be opened in M$ Office and OpenOffice.

Mike

awstats

[Shaheen]

I use AWStats on a home web server.

It's written entirely in perl so it can work on Windows just fine as well. To see a sample go here.

Friends don't let friends do Windows...

[Anonymous Coward]

If you are running IIS please just admit it.
Then we can all point and laugh.

D'oh.

[Rocky]

This was already invented a number of years ago.

It's called PERL.

Logs under Windows tend to be very thin...

[Spoing]

If the data isn't captured in the first place, getting a better analysis tool won't help.

I'm spoiled with the feedback from Unix-style systems.

That said, if there is a way to improve the volume and quality of what is being logged, I would appreciate hearing about it. (Yes, I've tweaked the default log settings for the system, and have enabled what I can for the apps I find. DR Watson is moderately useful when apps crash.)

Re:Logs under Windows tend to be very thin...

[barzok]

Did you even read the post? It's a web app. IIS logs to the same log format everyone else does, and captures all the same information.

A Microsoft tool for a Microsoft job?

[Jorkapp]

Consider using Microsoft's Log Parser 2.x (2.1 Included in IIS 6.0, 2.0 availible as Download for Win2k/XP/2k3, and is only 2mb.

It doesn't output to .xls files, but it does let you run SQL like queries against log files of any format. Its availible as command line and as a series of scriptable COM objects.

http://www.microsoft.com/downloads/details.aspx? di splaylang=en&familyid=8cde4028-e247-45be-bab9-ac85 1fc166a4

Some of these

[$exyNerdie]

You can try some of these:

http://awstats.sourceforge.net/

http://www.multicians.org/thvv/webtrax-help.html

http://www.ftls.org/en/examples/cgi/ewla.shtml

http://www.watchwise.net/

http://www.weblogexpert.com/

For a detailed list of web log analyzers, go to this page. It has listing of various platform specific and platform independent analyzers:
http://www.uu.se/Software/Analyzers/Access-analyze rs.html

Absoulte

[BenTheDewpendent]

Absolute log analyzer. exports to excel and many other things. Pro version is $250 but looks to be quite flexible.

WUSAGE is the best, I think...

[Cragen]

Wusage is $25 for a single server, $75 for 5 domains, & $275 - unlimited. I think it's the best you can get -- available for all web servers and OS's, I think. I used it for years back when I was web-mastering. Have fun,

*cragen.

Do it yourself

[muleboy]

So, basically you want someone to write something for you that works with all your proprietary programs so you can make money off of it? And you want them to do it for free or very little money?

Good luck.