Slashdot Mirror

← Back to Stories (view on slashdot.org)

Logfiles Made Interesting with glTail

Posted by ryuzaki0 on from the because-you-can dept.

Fudgie writes "My boss claimed it was pretty much impossible to create an entertaining way to visualize server traffic and events in a short time frame, so of course I had to prove him wrong. A weekend of neglecting my family produced a small ruby program which connects to your servers via SSH, grabs and parses data from Apaches access log and Ruby on Rails production log, and displays your traffic and statistics in real-time using a simple OpenGL interface (tested under Linux and Mac OS/X). It's a bit hard to explain over text, so please have a look at fudgie.org for an example movie, and more information."

131 comments

  1. Oh dear... by DamonHD · · Score: 5, Funny

    ...I'm afraid that's the nearest I've seen to a simulated pissing contest ever! B^>

    Rgds

    Damon

    --
    http://m.earth.org.uk/
    1. Re:Oh dear... by Anonymous Coward · · Score: 0

      And when the big blobs appear, does that mean he's passing stones?

    2. Re:Oh dear... by Nimey · · Score: 1

      I think those are blobs of spooge. Excitement from having such a large request, possibly.

      --
      Hail Eris, full of mischief...

      E pluribus sanguinem
  2. Oh great... by GodlikeDoglike · · Score: 5, Funny

    ...we just made his log screen look like a bukkake flick.

  3. Just took a look at the video by Centurix · · Score: 2, Funny

    And it looks like lots of things taking a wee. Once the site is slashdotted, it'll be a veritable golden shower...

    Nice work though.

    --
    Task Mangler
    1. Re:Just took a look at the video by v4vijayakumar · · Score: 1

      Once the site is slashdotted.. wtf? come on, what is your hit rate? million hits a day?

      how many users seem to read this? one tenth?

      how many users will find this interesting? one tenth?

      how many users will really vist the link? one tenth?

      so, let us say /. brings 1000 users to that site.

      how many of them will be regular user to that site? one tenth?

      how many of them will see ads in that site? one tenth?

      how many of them will click ads in that site? one tenth?

      so, what profit /. brings in, or, this 1 user can do to that site?!

      are they going to make fortune with this single user?!
  4. Wait, what... they're not interesting? by Anonymous Coward · · Score: 0

    I love looking at log files through less. Tons of fun.

    1. Re:Wait, what... they're not interesting? by Anonymous Coward · · Score: 0

      Growl.

      I spent a day last week tail -f'ing the access logs of two servers, and kick a couple of WebSphere 4 instances each time the xterms stopped scrolling, indicating the bloody JVMs had locked themselves up again. About every 30 minutes or so, significantly less as the load diminished in the afternoon. The time spent waiting was used to attempt to figure out just why I had to do this. To no avail, the next day we switched over to a different set of servers running the old version. Definitely not my idea of fun. Although, at least the access log is meaningful, compared to the appserver logs, full of exceptions and stack traces that result from "known errors" that haven't been fixed for almost half a decade.

    2. Re:Wait, what... they're not interesting? by Fudgie · · Score: 5, Interesting

      A lot of my time at work is spent looking at logfiles from webservers, applications servers, and databases looking for things about to break down, but after I introduced this I just need to glance at a screen to instantly see if some server has stopped answering, is taking too long to answer, or is generating way more exceptions than normal. I also add an event (the login text bouncing down the screen in the movie) on each money generating activity, which always amazes marketing people when they walk by.

    3. Re:Wait, what... they're not interesting? by mboverload · · Score: 1

      Very cool.

      This is the first time I have felt I needed to say anything on Slashdot in a while.

      Well done, sir.

    4. Re:Wait, what... they're not interesting? by zero_offset · · Score: 1

      Almost half a decade? Holy shit, that's nearly one-twentieth of a century!

      --

      Slashdot quality declines as the number of hot grits posts decreases. - Provolt's Law, Apr-09-2005

  5. engineering management 101 by Anonymous Coward · · Score: 5, Funny

    tell the engineer it can't be done

    1. Re:engineering management 101 by Anonymous Coward · · Score: 0

      Greatest comment ever.

    2. Re:engineering management 101 by mattgreen · · Score: 1

      It really is. Kudos to the original poster, I chuckled quite a bit.

    3. Re:engineering management 101 by H310iSe · · Score: 3, Funny

      Just want to give props, very nice you made my morning. Now to convert this to a heads up display for my helmet and I'm 1 step closer to becoming the motorcycle hacker I always dreamed I could be. And 1 step closer to earning a darwin award...

      --
      closed minded is as closed minded does
  6. Visitorville by Anonymous Coward · · Score: 4, Interesting

    The most entertaining way I ever saw to view logs was Visitorville-its kind of like SimCity meets web logging.

  7. Looks promising by Anonymous Coward · · Score: 0

    Can the same concept be used for graphing network traffic, perhaps from netflow?

    1. Re:Looks promising by Fudgie · · Score: 5, Interesting

      Anything put into a logfile could be parsed and shown. I've tried with emails, shoutcast listeners and server logins, but they're not as interesting to show in the movie as I don't have the kind of traffic to make it useful.

    2. Re:Looks promising by MarkRose · · Score: 1

      This would be really handy for MySQL queries. My shared MySQL server runs 10 to 200 queries per second for me alone. Finding a good way to represent the data could be interesting.

      --
      Be relentless!
    3. Re:Looks promising by DudeTheMath · · Score: 3, Funny

      So...how many hours of unpaid overtime did your boss get out of you?

      I like getting paid for my awesome work. Kudos, though.

      --
      You save only 59 seconds over 8 miles by going 75 instead of 65. Do you really have to pass that guy? Do the Math!
    4. Re:Looks promising by InsaneMosquito · · Score: 1

      On the website it mentions monitoring FTP, shoutcast, and DB queries. Have you coded these yet, or are those just to-dos? FTP and DB Queries would be interesting to see graphed.

  8. Not "Fudgie", glTail by gumpish · · Score: 4, Informative

    It's pretty obvious that fudgie.org is just the name of the site and glTail is the name of the program.

    1. Re:Not "Fudgie", glTail by Anonymous Coward · · Score: 0

      There is no such thing as a "forward slash".
      There is only "slash" and "backslash". Or if you're British, it's 'stroke' not 'slash'. I'm not sure if we've got an equivalent improvement on 'backslash' though.
    2. Re:Not "Fudgie", glTail by JamesRose · · Score: 1

      On the main page it says glTail, and when you click the link to read it with comments it says Fudgie, so actually, it seems both are there.

    3. Re:Not "Fudgie", glTail by Bert64 · · Score: 1

      Also, "slash" is also British slang for "urinate".

      --
      http://spamdecoy.net - free throwaway anonymous email - avoid spam!
    4. Re:Not "Fudgie", glTail by Aladrin · · Score: 1

      And American slang for 'fan-fiction with male-male relationships.' What's your point?

      --
      "If you make people think they're thinking, they'll love you; But if you really make them think, they'll hate you." - DM
    5. Re:Not "Fudgie", glTail by nacturation · · Score: 2, Funny

      Also, "slash" is also British slang for "urinate". And backslash is what happens if you urinate onto a parabolic surface?
      --
      Want to improve your Karma? Instead of "Post Anonymously", try the "Post Humously" option.
    6. Re:Not "Fudgie", glTail by LiquidCoooled · · Score: 1

      Actually, I think backslash occurs when you try pissing into the wind.

      --
      liqbase :: faster than paper
  9. Wow ! by cheros · · Score: 4, Interesting

    Obligatory jokes about 'taking the piss' aside, that is brilliant. It's the ultimate 'machine that does ping' (to name an old sketch) to keep management amused, but also provides real data. I bet that screen will go ballistic when you get Slashdotted (also a good way to visualise DDoS, maybe?).

    I was about to say that it's a sort of etherape on steroids, but I've just realised your visualisation could benefit etherape instead (if you don't know etherape, look it up. No tools identifies a virus infection quicker).

    Class, I'm impressed.

    --
    Insert .sig here. Send no money now. Owner may sue, contents will settle. Batteries not included.
    1. Re:Wow ! by bughunter · · Score: 2, Insightful

      I bet that screen will go ballistic when you get Slashdotted

      Look closer. It already is ballistic.

      --
      I can see the fnords!
    2. Re:Wow ! by Poromenos1 · · Score: 1

      Man, capitalize names. I got all sorts of things in my mind when I read about your EtheRape program...

      --
      Send email from the afterlife! Write your e-will at Dead Man's Switch.
    3. Re:Wow ! by nilbud · · Score: 0

      Looks like you said the secret part out loud again. You remember talking to the councillors in the hospital about the urges. Take one of the big green pills, have a good sleep and phone Dr. Green first thing in the morning. Whatever you do stay away from the zoo, remember what the judge said.

      --
      never let a man put his dirty how-do-you-do into your bajingo
  10. just a ploy to visualize the slashdot effect by molo · · Score: 2, Interesting

    Notice in the movie that one of the sites being monitored is fudgie.org, which is what is linked to here. This looks like a ploy to visualize the slashdot effect. :) Wonder what that must look like. Might tax the renderer pretty hard. I guess that is one way to get load testing done!

    -molo

    --
    Using your sig line to advertise for friends is lame.
    1. Re:just a ploy to visualize the slashdot effect by Fudgie · · Score: 5, Informative

      Still running at 30 fps with ~25 requests / second.

    2. Re:just a ploy to visualize the slashdot effect by Fudgie · · Score: 5, Interesting

      http://www.fudgie.org/slashdotted.jpg for how that looks.

    3. Re:just a ploy to visualize the slashdot effect by Anonymous Coward · · Score: 0

      Can you post UserAgent/OS statistics?
      Please!

    4. Re:just a ploy to visualize the slashdot effect by foobsr · · Score: 1

      Great work (How I love that I may contribute a positive remark ;)

      CC.

      --
      TaijiQuan (Huang, 5 loosenings)
    5. Re:just a ploy to visualize the slashdot effect by eclectro · · Score: 1

      Wow, it's like slashdot hurls chunks.

      --
      Take the cheese to sickbay, the doctor should see it as soon as possible - B'Elanna Torres, "Learning Curve"
    6. Re:just a ploy to visualize the slashdot effect by Fweeky · · Score: 1

      I just ran it through 10,000 Apache requests. After a minute and a half or so it stopped spewing dots from most of the graphs other than the "Content" ones, which spewed for about 8 minutes. In all those logs (about 60 seconds of activity) took 6 minutes 22 seconds CPU time on a 1.66GHz Core Duo Mac Mini.

      Most of that time seems to have been spent drawing dots at maximum speed spewing out of the "Content" lines; maybe they need to increase speed in response to higher request rates so it's not waiting for them to clear the screen so much?

    7. Re:just a ploy to visualize the slashdot effect by nacturation · · Score: 2, Interesting

      Very nice. One suggestion: rather than have each side's dots fall off at the bottom of the opposite side, how about matching up serving requests with the originating referral so that the dots go to the corresponding spot on the right? Also, if you're not familiar with Flight Patterns it's along the same lines. Borrowing from that, it'd be quite interesting to show a 2D map arranged in a hub and spoke model with the center being the site(s) and the spokes representing the top 10 (or 20... configurable) referring sites with a special case for search engines.

      Well, perhaps I'll have to learn Ruby and hack this myself. The script certainly looks clean enough.

      --
      Want to improve your Karma? Instead of "Post Anonymously", try the "Post Humously" option.
    8. Re:just a ploy to visualize the slashdot effect by Fudgie · · Score: 2, Informative

      Not sure why it stopped for you, I've had it running throughout a slashdotting without any problems at all. Peaked at 3500 req/min and still spewed dots from all the correct places at 30 fps.

    9. Re:just a ploy to visualize the slashdot effect by Fweeky · · Score: 1

      Well, each stream seems to have a maximum rate it can spew out dots; if you exceed that, they back up. If you can spew out 1,000 dots from each stream in a minute but you've got 10,000 to actually spew through it, it's going to take 10 minutes doing it.

    10. Re:just a ploy to visualize the slashdot effect by Fudgie · · Score: 2, Interesting

      Interesting idea. Shouldn't be too hard to try something like that, I already have some code in there doing something similar meant for incoming emails, uploads and other data going into the servers/sites. Try adding :type => 5 to the URL activities for an example. -- Erlend

    11. Re:just a ploy to visualize the slashdot effect by o2sd · · Score: 1

      Wow, it's like slashdot hurls chunks.

      I say hurl. If slashdot blows chunks and fudgie comes back, shes yours. If it spews and fudgie runs, it was never meant to be.

      --
      - Nothing to see hear.
    12. Re:just a ploy to visualize the slashdot effect by Fudgie · · Score: 2, Informative

      You're correct, and I will be adressing this in the next version. It's currently limited to 1000/FPS per second.

  11. doom by rucs_hack · · Score: 1

    didn't someone once do a version of doom that displayed network activity?

    I recall seeing screenshots, but that was years ago.

    1. Re:doom by xappax · · Score: 4, Informative

      perhaps you mean this: http://www.cs.unm.edu/~dlchao/flake/doom/

    2. Re:doom by rucs_hack · · Score: 1

      ah yes, that was it, not for networks then.

    3. Re:doom by aled · · Score: 1

      It's a shame nobody did a newer version with Quake.

      --

      "I think this line is mostly filler"
    4. Re:doom by Anonymous Coward · · Score: 0

      It's clever, but...

      Drastic action takes work. In a command line interface, all actions take approximately the same amount of effort. One can ls just as easily as rm -rf *, which is kind of unfortunate. In a cyberspace environment, the players are not omnipotent, so performing large actions takes time and effort.

      That sounds like a big disadvantage to me! In fact, it sounds remarkably similar to the Microsoft philosophy where the user interface is optimised for the most likely tasks, but with the side effect of making less likely tasks extremely difficult and non-intuitive.

  12. Oh, Sweeeetness! by avirrey · · Score: 5, Funny

    You gotta add an 'Asteroids' ship on the screen that lets you shoot down connections!

    "Oh, look! Bob just logged on... let's get 'em!"

    ...

    "IT support. How can I help you?"

    "Hi, this is Bob..."

    --
    X's and O's for all my foes.

    1. Re:Oh, Sweeeetness! by NFN_NLN · · Score: 2, Funny

      We're finally catching up to movies now... you know the cheesy and disconnected from reality sequence where some hackers enters a system by navigating a 3D maze... and the firewall is a monster you have to literally kill. The movie Masterminds comes to mind.

    2. Re:Oh, Sweeeetness! by quanticle · · Score: 1

      And maybe after that you can add a tool to allow you to kill "rabbits" with "flu shots" ;-)

      --
      We all know what to do, but we don't know how to get re-elected once we have done it
  13. Wow by its_me_ken_lai · · Score: 1

    Man this is cool. Very cool.

    --
    Ken Lai
    1. Re:Wow by Volatar · · Score: 1

      I highly agree, I am definatly going to check this out.

    2. Re:Wow by symbolic · · Score: 1

      Agreed. I saw something similar a few years ago, but this seems a bit more refined. I think there's actually a lot you can do when combining a graphics rendering engine with something like network activity. All it takes is a little creativity, a little time, and a boss who says it can't be done.

  14. Compiz for syslogs - ohmygawd ! by udippel · · Score: 2, Funny

    Luckily, I saw the movie before the meltdown of the server. It always pays to be on time. ;)

    For those unlucky and late, actually, you missed a competition of peeing coloured snowflakes from the right versus doing the same from the left.
    Only, the sources on the left are much better at aiming.
    Plus, you have some 'Login ...' scrolling top to bottom; like the cast of a movie.

    Heads up, Fudgie, it is truely the most amazing display of log files ever creeping across my eyes.
    Keep the good work up, and please post again when you have something actually useful for the sysadmin.

    I declare you 'King of Log Candy' !

  15. Ob quote by Provocateur · · Score: 4, Funny

    All I see now is blonde, brunette, redhead.

    --
    WARNING: Smartphones have side effects--most of them undocumented.
  16. Pretty, but? by dotancohen · · Score: 0

    It's pretty, but it is useful? Actually, I think that it is. Those who review the logs ("Firefox?!? We don't need to support a web browser with only 30% market share!!!") apparently have no idea what the data means anyway. So making it pretty certainly won't hurt. And if it's pretty it just might spark interest.

    --
    It is dangerous to be right when the government is wrong.
    1. Re:Pretty, but? by fmobus · · Score: 2, Interesting

      I believe this sort of tool is useful for realtime monitoring of net resources utilization. It can assist you giving graphic clues when something goes out of the usual parameters, like DDoS, slashdotments (sp?), router failure, etc. Depending on information being monitored and how it is displayed, it could also be used for long-term decision like buying more hardware or switching software because the current setup is not handling the load.

      One nice, but more local example is the "duck" activity monitor (a windowmaker classic): a duck floats by a mass of water. If the water gets to high, it means the memory usage is high; if it has too much bubbles, processor is being hit. No percentages nor text, just a simple graphic.

      A place I used to work is now trying to develop something like this: visualizations where you can tell trouble is brewing in a glance. This is useful for them because their services involved a lot of maintenance of third-party networks but having someone dedicated to nanny all systems is "dumb" and error-prone. Their solution consists of multiple screens around the office showing how the systems they are responsible for are behaving.

    2. Re:Pretty, but? by merreborn · · Score: 1

      A place I used to work is now trying to develop something like this: visualizations where you can tell trouble is brewing in a glance


      If you just install any of the standard RRDTool frontends out there, e.g. cacti, or my personal favorite, munin (far easier to install/extend/use than cacti), and check them regularly, it's not hard to tell when something's wrong. Traffic and usage patterns are pretty consistent from week to week on the boxes I've administered. After a month of checking graphs in munin daily, I could instantly tell when a CPU, network, memory, or process count graph was out of whack.

      After a few more months, not only could I tell at a glance that something was wrong, but I could use the information in them to figure out *what* was wrong.

      You'd be hard pressed to try to come up with a combined visualization that was actually *more* useful (i.e., equally data-rich) to a trained eye.
    3. Re:Pretty, but? by zero_offset · · Score: 1

      Even more to the point, the article says the boss challenged him to create a visualization tool that was entertaining... this is sort of interesting, but entertaining? Or did Fudgie misspeak?

      --

      Slashdot quality declines as the number of hot grits posts decreases. - Provolt's Law, Apr-09-2005

  17. Postfix? by JShadow21 · · Score: 1

    I'd enjoy a postfix version

    1. Re:Postfix? by Fudgie · · Score: 3, Interesting

      Shouldn't be too hard. I'll cook one up this evening.

    2. Re:Postfix? by JShadow21 · · Score: 1

      Awesome, thank you sir

    3. Re:Postfix? by Fudgie · · Score: 1

      At work I show about 30 logfiles, divided across 10 different servers running at 50+ FPS on an old Centrino laptop with a GeForce 5xxx mobile.

    4. Re:Postfix? by MarkRose · · Score: 1

      I have it monitoring Apache. At around 1000 requests per minute, I get about 10 fps on my 1750 MHz Duron. It's CPU bound, not GPU bound.

      --
      Be relentless!
    5. Re:Postfix? by Fudgie · · Score: 1

      A basic Postfix parser cooked up and introduced in v0.02. Also includes a simple IIS parser. More refined parsing of postfix will come. :-)

    6. Re:Postfix? by Anonymous Coward · · Score: 0

      Does this work on all versions of Linux or just the Debian based ones?

      I ask because I had problems getting it to work on Fedora 7, with up to date Ruby and Rubygem installed. In your instructions you state to install libopengl-ruby. On Fedora (at least with the default packages and Extra Packages for Enterprise Linux - EPEL) this package doesn't exist.

      So, either 1.) I'm missing the package by simply over looking it, or 2.) There is another way to get this to work. Any ideas?

    7. Re:Postfix? by Fudgie · · Score: 1
      Using the libopengl-ruby version provided has caused nothing but trouble, so I've changed to the gem version.

      gem install ruby-opengl
  18. That's no moon by rjamestaylor · · Score: 1

    Serious prostrate problems at Fundie.org, it appears... I'm looking forward to plugging this in to sysstat for some over-utilized servers I manage....

    --
    -- @rjamestaylor on Ello
  19. Comment removed by account_deleted · · Score: 1

    Comment removed based on user account deletion

  20. GNU GPL by wikinerd · · Score: 2, Informative

    #!/usr/bin/env ruby # gl_tail.rb v0.01 - OpenGL visualization of your server traffic # Copyright 2007 Erlend Simonsen # # Licensed under the GPLv2

    Hey, this is not the correct way to apply the GNU GPL licence. I don't know whether you had very little time available or just don't care, but the correct way is to explain exactly what licence (full title) the program is under and enable the user to find the licence (provide a copy of it and explain that the author of the licence is FSF, giving their address). We nerds of course understand completely what you mean, but other people may have no idea what you are talking about. To learn how to apply GPL on your program read this.

    Good work, by the way. Was there any reason you preferred GPLv2 and not GPLv3? Also from the wording of your licence I think that you intended this to be available only under v2 and not v3 (you say "Licensed under the GPLv2" without a "or any later version" clause).

    1. Re:GNU GPL by Anonymous Coward · · Score: 0

      Am I the only person tired of having people defile articles about technology with this inane "Why did you license it this way?" banter? Jesus christ, it's open source and even Free Software, just because he didn't use RMS' latest and greatest toy doesn't mean you need to badger him about it.

    2. Re:GNU GPL by kamochan · · Score: 1

      No, you are not the only one...

    3. Re:GNU GPL by makomk · · Score: 1

      I wonder if the author has spent time doing Linux kernel development? Variants of that statement are quite common on files in it, for some reason.

    4. Re:GNU GPL by Anonymous Coward · · Score: 0

      "No, you are not the only one..."

      That should be "Yes, you are the only one." The original question:

      "Am I the only person *huge snip of whining*"

    5. Re:GNU GPL by Anonymous Coward · · Score: 0

      You're kidding right?
      You should be pointing to ffs.org (hopefully that points to For F**ks Sake)

  21. I loved this line: by Anonymous Coward · · Score: 1, Funny

    I loved this line:

    "Certain processes are vital to the computer's operation and should not be killed. For example, after I took the screenshot of myself being attacked by csh, csh was shot by friendly fire from behind, possibly by tcsh or xv, and my session was abruptly terminated."

    1. Re:I loved this line: by lahi · · Score: 1

      I can see why that approach didn't take off. (The idea seems to be 8 years old.)

      However, the line you quote is quite satisfying: csh certainly deserves to be shot. Of course, so do users of csh. This also applies to tcsh of course.

      -Lasse

    2. Re:I loved this line: by Seumas · · Score: 1

      My sick great-grandmother uses csh, you insensitive clod!

    3. Re:I loved this line: by lahi · · Score: 1

      Pity. Perhaps if she stopped, she'd get well.

      Thanks for calling me an insensitive clod, btw.

      -Lasse

  22. Here's what it looks like when you're not ./-ed by chipster · · Score: 1
    http://chip.cuccio.us/gl_tail.png


    Perhaps the parser doesn't like my Apache logs?

    2437 frames in 5.000 seconds = 487.400 FPS
    Elements[0], Activities[0]
    2550 frames in 5.001 seconds = 509.898 FPS
    Elements[0], Activities[0]
    1182 frames in 5.002 seconds = 236.305 FPS
    Elements[0], Activities[0]
    987 frames in 5.001 seconds = 397.321 FPS
    Elements[0], Activities[0]
    2534 frames in 5.003 seconds = 506.496 FPS
    Elements[0], Activities[0]
    2506 frames in 5.000 seconds = 501.200 FPS
    Elements[0], Activities[0]
    2505 frames in 5.000 seconds = 501.000 FPS
    Elements[0], Activities[0]
    2603 frames in 5.000 seconds = 520.600 FPS
    Elements[0], Activities[0]
    2548 frames in 5.000 seconds = 509.600 FPS
    Elements[0], Activities[0]
    2561 frames in 5.000 seconds = 512.200 FPS
    Elements[0], Activities[0]
    2559 frames in 5.001 seconds = 511.698 FPS
    Elements[0], Activities[0]
    2567 frames in 5.029 seconds = 510.439 FPS
    Elements[0], Activities[0]
    2548 frames in 5.000 seconds = 509.600 FPS
    Elements[0], Activities[0]
    2193 frames in 5.001 seconds = 438.512 FPS
    Elements[0], Activities[0]
    2300 frames in 5.000 seconds = 460.000 FPS
    Elements[0], Activities[0]
    2508 frames in 5.001 seconds = 501.500 FPS
    Elements[0], Activities[0]

    1. Re:Here's what it looks like when you're not ./-ed by chipster · · Score: 1

      I figured it out.

      My apache config has the "HostNameLookup" feature enabled for the logs.

      The ruby script's apache log regex parser only allowed for IP's in the logs. I changed it from [\d.] to [a-z0-9.] (line 87).

      Bingo.

      PS: THis is a pretty neat script.

    2. Re:Here's what it looks like when you're not ./-ed by Fudgie · · Score: 1

      Ah. I turned that off in '00 and forgot all about it. Sorry. :-)

    3. Re:Here's what it looks like when you're not ./-ed by chipster · · Score: 1

      No worries! Ideally, hostname lookups introduce extra load and traffic anyway :) The a-z0-9 should capture IP's and hostnames.

      Nice work.

  23. Sorry, but the boss won this bet by nurb432 · · Score: 2, Insightful

    its still NOT entertaining.. Its more bizzare then anything else.

    --
    ---- Booth was a patriot ----
    1. Re:Sorry, but the boss won this bet by aftk2 · · Score: 1

      I'm afraid I'd have to agree. The first thing I thought was: "Hmm..how about trying to make logfiles more readable/understandable instead?" I'm impressed by the technical acuity though.

      --
      concrete5: a cms made for marketing, but strong enough for geeks.
    2. Re:Sorry, but the boss won this bet by Fudgie · · Score: 1

      It's both harder for me to track a scrolling display of text moving in erratic bursts, and processing the information in each line than it is to take a quick glance at a screen and see if there are many small dots or few large ones.

  24. syslog, not ssh+tail -f by allenw · · Score: 4, Insightful

    Why use ssh + tail -f when one can send the output to a centralized syslog server? There isn't any need to setup an account, keys, etc. when you can have the individual servers consolidate the data for you.

    1. Re:syslog, not ssh+tail -f by MarkRose · · Score: 1

      You could, like me, be using a shared host where you have access to the server logs, but not to the server configuration files. This is a fantastic way to monitor performance remotely.

      --
      Be relentless!
    2. Re:syslog, not ssh+tail -f by allenw · · Score: 1
      That is a very good point. I'm used to dealing with scales beyond a single node ;) where you have access to such things.

      In any case, I'm considering borrowing the idea and using it to 'watch' blocks on HDFS. I think it would be interesting to have a visual of blocks/files getting read/written/replicated. It might show patterns that we're otherwise not seeing.

  25. Green balls keep killing me. by Anonymous Coward · · Score: 0

    Those obstacles keep getting in my way!

    How do I shoot?

  26. User agents and OS by Wabbit+Wabbit · · Score: 1

    This would be very cool indeed.

    I guess we could download the source and do it ourselves!

    I don't know why so many comments were hating on this tool. As a big fan of "visualization" (Tufte books, etc.) I find Fudgie easy to understand and useful. The possibilities here are amazing.

    Kudos to you, Fudgie (er...that sounds kinda bad)

    --
    Nothing is inexplicable; only unexplained -Tom Baker, Doctor Who
  27. Accessing log files via sudo? by Anonymous Coward · · Score: 0

    Our servers don't allow the root account ssh access, and the log file ownership is root.

    Can this be made to work with sudo? (For instance, if the command argument is "sudo tail -f" instead of just "tail -f" then glTail could feed the password to the ssh session a second time? Forgive me, never used ruby.)

    1. Re:Accessing log files via sudo? by Anonymous Coward · · Score: 0

      Yes, this is theoretically possible.

    2. Re:Accessing log files via sudo? by Fudgie · · Score: 1

      Sure it can. You'd just need to send the sudo-command line, and send the password if you got a password prompt in return. Or you could just let other users read the access log for a while, so see how it looks before you decide if this is something you'd like to try.

    3. Re:Accessing log files via sudo? by Anonymous Coward · · Score: 0

      > Can this be made to work with sudo?

      No. The laws of physics would prevent this.

  28. seconded by Cheesey · · Score: 3, Insightful

    Remote syslog also means that your servers are more secure: (a) because it is harder for crackers to falsify remote logs as they need to compromise two machines, not just one; and (b) because your visualisation program doesn't need access to SSH keys for all of the machines it monitors, so a compromise on the visualisation computer doesn't automatically mean that all of the servers can also be compromised. However, you could presumably adapt this tool to use syslog quite easily.

    --
    >north
    You're an immobile computer, remember?
    1. Re:seconded by piranha(jpl) · · Score: 1

      I'm sorry; I know your comment is old, but: no. No, no. No no no no no.

      syslog is insecure; messages are unauthenticated. Don't believe me? Use the logger(1) utility to forge a message from any daemon on your system, as an unprivileged user. Send a UDP packet to an open syslog daemon to forge a message to look as if it came from any daemon on the originating host. Forge that UDP packet as if it came from any system in the world; there's no two-way handshake to verify the path to the sender is legitimate, like as with TCP.

      The only thing you couldn't do—given that the open syslog server is secure—is forge a message time, reliably cover up old log messages, or forge a message from a host where IPsec is used to protect against forgery. (Since syslog uses UDP, an unguaranteed delivery protocol, an attacker could use a DoS attack on the syslog server to overwhelm it with bogus messages, causing it to drop real messages. Or it could drop them anyway. UDP doesn't guarantee a thing.)

      DJB gets this right. daemontools devotes a separate security context—a separate user—for the logging needs of every single service that might wish to log a message. Filesystem security ensures no program can forge a message to look as if it came from an unrelated daemon.

  29. Re:SSH - jeesh kill the dinosaur by slyborg · · Score: 1

    Uh, wtf are you talking about?

  30. Re:SSH - jeesh kill the dinosaur by Anonymous Coward · · Score: 0

    Ignore the troll. Anyone with a PETA link for their site is obviously some type of sub-intelligent humanoid.

  31. Running glTail on Windows by Mazin07 · · Score: 5, Informative

    If you want to run glTail on Windows:

    1. Use the One-click Ruby installer from rubyforge (not Cygwin ruby)
    2. Make sure to `gem install net-ssh`
    3. Change "require 'glut'" to "require 'glut_prev'" to enable legacy GLUT ruby bindings

    Took me a while to figure this out.

    1. Re:Running glTail on Windows by 0racle · · Score: 1

      Thank you.

      --
      "I use a Mac because I'm just better than you are."
    2. Re:Running glTail on Windows by 0racle · · Score: 1

      Actually, now it runs but I get nothing but a blank screen on both Windows and OS X.

      --
      "I use a Mac because I'm just better than you are."
    3. Re:Running glTail on Windows by Fudgie · · Score: 1
      Probably one of these:
      • The parser you've chosen is unable to parse the log due to a wrong logfile format
      • You've failed to enter the path to the logfile correctly
      • net-ssh is failing to let you know about some problem with logging in
      Enable session_options[:verbose] = :debug and see what that prints out.
  32. Audialization by Aleksej · · Score: 1

    fastfinge> I used to have a program that would play a musical note every time someone hit a port. so for each port it would have a different note
    fastfinge> i put it in the dmz
    fastfinge> much musical entertainment
    fastfinge> I should find the source for that thing again. i could change midi intruments depending on the type of packet.
    fastfinge> or maybe create length and timbre data from the source IP?
    2006-09-20

    1. Re:Audialization by Wilk4 · · Score: 1

      cool... as I was looking at this I was thinking it ought to have sound...

      same principles as the visual could work perhaps, small requests get higher pitch discrete sounds, larger get lower (kettle drum for largest ;-)

      ... the fast the requests come in the faster the 'dits' sound...

  33. And replace it with what? by SanityInAnarchy · · Score: 1

    RDP? VNC? RSH???

    --
    Don't thank God, thank a doctor!
    1. Re:And replace it with what? by MarkRose · · Score: 1

      WTF? OMG? BBQ???

      --
      Be relentless!
    2. Re:And replace it with what? by DrSkwid · · Score: 1

      Those mechanisms are for RPC, you can choose whatever method you like when your whole IP stack is SSL encrypted, like mine.

      I'm constantly surprised at what people will plod along with!

      --
      There are places where the networks are not touching,and there are places where they are-Boeing's Lori Gunter
    3. Re:And replace it with what? by SanityInAnarchy · · Score: 1

      you can choose whatever method you like when your whole IP stack is SSL encrypted, like mine.

      That doesn't make it secure. SSH also has an authentication protocol, and it's per-user. If yours is per-IP-stack, you already lost -- both because we already have that (in the form of VPNs and ipsec) and because it's not secure (anyone who can connect to the server can authenticate).

      So once again: What method of RPC would you use instead of SSH? Telnet? Last I checked, it won't do RSA authentication -- it's really hardly more than netcat.

      --
      Don't thank God, thank a doctor!
    4. Re:And replace it with what? by DrSkwid · · Score: 1

      9p with centralised separately encrypted authentication, thank you for asking.

      --
      There are places where the networks are not touching,and there are places where they are-Boeing's Lori Gunter
  34. Booooring. by Angstroem · · Score: 0

    We did something similar like 10 years ago, hooking the log-file to the sound server where each port hat its individual sound and the frequency of connects directly related to the respective sound's volume.

    Was rather interesting as you actually could *hear* all those Windows trojans and worms trying to dig their way into your (Linux) system.

  35. Google called by Anonymous Coward · · Score: 2, Funny

    They heard about your cool project and want to subject you to a series of tedious interviews, ultimately not offering you a job because you didn't go to stanford.

  36. Not impressed by Anonymous Coward · · Score: 1, Insightful

    A weekend
    It *really* shows that this was hacked together over a weekend. I've spent 15 minutes trying to get it to run, and all I see are Ruby warnings about about obsolete code, and failed dependencies. I've installed about a dozen packages to try to satisfy this beast's dependency hunger, but to no avail. Behold:

    $ ls /usr/lib/ruby/1.8/net/ssh
    connection lenient-host-key-verifier.rb service userauth
    errors.rb null-host-key-verifier.rb session.rb util
    host-key-verifier.rb proxy transport version.rb
    $ ./gl_tail-0.02.rb
    ./gl_tail-0.02.rb:55:Warning: require_gem is obsolete. Use gem instead.
    /usr/lib/ruby/1.8/rubygems.rb:312:in `report_activate_error': Could not find RubyGem net-ssh (>= 0.0.0) (Gem::LoadError)
    from /usr/lib/ruby/1.8/rubygems.rb:246:in `activate'
    from /usr/lib/ruby/1.8/rubygems.rb:75:in `active_gem_with_options'
    from /usr/lib/ruby/1.8/rubygems.rb:60:in `require_gem'
    from ./gl_tail-0.02.rb:55
    $
    Nothing to see here folks, just some Works On My Machine crap by some Ruby fanboy who can't keep pace with Ruby's latest whims about how people should write their code.
    1. Re:Not impressed by Fudgie · · Score: 2, Informative
      Try and install the gem version of net-ssh or change the require_gem to plain old require so you use the packaged net-ssh instead? I've got net/ssh in

      /usr/lib/ruby/gems/1.8/gems/net-ssh-1.0.10/lib/net/ssh
      and

      /var/lib/gems/1.8/gems/net-ssh-1.1.2/lib/net/ssh
      depending on which Ubuntu version I'm running.

      It's not hard, and quite a few have been able to get it running on Linux, OS X and Windows. FreeBSD is still a no-go.

  37. I hate flash! by miknix · · Score: 0

    Damn! I hate flash..

    1. Re:I hate flash! by Fudgie · · Score: 2, Informative

      Grab the divx version of the movie, then.

  38. Next ? by billcopc · · Score: 1

    Of course, the logical next step is to make it into a game. Log File Centipede!!!!1!11

    Curiously, I spent a good half-minute staring at the animation. For a busy site this could actually be really useful, to watch a slashdotting as it unfolds in real-time, and to a skilled sysadmin it could offer the kind of timely info required to properly tune the server to survive the bursts.

    I'm not going to start watching this thing 8 hours a day (though some drones would), but it definitely does have limited uses that aren't currently matched by other analysis tools.

    --
    -Billco, Fnarg.com
  39. Two words... by John+Whorfin · · Score: 1

    Movie OS

  40. not as eye candy as Netcosm by catmistake · · Score: 1

    netcosm youtube vid

    --
    The Admin and the Engineer
  41. 64 bit prob? by Anonymous Coward · · Score: 0

    ./gl_tail-0.02.rb:479:in `BitmapCharacter': integer 206452306528 too big to convert to `int' (RangeError)
                    from ./gl_tail-0.02.rb:479:in `render'
                    from ./gl_tail-0.02.rb:479:in `each_byte'
                    from ./gl_tail-0.02.rb:479:in `render'
                    from ./gl_tail-0.02.rb:581:in `render'
                    from ./gl_tail-0.02.rb:669:in `draw'
                    from ./gl_tail-0.02.rb:668:in `each'
                    from ./gl_tail-0.02.rb:668:in `draw'
                    from /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in `to_proc'
                    from ./gl_tail-0.02.rb:805:in `call'
                    from ./gl_tail-0.02.rb:805:in `MainLoop'
                    from ./gl_tail-0.02.rb:805:in `start'
                    from ./gl_tail-0.02.rb:882

    1. Re:64 bit prob? by Fudgie · · Score: 1

      Ah, finally someone else with this error. Been bugging me, and the person that reported it didn't mention running 64bit. That could be it. -- Erlend

    2. Re:64 bit prob? by Fudgie · · Score: 1
      Solution is to install the latest ruby-opengl from gems, and replace all

      GLUT.BitmapCharacter(GLUT::BITMAP_8_BY_13,c)
      with

      GLUT.BitmapCharacterX(c)

      The next version will have an auto-fallback to this function if the exception is raised.

  42. Inverted spludge by bigpurpledick · · Score: 1

    I'd like to see a dDOS in action with this

  43. Fish, you gotta have fish by Tablizer · · Score: 1

    people love fish-tanks

    --
    Table-ized A.I.
  44. Move of the slashdotting... by Fudgie · · Score: 1

    Is available at the site.

  45. Netcosm doesn't exist yet by andrewd18 · · Score: 1

    The thing is, Netcosm is officially a "research project" that isn't public, it's not something you can install and use today.

    http://www.networkperformancedaily.com/2007/04/why_netcosm_is_not_a_product_y_1.html