Slashdot Mirror
Crazy Firewall Log Activity — What Does It Mean?
arkowitz writes "I happened to have access to five days worth of firewall logs from a US state government agency. I wrote a parser to grab unique IPs out, and sent several million of them to a company called Quova, who gave me back full location info on every 40th one. I then used Green Phosphor's Glasshouse visualization tool to have a look at the count of inbound packets, grouped by country of origin and hour. And it's freaking crazy looking. So I made the video of it and I'm asking the Slashdot community: What the heck is going on?"
Comment removed based on user account deletion
Wait, is this just an advertisement for Glasshouse? The voice in the video on Green Phosphor's website is exactly the same.
What gives?
it means that this is an ad for Quova and Green Phosphor's Glasshouse
Am I the only one who found the five minutes of this video to be about as interesting as listening to a stoned person describe the cracks on the ceiling?
You designed the visualization, buddy. If it's "freaking crazy looking," rather than yielding any useful insight, then obviously you did not visualize it in a meaningful way. You failed, in other words.
But as an earlier poster noted, this is just a Slashvertisement for the visualization tool in question. No doubt it will be quite effective on the kind of people who talk as slowly as the guy in the video.
Breakfast served all day!
(this is a guess, obviously. Full netflow data would tell me more, but only way to be really sure would be a full packet trace)
This just shows that you're being scanned with random source IP adresses (that's why the vertical stripe lights up). It is essentially a check to see if part of the botnet has more firewall access than other parts, or if a loadbalancer directs stuff to different firewalls, or if you have additional BGP uplinks, some of which might not be quite as secure.
Then the real scan starts, which uses the information gained in the first phase to make sure it tests out all the firewalls the target network has. Especially in the case of backup bgp links, where traffic comes in on physically and administratively different lines (say 1 verizon, 1 at&t, if you've got money to burn, and most govt. idiots feel the need to burn money). If the company in addition to the multiple uplinks outsources firewalls to those ISPs (or "security", not knowing what they're buying and getting nothing more than a smug false sense of security), again this is done by too many govt. agencies, you are bound to find holes this way. This uses actual bandwidth, and cannot be done on some networks. So what you're seeing is a disproportionate amount of scanning traffic coming from countries with fast networks and few watchful netadmins (or netadmins that just don't care, in Turkey's case), and many unsecured computers (and dear God, Turks and Russians really do not see any need for virusscanners, but generally you'd see a few other countries in there too. Heh the Russians are probably worried that running a virusscanner will interfere with their development of new viruses)
The regular repeats of vertical lines are probably to rescan reachability information, in case something changed. BGP can be twitchy, especially with incompetent local admins (on the botnet side of the network I mean)
From the (low) speed of the attack you can further deduce that it was an advanced attack, meant to stay below rate limiters, and presumably meant to stay below the radar. And from the resources required to pull this off you can deduce that this was not a lone hacker. Perhaps an organization (these days, tracing source ip's for security attacks almost invariably yields an IP address in far inland China, which is not because the russians have stopped attacking networks, but the Chinese are putting quantity above quality it seems these days).
And frankly, if someone has this kind of patience, generally they will find at least something, even in a well maintained network. Best hope it was only some files left out in the "public" folder or ~username folders. It's a good bet they probed the network security in other ways too (esp. googling), with IP's that will tell you much more about where the attack is coming from (using many hops is possible, but results in very slow page loads. And we're all human)
Btw : looking up a net's country can be done quickly via dns, no need for external company, no need for any tax dollars :
[kimmy@t61 ~]$ host -t TXT 104.79.125.74.cc.iploc.org
104.79.125.74.cc.iploc.org descriptive text "US"
(don't forget to reverse the IP address : looking up 1.2.3.4 is done by host -t TXT 4.3.2.1.cc.iploc.org)
"I happened to have access to five days worth of firewall logs from a US state government agency..."
"While skimming through my grandmother's cookbook, I stumbled upon a recipe for processing yellowcake uranium..."
"In passing, a close personal friend mentioned to me that he would deploy ~30k troops to a Mideastern country, but he's worried that the local restaurantuers won't serve fresh babaganoush ..."
"While I was talking to a famous adult film star about my successful experiment with cold fusion..."
"I was fighting against an alien invasion of the Soviet Union the other day. Natalie Portman and I prepared a platoon of sharks with frickin' hotgrits cannons on their heads, but the unwelcome overlords kept jumping the sharks..."
Why baffled? This is naught more than an advert for a graphic log analysis filter riding on the coattails of the google / China thing.
There are many others that go about the same task in different ways, most are free, this one is not.
You want complaining? How about this: This visualization is terrible.
The video took five minutes to watch and most of it was him rolling over the bars in the 3-D chart so you can see what each of the lines means. If that's supposed to be a useful visual aid, I'll eat my hat. It's bad enough that you have to manually roll over every data element to figure out what it is; scrolling through the graph seemed dead slow. I hope that's not a limitation of the product itself.
Simple labels on the axes of the graph would have been nice. Far be it from anyone to try stick little flags next to the lines to represent different countries. Hell, just color-coding them in a totally arbitrary way would have made the graph easier to read.
BTW, a quick look at the Glasshouse site reveals all their output looks pretty much just like this demo. And there's no evidence that you can export one of their rudimentary 3-D graphs to "pretty it up" in a real 3-D app. Instead, their raison d'être appears to be allowing you to run around looking at these graphs... in Second Life.
I'm sorry, but if you're doing something like plotting fractals, for example, where visual similarity to patterns is the whole point, I can forgive you for coming to the conclusion that "it's crazy looking." If what you're doing is trying to provide a visual to aid in the interpretation of data, then the visual should -- y'know -- aid interpretation. A glance at this graph, on the other hand, reveals nothing; not even what it's supposed to represent.
In summary, Edward Tufte will be rolling in his grave when he dies from looking at this graphic.
Breakfast served all day!
I don't even know why they Quova crap is mentioned since you can look up the country for *each* your IP locally using GeoIP.
c++;
I see no reason whatever that it would be necessary to use either Quova or Green Phosphor. Any competent programmer could have sampled the data, used whois to get location, and then used about 1000 different programs to visualize the data just as well. (Like Crystal Reports or Seagate.)
The fact that OP did neither, and is involved at a high level with one of the two companies, makes this whole post suspicious.
My best guess is that OP thought he had discovered a way to freely advertise via Slashdot, and victimized us as a result.
I get enough Spam. I don't need to see even more, on Slashdot. Can this user be blocked?
Bingo. My thoughts exactly.
Unless his gives up some more data, hard to tell for sure.
But, I agree, it sounds like someone is using their employer's (government)bandwidth to torrent. Could be a machine that someone shuts off the monitor on but P2P downloads overnight with a scheduled P2P app.
The peaks/valleys might be explained by reset packets introduced by the ISP temporarily killing the outbound requests and it takes the inbound requests awhile to trickle off.
You can see this same type of log traffic by simply starting a torrent, waiting a little bit, then stopping the P2P client, waiting awhile again, then restarting it. Rinse, repeat and you will see something that looks awfully close to what you have.
Reset packets essentially create the same traffic pattern, but for a different reason (ISP- introduced traffic "shaping").