Anatomy of the HBGary Hack

PCM2 writes "Recently, Anonymous took down the Web sites of network security firm HBGary. Ars Technica has the scoop on how it happened. Turns out it wasn't any one vulnerability, but a perfect storm of SQL injection, weak passwords, weak encryption, password re-use, unpatched servers, and social engineering. The full story will make you wince — but how many of these mistakes is your company making?"

Comments

Awesome

[cs668]

The story of their being hacked and how it was done has probably done more for systems security than they as a company ever have......

Re:Awesome

[hairyfeet]

Don't you just love it? Whether you are for the Anon guys or not you just gotta love a bunch that advises others on security that falls for every single bad practice in the book. They had badly coded CMS that didn't sanitize squat, no real rules when it came to passwords, passwords badly hashed, reuse of passwords, just on and on it is like a comedy of fail!

I have to agree with you that this should be a valuable life lesson for those that haven't paid attention before. Of course I figured that by the time SQL injection tricks had gotten so common XKCD was doing the "Bobby Drop Tables" bit that surely everyone had learned to sanitize? Apparently not and how sad but funny that it was a security group that was such a king of fail. Its like having the town drunk lecture you on responsible drinking while killing his second bottle of Jack! its just too funny!

Definitely interesting....

[jesseck]

I've been following this since I heard of it happening- definitely interesting. I like the idea of a custom CMS to avoid an open one (more security). And the poor admin who gave out root, dropped firewalls, and gave up the correct username all via email- that's a bummer. I bet that will be among his "worse day ever" collection. As for shared passwords, I'm sure a lot of us work at guilty companies. Hell, active directory exists partially to address the need for multiple passwords. In all, I enjoyed reading how it was done- quick, efficient work.

Re:Definitely interesting....

[NevarMore]

I like the idea of a custom CMS to avoid an open one (more security).

Its far easier to audit existing code than it is to build your own code. Even if you write it yourself you have to do the same auditing and testing that you would against an existing product.

Re:Definitely interesting....

[nodwick]

I've been following this since I heard of it happening- definitely interesting. I like the idea of a custom CMS to avoid an open one (more security).

Sadly the moral of the story is the exact opposite - the custom CMS HBGary commissioned was actually less secure, as it appears not to have been subjected to proper security audits, nor was it being updated to patch discovered bugs. Direct from TFA:

Rather than using an off-the-shelf CMS (of which there are many, used in the many blogs and news sites that exist on the Web), HBGary—for reasons best known to its staff—decided to commission a custom CMS system from a third-party developer. Unfortunately for HBGary, this third-party CMS was poorly written. In fact, it had what can only be described as a pretty gaping bug in it. A standard, off-the-shelf CMS would be no panacea in this regard—security flaws crop up in all of them from time to time—but it would have the advantage of many thousands of users and regular bugfixes, resulting in a much lesser chance of extant security flaws.

The very thing you consider a disadvantage in an open software system - the fact that anyone can discover bugs in it - also helps ensure that such bugs are publicized and fixed. With HBGary's custom CMS, the bugs were still there, but the only people looking were the ones specifically trying to break into their system. There can be a case for code obscurity, but if that's all you're relying on to protect yourself, I'd say you're really just burying your head in the sand.

Re:Definitely interesting....

[jamienk]

A non-custom CMS like WordPress is very often the target of massive automated attacks: a new bug is discovered in WP and a tool is written to seek out vulnerable installations and exploit that bug. If you have the skill or $$ to pour over the code, you can probably find your own bugs before they become publicly known.

On the other hand, if your site is specifically targeted, then your custom CMS is as vulnerable or more than the WordPresses out there. You might have a bit of security through obscurity (in a standard WP install, the attacker might know file names and locations, variable names, classes, etc.) but this will probably do you little good if you weren't able to harden the code.

Lesson: you are screwed if a rich, powerful, or smart attacker singles you out. A standard CMS can land you in hot water if you don't have a knowledgeable person administering it (and who has that?).

Re:Definitely interesting....

[Ihmhi]

What happened to HBGary is like a fire station burning down because the smoke alarms didn't work - you'd think they, of all people, would know better.

Re:Definitely interesting....

[PitaBred]

A custom CMS will protect you against most automated attacks against a "generic" CMS. But it will leave you more vulnerable to directed attacks, which is what happened here.

Re:Definitely interesting....

[benjamindees]

It's more like a fire station burning down because the fire chief was being paid by the mayor to make molotov cocktails and throw them at local teenagers and one day they decided to throw one back and instead of putting the fire out the firemen screamed and ran around in circles and poured gasoline on it and the fire station exploded. But, yeah.

Re:Definitely interesting....

[somersault]

You also need to make sure that the library you're using for parameterised queries implements them properly.. some libraries are apparently lazy and just concatenate stuff together behind the scenes rather than doing it the right way.

Re:Definitely interesting....

[SlappyBastard]

Considering the number of hacked major websites I've now heard of storing their passwords in plaintext, my faith in industry standards is shot. When sites the size of Gawker, Reddit and Plenty of Fish fail this really braindead obvious level of security, I think people who implement plain MD5 start to feel like geniuses.

Mistakes

[codepunk]

But how many of these mistakes is your company making?

Most companies probably make these mistakes, all except the biggest mistake which was poking a sleeping bear.

The real mistake

[Fex303]

The full story will make you wince — but how many of these mistakes is your company making?

Well, we're not going after 4chan/anonymous, so we're probably in the clear.

I think the biggest security mistake it's possible to make is antagonizing the largest collection of bored hackers/crackers/script kiddies/associated hangers on that exists.

Re: SQL injection

[naz404]

Looks like they got taken out by Little Bobby Tables...

http://xkcd.com/327

Incompetent

[Anonymous Coward]

I'm just amazed at how completely oblivious "Chief Security Specialist" Jussi Jaakonaho was during the email correspondence, AND that he was perfectly fine with sharing root passwords via plaintext email.

How do these people even get security jobs and be negligent in even the simplest security practices?

Re:Incompetent

[jesseck]

I also wonder though, how much of that was brought on by the corporate culture. My boss doesn't know what SSH is, so him asking about it would be a red flag to me. But executives at HBGary may have used it all the time. And maybe the required root access frequently. All it takes is one previous time of Jussi refusing to pass that info out and resulting in a "we pay your ass, do it when I tell you to!" reprimand, and Jussi will have been changed by the corporate environment to jump when the COO or CEO says to via email. Poor security practices, definitely. But often corporate culture leads to these poor practices. Everyone tries to start out doing the right thing, but often push it aside in favor of "the easy way".

Re:Incompetent

[Vaphell]

I'd hire him with no problem and most probably for peanuts. He got so burned that the paranoia will be eating him alive from now on. Yes, you can learn all you want about good practices and whatnot but sometimes you need to get really hurt to actually LEARN.

And What's next?

[rueger]

Gotta say, the linked article was a great education for me, one who's interested but never had time to dig into some of the arcana of stuff like SQL injection.

In watching Wikileaks, OpenLeaks, Egypt, the Palestine papers,and now HB Gary, I'm thinking that we're at the edge of something monumental. I expect we'll see a lot more formerly secret data become public, and see governments and corporations either clean up their acts, or become increasing desperate and hostile in trying to keep their inside info secret.

Either way we're in for a wild ride!

Re:And What's next?

[gman003]

Well, a Wikileak (that's the term for something Wikileaks leaks, right?) was one of the things that started the Tunisian revolution, which led to the revolt in Egypt, and protests in Algeria, Libya, Yemen, and Bahrain, and it seems to be spreading further, as far away as Iran, and Jordan. Add the fact that some pretty major corporations are also being attacked (), and this could be on the scale of 1848. I'm willing to bet that this chain of uprisings won't stop before it reaches Russia and Italy, and I'm hoping it goes all the way to the US.

We all know that America (hell, most of the world) has needed a major change in government for years now. Decades, even. It isn't bad enough that we need to start lining people against a wall, but at the very least, we need some changes that are big enough that the status quo would be upset.

Re:And What's next?

[LordLucless]

That's the end goal Assange always envisaged for Wikileaks. He wanted to make governments either become more open, or become so inefficient due to the security needed to hold their secrets, that Darwin would see them replaced with a more open one.

Was talked about in one of the interviews he gave.

Re:And What's next?

[gman003]

Quoth Wikipedia: "Another cause for the uprising has been attributed to the inability of the Tunisian government from being able to censor information from reaching the Tunisian people, such as information from WikiLeaks describing rampant corruption in the Tunisian government."

Main cause? No. Contributing factor? Yes. At the very least, it seems like it was the spark that brought all the other factors into focus.

Attack Summary

[Anonymous Coward]

1. SQL Injection

   The exact URL used to break into hbgaryfederal.com was http://www.hbgaryfederal.com/pages.php?pageNav=2&page=27. The URL has two parameters named pageNav and page, set to the values 2 and 27, respectively. One or other or both of these was handled incorrectly by the CMS...

2. Password Hashes didn't use salts etc.
3. Password hashing was done using MD5.
4. Password complexity policy was crap anyway.
5. Password recovery policy was vulnerable to social engineering (insider attack).

Re:Attack Summary

[Flyerman]

You forgot the part where the CEO of HBGFed used the same six letter pass in the CMS, his email, twitter, facebook...

Basically step 4->5 went lousy password to same password used for the email admin to another user's email account to the social engineering.

Re:Attack Summary

[dwarfsoft]

6. After targetting Anonymous they didn't invest in curtains.

7. After targetting Anonymous they didn't invest in a dog.

Surely they saw the FOX11 story on Anonymous when checking out the background of their quarry?

Re:Anonymous

[the linux geek]

Because social engineering is so totally an Uber Advanced Hacking Technique. Anyone who hands out a root password, enables remote root SSH access, and shuts off a firewall because of an email message is dangerously complacent.

They will be famous for a long time

[RelaxedTension]

They are the Tacoma Narrows bridge of the IT security world now. They will be the textbook case example of the generations of students, with the entire repertoire of what not to do every step of the way, especially the one about not pissing-off a malevolent, anonymous mass.

yeah, I can be an ass sometimes

[Thud457]

http://bobby-tables.com/

Re:Help me out here

[nedlohs]

http://lmgtfy.com/?q=SQL+injection

Re: SQL injection (I'm confused)

[Anonymous Coward]

You're missing something.

http://www.hbgaryfederal.com/pages.php?pageNav=2&page=27

Obviously the 2 and the 27 are not being validated before being appended into part of a larger SQL query, so construct your own URL substituting 2 (or 27) with something like 2';show tables; --

Find the one that looks like it contains user login information and then substitute again with 2';select * from user_table; --

Hey presto, you can now read all the user accounts and hashed passwords.

Re: SQL injection (I'm confused)

[GNUALMAFUERTE]

They are giving you the original URL where the injection was used, not a link to the actual injection.

They probably replaced some of those parameters with the injection code.

page is probably how many results per page they want, and pageNav is what page they want, so probably page landed straight into a LIMIT in a sql query, without any kind of treatment. Most likely, just passing that crap through mysql_real_escape_string() would have been enough.

imagine a conical bath...

[decora]

ok actually.

websites take input from users. like when i log in to slashdot, it asks me for input.

it will run the input through a program, which will talk to a database.

how does it talk to the database? it runs an SQL command, like 'SELECT * FROM TABLE USERS WHERE NAME=$username'

$username for me is 'decora' because thats what i type into my little login box.

but lets say i uhm, type into the 'username' box something like 'decora OR name=cmdrtaco'.

now, instead of just getting my info, it might spit back all of cmdrtacos info too! maybe even his hashed password.

to protect against this, most programs will take measures like:

0. validate input (does the username have spaces in it? reject if so)
1. check the SQL query to make sure its 'safe' and contains no parsable SQL commands.
2. dont write stuff like 'SELECT * FROM', only read stuff you need.
3. validate data returned from the SQL query before printing it to an html page.
  ie. if yr supposed to get one 5 datums back per user and instead you get 10, somethings wrong.

then again all that takes time and money and effort to do.
why bother, if nobody will ever care? the company that made the CMS for HBGary probably
contracted out the programming to some other company that hired people off a website,
(i have no evidence of course).

Re: SQL injection (I'm confused)

[Sulphur]

Watson: What is "http://www.hbgaryfederal.com/pages.php?pageNav=2&page=27" Alex

Re:Anonymous

[CodeBuster]

It's easy to monday morning quarterback this thing but consider the following two points (from TFA):

1. The social engineering portion of the attack originated from Aaron's company gmail account (HBGary used Google Apps for mail), which anonymous had gained access to through the gmail account of the admin who re-used his password from the hacked CMS. So the email to the Finnish sysadmin came from Aaron's gmail account (i.e. Anonymous was effectively impersonating Aaron using his own credentials).

2. The email exchange, which is repeated in TFA, shows that Anonymous used information from Aaron's old emails, including two previous root passwords, to further reinforce the notion that the email did indeed come from Aaron Burr who was in a jam before meeting clients in Europe and needed root SSH access asap.

So while the method itself may not have been sophisticated, the wording of the spear phishing messages, carefully chosen to create just the right combination of credibility and urgency, really was a master stroke. Obviously Anonymous has a few people who have done this before. Besides, have you ever tried to make credible pretext emails or phone calls to social engineer information? It's harder than it looks.

How Many Of Those Mistakes is My Company Making?

[Greyfox]

I think the big one is my CEO ain't talking shit about a bunch of hackers who are better at it than him.

Re:Morals?

[Zancarius]

Revenge is almost never the right thing to do. It is a thing to do, and it's an unfortunate human trait that the overwhelming majority of us feel when we have been wronged or perceive that a wrong has been committed against us. I've posted in a previous thread about my thoughts related to this, but I can essentially sum it up by stating that I think it was wrong what Anonymous did primarily because they have may have hurt many more people than just Barr. Though I do admit that I can't think of a more deserving target than Barr given his arrogance that could have lead to hurting innocent people with his own hands. And I think that's one of the more frightening underpinnings of this story that isn't getting due press--what happens if a (rogue) researcher is so confident he's identified a dangerous group of hackers that he's willing to do whatever it takes, even if his assumptions were completely wrong? What if they're so hungry for clients or press that they don't care if innocent people get slammed?

The other thing that surprises me about Barr is that he must have been greatly ignorant of basic childhood rhymes. There's plenty of sayings that come to mind that I'm sure he's heard before: play with fire, and you're going to get burned; don't poke the bear; let sleeping giants lie; and the list goes on and on.

Back to the discussion: sure, it's surprising that a security/consultancy firm like HBGary was hit so hard by something as simple as the attacks outlined in TFA, but I think it is far more surprising to me that Mr. Barr did not see this coming. From the previous articles I read on Ars Technica, it seems to me that the only level-headed person in the whole ordeal was Barr's programmer--a man who warned Barr numerous times not just about possible retaliation but that the names Barr had accumulated were almost certainly innocent people. But Barr was certain that they were the right names. Can you imagine the damage he would have done if he released them publicly or to the FBI (assuming that the FBI would take him seriously--I'd like to believe they wouldn't, but given the recent DHS fiasco with FreeDNS I have my doubts)? The potential for some poor innocent bystander who happened to friend the wrong person on Facebook may have found their doors knocked down at 6AM for something they didn't do. All that because of Barr's certainty he was right.

I guess it just surprises me that a security company whose job it is to analyze malware and is almost certainly well aware of the personality profile of the typical attacker didn't see this out of control freight train. I know that doesn't justify thoroughly destroying a company, but I don't think they're particularly deserving of much sympathy either. Barr's programmer warned company executives in the e-mails as reported by Ars, and I seem to recall at least one exchange where one of the higher ups told Barr to back off. He didn't, and he cost all of them dearly.

It's not unlike having a family reunion at the zoo where one of the overly curious bull-headed adolescents decides it would be funny to open the bear cage and poke it with a stick or throw rocks at it. He is then surprised when he and his entire family is mauled.

Family member: "Aaron, what are you doing?"
Aaron: "I'm going to poke the bears."
Family member: "Don't do that. They'll get angry."
Aaron: "I just want to open the cage."
Family member: "Are you serious? That's got to be the stupidest thing I've ever heard."
*bear cage opens, bear gets poked*
*assorted growls and screams*
Weeks later, in recovery; Aaron: "I don't know why it was so angry..."

Regardless, I'm with a couple of the previous posters. This is going to go into college books for the next 30-50 years as an example not unlike Enron. Further, as someone else also pointed out, Barr probably did more to further educate the technology-minded masses on exploits in a single week by screwing himself over than he has in decades.

On the plus side, I doubt he'll be poking the bear cage any time soon.

That was a great article

[dave562]

It's on par with what Sterling wrote in The Hacker Crackdown.

Re:Morals?

[Chas]

Who started with the vigilantism here?

Aaron Barr at HBGary. He's not law enforcement and as far as I know wasn't under contract by any law enforcement agency to root out the members of Anonymous.

Yet he's threatening to name names. To accuse people of participating in disruptive, possibly criminal activities.

Not in a court of law. But in public.

He's going all "Wild West" on people here and threatening to "pull his gun".

In this case, Anonymous responded in kind and Aaron Barr, shootist, is now laying in the street in a puddle of his own blood.

Unfortunately, Anonymous brought a gatling gun to a pistol fight. So lots of other people have huge bullet holes blown in them too.

Now I deplore "hacktivism" as the WORST possible way to convey one's message to people.

But I'm VERY familiar with the notion of making it painful for people who're harassing you to continue to do so.

What Anonymous did was wrong. Make no mistake about it.

But what did these jackholes THINK was going to happen?

Re: SQL injection

[PawNtheSandman]

HBGary and HBGary Federal are 2 different companies, that are related. HBGary was the one hacked.