Bots Scanning GitHub To Steal Amazon EC2 Keys

New submitter juniq writes: As one developer found out, posting your Amazon keys to GitHub on accident can be a costly mistake if they are not revoked immediately.

"When I woke up the next morning, I had four emails from Amazon AWS and a missed phone call from Amazon AWS. Something about 140 servers running on my AWS account. What? How? I only had S3 keys on my GitHub and they where gone within 5 minutes! Turns out through the S3 API you can actually spin up EC2 instances, and my key had been spotted by a bot that continually searches GitHub for API keys. Amazon AWS customer support informed me this happens a lot recently; hackers have created an algorithm that searches GitHub 24 hours per day for API keys. Once it finds one it spins up max instances of EC2 servers to farm itself bitcoins."

Comments

Now where have I heard this before...

[JJJJust]

Oh right: http://developers.slashdot.org...

Re:Now where have I heard this before...

[WWJohnBrowningDo]

There ought to be bots scanning /. to weed out duplicates.

Lesson: don't use root AWS API keys

[heypete]

AWS strongly discourages the uses of root API keys, as they give bad guys who find them the "keys to the kingdom". Why should the credentials for one's S3 account also work for creating EC2 instances?

Amazon provides extensive control over access credentials through IAM, so one can create (for example) an S3-specific user with limited privileges and generate API keys for that user. If they get compromised, the bad guy has limited access: they might be able to add new files to S3, which is bad, but it's less bad than them spinning up hundreds of servers for nefarious purposes, deleting all your files, etc.

Judicious user of IAM can also reduce user errors: I use Amazon Glacier for backing up certain critical files (e.g. wedding photos, baby photos, copies of wills, passports, etc.). I created an "upload, view, and restore/download" user for Glacier that explicitly does not have the "delete" permission enabled. I have a second IAM user with "view and delete" permissions. API keys for both users are stored in FastGlacier, with the "delete" user credentials stored encrypted so I need to enter a password to switch to that user. The user without delete permissions is the default user and the credentials are not stored with a password. This way I can do the standard backup/restore functions needed while working with backups but significantly reduce the possibility of my accidentally deleting backed-up files if I fat-finger the wrong key.

Re:Lesson: don't use root AWS API keys

[Anonymous Coward]

they could save themselves a lot of trouble by making a web UI for IAM that isn't shit, and actually documenting the system somewhere

Re:Lesson: don't use root AWS API keys

[Anonymous Coward]

This may be anon but mark it up... very insightful. Yes it is important to create role based accounts/permissions for AWS through IAM, but abso-fucking-lutely the IAM UI and the process for modifying permissions/authorization is positively fucking shit. Garbage. Crap. A cankerous sore on the taint of the internet. It's not very good. And the documentation for it is so poor to the point of being virtually non-existent. I did use it, and on an AWS project I worked on made sure we had specific accounts created permissions limited depending on the roles required. But it is basically hacking at the configuration until you get it right because it is the only real way to figure it out. Not what you want when it involves security. It is too easy to fuck up and leave yourself vulnerable... never mind the people who take one look at it and say WTF? and not set up security at all and just use the root access key/secret key. Saying that this is something that only experts should use so it's OK to be cryptic is not the answer... if only security experts should set up accounts Amazon would likely lose 75% of their business (i.e. a lot of it).

--theshowmecanuck ... posting anon because not only did I want to mod parent up, but hopefully someone from amazon reads /. and will help get their shit together on this.

HACKERS ARE SCANNING 24/7

[Anonymous Coward]

HACKERS, I TELL YOU

Sounds like multiple failures

[rebelwarlock]

But the user is still mostly to blame. Okay, so you might not find it intuitive that S3 keys can lead to new EC2 instances - I wouldn't have guessed that either, even though I've used both repeatedly. Maybe that shouldn't even be a possibility. But what howling insanity persuaded you to put those keys on github in the first fucking place? And if it was a mistake, why didn't you change them after? This isn't amateur hour, guys - there's real money at stake here.

Re:Sounds like multiple failures

[Paradise Pete]

This isn't amateur hour, guys - there's real money at stake here.

People make mistakes every day of their lives. We always have, we always will. It's how we learn what not to do. It's just that almost all mistakes are harmless... except on the internet. There it's like living in a minefield. Make a bad step and boom. It's not a question of amateur hour, it's a question of being human.

I guess i am old

[codepigeon]

I guess i am too old to understand how loose people treat the internet these days. 'I posted my credentials openly on the internet and am now shocked that I have been taken advatage of'... no way! You shared the keys to your kingdom and someone abused it?? Shocking.

As a complete side note: I hate when people like the author don't know the difference between 'where' and 'were'....fuck, no wonder he was easy fodder

Re:I guess i am old

[Kjella]

I guess i am too old to understand how loose people treat the internet these days. 'I posted my credentials openly on the internet and am now shocked that I have been taken advatage of'... no way! You shared the keys to your kingdom and someone abused it?? Shocking.

He wasn't surprised they were abused. He's surprised what he thought was keys to S3 unlocks the entire kingdom. Like you lose a card for electronic bus tickets and you think the worst they can do is ride the bus for free. Except the card doubles as a "secure ID" and they order thousands of dollars worth of goods from cooperating merchants, which you didn't even know was possible. Sure posting them on Github was ignorant and entirely his fault, but if you try to assess the risk then it's probability * impact and I think he just got a big surprise when it came to impact. I think you're being too hard on him.

Horrible design on part of Amazon

[iamacat]

When you sign up for a developer account, you should be asked how much you plan to spend per month. $2375/day would not be a common option for an individual. Given proliferation of free 15GB storage accounts, a very low end developer account with no credit card is not a crazy option. People will learn the API and use it in future, but neither them nor hackers will have enough quota to run a production site. This is just like limited data cell plans where a single buggy app can run up crazy charges. Good that they refunded money, but fundamental structural problem must be fixed.

Re:Summary without technobabble

[mwvdlee]

More like don't upload your PRIVATE keys to a PUBLIC repository.

Re:The Stupidity of Human Greed

[tlhIngan]

I assume the idea is that you make more money stealing $1 many times from more people over a year than you do trying to steal all of it from all of it at once.

It works in real life all the time, actually - companies do this quite routinely. Jack up your bill by $2 and they can rack up millions over the year, and it doesn't matter if it's a contract or not because how much are you going to spend trying to recovery $24/year? If it's a cellphone contract, the max they're going charge you is $48 more over the contract's 2 year lifespan. You going to sue them over that? Now repeat that for a million subscribers and that's an extra $2M a month in free profits.

Maybe you can jack it up by $5 ($60/year) because it's still too low to bother.

It's why they made class-action lawsuits, because someone stealing $48M/year would get sued/arrested/etc if it was against one person, but against 1M people? Worthless.

As for this, well, given it's still "free money", even at Bitcoin's deflated value of around $350 or so, it's still free money. Who cares about efficiency or anything when you can steal CPU cycles like that - just scan github checkins for the key, then use the APIs to automatically create sessions and all that and rack up the bitcoins. Even the github scanner doesn't have to be owned by the user - they probably stole some guy's EC2 credential and are using one of his instances for it unbeknownst to the user. Free money!

Give the man some slack

[hankwang]

To all posters who are blaming the man for being so stupid: please RTFA. He had just opened an amazon AWS 1-year free trial to practice what he'd just learnt about Ruby on Rails. He made a mistake:

I knew my API key needed to be safe, so I installed the Figaro gem (a rails API key security gem, which typically works great), and trusted it to keep my API key off of git when I pushed. (...) deleted all traces from GitHub. I was able to clean it up within about 5 minutes (...) After a close call, I went to bed.

Surely it is not that unreasonable to (1) realize that those keys will be scraped within 5 minutes after uploading to an obscure project, and (2) not realize that an S3 key in a free trial subscription wouldn't allow racking up $2375 in EC charges within 10 hours?