Slashdot Mirror
Bots Scanning GitHub To Steal Amazon EC2 Keys
New submitter juniq writes: As one developer found out, posting your Amazon keys to GitHub on accident can be a costly mistake if they are not revoked immediately.
"When I woke up the next morning, I had four emails from Amazon AWS and a missed phone call from Amazon AWS. Something about 140 servers running on my AWS account. What? How? I only had S3 keys on my GitHub and they where gone within 5 minutes! Turns out through the S3 API you can actually spin up EC2 instances, and my key had been spotted by a bot that continually searches GitHub for API keys. Amazon AWS customer support informed me this happens a lot recently; hackers have created an algorithm that searches GitHub 24 hours per day for API keys. Once it finds one it spins up max instances of EC2 servers to farm itself bitcoins."
"When I woke up the next morning, I had four emails from Amazon AWS and a missed phone call from Amazon AWS. Something about 140 servers running on my AWS account. What? How? I only had S3 keys on my GitHub and they where gone within 5 minutes! Turns out through the S3 API you can actually spin up EC2 instances, and my key had been spotted by a bot that continually searches GitHub for API keys. Amazon AWS customer support informed me this happens a lot recently; hackers have created an algorithm that searches GitHub 24 hours per day for API keys. Once it finds one it spins up max instances of EC2 servers to farm itself bitcoins."
Oh right: http://developers.slashdot.org...
How? They got $2000 worth of cpu time from amazon. If they'd waited the guy would have surely thought to change the damn API key the next morning (surely no one is dumb enough, to think just removing the stuff after the fact will help - I guess they also don't bother getting new credit cards when their wallet gets lost and returned by a nice stranger that very same day).
AWS strongly discourages the uses of root API keys, as they give bad guys who find them the "keys to the kingdom". Why should the credentials for one's S3 account also work for creating EC2 instances?
Amazon provides extensive control over access credentials through IAM, so one can create (for example) an S3-specific user with limited privileges and generate API keys for that user. If they get compromised, the bad guy has limited access: they might be able to add new files to S3, which is bad, but it's less bad than them spinning up hundreds of servers for nefarious purposes, deleting all your files, etc.
Judicious user of IAM can also reduce user errors: I use Amazon Glacier for backing up certain critical files (e.g. wedding photos, baby photos, copies of wills, passports, etc.). I created an "upload, view, and restore/download" user for Glacier that explicitly does not have the "delete" permission enabled. I have a second IAM user with "view and delete" permissions. API keys for both users are stored in FastGlacier, with the "delete" user credentials stored encrypted so I need to enter a password to switch to that user. The user without delete permissions is the default user and the credentials are not stored with a password. This way I can do the standard backup/restore functions needed while working with backups but significantly reduce the possibility of my accidentally deleting backed-up files if I fat-finger the wrong key.
HACKERS, I TELL YOU
But the user is still mostly to blame. Okay, so you might not find it intuitive that S3 keys can lead to new EC2 instances - I wouldn't have guessed that either, even though I've used both repeatedly. Maybe that shouldn't even be a possibility. But what howling insanity persuaded you to put those keys on github in the first fucking place? And if it was a mistake, why didn't you change them after? This isn't amateur hour, guys - there's real money at stake here.
They're going for The Big Dirty. One big score and they're out.
I guess i am too old to understand how loose people treat the internet these days. 'I posted my credentials openly on the internet and am now shocked that I have been taken advatage of'... no way! You shared the keys to your kingdom and someone abused it?? Shocking.
As a complete side note: I hate when people like the author don't know the difference between 'where' and 'were'....fuck, no wonder he was easy fodder
With his money refunded he probably didn't file a police report. Which means Amazon doesn't have to deal with nosy investigators.
“Common sense is not so common.” — Voltaire
I assume the idea is that you make more money stealing $1 many times from more people over a year than you do trying to steal all of it from all of it at once.
Wal-Mart!
I once came up with an algorithm that wasn't very good, so I just commanded the computer to "Enhance!", and it got much better.
When you sign up for a developer account, you should be asked how much you plan to spend per month. $2375/day would not be a common option for an individual. Given proliferation of free 15GB storage accounts, a very low end developer account with no credit card is not a crazy option. People will learn the API and use it in future, but neither them nor hackers will have enough quota to run a production site. This is just like limited data cell plans where a single buggy app can run up crazy charges. Good that they refunded money, but fundamental structural problem must be fixed.
Doesn't matter if their nuclear road flare gets their instances shutdown before a single shitcoin is mined. Given the speed of CPU hashing, even 1000 instances would take days to amount to anything. (the fastest dedicated miner does 6TH/s, and it would take a week to generate 0.5BTC -- worth about $150)
More like don't upload your PRIVATE keys to a PUBLIC repository.
Slashdot social media options: AIM, ICQ, Yahoo, Jabber and Mobile Text. Why no MySpace?
It works in real life all the time, actually - companies do this quite routinely. Jack up your bill by $2 and they can rack up millions over the year, and it doesn't matter if it's a contract or not because how much are you going to spend trying to recovery $24/year? If it's a cellphone contract, the max they're going charge you is $48 more over the contract's 2 year lifespan. You going to sue them over that? Now repeat that for a million subscribers and that's an extra $2M a month in free profits.
Maybe you can jack it up by $5 ($60/year) because it's still too low to bother.
It's why they made class-action lawsuits, because someone stealing $48M/year would get sued/arrested/etc if it was against one person, but against 1M people? Worthless.
As for this, well, given it's still "free money", even at Bitcoin's deflated value of around $350 or so, it's still free money. Who cares about efficiency or anything when you can steal CPU cycles like that - just scan github checkins for the key, then use the APIs to automatically create sessions and all that and rack up the bitcoins. Even the github scanner doesn't have to be owned by the user - they probably stole some guy's EC2 credential and are using one of his instances for it unbeknownst to the user. Free money!
I entirely agree but for some, namely the ones who still use symmetric keys, this has become an old school thought.
In Canada, the government bought 30 millions certificates for all its citizens in oder to authenticate for government on line services for a buck a piece. Total: 30,000,000$
I would have been glad to provide it to them for 10,000$ and guess what? All privaye keys were kept centrally ;-) Us, old school guys just couldn't believe it.
The big thinkers/marketing guys decided that it was just to complicated for citizens to manage and keep their secret key in a secure location.
Everything I write is lies, read between the lines.
Surely it is not that unreasonable to (1) realize that those keys will be scraped within 5 minutes after uploading to an obscure project, and (2) not realize that an S3 key in a free trial subscription wouldn't allow racking up $2375 in EC charges within 10 hours?
Avantslash: low-bandwidth mobile slashdot.
The big thinkers/marketing guys decided that it was just to complicated for citizens to manage and keep their secret key in a secure location
It is. Do not underestimate the ignorance of the common user, especially one who just wants to use their computer. Now if the government had charged $5 and sent a USB key with the certificate on it, maybe the end-user would take more care of it as they understand physical keys in a way that they don't with electronic versions.
Look at how many times you have to use the "I forgot my password" feature. For a service you use once or twice a year, the "forgot my password" link would be the login screen.
Secondly, if all keys are stored centrally, by the government, you can use them to decrypt end-user comms. I think someone must have been thinking ahead!
Thirdly, "bought" 30 million certs? They're the government, they can create their own certs and be their own authority. Then they can outsource the delivery of these to citizens to a private company for only a few hundred million dollars. (a company with a minister on the board as a non-exec director, of course)
I've lived all over the U.S.A. for decades, near plenty of stupid people, and I have never heard anyone use the phrase "on accident" in my life. That's the type of error you see in technical manuals from Asian countries. People do say "on purpose", which can easily confuse people who learn English as a second language into saying "on accident" instead of "by accident".
To spot Americans with two-digit IQs on the Internet, look for two unforgivable homonym goofs: confusion between "they're", "their", and "there", and also between "you're" and "your". Those are big warnings that you're reading something stupid written by a native-English speaker. Foreigners don't seem especially prone to goof over these words, but half-illiterate native English speakers just type the way they hear themselves talk, and if it passes the spell check they'll remain oblivious.
The mistake he made was not understanding the tools he was using. Apparently neither do you.
(1) The key could have been scraped at any time once it was pushed, because you can't actually "delete[d] all traces from GitHub" (some ways are more thorough than others, but nothing is foolproof with Google wandering the earth). He needed to revoke his keys immediately.
(2) There is no such thing as an "S3 key". There are only AWS API keys, which potentially have access to every service that you have enabled (plus the default ones). You need to use IAM profiles to restrict what services they can access, and what rights they have.
Signing up for a service and then using it without reading the documentation is foolish.