One In Six Amazon S3 Storage Buckets Are Ripe For Data-Plundering

tsamsoniw writes "Using a combination of relatively low-tech techniques and tools, security researchers have discovered that they can access the contents of one in six Amazon Simple Storage Service (S3) buckets whose owners had them set to Public instead of Private. All told, researchers discovered and explored nearly 2,000 public buckets, according to Rapid 7 Senior Security Consultant Will Vandevanter, from which they gathered a list of more than 126 billion files, many of which contained sensitive information such as source code and personal employee information. Researchers noted that S3 URLs are all predictable and public facing, which make it that much easier to find the buckets in the first place with a scripting tool."

Comments

I think it's booty

[alphatel]

You have done an excellent job of revealing the very loose fabric of the internet, especially those that would not set their own security properly. However, under current law, you have violated so many laws, with so many more to come, that your best way out is to stand on the last iceberg in the Arctic and hope it does not melt anytime soon. Just to clarify, here's a few of the things you've clearly done, and I don't even have to prove them.

Access and distribution of pornography (surely one of those buckets was full of porn, a felony in 20 countries)
Access and distribution of child pornography (well at least one of those buckets has it, or did, or will one day)
Failure to report a bucket full of child pornography
Conspiracy to distribute
Hacking every country in the world... let me explain, no wait let me sum up.
Amazon has storage in 193 countries
By accessing one you have violated the statutes of every country attacked
This is basically punishable by the rest of your life in prison in every country, except the Vatican, which will send you to hell.
So now you are going to hell, after spending the rest of your life kissing bubba's pants
Unauthorized access (fines from Amazon, billions $$$$ ($100,000 per bucket per country, ouch!)
Future crimes (as the future is soon you are already guilty of:
Discussing a hacking attempt
Intent to hack
Intent to exploit, list exploits, financially gain from exploits

I can't type anymore, and there's no doubt as far as most governments are concerned I'm as guilty as you are by now.

Re:I think it's booty

[Cryacin]

Your tin foil hat's come loose.

Re:I think it's booty

[Anonymous Coward]

It is not a hack if i was not circumventing a protection mechanism.

Work at Amazon? Any reason why Private was not the default setting?

Bad PR huh?

Re:I think it's booty

[WallaceAndGromit]

Did it? I'm not so sure...

http://yro.slashdot.org/story/13/03/18/1641221/41-months-in-prison-for-man-who-leaked-att-ipad-email-addresses

Re:I think it's booty

[Anonymous Coward]

At very least you're looking at 41 months.

Re:I think it's booty

[sgardner]

You have done an excellent job of revealing the very loose fabric of the internet, especially those that would not set their own security properly. However, under current law, you have violated so many laws, with so many more to come, that your best way out is to stand on the last iceberg in the Arctic and hope it does not melt anytime soon. Just to clarify, here's a few of the things you've clearly done, and I don't even have to prove them.

Access and distribution of pornography (surely one of those buckets was full of porn, a felony in 20 countries) Access and distribution of child pornography (well at least one of those buckets has it, or did, or will one day) Failure to report a bucket full of child pornography Conspiracy to distribute Hacking every country in the world... let me explain, no wait let me sum up. Amazon has storage in 193 countries By accessing one you have violated the statutes of every country attacked This is basically punishable by the rest of your life in prison in every country, except the Vatican, which will send you to hell. So now you are going to hell, after spending the rest of your life kissing bubba's pants Unauthorized access (fines from Amazon, billions $$$$ ($100,000 per bucket per country, ouch!) Future crimes (as the future is soon you are already guilty of: Discussing a hacking attempt Intent to hack Intent to exploit, list exploits, financially gain from exploits I can't type anymore, and there's no doubt as far as most governments are concerned I'm as guilty as you are by now.

The content owners granted access permissions to everyone by leaving their S3 buckets public; there's no way stumbling upon information made publicly available is considered hacking. Only the owners of these buckets could be convicted of anything, assuming stupidity is a crime.

Re:I think it's booty

[bloodhawk]

His post was mostly in jest, but many countries really do have extremely strict laws on data access which in many cases amount to if you don't have explicit permission to access the information then you are trespassing/hacking if you access it and poor security or settings by the company in question does not excuse the action. by saying they have scanned so many buckets they really have potentially opened themselves up for all sorts of legal trouble if any countries in question decide to make an issue of it.

Re:I think it's booty

[PRMan]

Tell that to weev. He'll be available to talk to you in 41 months.

Re:I think it's booty

[PRMan]

The content owners granted access permissions to everyone by leaving their S3 buckets public; there's no way stumbling upon information made publicly available is considered hacking. Only the owners of these buckets could be convicted of anything, assuming stupidity is a crime.

Again, tell that to weev, who did the EXACT same thing and has already been convicted of 41 months in prison.

Re:I think it's booty

[GryMor]

Private is the default setting?

Re:I think it's booty

[abirdman]

Tell that to Aaron Swartz. Even if the protection mechanism doesn't work, or isn't implemented, it's still hacking in the eyes of law enforcement. And also note, even if you win you'll spend every cent you have on lawyers.

Re:I think it's booty

[Anonymous Coward]

He could get out for good behavior after serving 85% of his sentence.

Re:I think it's booty

[DragonWriter]

Any reason why Private was not the default setting?

Private is the default setting. Which is why TFS notes that this is about buckets who users had set them to Public instead of Private.

Re:I think it's booty

[Anonymous Coward]

then blame the search engines... This is nothing more than a website. You want to find websites use a search engine.
You want to protect you documents? Don't use a webserver to do it. aka CLOUD
site:*.amazonaws.com

Re:I think it's booty

[Anonymous Coward]

That really depends on the countries laws. some countries require you to have permission to access the data and the data simply being poorly or not secured does not automatically grant you the right to access it.

Re:I think it's booty

[Anonymous Coward]

important note he plead guilty must of had bad lawyers and was scared

Re:I think it's booty

[similar_name]

The Innocent Defendant's Dilema

Re:I think it's booty

[Anonymous Coward]

they are hardly "stumbling" upon it. They had to write a script to find it, even though it was pretty simple to do. content being easy to find and access doesn't protect you from being prosecuted for accessing information you had no right to access. the fact they went out of their way to discover the information completely negates any possibility that they can claim they just "stumbled" upon it.

Re:I think it's booty

[nametaken]

The difference here is most of this shit is supposed to be public.

This whole article is goddamn idiotic. One of the primary uses of S3 is as an asset hosting service for websites. That doesn't mean there are a trillion public files that aren't supposed to be. It means people are using the service. So great, you found a trillion public files. You know what else does that already? Google.

It sounds to me like this "security researcher" is just some asshat that wasted time writing a script, and news outlets have zero technical standards.

Re:I think it's booty

[philip.paradis]

The difference here is most of this shit is supposed to be public.

Most != All. The entire point of this sort of exploit is to draw attention to widespread gross misunderstanding and misconfiguration of services used by a great many developers, content managers, etc. Maybe this is difficult to understand, so allow me to rephrase it: the entire point is that a large number of these files shouldn't be public, and at minimum this demonstrates incompetence on the part of those who put those files into public buckets. More interestingly, depending on the nature of the improperly stored data, the companies who employ these people can be held liable for severe legal penalties related to failure to properly secure their customers' sensitive information.

In short, you're either the sort of asshat who blames security researchers in general for finding and publishing security problems, or you're grossly ignorant of how these things actually work, or both. Have a nice day.

Re:I think it's booty

[Narcocide]

Oh they search engines have been attacked legally too but they usually:

1) are corporations so they can't be sent to jail
2) have resources to defend themselves in court or settle out of court

Re:I think it's booty

[davester666]

It's cute how naive you are.

Re:I think it's booty

[Lennie]

Maybe in your country, not mine.

An analogy of this law is: You will be punishable by law if you open a unlocked door when unauthorized.

Re:I think it's booty

[ewenix]

I think you'll have a hard time getting a tech savvy person to consider this hacking when the users are allowing public access. You'd need to prove that they opened an image file for any of the pron charges. There is no evidence of conspiracy to distribute unless you prove they opened any of the files. You could possibly get in trouble for intent to exploit, depending on the the laws of the country of the owner and yours. (Throw in a jurisdiction issue or two.)

Re:I think it's booty

[Anonymous Coward]

YES. Private is the default setting.

Re:I think it's booty

[Anonymous Coward]

I agree exposing files with confidential information is a security threat and needs to be addressed. But:

1.Amazon S3 has nothing to do with the problem. This problem is not inherent to Amazon S3.
Same thing happens with anonymous FTP servers, with regular web sites, etc.
The author could have scanned regular websites...

2.The problem is people willingly exposing confidential information. This is an ancient issue. nothing new. Still, yes, it needs to be addressed.

3. Amazon S3 was used to generate 'hype' IMHO

4. Files uploaded to S3 are by default private

5. S3 bucket names and files are not 'guessable', because there never was the intention to 'hide' things. S3 is like an 'online' 'filesystem', there's no intention to 'hide' files because files are supposed to be exposed (they mostly are files used in web sites, etc etc). that's what the service is for.

More importantly, security of files is done by means of permissions. Not by obscuring their names, which
would only achieve relative security by obscurity,

When the article mentions that 'it does not help' that S3 URLs can be 'guessed' , the article is
misrepresenting the truth IMHO. This is not accurate. It is not a 'weakness'. This assertion is IMHO incorrect.

I guess the article is trying to compare S3 with services like dropbox or sharefile, where to share a file, you click a button and a 'special' URL is generated to make the file selected public.

However,

A) this 'special' URL and PERHAPS (we'll see) 'not so easy to guess' URL is not adding real security. If you are trusting that, you are doing it wrong. The file is supposed to be public. that's why you click the 'share' button.

B) S3 works in a different way, and it is very clear how it works. It is not comparable to services like dropbox or sharefile, they both have very different use cases.

Again, yes, this is something people are doing wrong evidently, I'm not shooting the messenger, but the article was created to create hype. IMHO.

Re:I think it's booty

[omarius]

This. As soon as I read the stub, my first thought was, "This is exactly what they jailed Weev for." Mod thou the parent up.

Re:I think it's booty

[Anonymous Coward]

Even if the protection mechanism doesn't work, or isn't implemented, it's still hacking in the eyes of law enforcement.

Wait a moment...are you telling me my house is still legally protected against burglary and trespass, even if the locks are broken, weak, easily bypassed, or not installed?

What kind of ignorant luddite came up with that stupid system?

Re:I think it's booty

[cant_get_a_good_nick]

Massively offtopic, but...

"Professor, what's another word for pirate treasure?"
"Well I think it's booty" "booty" "booty that's what it is"

-- Beastie Boys "Professor Booty"

Re:I think it's booty

[sgardner]

There is a huge difference here. The owners of these buckets have given express permission (whether intended to or not) to the public through their privacy settings; the details of which can be found in the ToS and/or Privacy Policies they agreed to when they opened their accounts. On the other hand, weev accessed what is already defined as private data through a loophole, exploit, or whatever you want to call it. There was a way to establish guilt, which is not possible in this case, as the bucket owners have already agreed to a legal contract stating that public storage is what it is—public.

Re:I think it's booty

[sgardner]

Like the others, you are overlooking the fact that the bucket owners marked their storage contents as public, and that they've already agreed to a legal contract which describes what public storage entails. This is not difficult to understand; you guys might want to read articles more thoroughly before commenting.

Speakig of plundering

[Anonymous Coward]

I plundered your moms ass last night.

URLs?

[K. S. Kyosuke]

"Researchers noted that S3 URLs are all predictable and public facing"

I thought that was the whole effing point of URLs/URIs? Whether or not you get authorized to access them should be a completely orthogonal issue...or not?

Re:URLs?

[Nidi62]

Researchers noted that S3 URLs are all predictable and public facing, which make it that much easier to find the buckets in the first place with a scripting tool.

So basically they walked down the street checking door to see which ones were unlocked then looked inside the unlocked houses?

Re:URLs?

[ckedge]

You know that.
I know that.

90% of people who "code" or deal with software -- should not be allowed anywhere near anything that has aspects of security aspects of systems and software.

Good luck trying to find a manager that a) understands that, b) can identify the 10% vs the masses, c) is willing to pay that 10% what they're worth.

Shit, the 50% of managers and "architects" and developers who create unscalable crap are long gone off to their next task before whatever they last created gets to the point of implosion.

Re:URLs?

[BradleyUffner]

So basically they walked down the street checking door to see which ones were unlocked then looked inside the unlocked houses?

It would be like walking down a street and peeking in to public restaurant to see what's on the menu.

Re:URLs?

[wvmarle]

These storage buckets are presumably meant to be private, not public. So the private houses analogy is much better than the public restaurant analogy here.

Re:URLs?

[x3CDA84B]

I thought that was the whole effing point of URLs/URIs? Whether or not you get authorized to access them should be a completely orthogonal issue...or not?

In systems with URLs that contain some sort of object identifier, using a non-predictable identifier is a great way to add another layer of security. It doesn't replace actual authentication or authorization checks, it just complements them.

For example, if I have a REST URL like this:

http://someserver/users/ID

If I use sequential numbers, or actual usernames as the identifier, it becomes trivial for someone to enumerate all of them by iterating through numbers, or a dictionary. However, if the ID is a 128-bit (or longer) random UUID, then that is no longer possible, because it would take millions of years. So even if I (as the developer) make a mistake that allows someone to view or change data that I shouldn't have access to, that attacker may not even get to the point of being able to exploit it, because they may not have any other valid UUIDs to work off of.

This is why Microsoft moved towards using random/non-sequential identifiers for things like IIS website IDs and so forth in the early 2000s. It's one of the few choices of theirs I really agree with.

Re:URLs?

[Anonymous Coward]

Any person that presumes it will be private while setting it to public is a fucking idiot. The private houses analogy is not valid here.

Re:URLs?

[Anonymous Coward]

YES. you are correct.

Re:URLs?

[Anonymous Coward]

This is not true. S3 buckets are not supposed to be private.

When you upload files to S3 they are marked as 'private' as a security feature. Which is good.

But users of the S3 service upload files to make them public. They are mostly used in websites, etc.

S3 != dropbox

Re:URLs?

[Anonymous Coward]

If the file is 'public' and hosted via http (very common, much more common than https, specially for the use case of the S3 service) then MANY people know your 'secure' URL.....intermediate hops, people on the same lan segment, public wifi hotspot maybe? etc etc.

So, adding a UUID is adding security by obscurity, but no real security. It does not hurt to have it. But NO, it does
not add real security.

Also, S3 does not work that way. You are thinking of services like dropbox.

S3 is used to upload files and make them public, it's like a filesystem. it is not like dropbox.

Re:URLs?

[BradleyUffner]

These storage buckets are presumably meant to be private, not public. So the private houses analogy is much better than the public restaurant analogy here.

By default every bucket and file is marked as private. If something is marked as public then it has been explicitly marked that way by the user.

Morons Don't Read Slashdot

[BoRegardless]

Amazon's Jeff Bezos must not give much direction to his crew about running things right.

Re:Morons Don't Read Slashdot

[Anonymous Coward]

Amazon's Jeff Bezos must not give much direction to his crew about running things right.

I don't see how third party consumers of S3 misusing Amazon's API has anything to do with Jeff Bezos or his engineers.

Re:Morons Don't Read Slashdot

[girlintraining]

Amazon's Jeff Bezos must not give much direction to his crew about running things right.

The default policy is set to private and Amazon provides extensive documentation and support should customers wish to secure things properly. 5 out of 6 did, and think the sixth is a blithering idiot. How is Bezos responsible for the sixth guy shooting himself in the foot as when he was handed the gun it clearly said "Do not pull trigger while pointing at self."?

Re:Morons Don't Read Slashdot

[Anonymous Coward]

OHHHH so then the bucket heads granted permission then with intention.

So they said it is ok to view they chose to allow right?

Re:Morons Don't Read Slashdot

[Luckyo]

Being an idiot does not automatically revoke rights to things like privacy and property. Else you could argue that stealing from people who don't lock their doors is not a crime.

Re:Morons Don't Read Slashdot

[Anonymous Coward]

> The default policy is set to private

It is now, but it wasn't in Dec 2006 when I first started using S3. I looked through my buckets, and all of them I created that month are all public.

Re:Morons Don't Read Slashdot

[advocate_one]

where's my mod points when I need them... MOD parent up

This just in...

[girlintraining]

People don't bother reading the manual. Then, everything explodes. How is this news? Please, find me a person in this industry who doesn't know what RTFM means. "Idiot who didn't RTFM exposes personal info." Those of us in the industry have a term for when things like this happen: Tuesday.

What'll be news is when they say "And then the manager and personnel responsible went to jail, because their idiocy cost tax payers millions in lost productivity spent fixing their credit reports and financial lives."

Hacking

[Anonymous Coward]

I thought white-hat hacking was illegal unless you got the owner's permission...

Re:Hacking

[PhamNguyen]

It's ok, he's working for a company so it's a civil matter at worst.

What's the news here?

[viperidaenz]

A billion out of a billion Facebook accounts are ripe for the plundering too. Just wait for the next feature change and the inevitable default setting of "public" applied to every account.

Data Access

[Anonymous Coward]

Data put up to be available for access, is accessible. Why is this news?

Will these guys get 41 months in jail too?

[bennini]

This sounds an awful lot like what Andrew Auernheimer did.

If the justice department or any company affected by this wants to, they could claim Computer Fraud and Abuse.
Yet somehow I doubt the "researches" will get any jail time.

Public v. Private

[DragonWriter]

Using a combination of relatively low-tech techniques and tools, security researchers have discovered that they can access the contents of one in six Amazon Simple Storage Service (S3) buckets whose owners had them set to Public instead of Private.

So, if you want your bucket to be private, you shouldn't actively set it to be Public instead of Private. Okay, I can see that, but I'm trying desperately to figure out how this is news.

Re:Public v. Private

[Anonymous Coward]

I agree.

Not a security hole

[rgbrenner]

The default in s3 has containers set to private. The 'flaw' here is that public containers can be listed by anyone.

1) set container to public
2) shout loudly that the public can see inside your public container

I'm tempted to call the author a moron.

Re:Not a security hole

[philip.paradis]

It's not a problem with Amazon. The issue is with developers not bothering to think about what they're doing when they chuck data into buckets that are expressly set to public. It's potentially a very large problem for companies that expose sensitive customer information or things like access credentials in this manner. If you think I'm kidding about the latter, I'm not, having seen that happen.

S3 - MegaUpload...

[linatux]

but with a crapier business plan?

Public Stores or Private Houses

[Anonymous Coward]

So basically they walked down the street checking door to see which ones were unlocked then looked inside the open stores. These are marked public. They're public.

~orb

Newsflash!

[Anonymous Coward]

If you leave your assets "public", anyone can see them!
Film at 11...

Some people shouldn't be allowed on the Internet.

Bookem

[Anonymous Coward]

Dano! Throw all hackers in jail NAO NOWWWWWWW!

And in other news

[Anonymous Coward]

"Using a combination of relatively low-tech techniques and tools, security researchers have discovered that they can enter one in six homes whose owners didn't lock their front door. Researchers noted that home locations are all predictable and public facing, which make it that much easier to find the houses in the first place with their own friggin two eyes ."

Intentionally Public

[Anonymous Coward]

I have a public bucket and it is intentionally that way. It has some source code, some images, etc. Nothing proprietary, but it is convenient to store all of this in one place so I can refer to it from my websites.

Many people use S3 buckets to take care of all their webhosting. This is quite reasonable to do if you don't have any server-side scripting needed. This isn't a 'security breach'.

Finding a public bucket and even source files is not enough to determine that it is a security problem. It would not surprise me to learn that 1/6th of buckets are intentionally hosting files publicly.

Duh...

[Captain_Chaos]

"Public websites are public! News at 11!"

Slashdot has the same issue

[TwentyCharsIsNotEnou]

Looks like Slashdot has the same issue.

For example, this. No user authentication required at all.

But they were set to public...

[Anonymous Coward]

Ok, I read this and don't see where there is any kind of breakdown or failure on anyone's part but the end users who set up their buckets. I'm sorry but if you leave your Mercedes parked outside, unlocked with the keys on the dash and a brand new Rolex sitting in the box on the passenger seat and someone steals your car and your Rolex that is your fault. In fact, I have a hard time accepting that a crime was committed or that anything was stolen. If you don't lock it up you are essentially giving it away and that is your own fault, not the fault of the person who takes it. It would be like a public website complaining that they were getting hits. YOU, the END USER are responsible for YOUR data online, period. If you leave the "public" box checked and other people look at your stuff, that is not a service failure its a user failure. HOW IS THIS EVEN NEWS?!

Not relevant to everyone

[RevDisk]

I do weekly backups of my web servers to Amazon S3. I'm not overly concerned because I encrypt (AES-256) the tar files before upload.

While I admit, folks have their own priorities and needs... I only tend to trust "the cloud" for things that are public or well encrypted.

Re:Attention mods, impersonation warning... apk

[Anonymous Coward]

It's official, /. is now trolls all the way down.

Now you know how china caught up.

[Anonymous Coward]

We really are retarded.

"Big Drama" strikes again!

[Anonymous Coward]

This story proves one of two things:
The public buckets are actually intended for anyone to read and use as they see fit, or,
The people running those buckets are imbeciles, and deserve what they get.
It is the same reason that people complaining about the Google Streetview wifi thing are morons. Too lazy to use even the basic security for your wifi? Sucks to be you.
I'm going to guess that for the most part the public buckets are intended as public information, and that this article was written by an overly dramatic person (who either works for "Big Drama" or discovered that "Drama Sells" on their own).

It's not I folks: It's Jeremiah Cornelius... apk

[Anonymous Coward]

THIS is why he's doing it & proof of it, here -> http://interviews.slashdot.org/comments.pl?sid=3585927&cid=43295193 when others pointed out Jeremiah Cornelius forgot to submit one of the "first post spams" (masquerading as myself, by posting as AC & using some old posts of mine or other b.s. he put up), & JC mistakenly submitted one of the impersonations of myself as his registered 'luser' name here on /. forums.

Pretty pitiful actually, but like every up to no good idiot does? He screwed up & submitted it under his registered 'luser' name here, instead of his ac submittals he's been doing.

* Jeremiah Cornelius: DO YOURSELF, and the rest of us, A GIANT FAVOR MAN: Seek professional psychiatric help!

(Since Jeremiah Cornelius obviously can't get over the fact he made a spelling error on what it is HE ALLEGEDLY DID FOR A LIVING? That's not MY fault... it's HIS!)

APK

P.S.=> I seriously must have dusted JC (in his mind @ least) for his BAD spelling error & it "got his goat"...

I.E.-> Catching what he claimed to do as a job, for YEARS he left "PENETRATION" (correct) spelled as "PENTRATION" (incorrect) on his resume on LinkedIn & I pointed it out as he & his friends trolled me as usual (webmistressrachel, gmhowell, & crew (probably ALL JC no doubt using alterate emails or TOR to do it as a possible - I've caught "them & theirs" doing it before, ala Barbara, not Barbie = TomHudson (same person))).

So THAT is what has gotten his goat in a technical debate & his "geek angst" could only come up with *trying* to "impersonate me" in every news thread on /. for the month of March 2013 so far!

(Just to attempt to 'discredit me' as a spammer here obviously)

Doing so, by posting that "$10,000 challenge" &/or reposts of my old posts on hosts file value to end users into EVERY SINGLE NEWS ARTICLE POSTED on /. ...

It's all I can think of that *might* cause such a mentally troubled 'reaction' like the Jeremiah Cornelius is doing & there's NO QUESTION he's the one doing this spamming of nearly every posted article masquerading as myself...!

... apk