Cache On Delivery — Memcached Opens an Accidental Security Hole

jamie spotted this eye-opening presentation (here's a longer explanation) about how easy it is to access sensitive data on many sites using memcached, writing "If you already know what memcached is, skim to slide #17. The jaw-drop will happen around slide #33. Turns out many websites expose their totally-non-protected memcached interface to the Internet, including gowalla, bit.ly, and PBS."

Comments

Firewall?

[chx1975]

I run my memcacheds behind firewall. I thought that the basic server security rule was that you firewall everything opening ports very cautiously as necessary.

Re:Firewall?

[MikeFM]

My memcached server is on the private network only accessible to other servers and is firewalled to everything but the servers that need access. Not exactly rocket science.

Re:Firewall?

[PIBM]

Ive been running memcached since it's out, even sent some patches in.

The thing is, why aren't they running this on a private network ?? Memcached is designed to be fast AND non-secure, to be run on your local network. Running it on a server farm with thousands of people having access to your computers and ips is not a private network.

I had heard about people running it on the local interface and still getting problems before (somebody else with the same computer ran it too and forgot to pick the good port and finally used the same key ...) but that's because IT'S NOT BUILT TO BE USED ON AN UNSECURED NETWORK.

Nothing new, bad admins get bad things done to them, move along.

I fail to see why this is news

[OverlordQ]

Much less 'memcached' being at fault. They say it themselves:

Memcached does not spend much, if any, effort in ensuring its defensibility from random internet connections. So you must not expose memcached directly to the internet, or otherwise any untrusted users.

All this is is stupid admins doing stupid things story and those are dime a dozen.

Re:I fail to see why this is news

[TheRaven64]

A good tool is designed in such a way that the correct use is the easiest. Does memcached, for example, default to only accepting connections from the local host and require other IPs to be explicitly added? That would be trivial to implement and, if done, would require the admin to implement something like the correct security policy. In contrast, defaulting to accepting connections from anywhere means that the admin can use it incorrectly without needing to do any thinking, but needs to think before using it correctly.

Re:I fail to see why this is news

[MikeFM]

The difference is that in this case a non-retarded admin can secure things. With Microsoft products it often takes an act of God to secure them (the best security feature of a Windows system is a blue screen of death). And memcached isn't meant to be a public service. It's very plainly described as not being secure. Completely different than a service that is meant to be public such as web or email not being secure.

Re:I fail to see why this is news

[MikeFM]

It defaults to not being installed and running. Memcached is meant to be ran from one or more caching servers (not really on the web server itself). It isn't really meant to be ran on localhost under ideal usage.

Re:I fail to see why this is news

[nebular]

I believe the point here is that software designers should assume that in terms of security their users are complete idiots and WILL install and setup the program in an unsecure manner unless they are specifically beat over the head with the notion that what they are doing is BAD!

Re:I fail to see why this is news

[TheRaven64]

Which is exactly the point. The default install should never be working-and-insecure. It should be secure, and ideally it should be working. If it is not possible for the default install to be both useful and secure, as appears to be the case with memcached, then it should install only listening on localhost and require explicit intervention by the user to accept connections from other hosts.

If you can install it and have it work by default, then there is no reason for the user to bother reading the manual, so they won't learn that it needs to be specially configured to be secure. If the default is secure but not particularly useful, then the user needs to explicitly adjust the setting that makes it insecure, and in so doing needs to read the documentation explaining that this will make it insecure and how to mitigate it.

More Boiled and Distilled.

[SuperKendall]

Memcache allows anyone to overwrite a cache instance. Seriously? It does not authenticate a write to the cache? And they didn't see this as a problem when desgining memcache? Really?

Anyone can write on your underwear too, if you are stupid enough to wear it outside your pants.

Is that an underwear design flaw?

Re:More Boiled and Distilled.

[Farmer Tim]

Best. Analogy. Ever.

Re:Admin or distro?

[TheRaven64]

default to INDRR_ANY

And this is why they're to blame. Default should be the loopback, and enabling external access should require explicit configuration.

Re:Admin or distro?

[bill_mcgonigle]

Memcached is not meant for single-server configurations

That's silly, it's a generic object store. There's no reason not to use it to cache expensive local operations. Of course it shines across a farm of caches, but the server mapping hash will work just fine with one machine.

If you're a startup with just one webserver and starting to hit performance problems, memcached will likely buy you a few more months.

Going from one server to two is hard, three is a bit more work, and after three it's roughly all the same until you start adding more data centers and then it's all the same until you're Facebook. Taking on that 'hard' expense too early would be a poor allocation of resources.