Slashdot Mirror
Package Managers As Achilles Heel
An anonymous reader writes "Researchers from the University of Arizona have released a study that takes a look at the security of ten popular package managers. They were able to show all ten were vulnerable to attacks from a mirror or man-in-the-middle that allow an attacker to (along with other things) crash the system or obtain root access. Furthermore, the researchers created a fictitious administrator and company name and were able to lease a server and get it listed as an official mirror for all the distributions they tried (Ubuntu, Debian, Fedora, CentOS, and OpenSUSE). This raised the question: What keeps you up at night, the thought of attacks on your package manager or previously discussed and patched vulnerability in DNS?" justin samuel (one of the Arizona researchers) also points out a synopsis on CERT's blog.
What keeps you up at night, the thought of attacks on your package?
Yes.
... oh wait!
Love many, trust a few, do harm to none.
I think a lot of these risks could be reduced if people downloaded from one site and verified against one or more other sites. Furthermore, if the checksums were verified over SSL, some attacks would be harder.
Right now, verifying packages against a site other than the one they were downloaded from seems cumbersome with apt--or does anybody know of a simple command line to do so?
Just in case anyone thought (like me) that the vulnerabilities they're talking about might let an attacker install arbitrary software just through the package manager, this doesn't seem to be the case.
The attack might go like so:
The simple fix is to change the client so that it never regresses (e.g., never installs software older than what it already has installed).
I can't sleep because of the "thought of attacks on my package"...
RTFA
How does compiling from source help? Trojans can be introduced in source code just as easily as in binaries.
The headline should be project managers as achilles heal.
Until I RTFA, I was ready to dismiss this as 'failing to understand signed packages. Wrong, they understand package signatures all too well.
The basic attacks seems to be.
1. Obtain old, signed packages.
2. Become a mirror for debian|fedora|ubuntu|$distro.
3. Wait for vulnerabilities to be found in some package.
4. Do not serve the updated packages, continue to serve the vulnerable version.
5. Log IPs of machines downloading from your mirror.
6. Root them.
This works because some package manager software will download and use package metadata even if it's older than what's cached.
One long term solution would be to sign package metadata and serve it only from one central location, over https/sftp. There may be others.
Alex
The article actually discusses attacks that can be made by a malicious mirror even if you are only accepting signed packages.
Considering Fedora (at least) signs all packages, seems like the authors should have been clear if the attack still works if the packages are not signed.
"Thanks for all the money you paid to us. We've used it to buy off ISO among other things" -Microsoft
I like to have the latest versions of programs, with the latest features and bug fixes, without having to check the website for the latest available version, download it, compile and install it. Multiply that by tens of programs, and it's no fun at all. Additionally, when installing programs, there's no hunting down dependencies.
It's also very easy to hunt down hundreds of dependencies, right?
I really don't understand what the big advantage is in using package managers. It's dangerous because you never know what "updates" will come down the pike.
Normal people don't want to have to keep up on what the latest version of everything is as it comes out and what security holes are in it. That's one of the main reasons people use distributions instead of hacking together their own. People want knowledgeable experts to make the decisions and let the computer do the rest.
Thanks for reading the article.
The point is that there are plenty of signed (but old and vulnerable) packages that can be given to your system that it will gladly accept leaving you open to attack.
That's true, but are you sure that you're getting the keys in a secure way in the first place? Since people generally download the ISO without encryption, the ISO may have been altered as well. And are you sure the commands you display things with haven't been altered either? To really be sure, you probably would need to get a physical CD from a trusted source, including all keys, and then verify your entire system from that.
Still, I think that the larger risk for trojans still comes from people planting them in sources or binaries; any small open source project and any packager could introduce a trojan, and it only takes one.
They're saying that signing the package isn't good enough.
I think the threat they're describing is based on this idea: your Linux Vendor signs package foo-1.0. People being fallible, a security vulnerability is eventually found in foo-1.0. It gets patched. foo-1.1 comes out, and your vendor signs that too, and you install it. You are happy.
Then, one day, you tell your system to update everything. Your computer talks to some mirror, and gets a list of the latest packages. Then you download the latest packages. Oh look, here's foo-1.0. Oh good, it's signed by the vendor, therefore it's safe. You install it.
Oops, the mirror just tricked you into installing software with a known bug. The package was signed, but the information about what is the best package to download and install, wasn't signed.
As copyright owner of this comment, I authorize everyone to defeat any technological measure which limits access to it.
they basically say so themselves
Use signed repository metadata. If your package manager or distribution does not yet support signed metadata but only signed packages, at least require signed packages until signed metadata is supported.
Debian does that. The Release file defines what Package database files are available, and their md5sum's. It is signed.
the Package files include md5sum for the packages (can't modify the Package file otherwise signed md5sum in Release file will fail, can't modify the actual package as wont match md5sum in package file).
I would argue, that manually using the package manager (dpkg over apt) is less secure if one has apt set up correctly.
It doesn't help at all. The grand parent is a twelve year old that discovered Gentoo yesterday. Gentoo is '1337' yo.
Unfortunately it's not nearly as easy to type: ... ... x20
wget http://blahblah/dep1.tar.gz
wget http://foobar/dep2.tar.gz (404 Sigh)
wget http://woobar/dep2.tar.gz
wget ftp://wowowow/dep2.1.tar.gz
Followed by:
tar -zxvf files.tgz x 20
THEN followed by: ./configure && make && make install x20 IN THE RIGHT ORDER
times
Plus however long it takes you to debug the eventual compile problems because one of those obscure libraries that only one program uses was installed earlier with the wrong version.
All said and done, I'd rather take my chances with package managers, thanks.
Considering Fedora (at least) signs all packages, seems like the authors should have been clear if the attack still works if the packages are not signed.
The article seemed pretty clear to me. The most vulnerable systems don't have signed packages or metadata or expiring signatures. The least vulnerable systems have all of those, but are still potentially vulnerable for a window of time before the metadata signature expires but after a vulnerability is found and fixed in a package.
If you are downloading from a mirror, someone could apply similar methods to insert malicious code in your version of the source. And checking all of the source code is no trivial matter, especially if you are looking for an intentionally obscure bit of code.
Also, you have to trust your compiler, which you *had* to get from someone else. Your compiler may be inserting malicious code.
Yes. Yes you must.
RTFA. The key is valid forever. A server could have your machine load a previous version of software that has already been signed by the vendor. The previous version may not have a security fix and may be exploitable. Then your machine is compromised.
1. Set yourself up as a mirror
2. Put old packages up with known vulnerabilities.
3. Distribute "updates" listing the old packages as new updates.
4. Watch your logs to see who updated with old packages, then go PWN them.
It also counts on lazy admins, but garsh how rare are those.
I guess it comes down to controlling distribution of the updates. Kudos to these Arizona guys. This is a really simple method that can cause complete mayhem in uncountable ways.
What package manager silently downgrades packages?
I can see a package mirror (maliciously) refusing to stock updates, but yum at least picks a mirror at random by default. Apt didn't last I saw, but if you picked your own mirror, you already trust them.
... And so it comes to this.
You've got him all wrong. He's a twelve year old with a PhD that reads and understand every line of code before he allows it to execute on his own hardware.
Your firewall won't stop this type of attack... RTFA.
You don't type './configure', 'make' or 'make install' when installing stuff on Gentoo, it has a package manager, but thanks for playing
I've been using Mandr(ake/iva) for years now, and some of their releases were OK, while others crashed and burned. It was cool though, because it was usually fast and breezy to set up. When Mandriva One 2008.1 came to town, I was impressed! Hardware detection and support was faster and better than ever! It set up so smooth and dreamy, you would have to be crazy to use anything else. Then the updates started happening... Sometimes several every week. Then it got cranky. Web access isn't quite so responsive as it used to be. I had to give it up on my EEE and resort to using XP. My desktop is lumbering through it, but I'd still like to format and re-install with 2007.1. The major trouble with Mandriva One 2008.1 is they don't actually include your make/install, build tools so you have to use their package manager over the net. It's not such a dreamy feeling anymore.
All that, and I'm not so sure Google or Firefox are necessarily my friends anymore either... Did the good guys lose or something? What's happening here?
Blessed with all the brains that God gave a duck's ass, and twice the charisma.
I could answer that, but the short article you didn't read already does.
I bet your girlfriend doesn't keep you up at night either... It's ok, someday you'll find someone who loves you
I'm almost certain I posted that as AC...
Yes, the vulerabilities are there. Not surprising to anybody that ever did a Debian mirror, for example. DNS spoofing already is enogh, not even a man-in-the-middle needed. Still, these are high-effort attacks. And with the IP addresses of the update servers used (instead of their names), MiM becomes neecessary. From my observations, MiM attacks oten need to be done in close physical proximity of the attacked machine. If somebody manages to get there, they can direcly manipulate my servers, with me possibly not even noticeing. The effort for that is hogh enough that I do not care, since the stuff on the servers is not worth enough to justify the attack cost by a fair margin.
So, no, I have 2 v-servers and one remote and one local dedicated server, all with Debian and automated updates and I sleep fine.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
you download the hashes from the project page over SSL
Snowden and Manning are heroes.
....the more they stay the same. I was a Mandrake user 5 years ago. Compared to everything else around, it was the slickest out of the box. But the the Drak* tools were always flaky, as were the update tools, and I could never find a mirror that was worth a damn even if the package manager was working well that day. Hell, it made Windows ME look like a paragon of polish and stability.
Going to Debian after that was like a breath of fresh air. Sure it wasn't packed full of wizards and GUI config tools but once configured into a working configuration it STAYED working. I've been running either Debian or a Debian derivative since. I've thought that maybe by now Mandr(ake|iva) would have improved. It looks like they're still up to their old ways of surface polish atop a creaky collection of dirty hacks.
Comment removed based on user account deletion
I'm almost certain you didn't.
+1 IDisagreeSoHeMustBeATrollOrAnAstroturferOrAShill
TFA doesn't address Windows much. In the FAQ they say something like "since the vendor controls the repos for Windows and OSX, they are less vulnerable".
True, I can't set up a bogus mirror for Windows -- except in a corporate environment, where I believe MS has some facility to allow a local cache of patches to reduce external bandwidth usage.
But the authors keep talking about man-in-the-middle attacks on FOSS repos. Couldn't someone just as easily do that for Windows? And then use it to only offer outdated (known-vulnerable) versions of patches?
I'm fairly certain you didn't.
c++;
Good thing I'm running Slackware!
The metadata is signed as well as all the packages. This mitigates the situation, i.e. a malicious mirror cannot host arbitrary combination of packages to suit their needs, they must mirror some permutation that I had at one point approved all together.
XML is like violence. If it doesn't solve the problem, use more.
> The article actually discusses attacks that can be made by a malicious mirror...
Yes a mirror can keep you from getting a security update. But if you don't contact that mirror every day you will eventually get a good mirror and update, and since none of the package managers will downgrade automatically this is a mostly theoretical exploit.
Yes if a really BIG bug hits somebody could keep some subset of machines from updating, and since they would also KNOW the ip of each vulnerable host it could be very bad. That is the part that worries me, hell, they could even deliver the update from their perfectly up to date repo of signed packages, signed metadata AND perfectly in sync with the distro prime mirror.... and root your ass while the update is in flight. This gets to the real security vuln involved, telling an untrusted entity exactly which version (sorta) of a package you are running.
Democrat delenda est
A server can't make your machine do jack shit. Your machine is the one that pulls an old version and downgrades to it. If your package manager silently downgrades packages then that's a flaw in all cases. What if a mirror just happens to be out of date?
FTA:
Q: I'm worried about security and have a choice of package managers. What package manager should I use?
A: It depends on your situation. If you are using a distribution like RedHat Enterprise Linux or SUSE Linux Enterprise that support HTTPS and do not have mirrors run by outside organizations, then you are most likely safe from the attacks described here.
If you use a distribution that does not have these features, we strongly recommend using a package manager that signs the repository metadata and supports HTTPS. Of the package managers we examined, APT-RPM, Stork, and YaST have this support by default and APT has an optional package to support HTTPS.
Forgive me for my lack of *nix prowess, but I've noticed that Package managers seem to be a predominantly Linux thing and unfortunately, I've not had the chance to get properly acquainted with the OS yet (I know, wtf am I doing here?). I've never encountered one in Windows, unless you count the handy package manager in DevC++.
Are there any good windows alternatives, or is it simply not needed thanks to windows having many a dedicated installation packaging program?
I wouldn't mind having one so I could get the latest and greatest Open Source/Free software that I may or may not have heard of, would be a great help for when I want to switch over to Open Source only apps.
My thinking is if there's an easy way to find/install good OSS, then I can get used to that first and thus my eventual transition to an Open Source OS will be much easier as I will only have to figure out about 200 things to do differently instead of 300.
+1 IDisagreeSoHeMustBeATrollOrAnAstroturferOrAShill
The short answer : yes
The longer answer : every OS is vulnerable one way or another. The difference lies mostly in the response and the response time by the vendors.
Linux : take the debian ssh disaster a few month ago as example. I read about it at Google News, head over here to check how the linux bashing was coming along, and while I was reading, the "update available" icon appeared. A few minutes later and the vulnerability was no more. .. people who actually CAN fix this kind of problems, not your average /. reader)
Admitedly, it took a *VERY* long time to find out about the problem in the first place, but the response time from then on was very short, and the update contained concise information about the whole mess.
Today's vulnerability will probably take a bit longer to be fixed, as it requires some primordial changes in the way packet manager work to be fixed. But I'm rather sure people are already looking for a solution (you know
Apple Mac : when Apple admits that there is a vulnerability in their products, they take their dear sweet time to fix it. As a matter of fact, Apple just released a security fix for Apple TV, covering vulnerabilities dating back to, at least, January 2008 (at which time it was fixed for OSX, but NOT for Apple TV). I can't comment on how detailed the security fixes are, as I don't own apple products
Microsoft : the Zero Day initiative still lists 12 issues concerning Apple product, classified as "high severity", but the oldest item is a Microsoft vulnerability dating from September 2006 (more or less quoted verbatim from the iWire article I'll link to a bit later). Microsoft updates are particularly obscure in their descriptions, and, if I remember correctly, they are sometimes even applied without asking the user first, and have a bad habbit of breaking other stuff.
So, is Linux 100% secure? No, and it will never be. But at least the devs react in a timely manner, and they don't just install something without telling you what it is or that they are patching at all. Therefore it is better secured than Apple and Microsoft products whose vulnerabilities are often left open, for the sake of obscurity I suppose.
"Superiority" is a highly subjective term, so I won't even start to thread on this subject. It is for me, but your mileage might vary
Apple TV fix article
Zero Day Initiative upcoming advisories
"DRM is like the Ford Pinto: it's a smooth ride, right up the point at which it explodes and ruins your day."-C.Doctorow
from the cert overview:
Another potential solution to this problem is P2P technology. If widely used, programs like DebTorrent may allow official repositories to distribute metadata and tracker information while decreasing bandwidth costs.
I wonder why Ubuntu doesn't make this a standard option. Is port forwarding too complicated? From what I understand, if the package is not available via bittorrent, DebTorrent degrades to using http.
It is really risky not to have a revokation mechanism for packages, but definig this by availability of a new package somewhere. The only solution would be to have servers, which contain a list of revoked packages. If connecting to these list fails, the update should stop.
Noob, he doesn't need source. I'm a 13 year old PhD and I assure you even I can read binaries.
so you grab the same code from 3+ different mirrors and check them against each other...
comment first, facts later. http://chem.tufts.edu/AnswersInScience/RelativityofWrong.htm
Neither keep me up at night, I have a girlfriend and things to get done.
Perhaps you should consider changing your girlfriend, mine can always keep me up.
(for those without sense of humour up is used as erected)
Neither keep me up at night, I have a girlfriend and things to get done.
Perhaps you should consider changing your girlfriend, mine can always keep me up.
(for those without sense of humour up is used as erected)
Do you often find yourself having to explain your erections?
Ok, so foo-1.1 is uploaded on security.debian.org.
What are the official mirrors for security.debian.org ?
None. And if you decide to use some mirror anyway, try at least to select one you trust.
End result : everyone gets the patched foo-1.1
No need for anything as complicated as verifying packages accross several source :
most modern package managers use key-signing on package.
You can't setup a bogus repository and start serving malware to unaware users - those packages will fail the key check.
For that to work, the crackers would also have to find a way to inject their own keys into the ISO of the distribution, and they'll have to find a way doing it that still pass the checksum of the peer-2-peer client with which the users downloaded the ISO.
That might be possible with older p2p protocols relying on older weak hash, like eDonkey2k whose MD4 has known collisions, or even older protocols lacking checksums. (Some companies working for the media corporation back then used such possibility to pose as peer on the network and poisoning it by injecting bogus packets).
But distribution currently rely on bittorrent which use SHA-1 hash and it's (currently) much harder to find a way to inject tampered data and have the resulting file still pass the checks.
Another solution would be to trick the users into accepting a new set of keys to get onto the fake repository. For this, this repository will have to pose not as a mirror (as proposed in the TFA) but as an additional 3rd party repository hold functionalities not available in the original source (this is something that would benefit from the harsh imaginary-properties laws as most distro can't provide packages processing some media, and users have to rely on 3rd party repositories for this).
Besides, the summary is misleading. They didn't actually setup a bogus mirror that served maliciously crafted files, or otherwise injected malware (that would be impossible given the signatures).
They simply setup a mirror, that wasn't up to date on purpose, potentially exposing computer to exploit due to only older versions of software being updated.
As actual legit mirror may lag behind the release, it is nevertheless preferable to always add the original repositories to the list of source : thus get the files from the mirror if available there, or straight from the original website if not replicated everywhere.
This work around is even useful when there's no malicious intent.
"Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]
Hear! Hear!
When it becomes a problem like it is for Windows, I will then take notice.
From my point of view, it's all 'nothing to see here, move along.'
I will admit that the increasing popularity of *nix will eventually will make this an issue, the time is not now. Give it a few years and marketshare increase on the desktop.
The source is always available to be checked, so I don't see a problem here unless someone like MS is trying to 'poison the well' to try to increase their marketshare. This is FUD as far as I am concerned. (I maintain backups, so WTF?!- let's see a problem with this before we jump the gun(or shark))
Down With Slashdot BETA!!! I've been around the corner and seen the oliphant; you can only abuse me from your perspecti
...exactly what happened to PHP a couple of years ago.
But the general theme this post rings is: in FOSS, if you don't (as hardly any end user does) inspect the source code, know and trust the origin.
It won't be long before those who write `malware' will start searching for ways to get hold of linux desktops. And for them, prospects aren't that promising with blunt exploits against whatever is running on your linux desktop with some ports open. Neither, I earnestly hope, will an average linux user be so outright... er... end-userish as to go happy-clicking on links in spam.
Instead, setting up or overtaking some ftp.xx.debian.org is only sensible and effective to this end.
Except that it is solved by MD5 checksums as noted by several people. A sophisticated attack of this nature would require being added as a mirror, accepted by your target as being genuine, and then taking on the provider of the checksum to provide a false one.
Of course, someone might take the time to develop a package with the same checksum as the actual file I suppose, but it has been so long since I've read up on encryption that I don't know how feasable this is. Anyone else? I'm fairly certain that by the time you get the package developed now, a new version will be released, killing all that time and effort.
Rather than going through all that effort, I'd just take an assault team to the datacenter and take what I need by physical means. A few chainsaws, some C4 (or close equivalent thereof courtesy of roguesci&company) , and a few Walther PPKs handed out so we can feel like James Bond while going in, and call it good.
If someone is truly determined to take on your company and steal data, they will. The trick is to mitigate all the easy to use venues of attack. As you start to protect against more and more esoteric attacks that require the subversion of a good amount of external access sites, the usability of your systems will decrease.
---- Liquid was a patriot ----
Two minor points to object :
- Current package managers don't downgrade automatically. (Yast will show the version numbers in red, but keep the highest version anyway).
-People have at least 2 source for packages :
1. a mirror of the distribution (containing all the packages)
2. a "security" update website, which generally is also able to use delta-packages.
some even add :
3. the original repository, in case the mirror is lagging and doesn't have the latest version
Under those circumstances, the packages manager only considers the latest version between the one already installed and the one on the 2 or 3 mentioned repositories. With a slight preference for the delta package on the security site, to save some space.
"Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]
Also, you have to trust your compiler, which you *had* to get from someone else.
Nah, I wrote my own compiler directly in machine code. I didn't trust my keyboard manufacturer either, so I tapped out Morse code on a homemade key. I made the BIOS out of coconuts, but that was just because that douchebag Gilligan said it couldn't be done.
-Roy Hinkley
Furthermore, the researchers created a fictitious administrator and company name and were able to lease a server and get it listed as an official mirror for all the distributions they tried (Ubuntu, Debian, Fedora, CentOS, and OpenSUSE). What keeps you up at night, the thought of attacks on your package manager or previously discussed and patched vulnerability in DNS?
The man-in-the-middle attack is valid enough - particularly with ISPs now regularly committing that kind of attack (under the guise of traffic shaping and ad injection attacks) and not being cruelly and unusually punished.
As for the fictitious company? I'm going to go with option 3: Thank you, Debian, for not wasting money on something I can do myself.
It is not Debian's job to vet the sources.list. For me, it's my job. I use U.C. Berkeley's repositories, because I personally trust UCB. Some people hire systems administrators to do the vetting. Others buy commercial distros like RHEL. Lots of freedom of choice - that is a feature, not a bug.
It is the beautiful world of FLOSS - where self-reliance, personal accountability, and Darwinian smackdowns are available for free. The water is absolutely sublime - but be careful not to attempt to breath while your head is under water, as you will drown (or give it a try if you like - you are free to do so if you wish). If you are not sure that you know how to avoid breathing while under water, you may wish to hire a personal lifeguard, or join a fitness club that provides communal lifeguards. You may even want to write your city council person and ask them to provide a public distribution, akin to a public swimming pool. The opportunities are endless. This is Free. This is beautiful.
Stop-Prism.org: Opt Out of Surveillance
In package maintainers we trust. Amen.
I'm moderately certain you didn't.
Do it yourself, because no one else will do it yourself. [beta blockade 10-17 Feb]
Hmm. How to defeat that. Easy enough.
Firstly, the catalog would have to be signed. (The list of the current versions of the packages, I guess it's the metadata rather than the data.) And it would have to have a datestamp in it. And the catalogue would have a copy (or the copy of) the signature that would otherwise be attached to the packages themselves.
Then it's easy for a client machine to verify the catalog is correct (is correctly signed by a trusted key). Validate the datestamp in it and alert if the catalog is old (say, older than three months ago). When you download a package, it takes the signature from the trusted catalog and uses that signature to verify the package, proving the package file is a currently supported version.
(If the package download doesn't match the signature then it will fail the signature check like any other modification. Technically, checking this is a fullly signed singature isn't required, merely that it matches a cryptographically secure hash in the catalog but I imagine it's safer and easer to just verify the signature on the package in its own right. Packages may carry the signature themselves which would allow stand alone installations of the package - but of course, no guarentee that it is the latest version.)
Yes, this means monthly updates to the catalog, but the only parts changing (aside from actual updates) are the catalog datestamp and catalog signature, which I imagine would be reasonably small if something like rsync copied the data.
And, yes, this does mean a three month window for this attack. One way around that is for the 'current datestamp' to be available on a trusted central site (signed and/or SSL protected) and the package manager refuse to load a catalog until it has the catalog version that matches that current datestamp. This would probably only be a couple of hundred bytes at most. Maybe a weekly check would be sufficient.
Comment removed based on user account deletion
I'm somewhat certain you didn't.
That's what she said!
For anybody interested on this in Gentoo, I have updated my GLEPs about signing the portage tree to include a Gentoo-specific solution for this, by distributing a copy of the top-level signatures by a trusted system:
http://robbat2.livejournal.com/226512.html
ICQ# : 30269588
"I used to be an idealist, but I got mugged by reality."
I read that, but what package manager _updates_ from foo-1.1 to foo-1.0?
http://marriedmansexlife.com/
It's also plausible that a package manager could be compromised. It's unlikely with a major distributor like Canonical, Redhat, Novell, etc. but more likely if you use third party repositories.
Packages managers put one more link in the security chain. In most cases, it's probably not the weakest link, but it does add one more vector for attack.
Don't get me wrong. I use Ubuntu because of the package management. I could get slightly better performance if I compiled everything for my specific processor, and it would be a better security practice to compile my own packages, but for my personal systems the convenience of being able to say "install this package" outweighs the benefits of compiling everything myself.
ever heard of wildcards? you should look into them.
This isn't to say that I disagree with you about package managers, I use ubuntu as well.
I would agree that it's too bad there could be vulnerabilities in the package manager of several systems.
But on the other hand, it's pretty cool to think of the package manager as being a kind of bottleneck for attack vectors. In other words, though it may allow for some possible attacks, it's interesting to think that attacks could be concentrated there, and thus extinguished in one fell swoop --- once the package manager is "perfected" and all vulnerabilities taken care of, you could argue that the system as a whole is incredibly more secure for having discovered and destroyed the main source of them.
This could be compared to systems without package management (*cough* we know who you are *cough*), where EVERY SINGLE installed program has its OWN attack vectors that come with it, since every program comes with its own installer, running native, unmanaged, and often priviledged code.
Does that make any sense? It's late...
I run Debian at home, and security.debian.org doesn't have any mirrors, so it would be a little hard to set up a fake mirror and refuse to distribute security updates (or distribute old versions of packages).
I chose my mirror to be the university I attended, and I'm pretty sure they are not going to screw with the packages.
They don't grade fathers, but if your daughter's a stripper, you fucked up. --Chris Rock
LFS was my first ever Linux distribution, and I had no troubles whatsoever, despite being utterly unfamiliar with bash. Only problem I ever had was
Anybody who thinks that compiling from source provides security should read Ken Thompson's Turing Award talk titled Reflections on Trusting Trust.
BSD Ports
Now you get your package management AND you get to compile from source. Best of both worlds.
Me failed English...
FreeBSD over Linux. If my comments seem odd, this may explain...
I'm entirely certain I just added you to my foes list. Thanks for outing yourself as an AC troll.
"You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
and in compilers, too.
As far as I can tell, Debian is not vulnerable in its default install mode.
a - Debian will never automatically downgrade a package.
b - Security problems (as opposed to mere changes) get published in the Debian security repository.
c - The Debian installer automatically adds the debian security mirror to the end of your sources file.
So, you can create a mirror, easily inject it into the list of official Debian mirrors, and freeze the packages offered on it. You can do all that easily. It won't help you. If you take the CERT example, of the vulnerable openssl package, the fix was pushed through the security repository, and that is added by default to all new Debian installations. The attack would fail.
Furthermore, even if your mirror offered a security mirror as well, I, personally, always keep the Debian source as well, lower in the list. This way, if the local security mirror is up to date, I take stuff from there. If it's not, I take it from the Debian server. This means that even if you deviate from the default behavior, you can still be not vulnerable.
Shachar
You joke, but I've actually built gcc with a C interpreter, and used that to build gcc binaries. We did it primarily as a stress-test of the interpreter, but it was nice see that the resulting gcc binaries were the same as those built with a compiled gcc.
They're saying that signing the package isn't good enough
That's why Debian doesn't sign packages; they sign repositories. (Actually, they sign a list of md5sums of all avaiable packages.) You can actually argue that signing a package is counterproductive--it means that an insecure package out in the wild will still have a valid signature.
When you sign the repository, then an attacker has to freeze their whole repository if they want to keep the insecure package available. For Debian Unstable, people will get very suspicious if more than two days go by without any updates. And for Debian Stable, well, the security repository isn't mirrored, so owning an "official" mirror won't help you there. (The security repository could be vulnerable to a MITM attack, but then so could MS's system.)
So, while the attack vector they mention for Debian seems superficially plausible, in practice, it's unlikely to be as effective as they suggest.
Here is my solution:
When you go to the official distro website and click on the download link, ask the user for two, 6 digit numbers (xxx-xxx format to make it easy) over a secure https connection and tell the user to write it down because he/she will need it later. Now let the server automatically check the database to see if the first submitted 6 digit key matches with another first 6 digit keys another user has already submitted. If that is true; ask the user for another first 6 digit key and keep doing this until the key is unique. You'll understand later in this post why). Now hash those two 6 digit number and put it in a database and link the two keys (which is rather easy by doing 6+6 is one 12 digit key). In the meantime redirect the user to the actual download page.
When the user runs into the installer upon booting the image, ask him/her for the two 6 digit keys. When inserted, hash the keys and write the hashed keys to the medium you're installing the distro on (6+6=12). Now ask the user to get rid of the two 6 digit number he/she has written on a piece of paper because he/she will not need it anymore.
Now that the user and the official server have two identical 12 character (because it got hashed) keys each, pass the first 6 characters of the 12 character key to the server, over a secure connection, upon checking for updates for the first time (which can only be done by the user or the user's distro and not by any server. Let the distro ignore any incoming update notifications that it or the user didn't ask for him-/her-/itsef). The server will then check for a 12 character key that starts with these first 6 characters and send back the other 6 characters plus another 12 randomly generated characters (1). The distro will now check if the received first 6 characters matches the last 6 characters with it's own 12 character key. If it does; check for- and download updates and replace it's own 12 character key with the last 12 characters (1) that were send along with the first 6 characters. If the it doesn't; do nothing and tell the user what's up and direct him/her to a security section of the official distro website to see if the server is down and if not tell the user that a cracker tries to hack his system (repeat this sequence every time a check for updates is done, but do it only about max 3 times a day because a cracker can brute force the other 6 characters).
(1): The order of the (6+12=) 18 characters in this package that the server sends should be determent by the previous 12 character key, so that if this package is intercepted, no one will know what characters belong to the 6- and 12 character keys. If you want to increase the level of security even further, then you can add meaningless characters to the package so that if a cracker intercepts the package, and he knows the current/old 6 key, he can't just filter out the characters of the 6 character key and brute force the order of the 12 remaining characters, so brute forcing will take longer. And another problem is that each time the user check for updates, the key changes, so it will be a pain in the ass to get the key combination right in one time, and it gets harder for the hacker to predict the next 12 character key because he can never know what the last one whas for certain.
Ofcourse this method is not crack-proof, but aint security a bitch?
Here be signatures
And how do you know that you can trust your browser to display the correct hashes? The attacker knows where the correct hashes are available and what the correct hashes are and can substitute his hashes for the correct ones.
But both Ubuntu and Debian have central security mirrors for security updates, which are added by default.
For this it doesn't matter (much) if a regular package mirror doesn't have the latest openssh, the security mirror will have one with the security fix in.
Now, there could possibly be some cases where security fixes gets published as regular updates where a replay attack could be successful. If the story would load off the apparently slashdoted web server, I could see if they had seen such issues, etc.
But what is funny is the 25-years old bug in all of the BSDs, which has been spotted and hacked around (but not reported and fixed) by the SMB guys years ago :)
One of the safer ways to do solve this, is by having a pre-determined set of key-pairs between the OS-enduser and the distributor of updates. But I very much doubt that the distributors would do such a thing for free, as you would have to use an alternate channel (like email), or pre-printed media (certified CD's), and perhaps certification by a CA. A cheaper alternative would be to have a distribution come with a public key of the distributor, and establish a public key on your behalf during the installation. You would only have to trust the downloaded installation media then. Plus, the distributor would have to maintain a database of joined subscribers for updates.
Religion is what happens when nature strikes and groupthink goes wrong.
A package for safe building is made in Russia,
used in AltLinux/Sisyphus especially to prevent those security hacks. Named Hasher.
Here is source code and FAQ of this project
http://git.altlinux.org/people/ldv/packages/?p=hasher.git;a=blob;f=hasher/FAQ;hb=HEAD
There seem to be a lot of distributions whose package managers rely on the keys/hashes to "secure" the packages, where the mirrors are providing those keys/hashes. Silly idea - always has been, what makes this news?
What this article is basically saying is "anyone can be a mirror" (Yes, that's the point, without that you wouldn't be able to get a release/security update for ages after its release, or you could just take one website offline and stop all package updates worldwide - both are worse for security than the alternative) and "some package managers don't properly check the authenticity of a package from a mirror". The second is a problem that's easily fixed and shouldn't be present, granted. What idiot would accept a valid signed package without first checking that the root key it's signed with isn't current, valid and not revoked? This is like your web browser navigating to random SSL websites without first checking that the certificate chain is correct and valid and ends in a trusted root.
1) Download key from "redhat.com" or wherever (DO NOT USE MIRRORS), or from the CD, on first install.
2) Everytime you are asked to fetch a list of available packages, check the authenticity of that key against "redhat.com" directly (DO NOT USE MIRRORS), check the expiry date, check for revocation information, or allow the user to override (their fault), or check public keychains.
3) If the key ever changes, ALERT THE USER before you do anything with it (don't try and get smart by signing a package with the old key that introduces the new key to the world, or automatically accepting a new key). Then the distribution should NEVER use that old key ever again, and maybe even resign all old packages with the new key.
4) Use that key to verify all packages downloaded from any mirror against the GPG signature from that package (for ultra-security, get that from a second, random mirror, or the official website).
There's a small window of opportunity for compromise on machines that install packages without checking first, or assume that because a key was once valid it always will be, or are significantly out of date and don't check the "definitive" source for a new key first but you'll never remove that completely. The way to do that is to check the root key, reliably, on a regular basis. But then if you have an old machine that still has the key from the install CD, and that key is compromised, and the machine does not have reliable (read SSL with significant trusted chain) means of contacting the main server without possibility of spoofing for a new key, then you have a problem.
I don't see why this is "news" or even vaguely important except to show up distros that aren't ALREADY doing it this way.
BTW: This is why I do package management on Slackware using a script that makes *me* get the GPG-KEY from slackware.com first if it ever changes (including checking if the IP of ftp.slackware.com has changed, the date of the file has changed, the key itself has changed etc. but if they can compromise my access to Slackware.com without hitting one of those checks, I'm stuffed anyway), why I have an RSS feed of Slackware and check the website myself regularly (just in case the key requires changing and news is posted about it), why I only use trusted mirrors, why I download the GPG signature and package from seperate mirrors and why nothing gets installed without that GPG check succeeding. To be honest, I see this just as being sensible for anything that goes into production. I'm still waiting for a rogue Windows Update, but it surprises me that some distros (or administrators) don't ALREADY do all this...
Why do we still have 'mirrors' in this day and age? It made some sense when Internet users had beards and understood the network topology and how many hops they were away from Kevin Bacon's VAX. In those days you also needed to specify email addresses giving the routing explicitly, as foovax!kremvax!buffy!user. But that long since got replaced with giving everyone a fixed address and letting the computer figure out the boring details of how to route it.
Why not just use Bittorrent to distribute package updates and forget about the whole mirror network? People who spend time setting up and maintaining mirrors can enjoy more free time to hack on more interesting things.
-- Ed Avis ed@membled.com
I think the response time of most Linux or BSD distributions is admirable where security patches are concerned, but the Debian OpenSSL fiasco was a longstanding bug that went undetected for years and will have ramifications for a very long time. It didn't merely affect running SSH servers, it made it easier to crack a variety of keys, regardless of their strength. Many of these were generated a long time ago and will remain in production until some overworked admin gets a chance to test the keys for the vulnerability. The test itself is pretty easy, and victims will be open to a variety of attacks. Those in the know are already incurring a labor cost of replacing SSL certificates (thank god the commercial CAs stepped up to the plate and offered free replacements), and face the even harder task of identifying vulnerable SSH keys used for public key authentication to their systems (any system, not just Debian-based). There may even be mounds of data that's been encrypted and archived, but not really safe. I'm extremely happy I switched from Debian before this bug was introduced (I loved Woody, but couldn't stand the way they were tampering with upstream source in Sarge, an instinct that I'm glad I followed). But I've watched other shops jump on the Ubuntu wagon for servers, because it's "easy", and all they've done is run the update. They haven't made an attempt to replace vulnerable keys. This has set the stage for a smear campaign against Linux in the enterprise, which is undeserved for most distributions. Kudos to Debian for their response, but this mess is reminiscent of the Victor Hugo story, "A Fight with a Cannon", where the hero is given a medal, then executed for being the cause of the catastrophe in the first place.
The problem is not old or modified packages being supplied. That was somebody's early lazy reading.
This has been noted above, and I'll note it here to hopefully clog one loose parse.
The problem is the ability to harvest IP addresses of vulnerable machines, and the time it might (not) take to succeed in an attack.
In other words, before the package is updated.
1: Bad guys watch a perfectly good mirror sending updates out. They own the mirror, but they maintain it properly, so no one has reason to suspect the mirror is dangerous.
2: Bad guys have installed on the mirror a script that logs the IPs of machines downloading updates. The script probably analyzes the IPs, probes a little in a non-invasive way, then, when it finds something that looks like it has left the default port open to the internet, launches the attack. With a little luck (and a finite time to complete the update), the attack completes before the update.
It's a (well-known, really) window-of-opportunity issue, supposedly exacerbated by the volunteer nature of mirrors. Volunteerism is the so-called vulnerability.
The "workaround" is to (follow best practices and) shut down default ports. (This has also been mentioned above.) That will probably close the window of opportunity. And, then, there's also the technique of running incoming and outgoing ports on separate port numbers, etc.
If the attacker can't find your ssh port, it's going to bee difficult for him/her to take advantage of a vulnerability in ssh, etc.
Now, to consider why this is FUD against voluntarism, consider whether Microsoft's paid update network employees are paid enough to refrain from pulling a trick like this. Can money prevent this better than voluntarism can?
(There is also the issue of whether Microsoft can fix vulnerabilities in thei)r own update servers to avoid them getting owned between the vulnerability being discovered and the vulnerability being fixed..
Computer memory is just fancy paper, CPUs just fancy pens with fancy erasers; the 'net is just a fancy backyard fence.
How do you even know the world exists? Maybe you're part of an experiment by an evil genius who has your brain in a jar and all of this is just a creation of his making? Nothing you know actually exists.
Let me know when you have a 14lb. cat with bigass claws who likes to sit in your lap and use you as a scratching post.
"When information is power, privacy is freedom" - Jah-Wren Ryel
... converting CRLF to newlines...?
I don't think you did.
"The cup is in turn designed for holding hot or cold liquids, and has an open rim and closed base." --US Patent #5425497
If I'm not mistaken, using Debian you can grab source packages as opposed to binaries simply by running apt-get source packagename rather than apt-get packagename
Tricking people into believing that the hash they get from the web site is the correct one is a tiny alteration to Firefox. Any hacker worth their salt would, of course, do that if they tried to take over your machine, and checking the hash from a machine that's potentially compromised is useless.
In yum (which I will assume you are using since it's the most popular RPM based package manager and you refer to rpms) there are no signatures on the repository metadata or package metadata. There are signatures on the packages themselves, but not the extracted package metadata.
The fundamental way the replay attack works is that I can have my mirror host outdated versions of packages that are correctly signed. I can use an old repository metadata file or write my own repository metadata file that refers to package metadata and packages that are out of date. A client may ask to install firefox and I would provide a correctly signed version of firefox-2.0. Clearly if this package contains a vulnerability that I can exploit remotely, I now can control your system.
The fact that the package metadata in the tarballs isn't signed also enables some other attacks. Yum will blindly trust any package metadata it receives. So when you download the package metadata for firefox, I can modify the package metadata to say it depends on additional packages (which will cause you to install them as well), say that it has a dependency on a package that does not exist (which prevents you from installing firefox at all), or return a huge list of packages to attack another repository (details on website / papers). Yum does not check any signature until after it finished dependency resolution and downloads all of the packages.
Surprisingly yum does not verify that the package metadata it received from the repository matches the metadata embedded in the signed package. So if I say firefox depends on apache (both packages correctly signed), yum will install both even though after downloading it could check the firefox RPM to see it doesn't depend on apache.
We think that the right way to tackle this problem isn't to sign the package metadata, but the repository metadata. Since the repository metadata has the secure hash of the package metadata (in most repository layouts), signing the repository metadata covers all of the content on the mirror / repo. We advocate using timestamps and expiration times to prevent an attacker from providing an older but correctly signed version of the repository metadata or freezing the metadata perpetually. (please see our papers for more information)
Anyways, we understand the skepticism and appreciate the spirited emails and slashdot comments we've received about these vulnerabilities!
Justin Cappos
(*) Not all package managers download package metadata in separate tarballs (some actually scrape information from the headers of the packages on the repository). Regardless of how the information is stored, if it's not signed then an attacker can substitute malicious data. I'm simplifying here to make the discussion clearer and more generally applicable to a wide range of package managers.
if you picked your own mirror, you already trust them.
While this is a fairly common opinion, we believe this is the wrong security model for package distribution. I mean, if you trust your mirror, why sign packages in the first place? I would argue that if a distribution is going to support outside mirrors it needs to ensure that a mirror cannot compromise the security of the clients in any worse way than refusing to serve requests.
Also it's worth noting that there are many tools like Source Selector, netselect-apt, etc. which automatically choose a single mirror for you based upon bandwidth, etc. An attacker with a well provisioned system could obtain a huge number of clients.
Justin Cappos
I can only talk about the distro I'm using, which is Ubuntu 8.04. The ~stupid code~ was fixed, but the stupid patch also tried to find all the keys on your system and generate new.
Still a pain, but as much as was humanly possible was fixed automatically.
I agree that the ssh bug was *VERY* bad and had a particularly stupid source, but the way it was handled was adequate.
Now, how do you think Apple or Microsoft would have handled a similar problem?
"DRM is like the Ford Pinto: it's a smooth ride, right up the point at which it explodes and ruins your day."-C.Doctorow
Well, you can save yourself quite a bit of time by combining the downloading and untarring in one streamlined step:
(Some non-GNU tars require "| tar xzf -" instead, or even "| gunzip | tar xf -".)
Which brings me to another question: why the heck does everyone include the "v" in their tar invocations? Are you that obsessed with watching the contents of the file scroll past your eyes, cluttering up your terminal and actually slowing down the untarring process measurably with all that pointless I/O? Or do people just cargo-cultishly copy everyone else without the faintest understanding of what those letters mean?
Right on! It is amazing how current that article remains.
How far down the food chain do you have to go to insure the integrity of your systems? Are you even sure that the main CPU on your system doesn't have extra instructions that remain undocumented? As complex as the current CPU's are, they could have entire subsections devoted to malware and no one would notice until those sections were activated and did what they were designed to do.
For every problem there is a solution that is simple, obvious and wrong.
As an aside, here's the famous "don't use kill -9" letter:
No no no. Don't use kill -9.
It doesn't give the process a chance to cleanly:
1) shut down socket connections
2) clean up temp files
3) inform its children that it is going away
4) reset its terminal characteristics
and so on and so on and so on.
Generally, send 15, and wait a second or two, and if that doesn't
work, send 2, and if that doesn't work, send 1. If that doesn't,
REMOVE THE BINARY because the program is badly behaved!
Don't use kill -9. Don't bring out the combine harvester just to tidy
up the flower pot.
Well, you could look at the GDI issue from the Microsoft side. They released a tool to try to find vulnerable code in libraries used by the third parties.
You can of course do that. Hell, if you really wanted you could even do the other steps on one line too, but that's not quite what I was trying to illustrate with my example. Also, there is the fact that you're downloading data from the internet; It's a lot easier to resume when you have a partial file.
As for the -v option, I'm not sure about everyone else, but I simply feel better when I see signs of progress on my screen. It's a lot more relaxing to see a few thousand files scroll by than to sit there and wait while your operation finishes. Silence can, after all mean many things: it may have crashed, your SSH may have timed out, or any number of other nasties could have happened. True, it's not as much of a concern for tar, but to be fair, I do quite a bit of debugging for a huge and quirky program. For me an empty screen often means some very unpleasant problems.
if you are already compromised then there is fuck-all you can do to prevent further compromise anyways.
Snowden and Manning are heroes.
"Debian security updates ONLY come from a trusted server (security.debian.org)"
So that means...
1) If I owned a single server (security.debian.org) I own all updating machines in the whole world (good to know).
2) If I DoS security.debian.org I'll win a window of opportunity to all debian servers in the whole world (good to know).
3) Since security.debian.org serves through http and ftp it's opened to a MiM attack (good to know).
"It is impossible to forge a "newer" version than those on the security.debian.org server, so you won't miss the patch."
Who told you that? Say that openssh-server is currently at version 1:4.3p2-9etch2. Say I own apt.mymirror.com and that you use it on your sources.list file. Then, if I host an 1:4.3p2-9etch2 on my mirror you will update it next time you aptitude update && aptitude upgrade. There's nothing within apt structure that will restrict updates to just those from security.debian.org.
Now, *if* security.debian.org provided a signed packages.list (both for the "base" distribution and security updates) through https *and* packages.list both provided current packages list *and* their checksums you would:
a) Allow for security.debian.org to be securely mirrored (this way you'd avoid for the most part DoS, either deliberated or due to excess of "legal" traffic -I think to recall this already almost happened last year). Only packages.list would be forcibly downloaded from the security.debian.org master itself.
b) Avoid MiM-based attacks
c) A (currently to-be-developed) mechanism could be hooked to apt and related client-side utilities so for security update lists provided by a trusted source (https://security.debian.org by default) nothing but the package versions referred on that trusted packages.list would be accepted.
Note that even this wouldn't cope with the fact that if security.debian.org is owned, then everything is lost. But then again, security.debian.org would service just one port and service (https) and only a short list of trusted people would need access to it, so it could be reasonabily easy to secure against all but the most determined attacks.