Slashdot Mirror
Exploiting Wildcards On Linux/Unix
An anonymous reader writes: DefenseCode researcher Leon Juranic found security issues related to using wildcards in Unix commands. The topic has been talked about in the past on the Full Disclosure mailing list, where some people saw this more as a feature than as a bug. There are clearly a number of potential security issues surrounding this, so Mr. Juranic provided five actual exploitation examples that stress the risks accompanying the practice of using the * wildcard with Linux/Unix commands. The issue can be manifested by using specific options in chown, tar, rsync etc. By using specially crafted filenames, an attacker can inject arbitrary arguments to shell commands run by other users — root as well.
Who does NOT use -- in their scripts, if they're safety conscious?
rm -i -- *
Problem solved?
Normal programs should stop processing options after a (standalone) "--" and take everything following it as regular parameters. getopt and similar libraries handle this automatically.
I really wouldn't class the "use of wildcards" as a security risk - the security risk is the developer that doesn't know what he's doing. /" as root immediately afterwards?
Would command line handling be a security risk, if someone would add a --superuser-rm option to his code and execute "rm -rf
posting the answer to this useless story that was posted to FD
Date: Thu, 26 Jun 2014 12:55:42 -0700
From: Michal Zalewski
> We wanted to inform all major *nix distributions via our responsible
> disclosure policy about this problem before posting it
I'm not sure how to put it mildly, but I think you might have been
scooped on this some 1-2 decades ago...
Off the top of my head, there's a rant about this behavior in "The
Unix-Haters Handbook", and there are several highly detailed articles
by David Wheeler published over the years (e.g.,
http://www.dwheeler.com/essays/filenames-in-shell.html).
Yup, it's a counterintuitive behavior that leads to security problems. /mz
The odds of changing the semantics at this point are very slim. Other
operating systems have their own idiosyncrasies in this area - for
example, Windows it not a lot better with parameter splitting and
special filenames.
drop tables *
After years of using command line programs daily I never heard of -- before today. It was never brought up in school, nor did I see any specific thread / blog post on the subject. So to answer your question, I don't. I've never heard about that before. Where did you learn about that ?
So why would the expected method not be the default? This is exactly how security problems are born.
While the paper is an interesting writeup, there's nothing really new in there. A colleague used to exploit the same issue "for good," by touching a file named "-i" in directories he deemed important. Obviously, one could undo that by touching a file named "--". For users, I'd usually recommend always using ./* instead of just *, as well as directory/. instead of just directory
Is the wildcard expanded by the shell in PowerShell?
Really, this is well-known, non-surprising and will not happen to anybody with a security mind-set. Of course it will happen in practice. But there are quite a few other variants of code injection (which this basically is) that the same people will get wrong. Complete input sanitisation is mandatory if you need security. I mean, even very early Perl-based CGI mechanisms added taint-checking specifically for things like this. If people cannot be bothered to find out how to pass parameters from an untrusted source securely, then they cannot be bothered to write secure software.
The fix is not to change the commands. The fix is to exchange people that mess things this elementary up against people that actually understand security. Sorry, you cannot have developers that are cheap and competent at the same time and even less so when security is important.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
There is one great evil that Unix let into its filesystems long ago, one which Apple (which loves generate or perpetuate evil) put into its filesystem and that later Microsoft allowed because it was expedient to align with earlier Apple practice: spaces in file names. If we forbade spaces as well as control characters, things would be much better.
That is complete BS. Preventing users from doing things they legitimately want to do is not a valid approach to securing untrusted interfaces. The valid valid way is to sanitize the untrusted input before using it and only a complete moron will pass a wildcard from an untrusted source, unless it cannot do any harm where it is going.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
I remember reading about this in the 1991 release of "Practical Internet and Unix Security," from O'Reilly back in 1991. I'm pretty sure they even gave examples. They also laid out a number of suggestions to mitigate risk, including not specifying the current path, ".", in the root user's path so they must explicitly type the location of an executable script, and so on.
They also pointed out that some well-behaved shells eliminate certain ease-of-use-but-exploitable features when it detects that a privileged user is running it, and even on systems where that's not the standard, the default .bashrc or equivalent files often set up aliases for common commands that disable features like wildcard matching, or color codes (which could be used if you're very tricky, to match a filename color to the background color of the screen, among other things), the path restriction listed above, and many many others.
It's really hard to secure shell accounts on systems, no matter how you try. Is this article just proof that the current generation of unix admins is rediscovering this? Should I be shaking my fist and telling the kids to get off my lawn? This was old news 2 over decades ago.
I still hate the trickery I have always to put into my scripts just to deal with spaces in filenames.
find /my/files -print0 | xargs -0 do_some_stuff
With zsh you can type: do_some_stuff /my/files/**/*
I'm pretty sure you'll have a hard time trying to find a user who legitimately wants to pass arguments to command line tools by naming a bunch of files according to those arguments ;)
Maybe, but I don't think so. First of all, for this "exploit" to have been around so many years, it's interesting how I've never heard of it actually being used to hack or vandalize a system... second, if someone is already able to write arbitrary filenames, they're already into your system; if it's a normal user, you'd be able to track down who it is... it just seems like a really "weak" exploit, if I'd call it an exploit at all. IOW, IMO, nothing to see here.
Stupid sexy Flanders.
I understand why this works and I understand the need to sanitize user input, but this is dumb. Even if there are workarounds. It's obvious what the intent of "tar cf archive.tar *" is suppose to be, it shouldn't be treating file names as additional arguments. Anyone actively using this "feature" for anything legitimate is dumb too.
This seems very similar to the whole "we need some other language than C" argument. Sure, you *can* make secure code with zero overflow vulnerabilities, but damn near all software has them. You can only blame the user/coder for so long for doing something "wrong", but when 90%+ people are doing it "wrong" then you probably need to change how the thing works.
This is not an exploit, it's basic stuff any first year sysadmin should know. You know... quoting, ./, --, -i, -inum, IFS, etc. The author's stupid Linux 'ls' doesn't even order lexicographically correct. And dashes at the end of filename arguments are not interpreted as dashopts, that's not even posix.
If you want to learn unix and blog your newfound knowledge, great... just don't try to call it an exploit or secadv, you'll just make yourself look stupid.
Systems where user data can accidentally get mixed in control commands are dangerous. In addition to this shell trick, another example would be HTML, where you have to be careful to not let raw HTML data through your guestbook messages so that visitors can't inject HTML into the messages.
With competent and careful system administrators you can avoid problems, but it's still kind of a fragile design in my opinion.
Wake up. Not everyone is a developer. Not everyone has even 2 minutes of unix philosophy.
My Users are scientists, and they get to trash their home space here. These types of issues are most likely to happen when they are writing a script and it makes files for what should have been options.
My job isn't to teach them unix, it's to keep them happy and productive. They make mistakes, I clean them up and help them through the frustration of things going wrong.
This would seem a problem with universal implications, and one that largely depends on local access by the malefactor.
Er.. most of the exploits are only possible if one is root and/or the directory is writable for some other user (e.g. leon in this case).
Since one is root, one can do anything anyway so why bother with all this misdirection? If someone leaves world writable directories lying around (especially without the sticky bit set), then they deserve everything they get. Or is this some kind of "trap the (completely) unwary sysadmin" wake up call? If I see some strange named file (especially if I know I didn't put it there) I would investigate very, very carefully what is going on. I can't be alone in this - surely?
While that is indeed the solution, it is also true that it is too easy to forget. Perhaps one could modify all commands to require the use of the "--" separator, or to warn if it's not present, at least if some environment variable is set. That could be very helpful for people trying to write more secure code.
"the security risk is the developer that doesn't know what he's doing."
Not the hacker who does know what he is doing.
Unpacking 'shar' archives via 'sh' considered dangerous.
This is because the linux commands do not respect what the manual says:
man rm...
rm [OPTION]... FILE...
but in realitiy it's rather:
rm [OTION|FILE]...
whereas on other unix systems it works as expected, first the options, then the arguments
HP-UX
rm *
rm: DIR1 directory
Solaris
rm *
rm: DIR1 directory
So screw the GNU tools, they mess things up for the "old unix sysadmins"
Here is a nice linux/unix trap:
x=a
y="rm z"
f=$x $y
So you expect f to contain: a rm z
not really...
z: No such file or directory
so the rm actually was executed
a=$x is an environment variable attribution, so $y becomes an executed command...
And that one works on any unix/linux
Recently patched in chkrootkit (CVE-2014-0476)
Atari rules... ermm... ruled.
Nop, you can not just use --. because many commands do not understand --
Here is an article by dwheeler (a frequent slashdotter; often cited for his technique countering the trusting trust problem) about filenames.
http://www.dwheeler.com/essays...
I believe he is mostly right. We should move to file systems that do not allow "stupid" names and be done with it.
After years of using command line programs daily I never heard of -- before today. It was never brought up in school, nor did I see any specific thread / blog post on the subject. So to answer your question, I don't. I've never heard about that before. Where did you learn about that ?
RFM
...so wouldn't it be more accurate to to say that computers, like bull-dozers, can be dangerous in the hands of malicious, ill-informed, inattentive, or incompetent users? If you know of any of these archetypes, try to make them smarter, but don't allow them root privileges to anything taller than an ankle-high weed. Give them some locked-down version of Windows, without admin privileges, lots of monitoring tools and features. Consider helmets, knee-pads and child safety locks.
With zsh you can type: do_some_stuff /my/files/**/*
... provided that the number of files fits into the command line argument space (a common reason for using find/xargs rather than, say, wildcards/backticks, aside from the security issues).
Need to type accents and special characters in Windows? Use FrKeys
Let me check my dictionary for the defintion of idiot:
1. n: A user, especially super user, who uses * as an agument without first checking to see what * expands into.
2. n: A user who leaves his directories world writeable so others can put random garbage in them.
The one line summary for this story is bad things happen to people who use a command without knowing what the command does.
Don Dugger
"Censeo Toto nos in Kansa esse decisse." - D. Gale
Linux/Unix are an old design of an OS.
Old does not always equal worse. You ever hear the saying, "Those who do not know UNIX are doomed to reimplement it, poorly?" Similar to how rewriting code from scratch is very rarely the correct approach because you'll make a lot of the same mistakes over again.
Unity? Screw that: XFCE. Slashdot Beta? Screw that: SoylentNews. Australis? Screw that: Pale Moon. UX developers DIAF
Humans beings use spaces in the names they give to things or to other human beings. So, why their computers would have to behave differently?
Religion: The greatest weapon of mass destruction of all time
I remember reading about this in the 1991 release of "Practical Internet and Unix Security," from O'Reilly back in 1991. I'm pretty sure they even gave examples. They also laid out a number of suggestions to mitigate risk, including not specifying the current path, ".", in the root user's path so they must explicitly type the location of an executable script, and so on.
They also pointed out that some well-behaved shells eliminate certain ease-of-use-but-exploitable features when it detects that a privileged user is running it, and even on systems where that's not the standard, the default .bashrc or equivalent files often set up aliases for common commands that disable features like wildcard matching, or color codes (which could be used if you're very tricky, to match a filename color to the background color of the screen, among other things), the path restriction listed above, and many many others.
It's really hard to secure shell accounts on systems, no matter how you try. Is this article just proof that the current generation of unix admins is rediscovering this? Should I be shaking my fist and telling the kids to get off my lawn? This was old news 2 over decades ago.
Probably because anybody who's used the various Bourne-style shells for a while
considers it a feature, not a bug. This is a case where the Principle of Least
Surprise comes up with different answers for novice users and for experts:
"What? A * can expand into an unintended command argument?" "Yeah, what *else*
would it do - the shell is just globbing, it doesn't know for sure what the
command will do with the parameter".
Who asked for this feature? Can anyone give me a legitimate use case for "tar cf archive.tar *" evaluating as
tar cf archive.tar admin.php ado.php --checkpoint=1 "--checkpoint-action=exec=sh shell.sh"
instead of
tar cf archive.tar "./admin.php" "./ado.php" "./--checkpoint=1" "./--checkpoint-action=exec=sh shell.sh"
Back in '83, a friend challenged me to remove a file name "-rf *, without causing collateral damage.
General Relativity: Space-time tells matter where to go; Matter tells space-time what shape to be.
The examples given are potential admin tasks doing filesystem maintenance on a user's folder.
Is a basic feature of how unix shells work, and there's no way to change it.
How about adding ./ to the beginning of each filename output by *? Is there a single thing that would break by doing that?
Article should be called 'Exploiting wildcards on GNU'.
All of the examples in the original document work because GNU software can intersperse options and filenames freely. None of those exploits would work in BSD userland, for example.
Knowledge is power; knowledge shared is power lost.
If computers were conceived to execute user commands, then why is a command for matching file and dictionary names returning them in such a form that they become executable parameters, when they could easily be explicit filenames by adding ./ at the beginning?
Is making what should be basic and safe housekeeping functions like chmod * and tar * dangerous really something you actually want in Linux?
Because a lot of options take a filename as an argument, such as the "-o" option of any compiler or linker. How do you pass one of those without it getting expanded as a wildcard?
The real bug here is the same as in SQL injection attacks: A failure to safely distinguish between program and data!
I.e. when doing chown usr:grp *.php, the wildcard globbing should escape any special letters, particularly including white space and wild card characters.
This is the same idea as when you use prepare(... ?,?) on any sql statement with replaceable parameters, then execute() with the relevant dynamic values.
Terje
"almost all programming can be viewed as an exercise in caching"
Agreed. The solution would be for the OS to store underscores '_' but to display them as spaces ' '
Because, since the mainframe days, a new and unforeseen computer/user pairing had evolved. For emphasis, I'll say it again, evolved; never designed from the ground up. Smaller cheaper, but nonetheless ever more powerful, computers became available to a class of consumers spanning the general population, now networked to many other such computer/user pairs. These general-population consumers find themselves operating systems far beyond their ability (or time, inclination, etc.) to understand and safely control except as mediated by a deceptively enabling and presumed (to the user) protective interfaces. The problem is that computers are mechanisms designed to automate and execute instructions. That's what they are; and, networks are necessarily designed to facilitate communication. That's what they do. Yet, underlying system architectures and interfaces, though enabling, never evolved commensurately to the task of satisfactory trade-offs between function and security for ubiquitous employment in a general population. I have no idea what the solution is, but I suspect that we need to do some fundamental rethinking of secure architectures and user interfaces. Architectures need to more safely isolate data and logical functionality, and interfaces need to more safely mediate users interaction with devices. I confidently assert that the current architectures simply can't be secured, no matter how much junk is kludged to the task. Prove me wrong, please.
As may times DOS/Windows has been burned by bad design (Explorer hiding extension yet action dependent on extension being a big one) the lack of functionality helps it here. The * is not interpreted by the shell, but by the app. It sees a *, and does a library call to do the glob. So there's no chance of it being interpreted as a flag. My guess is that DOS was so restricted in memory, that executing a glob may have blown the stack, much like we use xargs on UNIX when a fileglob is too much.
Great suggestion, and much better than the one in the discussion thread linked from the summary about refusing to expand names starting with -. I don't think your suggestion would break anything, and it would eliminate the problem. It should be the default.
I know this is a Unix discussion but it's worth pointing out for those that don't know that DOS (or cmdshell) doesn't do wildcard expansion and if you want that, you have to take special steps (when I played with it, it involved linking with a different library). DOS commandline processing is very weird and (was) hard to find documentation for. I think I've spent many hours on trial-and-error trying to get things to work.
After years of using command line programs daily I never heard of -- before today. It was never brought up in school, nor did I see any specific thread / blog post on the subject. So to answer your question, I don't. I've never heard about that before. Where did you learn about that ?
man bash:
"-- A -- signals the end of options and disables further option processing. Any arguments after the -- are treated as filenames and arguments."
Because it complicates argument processing, and path names. How do you separate / detect program arguments and filenames without a meta character?
To delete a filename with a space we have to quote it or double quote the whole filename
The whole dam issue would of been a non-issue if we could just specify underscores to signify spaces:
How is a shell supposed to know the complete path name?
There are other problems detecting valid filenames from options:
i.e.
I am a developer too, and you did not understand my point. I know how complicated it is for a computer to handle this situation, but you as a developer should make things easier/simpler/intuitive for the user, not to yourself (as a developer) or to the computer.
Religion: The greatest weapon of mass destruction of all time
I'm glad that people are learning about this problem. Sadly, it's not new, it's been known for decades. CERT’s “Secure Coding” item MSC09-C (Character Encoding — Use Subset of ASCII for Safety) specifically discusses the vulnerabilities due to filenames. The Common Weakness Enumeration (CWE) includes 3 weaknesses related to filenames (CWE 78, CWE 73, and CWE 116), all of which are in the 2009 CWE/SANS Top 25 Most Dangerous Programming Errors. My freely-available book on writing secure software has a whole section about filenames. And so on.
We need to fix the problems with Unix/Linux filenames, not just keep rediscovering them. In particular, ensuring that filenames had no control characters, no leading dashes, and used UTF-8 encoding would simplify developing correct programs. Most people writing software already follow these rules. We don't need to make it easy for attackers.
- David A. Wheeler (see my Secure Programming HOWTO)
The -- end of options option is a GNU getopt extension. It is, if memory serves me correctly, not part of any standard. This means that program that were not compiled with glibc, or programs that do not use getopt/getopt_long, may or may not honor it.
Even by simply looking at the man page, it is easy to spot programs that don't use getopt. Any program that accept multi-letter options with one minus sign is, obviously, not using getopt (e.g. gcc, find).
Then there is git. Git uses -- to mean "no revisions after this point. Any remaining argument must be a file name". This is almost, but not quite, what -- means for getopt. I don't even know what underlying parsing engine they are using (could be getopt with the "no options after first argument" option set).
The "./*" feature is a more global workaround, if it is applicable.
Shachar
And not everybody has any business writing security-critical code. You are asking for the illiterate to be allowed and encouraged to write poetry. That cannot ever work.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
You do not seem to understand what this discussion is about. It is about security, not safety. And when it is about security, the wildcards come from somewhere else and need to be sanitized in that path. A user doing this to himself is just stupid, but not a security issue. (And yes, as somebody that once had to recompile bash to get a longer commandline-buffer, I know exactly where the expansion happens.)
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
You have noting to contribute but could not keep your mouth shut?
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
Yeah, by not giving people a commandline worth anything in the first place. But you are right: MS realizes their customers are incompetent, immature whiners that would blame MS is they used MS products to shoot themselves in the foot. So the function to shoot anything was removed.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
I have no idea what the solution is, but I suspect that we need to do some fundamental rethinking of secure architectures and user interfaces. Architectures need to more safely isolate data and logical functionality, and interfaces need to more safely mediate users interaction with devices. I confidently assert that the current architectures simply can't be secured, no matter how much junk is kludged to the task. Prove me wrong, please.
On the other hand this specific issue could be easily solved by * prefixing all filenames with ./
So far I've not heard of anything that would break, and it's silly arguing that this specific problem is part of required functionality and not something that can/should be fixed when it appears to have such a simple solution.
... is there an example you can point to?
Yes, there is a workaround you can use, if you know about it and remember it every time, to enable the safe behaviour. That does *not* count as 'problem solved'. To solve the problem, the safe behaviour needs to be the default, with the funky and unsafe behaviour of treating filenames as extra switches being the one you have to enable specially. Really - what are the odds that the user or programmer *intends* for a file called --foo to be treated as an option specifier when they expand a wildcard? Conceptually the fix is not hard. Each element in argv gets an associated flag saying whether it is a filename - and if it is marked as a filename, getopt() or whatever does not treat it as an option specifier even if it begins with the - character. Alternatively, filenames beginning - could simply be disallowed.
-- Ed Avis ed@membled.com
you are not precise on one point
In unix world, this long known issue was only about the first filename. Typically with classig usage of tar, the first file is your archive, and all others are just filenames. unix/bsd treat them as filenames.
Now comes GNU getopt and "suddenly" decides that from now on it would be cool to have options anywhere on the command line. Result is that in GNU-world, tar is now vulnerable (and many many other commands) which had had no issue at all previously.
I would really like that getopt changes back to "if we see filenames, then consider being after --" or rather: stop parsing options if you find a parameter which does not match any option. That's the last one. Just stop there, do not continue. Leave the rest as it is.
Atari rules... ermm... ruled.