Exploiting Wildcards On Linux/Unix

An anonymous reader writes: DefenseCode researcher Leon Juranic found security issues related to using wildcards in Unix commands. The topic has been talked about in the past on the Full Disclosure mailing list, where some people saw this more as a feature than as a bug. There are clearly a number of potential security issues surrounding this, so Mr. Juranic provided five actual exploitation examples that stress the risks accompanying the practice of using the * wildcard with Linux/Unix commands. The issue can be manifested by using specific options in chown, tar, rsync etc. By using specially crafted filenames, an attacker can inject arbitrary arguments to shell commands run by other users — root as well.

Comments

Question... -- ?

[beh]

Who does NOT use -- in their scripts, if they're safety conscious?

        rm -i -- *

Problem solved?

Normal programs should stop processing options after a (standalone) "--" and take everything following it as regular parameters. getopt and similar libraries handle this automatically.

I really wouldn't class the "use of wildcards" as a security risk - the security risk is the developer that doesn't know what he's doing.
Would command line handling be a security risk, if someone would add a --superuser-rm option to his code and execute "rm -rf /" as root immediately afterwards?

Lets quote FD while we're at it

[Anonymous Coward]

posting the answer to this useless story that was posted to FD

Date: Thu, 26 Jun 2014 12:55:42 -0700
From: Michal Zalewski

> We wanted to inform all major *nix distributions via our responsible
> disclosure policy about this problem before posting it

I'm not sure how to put it mildly, but I think you might have been
scooped on this some 1-2 decades ago...

Off the top of my head, there's a rant about this behavior in "The
Unix-Haters Handbook", and there are several highly detailed articles
by David Wheeler published over the years (e.g.,
http://www.dwheeler.com/essays/filenames-in-shell.html).

Yup, it's a counterintuitive behavior that leads to security problems.
The odds of changing the semantics at this point are very slim. Other
operating systems have their own idiosyncrasies in this area - for
example, Windows it not a lot better with parameter splitting and
special filenames. /mz

Re:Lets quote FD while we're at it

[gweihir]

It may be counter-intuitive for people that have very little experience with a UNIX commandline. All others did run in the issue at some time that they could create, but not easily delete a filename "-v" or the like. But people with very little UNIX commandline experience have zero business writing security critical software that uses the commandline tools!

This is a complete non-issue. Incompetent people will usually screw security up and this is just one of the countless ways to do it.

Re:Lets quote FD while we're at it

[Anonymous Coward]

But people with very little UNIX commandline experience have zero business writing security critical software that uses the commandline tools!

The problem is that bugs with quoting are non-obvious and turn things that should definitively be trivial and secure into potential security problems. And it's not just scripts, it's possibly everyday use of the shell that may get subverted.

Just consider all the Cross Site Scripting flaws, they are just another flavor of the same problem.

Re:Lets quote FD while we're at it

[Gunstick]

I just want to state the UNIX does not behave like that, it's GNU who does.

Re:Lets quote FD while we're at it

[gweihir]

If you do not do input sanitization, you are screwed anyways. If you do input sanitization, this is not a security problem anymore. Stop trying to fix problems in the wrong place.

Re:Lets quote FD while we're at it

[badkarmadayaccount]

I use z/OS you insensitive clod!

If only this was a Microsoft issue.

[jellomizer]

If it were a Microsoft Issue, this would be so a bug and not a feature.

Linux/Unix are an old design of an OS. There are some designs in its main way of doing things that do not work in today's much more secure environment.

Things have been upgraded Telnet replaced with SSH, hacks on FTP to make it more secure. But the underpinning is still there. Back in the day where computers needed to do things.

Re:If only this was a Microsoft issue.

[Anonymous Coward]

There is one great evil that Unix let into its filesystems long ago, one which Apple (which loves generate or perpetuate evil) put into its filesystem and that later Microsoft allowed because it was expedient to align with earlier Apple practice: spaces in file names. If we forbade spaces as well as control characters, things would be much better.

Re:If only this was a Microsoft issue.

[gweihir]

That is complete BS. Preventing users from doing things they legitimately want to do is not a valid approach to securing untrusted interfaces. The valid valid way is to sanitize the untrusted input before using it and only a complete moron will pass a wildcard from an untrusted source, unless it cannot do any harm where it is going.

Re:If only this was a Microsoft issue.

[jones_supa]

I still hate the trickery I have always to put into my scripts just to deal with spaces in filenames.

find /my/files -print0 | xargs -0 do_some_stuff

Re:If only this was a Microsoft issue.

[itzly]

With zsh you can type: do_some_stuff /my/files/**/*

Re:If only this was a Microsoft issue.

[beelsebob]

I'm pretty sure you'll have a hard time trying to find a user who legitimately wants to pass arguments to command line tools by naming a bunch of files according to those arguments ;)

Re:If only this was a Microsoft issue.

[gfxguy]

Maybe, but I don't think so. First of all, for this "exploit" to have been around so many years, it's interesting how I've never heard of it actually being used to hack or vandalize a system... second, if someone is already able to write arbitrary filenames, they're already into your system; if it's a normal user, you'd be able to track down who it is... it just seems like a really "weak" exploit, if I'd call it an exploit at all. IOW, IMO, nothing to see here.

Re:If only this was a Microsoft issue.

[Anonymous Coward]

Why are spaces bad? All it needs is a escape sequence.

Re:If only this was a Microsoft issue.

[Anonymous Coward]

You don't seem to understand shell variable expansion very well. The wildcard is not passed to the invoked application.

The underlying applications are not broken as they have no concept of the user shell. They take a list of arguments and process those arguments.

Expansion of a wild card occurs within the shell and the shell has no pre-wired knowledge of how an application should work. ie, it does not know that rm cannot determine the dash prefixed argument is really a file name. It simply passes on the arguments to an application. It does not matter that strange and powerful shell magic was used to create those arguments.... ie *

Re:If only this was a Microsoft issue.

[Anonymous Coward]

Wildcard expansion happens in the shell, not in the program.
The program doesn't know if a particular argument came from a wildcard expansion or from somewhere else.

Re:If only this was a Microsoft issue.

[psmears]

With zsh you can type: do_some_stuff /my/files/**/*

... provided that the number of files fits into the command line argument space (a common reason for using find/xargs rather than, say, wildcards/backticks, aside from the security issues).

Re:If only this was a Microsoft issue.

[TangoMargarine]

Linux/Unix are an old design of an OS.

Old does not always equal worse. You ever hear the saying, "Those who do not know UNIX are doomed to reimplement it, poorly?" Similar to how rewriting code from scratch is very rarely the correct approach because you'll make a lot of the same mistakes over again.

Re:If only this was a Microsoft issue.

[TheDarkMaster]

Humans beings use spaces in the names they give to things or to other human beings. So, why their computers would have to behave differently?

Re:If only this was a Microsoft issue.

[Anonymous Coward]

It's simple to forbid spaces and control characters in file names -- just don't use them, and avoid software that does (which is usually pretty easy, since avoiding spaces, uppercase, symbols etc in filenames has been standard unix convention since the epoch).

Re:If only this was a Microsoft issue.

[UnknownSoldier]

Agreed. The solution would be for the OS to store underscores '_' but to display them as spaces ' '

Re:If only this was a Microsoft issue.

[Richy_T]

I know this is a Unix discussion but it's worth pointing out for those that don't know that DOS (or cmdshell) doesn't do wildcard expansion and if you want that, you have to take special steps (when I played with it, it involved linking with a different library). DOS commandline processing is very weird and (was) hard to find documentation for. I think I've spent many hours on trial-and-error trying to get things to work.

Re:If only this was a Microsoft issue.

[Anonymous Coward]

Many humans do not put spaces between the names of things or other humans beings.

Sincerely,

Your_Friendly_Robot_Overlord

Re:If only this was a Microsoft issue.

[UnknownSoldier]

Because it complicates argument processing, and path names. How do you separate / detect program arguments and filenames without a meta character?

To delete a filename with a space we have to quote it or double quote the whole filename

rm foo\ bar
rm "foo bar"

The whole dam issue would of been a non-issue if we could just specify underscores to signify spaces:

rm foo_bar

How is a shell supposed to know the complete path name?

C:\Program Files\foo.exe # doesn't work .. Files\foo.exe is an argument to C:\Program
"C:\Program Files\foo.exe" # need to double quote the path
C:\Program_Files\foo.exe # my solution

There are other problems detecting valid filenames from options:
i.e.

echo "Filename is called minus f" > -f
rm -f # doesn't actually delete file "-f"
rm ./-f # this one does

Re:If only this was a Microsoft issue.

[TheDarkMaster]

I am a developer too, and you did not understand my point. I know how complicated it is for a computer to handle this situation, but you as a developer should make things easier/simpler/intuitive for the user, not to yourself (as a developer) or to the computer.

Re:If only this was a Microsoft issue.

[gweihir]

You do not seem to understand what this discussion is about. It is about security, not safety. And when it is about security, the wildcards come from somewhere else and need to be sanitized in that path. A user doing this to himself is just stupid, but not a security issue. (And yes, as somebody that once had to recompile bash to get a longer commandline-buffer, I know exactly where the expansion happens.)

what about?

[Joe_Dragon]

drop tables *

Re:what about?

[Penguinisto]

# rm -rf *.*

(I actually saw a Windows-centric guy do that once as root while he was learning Linux. The look of horror on his face as the entire box began to delete itself was hilarious...)

Re:what about?

[Penguinisto]

Oh, thought of another one, just to mess with other admins:

# chattr +i /*.*

Re:what about?

[Bradmont]

Unless you were, say, in /etc, this wouldn't really do much harm. The only file containing a . in my / is initrd.img, which even if it weren't a symlink, is easy to to regenerate.

Re:what about?

[Anonymous Coward]

Depends on the version of Unix that you're using. There a lot of non GNU variants of rm that will happily resolve .. and traverse it. In effect it became a rm -rf /

Re:what about?

[Anonymous Coward]

To be fair, if he was running on an old Sun OS 4.x box, in any directory right below root (such as /tmp), it would recurse upward (via ..) and begin deleting everything in root. I experienced this early in my career when I typed 'rm .*' in /tmp to clean out a bunch of hidden files (I'd only out of college for a few weeks and didn't realize Sun OS would behave this way...thankfully modern OSes no longer do), realized what was happening and managed to ^C before the system was completely unusable (though lots of commands, like ls, were already missing), and spent the weekend painstakingly recovering the missing bits. These days, with SaltStack it would take a few minutes with a rescue CD or thumbdrive to rebuild the machine with a vanilla install and recover all of the customiztions, but back then the systems were so customized and so unique (and so undocumented), it was actually faster to identify and recover all the missing bits than it would have been to rebuild the box and recover from (slow and possibly unreliable) tape backups.

Re:what about?

[Anonymous Coward]

Unless you were, say, in /etc, this wouldn't really do much harm. The only file containing a . in my / is initrd.img, which even if it weren't a symlink, is easy to to regenerate.

-r is the recursive switch causing it to visit every sub directory.

So, rm -rf *.* would delete any file in the entire file system with a . in it....including, as you point out, /etc.

Re:what about?

[Lorens]

Oh, thought of another one, just to mess with other admins:

# chattr +i /*.*

Wouldn't notice until kernel upgrade time:

$ ls -d /*.* /initrd.img /initrd.img.old /vmlinuz.old

Re:what about?

[Anonymous Coward]

Unless you were, say, in /etc, this wouldn't really do much harm. The only file containing a . in my / is initrd.img, which even if it weren't a symlink, is easy to to regenerate.

-r is the recursive switch causing it to visit every sub directory.

So, rm -rf *.* would delete any file in the entire file system with a . in it....including, as you point out, /etc.

No, it wouldn't!

The shell expanse the wildcards before calling the command. All rm sees is "initrd.img" as argv[2].

rm will not see the *.* at all, unless the shell can't expand the wildcard to any valid file- or directory name and even if the shell had to forward the *.* as-is to rm (if *.* didn't match anything in /), rm still wouldn't find anything matching /etc/*.* as it doesn't do glob()'ering itself. Why would it? The shell already did that.

Furthermore -r means "visit any subdirs of the dirs given as arguments to rm ", not "all directories there ever was and ever will be".

TL;DR: You have no idea what the hell you're talking about.

Damn it, /. You used to be cool and know this stuff. :-(

Re:what about?

[Penguinisto]

Most wouldn't run it in / , but rather in some odd subdirectory, in which case it will stumble across "..", work its way up, then find "." - whereupon most *nixes will immediately devour ./usr , ./etc, ./var, and suchlike.

I think they finally fixed this behavior in recent distro releases, but the incident in question happened in 2006-ish; can't recall exactly if it was an RHEL 2.x box or an older FreeBSD one, but it was hella fun to watch.

Re:what about?

[Anonymous Coward]

Whoosh! No one is saying *.* gets passed to rm. They're saying *.* resolves to "initrd.img . ..". Meaning rm sees . and .. as well as initrd.img. Makes your post pretty funny, though.

Re:what about?

[jandrese]

You forget that in modern RE, '.' is a wildcard as well. *.* is a bastard of old and new styles, but most shells will take it and run with it, happily globbing every single thing in the directory.

The -f flag to rm should be using sparingly. Also, people just learning how to work on machines should not have administrator accounts.

Re:what about?

[Sun]

On my system (Debian GNU/Linux Jessie with Bash version 4.3.11), ls -lad *.* does not return either "." or "..". This is unlike ".*", which return both.

I have not found the precise rules of how * is expanded, but it seems pretty universal that if the pattern doesn't begin with a dot, files beginning with a dot are not matched.

Other systems (and, more to the point, shells) might behave differently, of course.

Shachar

Re:what about?

[Sun]

At most, this will delete one directory above the one you're in. If you're in e.g. /home/shachar, this will delete /home, but not /usr or /etc.

Shachar

Re:what about?

[John Allsup]

*.* is a DOSism -- in *NIX shells it matches all files who have a . in their name, such as 'file.png', but will not match a file called 'hello' which does not have a dot. sudo rm -rf /*.* won't do a lot, whereas sudo rm -rf /* will.

Re:what about?

[Anonymous Coward]

Whoosh! No one is saying *.* gets passed to rm. They're saying *.* resolves to "initrd.img . ..". Meaning rm sees . and .. as well as initrd.img. Makes your post pretty funny, though.

... and that's wrong too. If it were true echo *.* should mention . and .. too, right? Check for yourself...

Question... -- ?

[Marneus68]

After years of using command line programs daily I never heard of -- before today. It was never brought up in school, nor did I see any specific thread / blog post on the subject. So to answer your question, I don't. I've never heard about that before. Where did you learn about that ?

Re:Question... -- ?

[drinkypoo]

Where did you learn about that ?

RTFM[anpages.] It's literally in the system documentation. Granted, not all commands have such an option. Knowing which do is your responsibility. Arguably, all commands should have such an option.

Re:Question... -- ?

[Marneus68]

I just check and indeed. Every application made with getopt implements this mechanism. It's good to know.

Re:Question... -- ?

[locofungus]

Back in the (iirc) bsd 4.2 days, su was a suid shell script - at least on the machines I was using at the time.

Setup a symlink to su called -i

$ -i
# rm -- -i
#

There was a security bug handling suid shell scripts where the user was changed and then the #! interpreter was run, i.e. /bin/sh -i

and you got an interactive root shell :-)

Was very informative when the 'script kiddies' (although I don't recall that term existing in those days) had symlinks called -i in their home directory that they didn't know how to delete ;-)

Re:Question... -- ?

[PIBM]

Are you running commands, with root on stuff you don't know where it comes from ?

If you absolutely have to go run the query in the folder into which someone has upload/file creation, then at least use ./* if the tool doesn't support --.

This is not an issue if you work recursively on the directory holding whatever they want, which should cover most of the situations. A bad admin can run queries that are as dangerous or worst pretty fast!

Always be cautious when running someone else shell script, that's even more dangerous..

Re:Question... -- ?

[Anonymous Coward]

You've never had a stupid program crash and create a file named "--" or something similar in its working directory? Now try to remove the file without knowing about the "--" command line options.

Re: Question... -- ?

[Anonymous Coward]

rm ./--

Re:Question... -- ?

[Anonymous Coward]

You went to school to learn bash? Is this community college? Doing.it.wrong.

Re:Question... -- ?

[hawguy]

You've never had a stupid program crash and create a file named "--" or something similar in its working directory? Now try to remove the file without knowing about the "--" command line options.

rm ./--

Re:Question... -- ?

[TheCarp]

About as I would expect. A fellow admin and I were recently talking about the disown command and how after more than a decade on the job we are still finding out about commands that have existed since we were kids running around on the playground.

Most admins find out about -- after they run into a situation where they accidentally created a file with a name like "-f" go ahead, try and delete a file named "-f" any other way.

It works in many unix commands, actually "--" is a very common "end of options" signal. Really, any command that doesn't have a good syntax reason to not support it, really probably should. Many of the old ones do.

Most people I know who have learned shell learned it on the job in one way or another. There are often a lot of gaps. I did too, it took me a long time to get in the habbit of proper quoting, escaping etc, and this is definitely an easy one to miss.

Re:Question... -- ?

[Anonymous Coward]

I learned about it when I configured samba to share to windows clients, the issue was in that the customer had some files starting with dashes, and thus an older Red Hat distro couldn't handle the prefix depending on the command used unless I had -- added before the final directory parameters

Re:Question... -- ?

[TheCarp]

ROTFL I never even thought of passing it as a path name!
Its amazing how the solution that is right there can be so hard to see. Like many, I found out about -- years ago after swearing at my terminal for a while before resorting to reading the rm man page.

In fact, if your script generates path names instead of changing working directory, then it solves the entire
original issue as well.

I might start using ./ a lot more now.

Re:Question... -- ?

[Anonymous Coward]

You are the kind/level of admin who would never be granted access to critical systems. You don't learn Unix in school. You learn it by reading. And '--' is right there in sh(1). Of all the things you should read in Unix, that's the most important one, and you never read it. Neither did your 'teacher'. That's bad.

Re:Question... -- ?

[Anonymous Coward]

If the default behavior of a command was --, then you couldn't pass argument flags.

Re:Question... -- ?

[mooingyak]

Can't speak for the OP, but I once accidentally created a file name '-r'. Trying to remove it eventually led me to discover '--', but I don't expect most people to know about it.

Re: Question... -- ?

[mooingyak]

The painfully obvious solution I never thought of....

Re:Question... -- ?

[Gunstick]

this is the main problem cause: getopt messes it up, or something else in GNU

the oldschool unix systems do not behave that way. First the options, then the files.

Re:Question... -- ?

[Gunstick]

ordered ls:
LANG=C ls

Re:Question... -- ?

[Anonymous Coward]

I might start using ./ a lot more now.

So, you learned about ./ on /.?

Re:Question... -- ?

[Bazman]

Any other way? How about this way:

rm ./-f

Re:Question... -- ?

[Anonymous Coward]

/. too

Re:Question... -- ?

[Gunstick]

no it does not
The shell is not doing the -rf
But it does the pipes, the rm does not do the pipes.
So a pipe charater will not open a pipe
And I also tested it, to be sure what I say.

Re:Question... -- ?

[beh]

Sorry, if that appears harsh - but sometimes it pays to read manuals and try and understand what you're doing and how the stuff works.

I don't exactly remember when I learnt it first - but I DID already know when I also got told about it during my CS BSc degree course (probably 1st or 2nd year - which would place it about 1998-2000).

If you need to code stuff "securely", you need to understand how stuff works -- I don't think of myself as a particularly apt security coder or hacker - I mainly specialise on internal systems integration, not so much web or other front-end stuff, so I have the luxury that I already know the data is "sane", before it gets to me - and I "only" need to figure out how to transform it and where to send it on to.

Here are a few pointers, where you might read about it:

http://pubs.opengroup.org/onli...
"Guideline 10:
The first -- argument that is not an option-argument should be accepted as a delimiter indicating the end of options. Any following arguments should be treated as operands, even if they begin with the '-' character."

Even wikipedia mentions it - even though not strictly a "developer" resource:

http://en.wikipedia.org/wiki/C...

"In Unix-like systems, the ASCII hyphen-minus is commonly used to specify options. The character is usually followed by one or more letters. Two hyphen-minus characters ( -- ) often indicate that the remaining arguments should not be treated as options, which is useful for example if a file name itself begins with a hyphen, or if further arguments are meant for an inner command. Double hyphen-minuses are also sometimes used to prefix "long options" where more descriptive option names are used. This is a common feature of GNU software. The getopt function and program, and the getopts command are usually used for parsing command-line options."

If that's too far to go - try "man getopt" on your linux machine:

"
The parameters getopt is called with can be divided into two parts:
options which modify the way getopt will parse (options and
-o|--options optstring in the SYNOPSIS), and the parameters which are
to be parsed (parameters in the SYNOPSIS). The second part will start
at the first non-option parameter that is not an option argument, or
after the first occurrence of `--'. If no `-o' or `--options' option
is found in the first part, the first parameter of the second part is
used as the short options string.
"

man rm - and even rm --help on linux show it:
"
To remove a file whose name starts with a '-', for example '-foo', use
one of these commands:

rm -- -foo
" ...though without explaining the "--" in general...

man chown doesn't mention it, but refers to the full documentation in texinfo and how to access it - that one says under "Common options"

"
`--'
Delimit the option list. Later arguments, if any, are treated as
operands even if they begin with `-'. For example, `sort -- -r'
reads from the file named `-r'.
"

The information is there - and in _lots_ of places - but it DOES requ

Re:Question... -- ?

[jones_supa]

Use "find" to delete the files. This way you avoid all the wildcard bombs. Look in /etc/init/mounted-tmp.conf in Debian/Ubuntu for an example:

# Remove all old files, then all empty directories
find . -depth -xdev $TEXPR $EXCEPT ! -type d -delete
find . -depth -xdev $DEXPR $EXCEPT -type d -empty -delete

Re:Question... -- ?

[fnj]

Most admins find out about -- after they run into a situation where they accidentally created a file with a name like "-f" go ahead, try and delete a file named "-f" any other way.

rm ./-f
Is the most dead-simple way of doing it and is portable to non-gnu-based systems, although even BSD has the double-dash option nowadays.

And there is always the old standby of insulating the operation from bash command line expansion:
perl -e 'unlink "-f";'

You could also, within a minute or so, write and compile a literally two-line C program to remove it. I don't understand the mystery.
#include <unistd.h>
int main() { unlink("-f"); }

Re:Question... -- ?

[Ocrad]

Where did you learn about that?

In the manual of any argument parser.

Re:Question... -- ?

[Anonymous Coward]

That is B.S.

If someone reads that, they do not think security. They think it is an escape to deal with files that start with - and that is where they file it in their head. You also have to understand about '*' and think about how the two would work together.

This is exactly why computer code is insecure.

Re:Question... -- ?

[TangoMargarine]

after swearing at my terminal for a while before resorting to reading the rm man page.

I find that half the time the swearing comes after trying to read the man page. Then it's time to fire up the old Google...

Re:Question... -- ?

[Anonymous Coward]

If the first file name starts with a dash, how do you know it's a file name?

Re:Question... -- ?

[TangoMargarine]

Can someone explain to me why all these program manpage references have e.g. "(1)" after them? Is it referring to a specific prototype with X number of arguments to invoke it or something? There aren't multiple programs in the PATH with the same name in a Linux install AFAIK.

Re: Question... -- ?

[Anonymous Coward]

Double dash is specified by POSIX and it sure as heck didn't originate with GNU. I never knew it was absent from early BSD, but i'm pretty sure it was in some later AT&T versions.

Re:Question... -- ?

[Penguinisto]

Always be cautious when running someone else shell script, that's even more dangerous..

If you aren't capable of auditing an untrusted-sourced script before you run/use it, then don't run it or use it.

Seriously.

I grab (cut+paste) script bits from online when I'm lazy, but I always take the time to audit the chunk of text, and insure that it doesn't do anything dumb before I use/incorporate it. Doing this gives me two benefits: First, I can double-plus insure that it doesn't do anything I don't want it to, and second, I learn a bit about the person who wrote it (and in some cases, I discover a trick or bit of info that I didn't know before.)

Re:Question... -- ?

[Chris Mattern]

I might start using ./ a lot more now.

I suppose you could say you learned about ./ on /.

Re:Question... -- ?

[Chris Mattern]

Can someone explain to me why all these program manpage references have e.g. "(1)" after them?

It's the manpage section. Section 1 is general commands, for example, while section 3 is library calls. Thus, if you want to see the man page for the printf command, you can say "man 1 printf", while if you want to see the man page for the printf system call, you can say "man 3 printf".

Re:Question... -- ?

[tlhIngan]

Nop, you can not just use --. because many commands do not understand --

"--" is understood by getopt(), a library function to parse command line options. getopt() can simply print a warning out stderr when it's called and comes across un-escaped operatnds.

Bonus points if it defaults to printing errors when it's run in a non-TTY environment (e.g., shell script).

Extra credit for programs like rm that pre-scan their arguments to see if they match files - so if you passed in "-r" in the option area without a -- somewhere further along and a file named "-r" exists, error out and suggest using --. So "rm -r file" would work, unless there's a file named "-r" in the directory. "rm -r -- file" would generate no noise, as would "rm -r -- -r"

Re:Question... -- ?

[Geeky]

This. See manual sections. You might have a system call with the same name as a command, and the man pages will be in different sections.

Re:Question... -- ?

[Anonymous Coward]

Because reading man pages is not an exercise in frustration, patience, and mystical interpretation.

Re:Question... -- ?

[virx]

man man

              The table below shows the section numbers of the manual followed by the types of pages they contain.

              1 Executable programs or shell commands

              2 System calls (functions provided by the kernel)

              3 Library calls (functions within program libraries)

              4 Special files (usually found in /dev)

              5 File formats and conventions eg /etc/passwd

              6 Games

              7 Miscellaneous (including macro packages and conventions), e.g. man(7), groff(7)

              8 System administration commands (usually only for root)

              9 Kernel routines [Non standard]

Re:Question... -- ?

[TangoMargarine]

Interesting that point 7 is the first time they offer an example.

wat

Re:Question... -- ?

[Anonymous Coward]

That would mess up listing file/dirnames with unicode characters.

Re:Question... -- ?

[beh]

I did not say purely that reading about -- should tell you about security alone. IIRC my original incident with -- was a colleague setting me a teaser on trying to find out how to delete a file called '-f'; and me first having to figure out, that 'rm ??' reads like delete all files with two character filenames (of which there was only the '-f' file), but not seeing that the ?? actually gets expanded to all the two character filenames by the shell; rm never sees the '??' but instead only sees the filenames - and obviously, it can't discern whether a parameter of '-f' was expanded from the filename -f or intentionally given as a parameter.

If you learn that - you'll get a better understanding of how the system works - and that _in turn_ will help you get a better grasp on what could or would go on and particularly, what could go WRONG, in a system.

Re: Question... -- ?

[Anonymous Coward]

-- was in the original AT&T getopt source code (circa 1983). At the time, flags weren't quite standard (some people were butthurt that AT&T went with - instead of +). GNU originally used +long-options. --long-options become the GNU standard in the late 80s/early 90s.

And yes, get off my lawn!

Re:Question... -- ?

[the_B0fh]

Uh, it's only been around since the mid-90s if not earlier.

Re:Question... -- ?

[the_B0fh]

No, it is not. Granted, GNU doesn't like man pages any more, but in general, reading documentation is the RIGHT THING TO DO.

Re:Question... -- ?

[Anonymous Coward]

sh(1) is documented in manpage section 1 (general commands). ie. man 1 sh. For something like chmod, there's a chmod(1) and a chmod(2), ie a command and a system call.

Re:Question... -- ?

[the_B0fh]

If you don't learn it from school, and don't learn it from slashdot, you don't know about it?!?!

No wonder all software sucks.

Re: Question... -- ?

[chelberg]

Emacs dired mode can delete any file.

Re:Question... -- ?

[Kremmy]

Think of it a little like any other form of studying. The frustration, patience, and mystical interpretation part? That's called living.

Re: Question... -- ?

[TheCarp]

I figured it might be, looks like you are right: http://pubs.opengroup.org/onli...
12.2 Utility Syntax Guidelines:

Guideline 10:
        The first -- argument that is not an option-argument should be accepted as a delimiter indicating the end of options. Any following arguments should be treated as operands, even if they begin with the '-' character.

Re:Question... -- ?

[Darinbob]

This "--" is only for some commands that decide to implement it. It basically means "nothing after this is an argument", and it's common with GNU utilities (probably the default) but is more rare in other tools and Unix variants.

Re:Question... -- ?

[neurovish]

"How do you delete a file whose first character is '-'"?

Re: Question... -- ?

[jandrese]

It's built into getopt now I think, so anything that uses getopt should automatically support it.

Re:Question... -- ?

[Richy_T]

There are a few programs which allow per-file options so it doesn't necessarily make sense to do so (the imagemagick tools for example).

In fact, I think there's quite a lot. I often find myself adding options at the end and it seems to work more often than not. (wget also springs to mind).

Re:Question... -- ?

[shutdown -p now]

This indicates the problem with the overall design - the fact that it's the shell that's doing wildcard expansion, and doing in a way that is completely oblivious of the syntax of the command.

Windows, in contrast, doesn't have this problem because every program does its own wildcard matching, so you can never wildcard-substitute a switch. On the other hand, it means that 1) not all programs that should support wildcards, do, and 2) those that do often differ in tiny details (there's a standard OS API for this, but people still insist on reinventing it).

Arguably, the right way to do this is to unify the switch syntax for all programs, so that shell can catch such wildcard expansion and report it as illegal.

Re:Question... -- ?

[S.O.B.]

I've been using various flavours of Unix (commercial and free) for over 25 years and I've never had occasion to use '--'. I may have even come across it at some point but since I haven't needed it it's not something in my every day bag of tricks.

It does sound as though it's one of those things that once you need it you never forget it.

Re:Question... -- ?

[S.O.B.]

It's been pointed out numerous times so for consistency I might as well add it here:

rm ./-f

Re:Question... -- ?

[Anonymous Coward]

TLDP - The Linux Documentation Project. It has links to the Advanced Bash Scripting Guide which can introduce you to the BASH shell and its limitations. Another good read is the Power On to BASH Prompt which, though from last decade, isn't quite out of date.
 
oblig: csh considered harmful

Re:Question... -- ?

[Anonymous Coward]

There is no printf system call! There is only a printf library function.

Re:Question... -- ?

[Anonymous Coward]

the number tells what the man page is about.
(1) User Commands
(2) System Calls
(3) C Library Functions
(4) Device Special Files
(5) File Format
(6) Games
(7) Misc.
(8) System Admin tools

This is contained in the man(1) man page.

Re:Question... -- ?

[TangoMargarine]

Five identical responses and counting. Read the replies before you post, people :P

Re: Question... -- ?

[loufoque]

Using computers is not something you learn in school. It's something you further every day as you use new tools and their documentation.

Re:Question... -- ?

[John Allsup]

There are multiple manpage sections. For example

man -s1 stat

will get you the docs for the command line tool called stat, which will have STAT(1) in the top left, whereas

man -s2 stat

will get you the docs for the C library function stat, and this will have STAT(2) in the top left.

Re:Question... -- ?

[DarwinSurvivor]

after swearing at my terminal for a while before resorting to reading the rm man page.

I find that half the time the swearing comes after trying to read the man page. Then it's time to fire up the old Google...

... which more often than not returns a link to the online copy of the man page ...

Re:Question... -- ?

[DarwinSurvivor]

rm /my/directory/path/-f

find . -maxdepth 1 -name -f -delete

python -c "import os; os.remove('-f')"

Re: Question... -- ?

[allo]

rm -- --

Re: Question... -- ?

[mooingyak]

Yeah, that's how I did it after some research the first time around. But rm ./-- would have worked just fine without knowing about the -- option.

Re:Question... -- ?

[Gunstick]

> If the first file name starts with a dash, how do you know it's a file name?

here all the other poster's arguments are valid. The first file being a -file has always been a concern in the unix world and programmers worked around that.
But what's new with GNU, is that if you hid a command swithc in a long list of filenames, it won't be seen as a file but as a command switch. And that's where you shoot yourself in the foot.
On old unix system this will never kill you: rm /tmp/file1 /tmp/file2 *
On linux, you are ready for losing all in you current directory's subdirs (or any other trickery of the original post)

Re:Question... -- ?

[jd]

Of course it's an exercise in mystical frustration! Linus Torvalds was declared God at the first Linux conference, Richard Stallman is head of the Church of GNU (that's what it says on his website) and Eric Raymond runs a bazaar outside a cathedral.

Re:Question... -- ?

[jd]

Most of the Info pages are rips of the Man pages. Only a handful of programs have "real" Info documentation. Not that this matters, you just need to run man first for the summary and then info for more detailed stuff. The heavy documentation is only useful for really obscure stuff.

Re:Question... -- ?

[Anonymous Coward]

So why would the expected method not be the default? This is exactly how security problems are born.

nothing really new

[Anonymous Coward]

While the paper is an interesting writeup, there's nothing really new in there. A colleague used to exploit the same issue "for good," by touching a file named "-i" in directories he deemed important. Obviously, one could undo that by touching a file named "--". For users, I'd usually recommend always using ./* instead of just *, as well as directory/. instead of just directory

User supplied input problem

[Anonymous Coward]

The root issue boils down to user supplied input not being checked.

PowerShell

[jones_supa]

Is the wildcard expanded by the shell in PowerShell?

Re:PowerShell

[The MAZZTer]

I assume from this article that Linux replaces * with filenames before the command sees it. AFAIK DOS/Windows the wildcard is handled by each specific command. dir * displays the same listing as just plain dir does, while passing dir a bunch of directory names will display the contents of those directories (like ls does... I guess that explains that behavior! It always confused me). PowerShell, at least as far as Get-ChildItem, seems to work the same way as dir (except it does not take multiple directory names in parameters).

Re: PowerShell

[Anonymous Coward]

What if there's an actual file named *?

Re: PowerShell

[by (1706743)]

> echo you need to use an escape sequence > \*
> cat \*
you need to use an escape sequence
> rm -i \*
rm: remove regular file ‘*’? y
> echo $SHELL
/usr/bin/zsh
>

Re: PowerShell

[ray-auch]

* and ? are illegal characters in windows filenames, which prevents this. As is /, which is used to indicate parameters in windows command prompt (dos style), which effectively means that the style of attack in TFA doesn't work. Except maybe for unix (GNU, cygwin etc.) apps on windows which use "--" to indicate command option , and "--" is allowed in windows filenames, thus porting this Unix bug/hole/feature to Windows.

And of course Windows has other idiosyncrasies. Nothing is perfect.

Re:PowerShell

[Dishevel]

What I like about PowerShell is that it runs on Windows. Windows really sucks at command line.

Re:PowerShell

[jones_supa]

Well, yeah. The object-oriented approach is pretty clever for example. Do not have to sweat over spaces in file names breaking your scripts and things like that.

Re:PowerShell

[benjymouse]

Is the wildcard expanded by the shell in PowerShell?

No. This class of attacks will not work against PowerShell (nor for plain old DOS for that matter). The problem is the combination of text-centric shell scripting and shell expanded wildcards.

Re:PowerShell

[Anonymous Coward]

if you know what you are doing, you won't sweat over spaces nor newlines in file names.

Re:PowerShell

[Anonymous Coward]

But in the beginning you have to sweat over lots of silly parentheses you didn't expect to be needed.

Incompetent people will always mess things up...

[gweihir]

Really, this is well-known, non-surprising and will not happen to anybody with a security mind-set. Of course it will happen in practice. But there are quite a few other variants of code injection (which this basically is) that the same people will get wrong. Complete input sanitisation is mandatory if you need security. I mean, even very early Perl-based CGI mechanisms added taint-checking specifically for things like this. If people cannot be bothered to find out how to pass parameters from an untrusted source securely, then they cannot be bothered to write secure software.

The fix is not to change the commands. The fix is to exchange people that mess things this elementary up against people that actually understand security. Sorry, you cannot have developers that are cheap and competent at the same time and even less so when security is important.

He could have researched a bit harder.

[quietwalker]

I remember reading about this in the 1991 release of "Practical Internet and Unix Security," from O'Reilly back in 1991. I'm pretty sure they even gave examples. They also laid out a number of suggestions to mitigate risk, including not specifying the current path, ".", in the root user's path so they must explicitly type the location of an executable script, and so on.

They also pointed out that some well-behaved shells eliminate certain ease-of-use-but-exploitable features when it detects that a privileged user is running it, and even on systems where that's not the standard, the default .bashrc or equivalent files often set up aliases for common commands that disable features like wildcard matching, or color codes (which could be used if you're very tricky, to match a filename color to the background color of the screen, among other things), the path restriction listed above, and many many others.

It's really hard to secure shell accounts on systems, no matter how you try. Is this article just proof that the current generation of unix admins is rediscovering this? Should I be shaking my fist and telling the kids to get off my lawn? This was old news 2 over decades ago.

Re:He could have researched a bit harder.

[FudRucker]

To turn off wildcard expansion permanently: Utilize the ânoglobeâ(TM) option. To set the ânoglobeâ(TM) option, please execute the below command at the BASH shell: bash-prompt#>set -o noglobe; More often, the requirement is to turn off Path name expansion. This is especially useful if a wildcard is part of an argument to a program. http://ksearch.wordpress.com/2... Use set â"f in such cases. Execute âset â"fâ(TM) as below bash-prompt#>set â"f; To reset the wildcard expansion property of the BASH Shell, execute âset +fâ(TM).

Re:He could have researched a bit harder.

[Anonymous Coward]

Kids, libraries, publishing companies. That's the unholy combination of recurring issues in knowledge distribution.

Re:He could have researched a bit harder.

[FudRucker]

my copy & paste really borked that comment, but the link is good, sorry about that

Re:Question... -- ?

[Anonymous Coward]

Why are you using wildcards in a script processing publicly-accessible directories in the first place?

Re:Question... -- ?

[Anonymous Coward]

I never do what you suggest.
I have never written a script that needed "rm -i" for one thing.
But even ignoring that, I never remove "*" in a script. If I have a bunch of temporary files, I will create a temporary directory (TMP="/dev/shm/tmp_delete_me.$$.${RANDOM}") and put them in there. I then "rm -rf $TMP" (the temporary directory).
I have needed to use wildcards with some files before, but I usually do have more than just a "*" in there. For example, "rm -f *.stub" is more common.

Stupid headline, stupid article, stupid ...

[Anonymous Coward]

The article reveals a deep lack of thought. It might just as well have been headlined, "exploiting command lines". I mean, for goodness sake, this is a deliberate and oft-needed, oft-used feature. If you can't handle it then stop using computers because you lack the necessary skills.

-- david newall

Re:Question... -- ?

[Anonymous Coward]

Also, normal programs should stop processing options after encountering the first non-option argument. GNU tools often seem to stray from this convention for whatever reason, probably in order to stay compatible with some flavors of UNIX.

Re:Incompetent people will always mess things up..

[Anonymous Coward]

Typical Linux user reaction: "you're holding it wrong!"

At least Microsoft takes these things seriously.

Sanitize crazyness

[watermark]

I understand why this works and I understand the need to sanitize user input, but this is dumb. Even if there are workarounds. It's obvious what the intent of "tar cf archive.tar *" is suppose to be, it shouldn't be treating file names as additional arguments. Anyone actively using this "feature" for anything legitimate is dumb too.

This seems very similar to the whole "we need some other language than C" argument. Sure, you *can* make secure code with zero overflow vulnerabilities, but damn near all software has them. You can only blame the user/coder for so long for doing something "wrong", but when 90%+ people are doing it "wrong" then you probably need to change how the thing works.

Re:Sanitize crazyness

[itzly]

It's obvious what the intent of "tar cf archive.tar *" is suppose to be, it shouldn't be treating file names as additional arguments

The problem is that the * expansion is done by the shell, and the shell doesn't know the difference between file names and arguments.

Re:Sanitize crazyness

[Anonymous Coward]

It's obvious what the intent of "tar cf archive.tar *"

Obvious to who? You? Bash, which turned that into tar cf archive.tar file1 file2 file3? Tar, which never saw the *?

Re:Sanitize crazyness

[Gunstick]

no the problem is with gnu tar...

it sees cf file file file --whatever
and it usese --whatever as option

on unix (not linux) it also sees cf file file --whatever
and tries to put the file "--whatever" into the tar archive.

linux (gnu) broke stuff which worked for ages in unix world

Re:Sanitize crazyness

[jones_supa]

Yes, but what if the "--whatever" happens to be the first file name in the list?

Re:Sanitize crazyness

[DarkOx]

The problem is until we are willing to essentially throw everything out, baby bath water and all, we can't really fix it.

Take the C argument. The issue is really again one of input validation, buffer over flows happen ultimately because of of problems with input validation. Yes a language that enforces bounds checking everywhere and takes memory management out of the hands of the programer solves the specific problem of the buffer overflow; it does not make it much more secure though in the grand scheme of things.

There have been plenty of exploits and injects in software written in Java, perl, Python, Ruby, BASIC, etc. It almost always comes down to input validation, and that is because input validation is *HARD* for any non trivial range of allowed inputs.

Then start mixing other technologies and it gets even more fun. So your C program is on a system using UTF-8, how big a buffer do you need to handle data from the database server with a VARCHAR(128) field? What character encoding is it using? What else writes data to that field what character encoding do those things use?

Re:Sanitize crazyness

[AC-x]

The problem is that the * expansion is done by the shell, and the shell doesn't know the difference between file names and arguments.

But it could very easily make them explicit filenames by prefixing them with ./, and I can't think of anything that would break.

Re:Sanitize crazyness

[jones_supa]

Interesting idea. That might be used to mitigate the danger.

Re:Sanitize crazyness

[petermgreen]

Take the C argument. The issue is really again one of input validation, buffer over flows happen ultimately because of of problems with input validation.

Sometimes they do, sometimes they happen because a series of inputs while valid individually nevertheless result when combined in a value too big for an internal buffer.

There have been plenty of exploits and injects in software written in Java, perl, Python, Ruby, BASIC, etc. It almost always comes down to input validation, and that is because input validation is *HARD* for any non trivial range of allowed inputs.

So you have three options
1: write a validator and assume the validator produces "safe" output
2: don't validate the data and treat it as potentially hostile whereever it passes in your system
3: write a validator but neverthless treat the data as potentially hostile even after passing through the validator. That way you have to screw TWO things up to get a serious exploit.

Then start mixing other technologies and it gets even more fun. So your C program is on a system using UTF-8, how big a buffer do you need to handle data from the database server with a VARCHAR(128) field? What character encoding is it using? What else writes data to that field what character encoding do those things use?

So you have three options
1: try to work out what the maximum size is and use a fixed buffer of that size with no bounds checking
2: do the above but put in checking so that if you screw things up you get an error instead of memory corruption.
3: use a buffer allocated on demand of the size that is actually needed.

Re:Sanitize crazyness

[Gunstick]

It will still work... as it did in the pre-GNU world of things.
See this:
UNIX (tries to archive file -v instead of becoming verbose):
$ tar cf /tmp/a.tar a b -v
tar: cannot stat -v. Not dumped.

LINUX (suddenly is verbose, thanks getopt):
$ tar cf /tmp/a.tar a b -v
a
b

Re:Sanitize crazyness

[Gunstick]

it's not the bash problem
the problem is that getopt gives tar that there is a parameter.... which is in fact a filename hidden somewhere deep in the file list.

tar cf archive.tar file1 file2 -v file3 # is verbose on linux, is NOT verbose on UNIX (complains about inexisting file "-v")

GNU is wrong, GNU broke the inherent security in options parsing by allowing options anywhere. GNU is insecure.

Re:Question... -- ?

[Anonymous Coward]

This is not an exploit, it's basic stuff any first year sysadmin should know. You know... quoting, ./, --, -i, -inum, IFS, etc. The author's stupid Linux 'ls' doesn't even order lexicographically correct. And dashes at the end of filename arguments are not interpreted as dashopts, that's not even posix.
If you want to learn unix and blog your newfound knowledge, great... just don't try to call it an exploit or secadv, you'll just make yourself look stupid.

User data to control commands

[jones_supa]

Systems where user data can accidentally get mixed in control commands are dangerous. In addition to this shell trick, another example would be HTML, where you have to be careful to not let raw HTML data through your guestbook messages so that visitors can't inject HTML into the messages.

With competent and careful system administrators you can avoid problems, but it's still kind of a fragile design in my opinion.

I lost file ownership! What happened?

[Anonymous Coward]

WILDCARD Bitches! Yeeeeeeehaw!!

.

Re: Incompetent people will always mess things up.

[Anonymous Coward]

Wake up. Not everyone is a developer. Not everyone has even 2 minutes of unix philosophy.

My Users are scientists, and they get to trash their home space here. These types of issues are most likely to happen when they are writing a script and it makes files for what should have been options.

My job isn't to teach them unix, it's to keep them happy and productive. They make mistakes, I clean them up and help them through the frustration of things going wrong.

Wildcards are used on other OSes, as well.

[Rambo Tribble]

This would seem a problem with universal implications, and one that largely depends on local access by the malefactor.

in root? Am I missing something?

[gb7djk]

Er.. most of the exploits are only possible if one is root and/or the directory is writable for some other user (e.g. leon in this case).

Since one is root, one can do anything anyway so why bother with all this misdirection? If someone leaves world writable directories lying around (especially without the sticky bit set), then they deserve everything they get. Or is this some kind of "trap the (completely) unwary sysadmin" wake up call? If I see some strange named file (especially if I know I didn't put it there) I would investigate very, very carefully what is going on. I can't be alone in this - surely?

Re:in root? Am I missing something?

[phantomfive]

In addition, they're only possible to use as a privilege escalation exploit, not to gain entrance into the system in the first place. So this is mainly only useable on multi-user systems, of which there aren't very many anymore.

Re:in root? Am I missing something?

[jones_supa]

No, you don't need root access. Let's say that you are in a group called "students", which has R/W permission for /work/students and all its subdirectories. You are in directory /work/students, and you want to remove all the files from that directory. Now some wiseass has created a file called "-rf" and you unknowingly end up destroying all the subdirectories too. This happens because the shell expanded the asterisk, instead of the "rm" program. The "rm" program happily interprets the "-rf" as an argument, even though it was originally a file name.

Re:in root? Am I missing something?

[gb7djk]

But, but (splutter), since you have group access, then you can do rm -rf * anyway, so why bother with the cleverness? Especially as you have to leave a file there with your username handily stamped it ready for some more clueful person to catch you.

Re:in root? Am I missing something?

[jones_supa]

Yeah, you have a point there. But there's all kinds of scenarios. Maybe an upset employee leaves such a bomb somewhere and it is triggered after he has left the company.

Re:in root? Am I missing something?

[AC-x]

Since one is root, one can do anything anyway so why bother with all this misdirection?

Because you can trick a more privileged user into executing commands for you by writing files into your own folder. Most the examples given were of admin housekeeping tasks run against a user writeable folder.

Re:in root? Am I missing something?

[benjymouse]

Er.. most of the exploits are only possible if one is root and/or the directory is writable for some other user (e.g. leon in this case).

Since one is root, one can do anything anyway so why bother with all this misdirection? If someone leaves world writable directories lying around (especially without the sticky bit set), then they deserve everything they get. Or is this some kind of "trap the (completely) unwary sysadmin" wake up call? If I see some strange named file (especially if I know I didn't put it there) I would investigate very, very carefully what is going on. I can't be alone in this - surely?

The point is that this can be used to trick a root user into issuing what he believes is a safe command. The combination of the text-reinterpreting shell and specially crafted file names combines into a seemingly innocent command ending up allowing the attacker (the creator of the specially crafted file) root access on the system.

It doesn't help that some (on the surface) idempotent commands like find packs a number of dangerous options that can be used to execute shell scripts, commands or remove files.

Re:in root? Am I missing something?

[Anonymous Coward]

Yes you are... the security risk is (and has always been) tricking root to execute something somone else has prepared.
In the tar example a user makes the files and when root tar's them he will execute the shell.sh as root... the user just tricks root to tar them by calling and asking for a tarball of one of his directories.

Re:Question... -- ?

[PacoSuarez]

While that is indeed the solution, it is also true that it is too easy to forget. Perhaps one could modify all commands to require the use of the "--" separator, or to warn if it's not present, at least if some environment variable is set. That could be very helpful for people trying to write more secure code.

Re:Question... -- ?

[Anonymous Coward]

"the security risk is the developer that doesn't know what he's doing."

Not the hacker who does know what he is doing.

This just in!

[jlv]

Unpacking 'shar' archives via 'sh' considered dangerous.

linux problem NOT unix problem!

[Gunstick]

This is because the linux commands do not respect what the manual says:
man rm...

rm [OPTION]... FILE...

but in realitiy it's rather:

rm [OTION|FILE]...

whereas on other unix systems it works as expected, first the options, then the arguments
HP-UX
rm *
rm: DIR1 directory

Solaris
rm *
rm: DIR1 directory

So screw the GNU tools, they mess things up for the "old unix sysadmins"

Here is a nice linux/unix trap:
x=a
y="rm z"
f=$x $y

So you expect f to contain: a rm z
not really...
z: No such file or directory
so the rm actually was executed

a=$x is an environment variable attribution, so $y becomes an executed command...
And that one works on any unix/linux
Recently patched in chkrootkit (CVE-2014-0476)

Re:linux problem NOT unix problem!

[Anonymous Coward]

Typos in the bug description. Ironic

Re:linux problem NOT unix problem!

[Anonymous Coward]

> So you expect f to contain: a rm z

Wait, what? No. I wouldn't expect that at all. It's behaving correctly. If I wanted to concatenate like you're expecting, I'd probably
f=$(echo "$x $y")

Re:linux problem NOT unix problem!

[Anonymous Coward]

So screw the GNU tools, they mess things up for the "old unix sysadmins"

Screw bell labs people who released UNIX non-free.

Re:linux problem NOT unix problem!

[AC-x]

This is because the linux commands do not respect what the manual says:
man rm...

rm [OPTION]... FILE...

but in realitiy it's rather:

rm [OTION|FILE]...

And what happens if the malicious filename is first in the list?

Re:linux problem NOT unix problem!

[Gunstick]

the "first file in the list" problem is known to anyone doing unix since 30 years.
What's new here is that now the option can hide anywhere in the list.

Funny: in unix, the -file is lexicographically globbed onto first position (- comes before a, typically LANG=C)
in linux, you have some other oder... and it's first "a" then "-a" so the problem file does not come first. This would save one or the other exploit.... but getopt makes it so that the exploit works in any case.

Re:Question... -- ?

[godrik]

Nop, you can not just use --. because many commands do not understand --

Here is an article by dwheeler (a frequent slashdotter; often cited for his technique countering the trusting trust problem) about filenames.
http://www.dwheeler.com/essays...

I believe he is mostly right. We should move to file systems that do not allow "stupid" names and be done with it.

Re:Question... -- ?

[Anonymous Coward]

This doesn't solve the problem completely, but handles most cases of abuse. Remember, in Unix you can have a pipe in a filename. So the wildcard could activate a pipe to a script that ignores your initial input and executes nasty things.

Question... -- ?

[agent59517795]

After years of using command line programs daily I never heard of -- before today. It was never brought up in school, nor did I see any specific thread / blog post on the subject. So to answer your question, I don't. I've never heard about that before. Where did you learn about that ?

RFM

Re:Question... -- ?

[Anonymous Coward]

Anything can be put in a context such that it's not a security risk. For example, Heartbleed isn't a security risk unless you expose it to untrustworthy people.

But that's just it -- developers aren't trustworthy in this scenario. I cut my teeth on Unix decades ago and I didn't know of the "--" feature. And even then, not everything uses getopt. If I can't figure out how to use a feature securely, directly or through software others have created, then for me it's insecure. And it sounds like for most people who use wildcards it's insecure, too.

Computers were conceived to execute user commands

[INT_QRK]

...so wouldn't it be more accurate to to say that computers, like bull-dozers, can be dangerous in the hands of malicious, ill-informed, inattentive, or incompetent users? If you know of any of these archetypes, try to make them smarter, but don't allow them root privileges to anything taller than an ankle-high weed. Give them some locked-down version of Windows, without admin privileges, lots of monitoring tools and features. Consider helmets, knee-pads and child safety locks.

Re:Incompetent people will always mess things up..

[Anonymous Coward]

At least Microsoft takes these things seriously.

Was this supposed to be a joke, FUD, or BS ?

What's this... the 90's?

[Anonymous Coward]

Who's the writer? Some "security" guy that recently got a shell somewhere?

Any Unix admin professional is already familiar with this.
It's a pain? Not for me. Practicality hugely overcomes danger IMHO, but it's impossible to change right now.

Is a basic feature of how unix shells work, and there's no way to change it. Is like complaining about the rain.
Maybe someone invents something new and makes it popular.... Good luck with that.

Re:What's this... the 90's?

[AC-x]

Is a basic feature of how unix shells work, and there's no way to change it.

How about adding ./ to the beginning of each filename output by *? Is there a single thing that would break by doing that?

Re:What's this... the 90's?

[Anonymous Coward]

thousands upon thousands of scripts in existence, that don't account for ./ when deriving something from the assumed file name.

Re:What's this... the 90's?

[AC-x]

... is there an example you can point to?

Definition of idiot

[n0ano]

Let me check my dictionary for the defintion of idiot:

1. n: A user, especially super user, who uses * as an agument without first checking to see what * expands into.
2. n: A user who leaves his directories world writeable so others can put random garbage in them.

The one line summary for this story is bad things happen to people who use a command without knowing what the command does.

Re:Definition of idiot

[AC-x]

Right, so an admin tarballing the content of a user's folder is an idiot because he didn't check to make sure the shell he was using wouldn't pass any of the file names as executable attributes instead of, you know, file names?

The one line summary for this story is bad things happen to people who use a command without knowing what the command does.

The definition of the unix wildcard when used in the shell is:

"The character * is a wildcard and matches zero or more character(s) in a file (or directory) name."

Note that the definition doesn't include anything about translate filenames into other kinds of executable parameters.

Re:Definition of idiot

[Anonymous Coward]

> Right, so an admin tarballing the content of a user's folder is an idiot [...]
Don't tar with * then. It'll omit dot files too, another reason not to tar with *. Tar with a path.

> "The character * is a wildcard and matches zero or more character(s) in a file (or directory) name."

> Note that the definition doesn't include anything about translate filenames into other kinds of executable parameters.

It doesn't have to. You just quoted a section that says what happens when glob parses the an asterisk. What happens with the array of file + directory names it returns is the caller's business, not glob's (which is used in more areas than just shells.) There isn't and never was any "translation" for one token to mean "this is an executable parameter" rather than a filename.

There is no caller provision for stating whether an argument passed via argv is intended as a filename or not. "Unixy" shells use glob expansion and deliver the result as arguments to the binary with absolutely no meta-information on what their intended meaning is (as filenames). The exact same problem exists if you are manually typing in all the filenames as arguments to a binary.

In summary, the definition you quote and use to make your argument is not relevant and does not protect you from having to know how glob is used and where glob expansion can be exploited.

Re:Definition of idiot

[Anonymous Coward]

Well, it's the admin's job to know such things. It comes with the territory.

shell doesn't know anything about any parameters. It's all meaningless 'words' resulting from plaintext based operations. It's the command itself that decides what they mean, shell doesn't know nor cares, it just passes them along. Everything about parameters is just a convention, -X switches, long options, -- marker.

Would it be cool if these issues related to globs were solved better? Sure, it's pretty much impossible now, because we are talking about removing backward compatibility with countless scripts in existence.

root

[Anonymous Coward]

First they came for our Shell.
Then they came for our Emacs.
Now they have come for our *.
No, I shall stand! They will not take our noble asterisk.

Re:root

[Anonymous Coward]

First they came for our Shell.
Then they came for our Emacs.
Now they have come for our *.
No, I shall stand! They will not take our noble asterisk.

I regret that I have but one asterisk for this principle!

Tempest in teacup?

[Anonymous Coward]

Is this REALLY an important issue? Are quick and dirty shell scripts really a security risk? Who runs them other than the person who wrote them or other sysadmins?

Anything sophisticated enough for someone to put into production is probably written in Perl or something to give it a better user interface. In decades, I have never used wildcards in Perl for any kind of file processing. I get lists of files form the system, but if you can't trust the system's directory listing facility, then you're already hosed.

I'm encouraged that this is the only thing security researchers can come up with. Linux must be really secure now.

Re:Tempest in teacup?

[AC-x]

The examples given are potential admin tasks doing filesystem maintenance on a user's folder.

Re:Tempest in teacup?

[Anonymous Coward]

If you really are a fan of UNIX (or more specifically Linux), never downplay its deficiencies. That will only mean that it will stay unsecure. The folks at Microsoft certainly are taking notes and have the resources to make these things work properly if necessary.

He could have researched a bit harder.

[phocutus]

Kudos Sir kudos!

I remember reading about this in the 1991 release of "Practical Internet and Unix Security," from O'Reilly back in 1991. I'm pretty sure they even gave examples. They also laid out a number of suggestions to mitigate risk, including not specifying the current path, ".", in the root user's path so they must explicitly type the location of an executable script, and so on.

They also pointed out that some well-behaved shells eliminate certain ease-of-use-but-exploitable features when it detects that a privileged user is running it, and even on systems where that's not the standard, the default .bashrc or equivalent files often set up aliases for common commands that disable features like wildcard matching, or color codes (which could be used if you're very tricky, to match a filename color to the background color of the screen, among other things), the path restriction listed above, and many many others.

It's really hard to secure shell accounts on systems, no matter how you try. Is this article just proof that the current generation of unix admins is rediscovering this? Should I be shaking my fist and telling the kids to get off my lawn? This was old news 2 over decades ago.

Who ever asked for this "feature"

[AC-x]

Probably because anybody who's used the various Bourne-style shells for a while
considers it a feature, not a bug. This is a case where the Principle of Least
Surprise comes up with different answers for novice users and for experts:
"What? A * can expand into an unintended command argument?" "Yeah, what *else*
would it do - the shell is just globbing, it doesn't know for sure what the
command will do with the parameter".

Who asked for this feature? Can anyone give me a legitimate use case for "tar cf archive.tar *" evaluating as

tar cf archive.tar admin.php ado.php --checkpoint=1 "--checkpoint-action=exec=sh shell.sh"

instead of

tar cf archive.tar "./admin.php" "./ado.php" "./--checkpoint=1" "./--checkpoint-action=exec=sh shell.sh"

Re:Who ever asked for this "feature"

[Anonymous Coward]

add ./ on your own, problem solved.

What about
for x in *; do mv -- "$x" "backup_${x}"; done

default ./ would require explicit path removal and then reassembling the new name back. 1 step forward, 1 step back.

Re:Who ever asked for this "feature"

[AC-x]

Hmm, good point, didn't think of messing around with filenames in a for loop.

* could still use ./ as an escape code just for files beginning with - , that seems to me to be more of a step forward than a step back

Re:Incompetent people will always mess things up..

[Anonymous Coward]

Jesus Christ, get off yourself already.

Use of malicious filenames is at least 30 yrs old

[sconeu]

Back in '83, a friend challenged me to remove a file name "-rf *, without causing collateral damage.

s/Linux\/Unix/GNU/

[SigmundFloyd]

Article should be called 'Exploiting wildcards on GNU'.

All of the examples in the original document work because GNU software can intersperse options and filenames freely. None of those exploits would work in BSD userland, for example.

Re:s/Linux\/Unix/GNU/

[jones_supa]

It would still work if the first file name is interpreted as an argument.

Re:Computers were conceived to execute user comman

[AC-x]

If computers were conceived to execute user commands, then why is a command for matching file and dictionary names returning them in such a form that they become executable parameters, when they could easily be explicit filenames by adding ./ at the beginning?

Is making what should be basic and safe housekeeping functions like chmod * and tar * dangerous really something you actually want in Linux?

Yeah, that's Slashdot for you

[Anonymous Coward]

News for nerds, stuff that mattered two decades ago.

Re: Incompetent people will always mess things up.

[Anonymous Coward]

>My Users are scientists, and they get to trash their home space here. These types of issues are most likely to happen when *they are writing a script* and it makes files for what should have been options.
>*My job isn't to teach them unix*, it's to keep them happy and productive. They make mistakes, I clean them up and help them through the frustration of things going wrong.

Hi! I want to program computer thing but dont u dare try toi tgech me how i shot unix

Options take a filename as an argument

[tepples]

Because a lot of options take a filename as an argument, such as the "-o" option of any compiler or linker. How do you pass one of those without it getting expanded as a wildcard?

Re:Question... -- ?

[Terje Mathisen]

The real bug here is the same as in SQL injection attacks: A failure to safely distinguish between program and data!

I.e. when doing chown usr:grp *.php, the wildcard globbing should escape any special letters, particularly including white space and wild card characters.

This is the same idea as when you use prepare(... ?,?) on any sql statement with replaceable parameters, then execute() with the relevant dynamic values.

Terje

Re:Computers were conceived to execute user comman

[INT_QRK]

Because, since the mainframe days, a new and unforeseen computer/user pairing had evolved. For emphasis, I'll say it again, evolved; never designed from the ground up. Smaller cheaper, but nonetheless ever more powerful, computers became available to a class of consumers spanning the general population, now networked to many other such computer/user pairs. These general-population consumers find themselves operating systems far beyond their ability (or time, inclination, etc.) to understand and safely control except as mediated by a deceptively enabling and presumed (to the user) protective interfaces. The problem is that computers are mechanisms designed to automate and execute instructions. That's what they are; and, networks are necessarily designed to facilitate communication. That's what they do. Yet, underlying system architectures and interfaces, though enabling, never evolved commensurately to the task of satisfactory trade-offs between function and security for ubiquitous employment in a general population. I have no idea what the solution is, but I suspect that we need to do some fundamental rethinking of secure architectures and user interfaces. Architectures need to more safely isolate data and logical functionality, and interfaces need to more safely mediate users interaction with devices. I confidently assert that the current architectures simply can't be secured, no matter how much junk is kludged to the task. Prove me wrong, please.

Re:Use of malicious filenames is at least 30 yrs o

[Anonymous Coward]

And so you found the proper inode and removed it that way?

Mild Irony - CMD.exe immune to this type of stuff

[cant_get_a_good_nick]

As may times DOS/Windows has been burned by bad design (Explorer hiding extension yet action dependent on extension being a big one) the lack of functionality helps it here. The * is not interpreted by the shell, but by the app. It sees a *, and does a library call to do the glob. So there's no chance of it being interpreted as a flag. My guess is that DOS was so restricted in memory, that executing a glob may have blown the stack, much like we use xargs on UNIX when a fileglob is too much.

Mod parent up!

[amaurea]

Great suggestion, and much better than the one in the discussion thread linked from the summary about refusing to expand names starting with -. I don't think your suggestion would break anything, and it would eliminate the problem. It should be the default.

Question... -- ?

[Damnshock]

After years of using command line programs daily I never heard of -- before today. It was never brought up in school, nor did I see any specific thread / blog post on the subject. So to answer your question, I don't. I've never heard about that before. Where did you learn about that ?

man bash:
"-- A -- signals the end of options and disables further option processing. Any arguments after the -- are treated as filenames and arguments."

Re: Incompetent people will always mess things up.

[Anonymous Coward]

use ./* instead of * always
all problems solved forever

Old problem. Let's fix it.

[dwheeler]

I'm glad that people are learning about this problem. Sadly, it's not new, it's been known for decades. CERT’s “Secure Coding” item MSC09-C (Character Encoding — Use Subset of ASCII for Safety) specifically discusses the vulnerabilities due to filenames. The Common Weakness Enumeration (CWE) includes 3 weaknesses related to filenames (CWE 78, CWE 73, and CWE 116), all of which are in the 2009 CWE/SANS Top 25 Most Dangerous Programming Errors. My freely-available book on writing secure software has a whole section about filenames. And so on.

We need to fix the problems with Unix/Linux filenames, not just keep rediscovering them. In particular, ensuring that filenames had no control characters, no leading dashes, and used UTF-8 encoding would simplify developing correct programs. Most people writing software already follow these rules. We don't need to make it easy for attackers.

Re:Question... -- ?

[Sun]

The -- end of options option is a GNU getopt extension. It is, if memory serves me correctly, not part of any standard. This means that program that were not compiled with glibc, or programs that do not use getopt/getopt_long, may or may not honor it.

Even by simply looking at the man page, it is easy to spot programs that don't use getopt. Any program that accept multi-letter options with one minus sign is, obviously, not using getopt (e.g. gcc, find).

Then there is git. Git uses -- to mean "no revisions after this point. Any remaining argument must be a file name". This is almost, but not quite, what -- means for getopt. I don't even know what underlying parsing engine they are using (could be getopt with the "no options after first argument" option set).

The "./*" feature is a more global workaround, if it is applicable.

Shachar

Re: Incompetent people will always mess things up.

[gweihir]

And not everybody has any business writing security-critical code. You are asking for the illiterate to be allowed and encouraged to write poetry. That cannot ever work.

Exploiting Wildcards in Poker

[Anonymous Coward]

Aces are high, deuces are wild and the sky is the limit.

Place your bets.

Re:Incompetent people will always mess things up..

[gweihir]

You have noting to contribute but could not keep your mouth shut?

Re:Incompetent people will always mess things up..

[gweihir]

Yeah, by not giving people a commandline worth anything in the first place. But you are right: MS realizes their customers are incompetent, immature whiners that would blame MS is they used MS products to shoot themselves in the foot. So the function to shoot anything was removed.

Re:Computers were conceived to execute user comman

[AC-x]

I have no idea what the solution is, but I suspect that we need to do some fundamental rethinking of secure architectures and user interfaces. Architectures need to more safely isolate data and logical functionality, and interfaces need to more safely mediate users interaction with devices. I confidently assert that the current architectures simply can't be secured, no matter how much junk is kludged to the task. Prove me wrong, please.

On the other hand this specific issue could be easily solved by * prefixing all filenames with ./

So far I've not heard of anything that would break, and it's silly arguing that this specific problem is part of required functionality and not something that can/should be fixed when it appears to have such a simple solution.

Re:Question... -- ?

[Ed Avis]

Yes, there is a workaround you can use, if you know about it and remember it every time, to enable the safe behaviour. That does *not* count as 'problem solved'. To solve the problem, the safe behaviour needs to be the default, with the funky and unsafe behaviour of treating filenames as extra switches being the one you have to enable specially. Really - what are the odds that the user or programmer *intends* for a file called --foo to be treated as an option specifier when they expand a wildcard? Conceptually the fix is not hard. Each element in argv gets an associated flag saying whether it is a filename - and if it is marked as a filename, getopt() or whatever does not treat it as an option specifier even if it begins with the - character. Alternatively, filenames beginning - could simply be disallowed.

Re:Old problem. Let's fix it.

[Gunstick]

you are not precise on one point

In unix world, this long known issue was only about the first filename. Typically with classig usage of tar, the first file is your archive, and all others are just filenames. unix/bsd treat them as filenames.

Now comes GNU getopt and "suddenly" decides that from now on it would be cool to have options anywhere on the command line. Result is that in GNU-world, tar is now vulnerable (and many many other commands) which had had no issue at all previously.

I would really like that getopt changes back to "if we see filenames, then consider being after --" or rather: stop parsing options if you find a parameter which does not match any option. That's the last one. Just stop there, do not continue. Leave the rest as it is.