Slashdot Mirror
Secretly Monopolizing the CPU Without Being Root
An anonymous reader writes "This year's
Usenix security symposium
includes a
paper
that implements a "cheat" utility, which allows any non-privileged user to
run his/her program, e.g., like so 'cheat 99% program'
thereby insuring that the programs would get 99% of the CPU
cycles, regardless of the presence of any other applications in the
system, and in some cases (like Linux), in a way that keeps the program
invisible from CPU monitoring tools (like 'top'). The utility exclusively
uses standard interfaces and can be trivially implemented by any
beginner non-privileged programmer. Recent efforts to improve the
support for multimedia applications make systems more susceptible to
the attack.
All prevalent operating systems but Mac OS X are vulnerable, though by
this kerneltrap story,
it appears that the new CFS Linux scheduler attempts to address the
problem that were raised by the paper."
... allows any non-privileged user to run his/her program, e.g., like so 'cheat 99% program' thereby insuringI'm assuming that we're saying that this application can get 99% of the time-slices on an otherwise occupied system, starving other tasks for resources. I'd be interested in hearing how this plays with the latest scheduler for the Linux kernel, which is supposed to favor the most needy applications.
I run several websites off of a single host. If I need to login to do maintenance during peak hours, I'm slowed by Apache and MySQL. This would be a nice utility if it were wrapped into SUDO.
I'd rather you do it wrong, than for me to have to do it at all.
that others are starting to look after the *nix world for weaknesses? Once windows is equal or better than *nix in terms of security, then all the security and malware people will start looking at us.
For those harboring poisonous grudges against PDFs, the Googlerised HTML version is here.
Ha! I told you Mac OS was more secure. What? Of course I'm not a fanboy! What gave you that idea! Jeez.
The gnome desktop for years has been hiding processes that h0rk the cpu.
Using up 99% of the CPU's easy!
#include
int main(int argc, char *argv[])
{
while (1) {}
return 0;
}
Summation 2
Not quite sure what justifies a paper out of this.
If you check the linux kernel mailing list for Vassili Karpov, you should find test cases that demonstrate this behavior and tools for monitoring actual CPU usage for a variety of platforms, though I notice no mention of any of that in the paper.
See http://www.boblycat.org/~malc/apc/ for the tool and 'invisible CPU hog' test case.
Sanity is a sandbox. I prefer the swings.
Back in my day we called it renice.
Yes, I'm kidding. Please don't post a long reply explaining how renice differs from this cheat thing. It isn't necessary.
Finally, the "sue" command of PC UNIX has been implemented.
I seem to recall usenet discussions about this circa the time of !uucp!newsglop!..... It seemed the Unix scheduler would let certain IO operations hog the CPU. And if you somehow installed your app as a IO driver or IO completion routine, then your app could hog the CPU. Similarly since day one of Windows soundcards you could set your app to realtime_priority and everything else would suffer. Not exactly smokin' hot off the press.
This year's Usenix security symposium includes a paper that implements a "cheat" utility, which allows any non-privileged user to run his/her program, e.g., like so 'cheat 99% program' thereby insuring that the programs would get 99% of the CPU cycles, regardless of the presence of any other applications in the system, and in some cases (like Linux), in a way that keeps the program invisible from CPU monitoring tools (like 'top').
Next up, a virus which senses bad grammar and punishes you by using 99% of your CPU. Seriously, somewhere a middle school English teacher is crying, and doesn't know why.
Please help metamoderate.
Curious how this works.
I wasn't aware the schedulers for those systems were so deficient !
In my days (yes, I'm an old fart) - the schedulers had basic principles :
- Voluntary yielding led you to get accounted for the time you spent running.
- You could stay in the interactive queue for only a certain amount of time. After some amount of time had passed (a few secs) you were either bumped to non-interactive if you were running (with longer time slices but lower priority) or removed off the scheduler list for good (if the time spent there was idling). They had a special 'idle but interactive' (not eligible for dispatching) queue for that.
- Scheduling a new task restarted a new time slice
That particular scheduler even had a 3 queue system so that if you got accidentally bumped into the non-interactive queue or if your process was semi-interactive you had a better chance of gaining interactive status again. And they had a 'really' not interactive queue for those CPU hogging processes.
Of course this requires the hardware to have a precise timing feature (something with a granularity that is finer than the process interleaving time slice time and ideally in the magnitude of instruction execution). And this scheduler wasn't using time sampling and time quantums.. (but something more like the OSX timer on demand paradigm).
--Ivan
This is accomplished by sleeping for a fixed amount in between OS clock ticks. The timeline looks like this:
The most rabid believers in American Exceptionalism are the exact same people whose policies are destroying it.
We had a user who insisted on abusing the "nice" command, to run his jobs at a higher priority. Pleading and cajoling didn't work, so we decided to get creative.
We changed nice so that whenever this particular user ran it, it lowered his priority by exactly as much as he was attempting to raise it.
He stopped coming to work soon after that. I suppose he had the last laugh though -- NYIT continued to pay him for another six months.
Thad
I love Mondays. On a Monday, anything is possible.
Does it work on Solaris? If so I can run my sparse distributed memory simulator on the comp sci depts main server without waiting hours to get results!
"Good news, everyone!"
According to the paper, the reason Mac OS X is not vulnerable is that it uses one-shot timers scheduled for exactly when the next event needs to occur, rather than periodic "ticks" with a fixed interval between them. The "tickless idle" feature introduced in Linux 2.6.21 (currently only on x86, I believe) takes the same approach, and very possibly makes Linux immune too.
(Ironically, immediately after discussing OSX's ticklessness, the paper mentions that "the Linux 2.6.16 kernel source tree contains 8,997 occurrences of the tick frequency HZ macro, spanning 3,199 files", to illustrate how difficult it is to take a tick-based kernel and make it tickless. But those kernel hackers went and did it anyway.)
The tickless feature isn't yet implemented on all architectures that Linux supports, though. I think AMD64 support for it is supposed to come in 2.6.23, along with the new CFS scheduler.
If this program consumes CPU cycles, but it doesn't leave any indication that it does, how do I know that it works?
How likely is it that cheat has already been in the wild for a while?
I have noticed that my CPU on some tools show ~5 minute bursts of 100% usage, but top sorted by cpu usage peaks at ~5% usage and shows 90% idle.
Tick Based Accounting v.s. Time Sliced/Sample Based Billing
(Reminds me of some Zombies Processes I have seen in the past.)
They took too long to publish this. Linux 2.6.21 (released in April) added support for using one-shot timers instead of a periodic tick, so it avoids the problem like OS X does. In addition to resolving this issue, tickless is important for saving power (because the processor can stay in a low-power state for long enough to get substantial benefits compared to the power cost of starting and stopping) and for virtual hosting (where the combined load of the guest OS scheduler ticks is significant on a system with a large number of idle guests). As a side effect, while the accounting didn't change at that point, the pattern a task has to use to fool the accounting became impossible to guess.
The CFS additionally removes the interactivity boost in favor of giving interactive tasks no extra time but rather just quick access to their available time, which is what they really benefit from.
My mother is a gun-toting marxist redneck zealot astroturfer, you insensitive clod!
The creator of this post (Jacob Smith) hereby releases it, and all of his other posts, into the public domain.
The most rabid believers in American Exceptionalism are the exact same people whose policies are destroying it.
Sounds like somebody's discovered Java!
It's a baseball bat.
It doesn't even matter if these CPU-hogging processes can hide from "top" - you should already be making regular rounds of your users, even the ones you haven't caught doing anything wrong. Nobody questions it when you tell them, "You know what you did." Not when you're the one with the bat.
I recently saw a "tickless" option in the kernel config. Would using that solve this problem? I'm not a kernel hacker by any means; knowing enough to run a clean Gentoo with no issues doesn't necessarily imply programming talent.
~Eien no Inori wo Sasagete~ Searching for my Hatsumi...
(reply to self after RTFA)
What 'saved' the Mac OS was its different use of timing triggers. "All" other OS'es use one common steadily ticking clock as a dealer of time slots. This allows the cheat to "skip to the start of the line (queue)" every time it's had its turn.
OTOH, the Mac uses a stack of alarms set to specific points in the future, and polled in order as they occur. So the difference on Mac OS is that there's no skipping the queue, it's rather "there is no queue, we'll call you when it's your turn".
I don't know the details of the OpenBSD scheduler, but it's very likely the same (clock tick) method as used by the rest of the susceptible OS'es.
"Good news, everyone!"
The paper is quite long, so here's a summary (take this with a grain of salt, who wants accurate information should still RTFP):
Most OSes (Linux, Solaris, Windows but not Mac OS X) are tick-based. This means that the kernel is called from hardware periodically (this is the "HZ" value you set in the Linux kernel). Some of them (Linux) simply check which process is running at each tick and compute statistics based on that ("sample-based statistics"). This means that the process running when the tick happens is billed for the entire period of the tick.
Since ticks are typically "long" (typically 1-10 ms on Linux) more than one process may run during this period. In other words, using this approach leads to inaccuracies in the process billing. If all programs "play by the rules" this works quite well on average though.
Next thing: the classic schedulers typically maintain some sort of "priority" value for each process, which decreases whenever the process is running and increases when it's not. This means that a process runs for some time, its priority decreases, and then another process (which hasn't been running for some time) takes over.
You can exploit that by always sleeping when a tick happens and running only in-between ticks. This makes the kernel thinks that your process is never running and give it a high priority. So, when your process wakes up just after a tick happened, it will have a higher priority than most other processes and be given the CPU. If it goes to sleep again just before the next tick, its priority will not be decreased. Your process will (almost) always run when it wants to and the kernel will think that it's (almost) never running and keep its priority high. You win!
Another aspect is that modern kernels (at least Linux and Windows) distinguish between "interactive" (e.g. media players) and "non-interactive" processes. They do so by looking how many times a process goes to sleep voluntarily. An interactive program (such as a media player) will have many voluntary sleeps (e.g. inbetween displaying frames) while a non-interactive program (e.g. a compiler or some number crunching program) will likely never go to sleep voluntarily. The scheduler gives the interactive programs an additional priority boost.
Since the cheating programs go to sleep very often (at every tick) the kernel thinks they're "very interactive", which makes the situation worse.
Some of the analyzed OSes - even if tick-based - do not use sample-based statistics in the kernel but they do use sample-based statistics for scheduling decisions. So the kernel sees that a process is taking more CPU than it should but it will still keep on scheduling it.
Mac OS X is not affected because it has a tickless kernel (e.g. without periodic interrupts). Because of that sample-based statistics don't work and it has to use accurate statistics, which make it unaffected by the bug.
This bug can be exploited to (at least)
- get more CPU than you're supposed to
- hinder other programs in their normal work
- hide malicious programs (such as rootkits) which do work in the background
Here's a list with the OSes (this USED TO BE a nicely formatted table, but the darned Slashdot "lameness filter" forced me to remove much of the nice lines and the "ecode" tag collapses whitespace).
OS, Process statistics, Scheduler decisions, Interactive/non-interactive decision, Affected
Linux, sample, sample, yes, yes
Solaris, accurate, sample, ?, yes
FreeBSD 4BSD, ?, sample, no?, yes
FreeBSD ULE, ?, sample, yes, yes
Windows, accurate, sample, yes, yes
Mac OS X, accurate, accurate, not needed?, yes
I guess that Mac OS X doesn't need a interactive/non-interactive distinction because of its different (tickless) approach. I assume that interactive applications can (implicitly or explicitly) can be recognized as such in a different way. Does anyone have more information on that?
How does tickless Linux compare? What abo
A very simple patch is to issue RDTSC instructions at process restart and blocking syscall to count the cycles actually used. That way the extensive tick-code doesn't need to be modified.
Someone didn't preview and doesn't know how to use < and >.
Also, what's the deal with that empty block in between the "while (1)" and the "{fork ();}"?
Geez, if you're going to critique someone else's code, do a double-check on your own first.
If it's for-profit but free, you're not the customer -- you're the product (e.g., the Slashdot Beta's "audience").
The point of the paper is that you could have some malware using 99% of your CPU and it wouldn't even show up in top.
Besides the syntax comment the other poster said, it could've also been that the school implemented per-user process limits on the machine. Linux has had this capability for years and years; most people just don't bother setting it, but universities hosting machines for programming students pretty much have to set it for exactly this sort of thing, whether it be accidental or malicious.
If it's for-profit but free, you're not the customer -- you're the product (e.g., the Slashdot Beta's "audience").
n/t
Chris Torek gave a presentation at UseNIX about how a constant quantum could result in a process having its CPU usage unaccounted.
His solution was to use a randomized quantum. Not unique per process, but randomized when the kernel starts running each process. That gave you a better accounting of the CPU time (statistics, doncha know :)), but also made this kind of attach much, much harder.
I'm somewhat disappointed that I did not see Chris and Steven's paper referenced in this one. (I believe that the title of that paper was "Randomized Sampling Clock for CPU Utilization Estimation and Code Profiling," for those who care to find it.)
Now I have a *Real* job as part of a programming team working with a *mostly* real RDBMS.
you will be in my prayers, brother-in-arms. It's worse than that, I'm a software development major in college, working as an intern consultant. I don't even know VB, but using Real Languages gives me enough to fly by the seat of my pants. The dbase is done by some guy who can no longer be found, writes spaghetti code and has a fondness for loops while doing lookups. It's been 'upgraded' from Access '95, to '97, to 2K, yet I'm not allowed to just drop the thing into SQL Server and use
If I mod you up, it doesn't necessarily mean I agree with what you've said, sorry.
That might make an interesting sig, actually.
Red Light Green Light.
Your goal is to run as quickly as you can towards me. When I turn and face you and say "Red Light" you must stop moving and if I catch you moving I make you start over from the beginning. When I'm not looking and say "Green Light" you can move again.
In this case, the goal is to cover the greatest total distance instead of just reaching my position; so we could adapt it from running to eating: The "winner" is the one who eats the most and the losers end up hungry.
Democracy Now! - uncensored, anti-establishment news
Apparently windoze uses thread based scheduling, so a program with more threads gets more priority.. I belive that in most Unixes its process based, depending on the thread package implementation, this may or may not work.
M
This, and the earlier comments about the latest Linux versions becoming tickless are ruining my plans for making a thumbdrive of nifty utilities.
Your ad here. Ask me how!
How about a baseball analogy? You steal bases by running when the ball's not in the air.
A Car analogy? I got nothin.
No weapon in the arsenals of the world is so formidable as the will and moral courage of free men.-Ronald Reagan
Wait. Something's wrong here. Not if you have two cores, it isn't. If two cheating processes each take 99 percent of a core, then their total is 198 percent.
Nothing new here.
I remember seeing this done on the VAX/VMS mainframe back in 1987. In that environment, it simply meant that you kept track of your timeslice and voluntarily gave it up before the scheduler took it away from you. That meant you got put at the top of the run queue, and unless someone else was doing the same thing, you were the next program to run. Voila... 99% CPU for you!
Of course, ordinary users were given a limited amount of CPU time (as well as connect time, disk space, etc), so for the ordinary student, this just meant they used it up in a day or two instead of having a whole month. But then again, for class accounts, they could usually beg for more.
Under unix variants, one could do the same by implementing cpu quotas at the user level. I've seen network packet quotas, and I'm sure someone out there has done cpu quotas along the same lines.
Bravo, sir
This must be the best summary I've ever seen for any FA. That's all I wanted to know.
NEXT!
Who is General Failure and why is he reading my hard disk?
I could swear i have seen this already in windows XP, i work as an It tech so i see a lot of windows systems (pity me?) anywho, i have come across a few systems where task manager shows the CPU pegged at 80-100% but no process in the process list is using it (sorted by CPU time), even the idle process shows 75% or so.. the system is slow as heck so I tend to believe the over all usage stats but it should show "something" using the processor.
I would say (from my hazy memory) that the system suffered from malware too, as after a cleaning of the system it would be gone (the problem).
I don't know for sure if the system was just too busy to report actual process usage or there was something hidden from the task manager so you couldn't kill it, but i have seen this a few times..
like so 'cheat 99% program' thereby insuring that the programs would get 99% of the CPU cycles, regardless of the presence of any other applications in the system, and in some cases (like Linux), in a way that keeps the program invisible from CPU monitoring tools
So what's the deductible on this insurance?
this nation, under God, shall have a new birth of freedom. -- Lincoln, Gettysburg Address
That doesn't help if you can't upgrade to 2.6 on production machines...
Just what are the regulated computations that are being talked about?
I heard about the same VAX hack at about the same time. The idea of boosting process priority by synchronizing to the CPU ticks isn't all that new.
Of course, in 1987 Windows was completely immune to this problem. It had non-preemptive cooperative scheduling. :-)
A cattle prod works better than a baseball bat, but locking the users in the tape safe is the only sure way to ensure that they can't steal CPU cycles.
Just "gittin-r-done," day after day.
Is that you, BillyG?
TFA (pdf) says `the Windows family` is also vulnerable. Note how, in the list of operating systems, `the Windows family` comes last, looking almost like an afterthought, or an also-ran.
By the way, Bill. Any news on when Notepad is going to become a proper text editor, rather than just a toy?
My computer just freezed deadly when I was reading the paper. Have to use teh big red button.
If you use Solaris, the top replacement "prstat" has a flag -m that enables microstate accounting. This will give you the exact CPU time used, and not an estimate.
If you want to go further, you can use DTrace, which allows you to monitor in detail exactly what is going on. This is also unaffected by any tricks played by the process.
when its time to check what process is running,the program (which is cheating and already knows the time of check)just sleeps.
That way it looks like it never uses cpu.
Which is possibly true as this paper was published before Vista was out. However, Vista securety is more about preventing those proseses from turing your comp into a spam zombie.
Teenagers:
Start after parents leave, finish before the predicted time they come back.
Cars:
You SPEED when you are in areas without any cops watching.
Baseball:
Steal bases while the pitcher is not looking. diff: pitcher needs to forget what base you were previously on.
Red Light Green Light:
kid playing the sign is the scheduler. diff: being winner would be the one with the most distance traveled (could change game to eating candy.)
Democracy Now! - uncensored, anti-establishment news