Slashdot Mirror
When Sysadmins Go Bad
An anonymous reader writes "Here is a story about what can happen when you think you're being oh so clever. This sysadmin planted so-called logic bombs on the systems he was responsible for and then quit. He also tried to game the stock market, buying put options on his former company, hoping to cash in when the disaster he engineered struck. Who can companies trust if they're afraid that this kind of thing can happen? How can they prevent it?"
:eek:
Everyone died today? Large lack of posts!
Obviously, in the sake of security, you should NEVER provide system administrators with dangerous tools such as root passwords!
Seriously though, security is a very delicate matter which is entirely built on trust.
Ways to improve security is to limit access to only what you actually need to use. In the case of system administrators and the like, it's not quite as easy as they obviously need a high level of access.
One solution would be to have third party audits of the systems, perhaps with read-only access in order to prevent tampering, but even then you need to trust the integrity and skill of the auditors.
Another thing to remember is to have a solid disaster recovery plan, but that's only good AFTER something happens and the person designing and implementing this plan will likely be the person that has the most access.
There's no universal answer to this problem. If I knew of one, I'd be rich as heck from selling it to companies.
Many years ago one of our staff left at the end of the summer. Our boss said "Thank you very much for working for us ... [pause as the door closed, then turned to a coworker] ... delete his account."
Slashdot monitor for your Mozilla sidebar or Active Desktop.
What the hell just happened?
/. the [logic] bomb...
I go to post a comment and I get a page full of ads. I think someone set up
Sometimes I doubt your commitment to Sparkle Motion.
By making sysadmins unnecessary!
Have everyone running WINDOWS XP! That doesn't need any system admistration at all, it has perfect uptime and is fully transparent for even the dumbest user!
[/sarcasm]
Have two sysadmins, who work in different areas, and who a la "missle key firing system" both have to approve additions to important code bases.
Obviously, you could get two bad apples and have the same thing happen, but odds are slim.
Problem is, it tough to find ONE good admin, much less two, esp. with tough times for business... having to dole out twice the budget to protect yourself "just in case". Then again, it would double the job market =)
OR mabye CVS everything, and look through all changes an employee made after they quit... then again, the clever get around this, etc.....
*sigh* People just suck sometimes.
Department of Homeland Security: Removing the rights real patriots fought and died for since 2001
When you have reasonable salaries, reasonable work hours, and no one that runs everything.
First of all you'd have less disgruntled employees.
Second, you'd have less disgruntled employees.
Third, you wouldn't need to trust anyone 100%. Most egos of sysadmins wouldn't let them let someone else compromise their system. If you have 2 or more admins 100% responsible for the integrity of a system, and each performing checks on each other, you would reduce the occurences of these types of attacks.
I was disappointed to find that this was an article, and not a new show on Fox.
It's better to burn out than to fade away
> Who can companies trust if they're afraid that
> this kind of thing can happen?
Nobody.
> How can they prevent it?
They can't.
Employee misbehavior spans an entire spectrum of seriousness, from stealing paper clips to embezzling billions. You can't prevent a determined and dishonest sysadmin from sabotaging a system any more than you can prevent an accountant from diverting funds or an after-hours custodian from taking things off peoples' desks.
There is no panacea, technological or otherwise.
Preventing employee misbehavior has several parallels with Copy Protection. No affordable and practical scheme is bulletproof if the person is determined enough, so the best method is to remove the motivation. The same rules apply to all employees: treat and compensate people fairly and they will be less likely to want to hurt you.
But even that doesn't work in all cases. If your staff is large enough there will always be people who feel that you are mistreating them, or underpaying them, and who will feel compelled to get what is "rightfully theirs" in other ways, large and small. And many people steal/etc. without regard to the harm it causes the company or other employees; their motivation is purely selfish, so it doesn't matter how well they are treated and paid.
So even if you treat and compensate people fairly, and trust everybody you hire, you must monitor people's activity, investigate suspicious behavior, and, when necessary, prosecute wrongdoers to the fullest extent of the law.
I probably sound cynical, but I speak from experience.
Something similar happened to my Dad's business about 15 years ago. Back then, they just trusted the employees. For some reason I can't recall, they decided to fire the sysadmin that was running their billing systems and gave him a months notice. During that month, they let him take time off from work to interview at other places and were generally pretty nice about the whole thing.
A couple weeks after he left, the system started crashing and losing data. Apparently he used a rather well-known bomb because the company they used for support was able to dial in and found it rather quickly. He was charged, arrested, tried, and found guilty. It was a big deal because the state (South Carolina) had just passed some really though computer crime laws at the time, and the Attorney General wanted a "test case" for the law.
My Dad and his partner's requested that the guy not get any jail time since he had a wife and some kids, but he got major probation and a huge fine (something like $60,000, which was a lot back then). Plus he now has a felony charge on his record. Last I heard, he had gotten out of the computer biz and was working in a family business.
Anyway, the short lesson is: if you're a company firing someone with privileges, pay them the two weeks or whatever but don't let them back on site. And if you're the guy getting sacked, don't try to get revenge through sabotage; it's just not worth it.
As an aside: every place I've worked had a policy that whenever someone was fired they were led to their desk with a cardboard box, then escorted out of the building that very moment.
How is this different from any other kind of sabotage by employees or ex-employees? As long as there have been accountants, there has been embezzlement. A short-order cook could forget to wash his hands. A construction contractor can use sub-standard building materials.
You gotta trust somebody; just make sure it's somebody worthy of trust.
As for preventing this particular kind of sabotage, use the same principles as everywhere else: supervision, audits, bonds, insurance, and the threat of jail time if the rest fails. Oh--a good disaster recovery plan sure doesn't hurt, either.
Cheers,
b&
All but God can prove this sentence true.
... pull a stupid crime and spend the rest of your life in a state-funded institution.
For critical systems, nothing gets changed without an approved change request. All changes must be examined, tested and approved by someone other than the programmer. You can also have a separate group to maintain the source libraries and to do builds.
Mea navis aericumbens anguillis abundat
With the Paine Webber guy, I was amazed this guy didn't think the SEC could put 2 and 2 together.
"Hmmm, there's the guy who had access to the company's computers and made all those put options, but I don't know if there's any way we can prove motive or opportunity."
Trust in God; Everybody else pays cash
Who can you trust? -- Nobody. As our master said:
Machievelli, The Prince Ch 17.The answer to the question is no one, not even your mother. If you are not secure against being hacked by an insider, you are not secure. And that means everybody, Newspapers are full of headlines about CEO's ripping off their companies. Stories about long-time trusted employees who embezzle a few hundred thousand dollars are so common that they usually wind up on page 7 of the Metro section.
SysAdmin, as the word says, it's the Administrator of the System.
there's no technical way to restrict their actions, or we should restrict the computer's capacity.
people do bad things for money, that's all, how could we prevent this happen? how could we prevent crime? how could we prevent people shoot each other? these are analog.
it's political or human issue. not technical.
Don't keep disgruntled employees or employees that you keep hidden away in a back room and ignore. Management that keeps good relationships with its employees don't have as many problems with this sort of thing.
This means:
1) Help work to keep employees happily employed (not with bribes - with real career paths, personal interest, etc.). If you keep wage-slaves, expect mutiny.
2) Actively replace employees who can't be kept happily employed. Get others who are competent and glad to have the spot (which shouldn't be too hard in this economy). Keeping people around who don't want the position isn't doing them any favors. If no one who would be qualified would also be glad to have the spot, rethink the position.
"Management" should be helping manage situations like this. If this guy had been disgruntled for a long time, it seems to be their fault for keeping him (and keeping him unhappy and ultimately vengeful). Sounds like someone did a bad job at people-management . . . sounds like the type of willfull neglect that is inexcusable but all too common. Many people think that "management" is watching the bottom line -- that is a lazy, oversimplified way of looking at an important job.
C'mon -- this is really small potatoes ...
That this firm had a SIXTY year old sys admin.
There's hope for me yet.
It's Christmas everyday with BitTorrent.
If systems are so critical and secure, then you need to separate responsibilities, and dispense information to those holding the keys on a need to know basis.
--- have you healed your church website?
or something like that.
Best Slashdot Co
Makes my little cron job that changed the shell on this user's account three times a week look really mild in comparison.
That guy annoyed the hell out of me one too many times.
ACK
It is not equivalent to a real bomb. There was no destruction of property, no casualties. It's in a completely different league. The real solution here is to treat your employees with respect and not treat them as slaves.
Ethics aside, I have to admire this guys balls!
I'll put my ethics back on and fix the sendmail f'up I made this morning now :-)
Help children born unable to swallow - www.tofs.org.uk
From the article:
:)
So-called logic bombs are pieces of software code buried within another program and are designed to disrupt computer systems. They are often delivered by e-mail.
Ok boys and girls, would someone like to explain how this is different than a virus/Trojan?
Keep in-mind that I am not a financial expert, nor the general public that I can assume are reading this article. With that in mind... the following statement is even more mind boggling:
He allegedly bought more than $21,000 of put options, which grants an investor the right to sell a certain amount of underlying stock at a certain price. By giving the investor the right to sell underlying stock at a given price, put options increase in value when the stock value falls.
Christ.... wtf does that mean
All in all this article goes into no detail in regards to how he was caught, and how they in intend to prove it's him.
--Noodles
"Who can companies trust if they're afraid that this kind of thing can happen? How can they prevent it?"
Management: "We don't need a sysadmin, everything is working just great!"
TANSTAAFL.
I was amazed he didn't think to have his friend or his grandmother buy the options.
My God, it's Full of Source!
OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
this problem has nothing special to do with sysadmins. its a human resource problem of a entirely generic form.
"how do people prevent people with privledge from fucking them over?"
i think employee onto employer a mockery of the likelihood of employer unto employee.
i dont have much more of flame bait suggestions for answers, just more or less refined questions.
members are seeing something, your seeing an ad
Who can you trust?
Microsoft. Trustworthy computing.
At Microsoft, we make operating systems that administer themselves, so you don't have to hire those untrustworthy and expensive system administrators. Nearly any high-school graduate, or poo-flinging monkey, with the proper brainwa^H^H^H^H^H^H^H training can become a Microsoft-Only Operations Certified Omnipotent Worker. Get your own MOOCOW today, and let us handle your security problems. You shouldn't have to worry about these computer dealies - that's our job.
Microsoft. Trusted Computing since 2002.
My beliefs do not require that you agree with them.
No I understand that much.
The article is just not clear about the definition of a put option at all.
(Yes I know there is another definition linked in the slashdot post.)
Put option quick explaination:
Suppose that the stock of company FooBar is worth $80 today.
I buy the *option* of selling that stock at $80 in one weeks time (this of course cost me something since there is a risk involved for the entity that I buy this option from).
Let's say that priviledge costs me $1 (since everybody considers company FooBars stock prices to be quite stable).
Now, one week later the "bomb" has blown up their computer system and the stock has plunged to $40.
The option of selling one stock at $80 is now worth $40 since the stock is currently priced at 40$. I don't even have to own the stock since someone who does can buy the option from me instead.
In total I've made 39$ on an investment of 1$ in one weeks time.
Forget the sysadmins hosing the company, how many friggin execs run the thing into the ground looking to pad their stock options, then leave?
At a big EDA firm I worked at the sysadmin got into big trouble (I think he was fooling around on his old lady and was trying to run away with some other chick). He decided to hose the backups by placing a small magnet on the read/write head (IIRC). Then he did real backups, which he hid in the drop-down ceiling. His stupidity led him to try to blackmail the company (gold coins). The episode ended badly--high speed chase, crash, prison. Now that I think about it, yeah, a Fox mini-series!
doug
A lot of larger companies can have multiple admins, each taking care of a particular sector. By having a common methodology or plan, you can ensure that one admin can cover for another (in case of unforseen accident) or take over.
In the schools where I work, I can walk into another admin's school and be fairly comfortable with making fixes/changes to their system - since everything runs similarly. This is convenient if one of us gets sick, or has a holiday, etc, and a server goes kaput somewhere.
Some of us are more well-versed than others, and one of the other admins has a much better knowledge of most of the systems than me - in particular our main user repository.
I can get by fairly well the "armadillo book" (0'Reilly) when there's something I don't understand, but sometimes I still need to call him when things go awry. For those that need to catch up with other admins, I do recommend the O'Reilly books though. I've only been here a few months, and I expect that after time (and reading) I'll be much more confortable with some of the systems I'm not currently as fluent in as others.
It just sounds like to me the guy set up a nice little crontab entry that no one bothered to check that did a rm -rf /* on their systems. But, then again, the article did say...
Duronio, a computer systems administrator, resigned from PaineWebber on Feb. 22 after complaining about his salary and bonuses. The logic bomb he allegedly constructed from November 2001 until February of this year was activated on March 4, U.S. Attorney Christopher Christie said in a statement.
So this guy was clearly dumb, executing something like this only two weeks after he left. I could see how it would take him from November to February to figure out how to work cron.
--Chag
>> How can they prevent it?
> They can't.
They can at least reduce the chance a lot with redundency.
If you have a team of sys-admins, you have a good chance that the other might catch the bad one before it's too late. And if they feel treated well by the company and don't share the sentiment of the saboteur, the damage is usually contained.
Another policy I've seen in some banks is that all employees have to take 2 continuous weeks paid vacation each year (the rest of the paid vacation time can be distributed at will). This promotes cross-training and redundancy.
sounds like something right out of the Bastard Operator from Hell
"Facts are meaningless. You could use facts to prove anything that's even remotely true." - Homer Simpson
All of this costs money, but think of it as cheap insurance, compared to the cost of rogue sysadmin. Is it worth penny-pinching on salaries and benefits, while maxing out the workload if that results in disgruntled employees who timebomb your systems as they head for a new job?
If you paid the sysadmins $1 million per year, there would be zero theft, zero funny business, and zero turnover. Of course, nobody can do that and stay in business. At some level less than $1 million and higher than fast-food wages, you can retain decent people and discourage malicious tactics. The key to avoiding a technological meltdown is to treat people well enough so that your recruiting process lets you avoid the marginal candidates. Once hired, a properly compensated person should feel as if the "have something to lose", and therefore you can expect such a person to act as a professional. Paying hamburger wages and putting a person in the sysadmin seat would be like staffing a nuclear power plant control room with random selections from the phone book.
This is a very interesting topic, especially right now. We are in a down market, and there is an irresistable temptation for some employers to make lowball offers to currently-unemployed candidates. This allows the employer to cheaply refill vacancies (or exert leverage against current employees). Those employers who are gung-ho about bottom-feeding are setting the stage for big trouble later. Employee turnover is just the tip of the iceberg.
- Design the system so that it requires change controls
- Take daily md5 snap shots of systems
- Always keep off site duplicates of your monthly full back ups. It's not just for DR; it's also for versioning.
- Sue him out of existence and make sure EVERY employer in the area knows about it - not just for vengence, but also as a heads up to other rouge sysadmins.
In other words, follow best practices and procedures.
Democrats and Republicans only disagree about how to enslave you
My take on it would simply be that your employer did not pay enough attention to your activities abd subsequently due to their mismanagement you would not be at fault. Comments?
--Chag
You must not be a sysadmin...Or you must be working for the government?
This is unrealistic. When the fire is burning, you can't take 5 minutes to sit down and follow the procedures, you just jump in and fight it.
-- Leeeter than leet
He allegedly bought more than $21,000 of put options, which grants an investor the right to sell a certain amount of underlying stock at a certain price. By giving the investor the right to sell underlying stock at a given price, put options increase in value when the stock value falls.
I'm trying to figure this out. From the ABC article, it sounds like he bought stock in the parent company and expected to profit when things went bad? I could see how this works with buying into a rival company would work, but this sounds like a losing situation. Maybe the article is just weirdly worded, or I'm reading things wrong?
1) Buy stock
2) Logic bomb subsidiary company
3) ??? 4) Profit?
You can say that SysAdmins "own" the business, or at least, they control whether it runs or not. They can crash/corrupt/etc anything in less time it takes you to fart...
It is a common practice to delete any sysadmin account *before* they get the news.
Most people I know were even escorted out of the building.
Think about the bad things a secretary can do? Not much... Maybe call a few customers and piss them off? Bogus orders of pizzas? Now think about what a sysadmin can do? Create a disaster big enough to kill a company... It's too easy to "skip" some backups and then crash a few DB's. I'm sure there are tons of way you can "kill" a company... It's too easy for a sysadmin..!
-- Leeeter than leet
You can't. Next question.
Uh, but if the sysadmin was in charge of the backup system...
+1 Insightful, -1 Troll. What can I say, I'm an Insightful Troll.
"Be kind to your enemies; be peaceful. But if they lay a finger on you, send them to the cemetary."
My wife was consulting at the time and was called to a similar case. The network admin was fired and a few days later most of the workstations and a few of the servers just formatted themselves. She got there in time to save most of the servers and a few workstations, but it took weeks to rebuild. This was at the HQ of a regional company.... The last I heard the FBI was going after the guy after he ran off to New York.
This is the reason network contractors and admins almost NEVER get to work a notice.
How can you hire one person, give them God access, and trust they won't abuse it? you cannot, and you never will. Checks and balances -- hire a staff, not a person.
If co-admins can see the changes I've made and call me on them, my opportunity to screw with the company is dimished. Granted it's not completely gone, but it is less than if no one ever saw what I did.
You cannot keep one person happy forever. But with a staff you can attempt to control the unexpected life-events of your employees (which could cause someone to steal) with the decent salaries / work hours / conditions / respect / recognition that have been mentioned above.
If you blog it...
I presumed you're the type that think that corporate CEO who looted pension fund shouldn't get any time in jail, since they didin't actually use physical violence?
14 * * * * /bin/kill -9 $RANDOM
That would cause some pretty wierd things to happen from time to time. Kinda like bad ram, or something.
Wouldn't be that hard to find though.
Note it's similarity to my sig, too.
Get your own free personal location tracker
Here in Venezuela, when the Oil strike begun some sysadmins blocked and placed logic bombs in the critical computers. It is costing the country an average of US$ 15 million a day. The computers that control the fuel-load process in the tankers where so sabotaged that any try to get the system up would end up spilling fuel on every "island" (the place where the fuel truck loads). The only way to stop the spill would be to activate the emergency system in the plant. Gladly (it's already very known worldwide) the goverment set up a "hackers team" to take over all the sabotaged industry computers. Most of them are running Solaris or Windows NT 4, so it wasn't too hard to break all the systems. If you calculate: US$ 15 Millions * 16 days = 240 Million US$ ... and most of it is because the admins who sabotaged the critical computers.
Looks like one SysAdmin is thinking things were not as easy as in Office Space or Superman 3. Off to "federal pound me in the ass prison" for him.
~~ What's stopping you?
1985: A travel company with several offices (local big group) had only one sysadmin for their computerized booking system. He was this nasty guy who was related to one of the founders, and no one wanted to fire the guy because only he knew how to run the damn things. Not that he did a good job. He was lazy, rude, and demanding. Well, one day, new management got sick of him, and tried to get an "assistant" for him (read "learn his job so we can fire him"). Sysadmin was wise to that, and basically they went through several employees in as few months. Finally, they decided to fire the guy, and hire a contractor to replace the systems. The firing was ugly, they ex-admin had to get dragged out by the police in the end. Days later, the whole system went down. Guess what? No backups. No one knew how it ran, and years of data was lost, chaos among their customers ensued, and six months later the company went out of business.
1996: Our company bought out a competetor. They guy in charge of the call center was the only one we didn't lay off right after the merger because he was the only one who knew what went where, and he used this knowledge to leverage his job security. He was impossible to work with, never did anything on time, never answered his pages, and did just enough work not get fired, but it was really, really hard to get him to do anything else. Finally, we gathered a team of experts (our staff plus vendors) to go as a group, figure out what he was doing, then fire him. His response? He deleted all the call center tables, databases, and destroyed all paperwork... then quit. We had him arrested, but he posted bail, and we never found him again. It took half a month to get everything working right, which meant we had to tell 300 call center employees they couldn't come to work or get paid until we called them back. Boy, was that a clusterfuck.
I saw this button once, "Now that I have changed the master password for the database, it is time to discuss my salary." Heh.
1997: The head of our HR department was fired due to some political bullshit. Standard procedure was to take an ex-employee's computer, wipe it, and give it back to the tech department. Guess what we lost because no one thought about it? All employee records for the department. Backup was on a single floppy that wouldn't load, and she hadn't done backup since the first of the year anyway. We had to have every employee resubmit 1099s and W4s, plus tell us honestly what vacation and sick they already took.
1999: Same company, same situation, but this time it was the guy who kept the entire tech department hardware inventory records. It took a year to recount what we had, and re-enter serial numbers and license keys into a new database. The stupid thing was, this guy made regular backups on the network drive... which was on a server they wiped by accident. Doh!
2001: After a round of layoffs, one of our more brilliant and inspired programmers had "expiration dates" on all his compiled software. He wrote most of the tools we still use today. Months after he was laid off, all of them stopped working on September 17th, 2001 at 12:00 midnight. The only way we got saved was that no one wiped his original desktop box (which had the source code on it, which is how we found out about the "expiration date"). So we recompiled without the date, and everything worked again. Due to WHEN it happened, our whole company thought we'd been attacked by terrorists (the clever generic error only said there was a "network failure") until the truth was revealed. Later we found 9/17 was his birthday, and it was just coincidence it happened so close to 9/11; the layoffs were in March, and they were unexpected and sudden. I doubt this guy had Al-Queda (sp?) connections, so he must have been planning this "job security" (as the comment in the code labeled it) way in advance.
I see a lot of posts saying that if you pay people well, if you treat them better etc this won't happen. But it will, because even in the best environments, someone is unhappy.
What people need to remember is that personal integrity is important too. Two wrongs don't make a right.
...is 20 years in prison. It doesn't hurt to have national press coverage of the guys who have tried this and have failed. It's not like you can get away with this very easily.
Let's see? Who has had access to all of these systems? Who has recently quite or been fired? Who just sold a boatload of stock when we got hit? A smart admin realizes that there are other admins as smart or smarter. People can piece these things together, and obviously this employer and the government are taking this crime very seriously.
RP
I have no idea what buying put options means, but with my "touch", the stock market is mine!
Anyone want anything on my way up?
Keep your packets off my GNU/Girlfriend!
Sysadmins are the least of my worries. I'm more worried about directors who screw up companies, or people who are brought in to manage the company whose only intention is to sell and make money. Yes B.L. that means you!
20 years seems harsh only when viewed in the context of a "victim-less" crime. However, most white collar crime has the potential to affect a larger number of innocents than most people consider.
Consider the consequences of an irrevokable malicious act on a trading company. If damage is broad enough the perp shuts down said company for days on end. Thousands of clients are unable to do anything during this time. Employees waste thousands of man hours attempting to rebuild wasted systems. If the damage is extensive enough, it could put the entire company out of business.
Just take a look at the fallout of the Enron situation and you'll find countless people who have lost entire life savings because of some "victim-less" white collar crimes. Not only is Enron dead, their consulting firm has died, thousands of people are out of work, numerous support companies have gone under, and thousands of people have lost millions upon millions of dollars in retirement savings. The consequences of Enron's illegal practices touch many people who did not have anything to do with the crimes being commited.
Don't assume because a crime doesn't physically harm someone that it has fewer consequences or requires lesser punishment. In the broad perspective of total social impact, white collar crimes have the potential to an aweful lot of harm to a large number of people.
What if the employee is a good guy? What if they have discovered one or more security flaws in the company's systems(s)? Flaws that range from minor (Joe Random customer being able to format a sales terminal) to intermediate (changing employee paychecks or discounting merchandise) to major (stealing the entire payroll account)?
The question: How does the employee tell the company without getting in trouble? After all,the employee did gain... improper... access to the systems to find out this information. obviously, the employee is good or they would have taken advantage of this opportunity, but the company may not see it that way.
So, how can the employee (or anyone, for that matter) handle this?
When you are a huge corporation, even a day's downtime to restore backups can cost $3m in lost productivity and business opportunities.
- In Capitalist America, law violates YOU!
Their firing procedure: the boss invites you out to lunch. As soon as you are outside the turnstyle he says, "You're fired. Give me your ID badge." And you have to wait there a few minutes while a (former) colleague boxes up your personal effects and brings them outside to you.
Why the gratuitous cruely? To make recruitment of new employees so much the harder?
Opinions on the Twiddler2 hand-held keyboard?
Actually, banks are required by law to report to the Federal Reserve each year with a list of all officers of the bank (pretty much anyone in any manager role at all, plus major non-managers) who did not take 2 weeks of consecutive vacation that year.
In the past, this time was used to audit the person's desk. Nowadays, it's kept around under the theory that if someone wants to hide something, it's much more likely to show up if they can't cover their tracks for 2 weeks straight.
At least mafia-owned pizzarias make excellent pizza. Compare to Bill Gates.
They do that here, too. The catch is that to the rest of the employees, the firing can often look arbitrary, and everyone gets nervous.
For example, last month they let go of two people (for diff. reasons). Each of them had several meetings for "remediation" (warnings) for months in advance, but they had to finally let them go. The meetings were usually in private, so nobody else knew. All that anyone else knew was that suddenly they're packing up a box and saying goodbye.
The management isn't allowed to tell anyone it's coming since it's an HR policy, and the employee isn't likely to brag that they're "on the bubble", so all anyone else sees is that their own job appears to be pretty fragile. We all "know" differently, but the impression is there.
As a rule I never delete an account or remove user identification information.
Nuking an account kills part of your auditing trail and/or proper file associations when you do it. Besdies, if you need to check something as a specific user it can be a bear to undo the dammage. Temporarily suspending access can happen just as often depending on the environment, so why not simplify it to one process?
Besides the practical option of re-enabling the account if the person comes back, disabling accounts is a good habit preventing nasty problems fixing mistakes (John Smith vs. Johan Smith).
A firewall can not protect you from yourself. Turn off what you do not need. Do not use the firewall to do your work.
That's a very good question, it's too bad you were joking. You can fix the advert problem by adding "ALL: www.transfer.com" to your hosts.deny file. It uses CGI to load up images from other sites based on some hideous random number. Blocking the images from www.transfer.com does no good because the images come from other servers. Blocking all crap from them cleans the page up and eliminates their pop ups too. Now for the serious matter.
The article was a slam job. It has a byline of december 17th and says that they tried to contact the sixty year old perpetrator the same day he was due to go to trial. Duh, someone chruning through the justice system might be hard to reach. Yet we are unable to tell if he refused comment or was simply not reached. All we have is the accusation.
Presumption of innoncence is a nice thing to have. There are several reasonable explainations for this man's actions. He might have quit in disgust, having been overridden by management on several key issues and just known that the results would be catastrophic. We have no proof yet that he really planted "bombs", we have only the prosecutor's interpretation of what the company and software vendors told them. I wonder just how he will be able to defend himself without access to systems that have been manipulated by his accusers.
This case should send chills down your spine. There is no way to keep a responsible person from sabotaging a company. It's the same case in meat space, anyone can throw a monkey wrench into the works. In cyberspace much more is stacked against you. The evidence is not easy to explain, is easy to create and destroy, and is wholy controled by those accusing you. It can not be visited by your defenders and what they find if they could look can be modified without a trace.
Friends don't help friends install M$ junk.
- Design the system so that it requires change controls
So who has the "change control" if not the administrator?
- Take daily md5 snap shots of systems
Woot, the system stays the same and this dude's chron jobs execute on time.
- Always keep off site duplicates of your monthly full back ups. It's not just for DR; it's also for versioning.
I suppose your monthly full back up will save your bacon, as well as the chron job. Still, the chron job can be found and the data repaired. That's what happened here, right?
- Sue him out of existence and make sure EVERY employer in the area knows about it - not just for vengence, but also as a heads up to other rouge sysadmins.
Not so fast. First you have to prove that he did it. I have not seen anything but an accuasation yet. Imagine that you have a disagreement with your boss. The dumb dumb wants to do something you know will be a disaster, you disagree and quit. He does it, it's a disaster, then he blames and frames you siting you being dissatisfied with the subpar salary you put up with for years. Woops, you be very very rouge now, like third degree red, while your boss claims that you are a rogue.
In other words, follow best practices and procedures.
Words of wisdom to be sure.
Friends don't help friends install M$ junk.
In the same way one should prevent employees from placing out pieces of fish to rot in strategic places around the building, or other nice ways to sabotage: by taking care of their workers.
I don't see the difference between this way of sabotaging and my silly example above.
Remember the lesson of "Jurassic Park":
If you don't pay your programmers enough money, a tyrannosaurus rex will eat your lawyer.
We cook your meals,
we haul your trash,
we drive your ambulances,
we protect you while you sleep.
Do not fuck with us.
--Tyler Durden, Fight Club
We backup your servers,
we script your patches,
we don't mention the porn on your laptop,
we run your firewall,
don't fuck with us.
--Painehope
PC moderators can suck my White pierced, tattooed dick. If you think pride == hate, s/dick/Aryan meat mallet/g.
fund the policy from your employees salaries.
Allow employees to get a refund of most of the insurance salary deduction by being bonded for a few million dollars.
Alternatively, just take your chances and hope shit doesn't happen, or that if it does, you catch the person and they don't go bankrupt.
This story is about a large company my previous employer did work for. Of course I won't say the company's name, but it's often used as a verb, and their products are probably in your office.
:)
:)
We were hired to write software to show our customer's customer how our customer was doing. It kept track of when shipments went out, things like that. It was replacing an earlier attempt from the sole sysadmin at that location.
Now I must mention that the entire network was 5 years old. Everything was purchased at one time, when the location opened, and nothing had been bought since.
Anyhow, the admin gives us a Compaq P75 workstation with 24MB and NT Workstation to use as our production web/database server. Significantly below our requirements.
He refuses to give us access to their current data to convert/test. Etc, etc. The Manager then gives him the ultimatum to comply or quit, so he walks out. No one there knows any passwords, no network diagrams, not even what boxes do what.
So I had to own every device on their network to give them control again. While writing the software we were there to do originally. Lots of 80 hour weeks, and my previous employer is a bunch of bastards so I was not well paid for it. But to this day, the customer location is still in business, and I have a terrific reference on my resume from them.
A company I previously worked for treated me like absolute crap. Eventually they threw me out and I before they threw me out they let me go clean up my desktop. I copied a "logic bomb" that I had studied out of interests sake onto the firewall and then left. This one required a specific IP/request to set it off, but I never did it, because after I had calmed down it was just too childish and irresponsible. They had been scared however, that I would do something like that and deleted all my accounts, thereby shooting themselves in the foot when they needed to work on the webserver sometime later, I heard from a former coworker. For all I know that bomb is still there today.
Their definition of logic bomb isn't quite accurate, it's a little too specific. Logic bombs and trojans are highly related (you could argue that either is a subclass of the other), but viruses are quite different.
A virus is a program fragment which, when run, inserts that same fragment in other programs. Today's mass media enjoys the word "virus" and applies it to many other kinds of malware- the recent headliners like Melissa, ILoveYou, and Code Red were mostly worms, not viruses. (A virus rarely spreads very fast, since the delay before infected programs are restarted introduces a lagtime)
The difference between Trojan and Logic Bomb is a little vaguer. Trojans are usually inserted into software by a programmer who wishes to gain access to a computer he doesn't administer. The canoncial logic bomb is something left behind to impair a system long after the bomber has gone away.
Usually "Logic Bomb" implies that there is some kind of timer mechanism involved, so that after you're fired the payload can still be delivered, even if the target computer has no internet access.
For instance, a simple logic bomb might be to schedule a job to delete all a server's files in 6 months. As long as you're employed, you can keep cancelling that job and re-scheduling it... but a while after you leave, boom! (More subtle payloads would be both more damaging, and less likely to get you caught)
I got a +5 (Score:1, Funny)
by Anonymous Coward on Wednesday December 18, @12:12PM (#4916014)
And I forgot to be logged in.
Arrrggggghhhhh. Isn't that how it always goes.
Apparently, with you, yes it is. Jolly bad show, old chap.
I remember reading a comment by somebody, somewhere (gee, can I be more vague?)...
I think it was in SF, and they got called for a survey about crime. They got asked "how do you feel about crime rates?" They asnwered "I think they're going up, Enron is stealing millions WorldCom is stealing millions, so is Xerox and a bunch of other guys." The survey taker was taken somewhat by surprise by this. "Um, no I mean street crime". "Oh you mean some guy who's going to steal $6 from my wallet instead of a couple thousand dollars from my grandma's pension fund?" The survey taker sid "um, yes." "Oh, I think that's getting better, though crime overall is bad."
Some junkie jonesing for a fix steals some car parts, goes to prison. WorldCom execs lie and still get millions from bankrupt companies.
In the real world your company should have code documentation standards. Unfortunately most standards seem to focus on compiled code (C,C++) and not php, perl, bash or configuration scripts.
In any case, typically sysadmins work unpaid overtime to meet unrealistic delivery schedules set by marketing or management.
Is it better to have a working system or unfinished well documented code?
Supervisors should set a good example. Peer code reviews and team projects lead to better documentation.
Beware of the lone wolf and loose canon.
What about when you have been working for years with minimal documentation. Suddenly upper management wants you to document everything. Not too suspicious until you consider the amount of layoffs that has been happening recently. On the other hand new equipment is being implemented and there is more time during this slow economy.
So if "The writing is on the wall", do you take your time? Do you procrastinate? What quality do you provide? How much do you let your documentation interfere with your job hunting?
My boss was given this dilemma, right after setting up a W2K cluster. I think he followed the procrastination route. It seems management realized he is still worth what they pay him so they are not bothering about the documentation anymore.
Life moves pretty fast; if you don't stop and look around once in a while, you could miss it. -FB
[I'm guessing either nothing (the APC UPSs worked just fine and nobody noticed) or major disaster (APC wasn't using their own product).]
If all this should have a reason, we would be the last to know.
... which is why the SEC investigates any large options purchases which occur shortly before large short-term movements in stock prices. If you're one of these lucky devils, they will probably get your name and address from your broker and see if you are employed by the company in question, if you work for a law or accounting firm retained by that company, if you have the same last name or home address as someone who works for the company, etc., etc.
There is nothing sinister about this kind of investigation; it's routine police work. (Likewise, if you're the town layabout, and the day after a masked man robs the town bank you start spending money like it was going out of style, the sheriff will probably peg you as a suspect.) What is amazing is that people do not realize that it is the SEC's job to do this sort of investigation: they just blithely go ahead with their stupid criminal plans. Even lawyers, who ought to know better even if they are unwilling to behave better, do this sometimes.
The perfect inside trader would have 10 loyal friends located around the country willing to make small purchases of options on his behalf, to forward him all the profits, and to stonewall the SEC investigators who come knocking. Believe me, you don't have 10 friends like that.
That sounds very unusual. Typical US corporate procedure is not to give you a clue until you're done working, and then not to leave you alone until you're out the door. I know a guy at HP who is still technically an employee and doesn't have access to the site or his accounts. (IIRC, he has a couple months to look a job to transfer to within the company before he gets laid off and his severance pay starts.)
I guess it makes sense from the corporate prime directive of "maximize shareholder value". Presumeably the thinking is that you're loyal and you can't figure out you might be in line for the axe, but in the instant you get the news your loyalty evaporates. But it is not a reasonable model of how people work, and it is not humane.
We got back after one day, and had more than 20 (!) messages on our answering machine. The entire line was shut down because the software was not seeing any new orders. My boss had been going around, saying, "Well, he's finally left. I knew he would do something like that. We're screwed."
Turns out some fool had modified a record without using the proper indexes (ancient FoxPro for DOS). Because the indexes were no longer synchronized, the software's "do while order == opened" loop hit a closed record that was indexed as open, and exited prematurely.
I went in, fixed it in five minutes, and left. They were bankrupt within 4 months, and I was thankfully on to a new employer (that didn't trust employees any further, but that's another story).
You quitting proves that the karma kap worked. The most annoying of the whores shut up. --CmdrTaco
Aren't all sysadmins evil trolls that restrict user rights, sleep in server rooms and complain that they don't have enough control????
Okay, I have heard the term before, but it smells of fear mongering in this story. Trojan horse would have been more accurate. They use the term 'logic bomb' six times in a nine paragraph story.
CEOs and accountants do more damage to companies and steal more than this while getting less time in prison. I wonder if this guy is going to some cushy minimum security country club?
When rich people are caught stealing, the crime is getting caught, not stealing, and the punishment is light. When rich people's trusted tools are caught stealing, they are terrorists.
- None can love freedom heartily, but good men; the rest love not freedom, but license. -- John Milton
The presedent has his key, somone else has another key. The presedent doesn't have access to the second key, and the second key holder can't access the presedents key.You need both keys to launch the nukes.
----
I always like the other part of the same story, both the president and the vice-president gets same clerk to type in both the keys!
You still have to trust someone to implement the double-key system.
I'm a UNIX sysadmin and Oracle DBA. I've always had root (and sys, for Oracle) on all systems I manage. I've done this for years and have never compromised any data or any system. And I don't think I'm an anomoly. As the admin, I'm very proud of the work I do and the efficacy of the systems I'm responsible for. Employers have extended a trust to me and I wouldn't dream of violating it. No amount of money would be worth the loss of self-worth.
At my last job, I had unfettered access (at work and at home) to thousands of customer's credit card info. It was not even a temptation for me (it was a source of concern that the info might be compromised by others, and I brought that to management's attention on a number of occasions). When the company started layoffs and morale plummetted, I left, but on extremely good terms. The level of trust between us was so high that I was asked to keep my secured access to the system in my home for several months in return for a consulting retainer.
When we were getting new PC's, they let us spec what we wanted. The PC dept prohibited us from ordering the PC's with CDRW's because they were afraid that we would use them to steal company data or code. My boss chuckled when I pointed out that it would be safer and more convenient for me to download said data or code via the company provided ISDN to my house. I just bought a CDRW myself and installed it. Either the PC guys never figured it out or they were afraid to mess we me. Doesn't matter much now, as they are all unemployed anyway.
Hearing about this kind of abuse really pisses me off, it puts us Sysadmins that are legit in a serious bind, and we are less trusted.
The Sys Admins need to form some kind of honor system/group, that puts a code of ethics in place that group members need to follow, If they are suspected of malicious intent during a screening process or on the job, they are banned from the group and can never work in the IT industry again, that's how serious these types of actions should be taken.
Then employers could at lest be assured that we tried to screen out as many plp as possible that are shady.
Anyway just my 2cents.
But honestly, why is it that companies don't 3rd party audit departments that are so important to their continuation every year?
Do they just let the accounting department run wild?
Seems to me the company looks just as stupid as this guy for never picking this up.
If you could be told what you can see or read, then it follows that you could be told what to say or think - BoC
Man, the first example reads like a page out of BOFH! http://bofh.ntk.net/Bastard.html
(New boy comes in) "Here, hold this wire." (Bzzt!)
Yeah, it's a multi-faceted problem. I guess it comes down to "Don't hire jerks, and try not to be a jerk" as much as "redundant meat-ware".
Yow! I'm supposed to have a plan?
I like the fact that this article is titled "When Sysadmins Go Bad", as opposed to "If Sysadmins Go Bad".
ALTERNATE JOKE: What do you mean, go bad? I thought Sysadmins were all Chaotic Evil.
"Destroy science and religion. Science would re-emerge exactly the same; but not religion." - Penn Jillette, paraphrased
This is not a technology issue. These sort of problems cannot be solved with technology. Whatever you are entrusted to control or change you can destroy.
Don't put those you cannot trust in critical/important positions (same for the incompetent). There will always be critical/important positions. You can improve things by requiring cooperation/collusion between more than one party in order for things to be done. This has its costs. And if the untrustworthy are plentiful in your company, you might be doomed anyway.
Technology can help those you trust do their job properly - prevent/recover from mistakes, help manage people with various degrees of trustworthiness/integrity.
The AI proponents through their failures, have proven computers are no substitute for humans. Those pushing AIs created by modeling systems they don't understand, are laughable - I'd trust the resulting creations even less than humans, and definitely far less than a trained dog. And we all know software has bugs.
An organisation which cannot trust its people would have to spend a lot more money and resources vainly trying to extend the boss's capabilities and control (since the boss would then be the only one who can be trusted). However that scenario would render most of your employees capabilities redundant. And at a certain point the boss won't be able to oversee everything and would still have to trust someone else.