Slashdot Mirror
How To Manage a Security Breach?
Salvance writes, "A friend of mine has recently been stressed over a security breach at the company he consults for. The company maintains dozens of Windows 98 desktops to support legacy software that cannot be easily replaced. Due to the inherent lack of security in Win98, a worm was able to infiltrate almost every computer and send gigabytes of data (possibly including sensitive company data) to a 'redirector' in Eastern Europe. My friend was working on other security projects at this company and stumbled across this massive hole. He quickly convinced company executives to remove Internet access from all Win98 machines, purchase better firewalls, and implement other data protection strategies. However, the sticking point was client notification. Due to the nature of the legacy systems, there was no way to know what data was transferred. For this reason the company wanted to play it safe and disclose nothing. Of course, my friend is all for disclosure and preventing harmful use of the potentially leaked data. My friend doesn't know what to do, so I'd like to know what others here think."
Get the resume ready. If I were a client of a company that had such shitty protection of my data, I'd find another company ASAP. I expect that said person would do much better finding another place to work.
See my journal for slashdot ID's by year. Mine created in 2005. http://slashdot.org/journal/289875/slashdot-ids-by-year
Get a network tap and put a machine in front of everything. If anything ever happens again you will have raw data captures of all the traffic and will be able to identify what was sent depending on encryption.
Offsite, you need to have a spreadsheet or other document. Put in the date and write down everything that happened to the best of your knowledge.
If something is not documented, it didn't happen.
Then, do what the client wants you to. Include the client's wishes in your documentation.
He might want to dig deeper into which worm he was infected with and what types of docs it steals, this may help determine what data was indeed stolen. If his company wants to investigate this further he should notify the FBI. If they want to keep quiet well their isn't much he can do about that with out possibly losing his job.
Microsoft aggravates my tourettes syndrome.
So the company knows that there WAS a breach, and potentially sensitive data may have been leaked. The company probably doesn't have a technical obligation to disclose anything, since they don't know for sure that information that requires (or should require) disclosure (like customers' billing data, social security information, credit card info etc) was compromised.
That being said, the right thing to do is to be forthcoming and disclose the nature of the breach, emphasizing that no specific information about what was leaked is available.
Of course, this being a corporate setting, if they can get away without telling anyone, they will. Especially if it's publicly held; while the stockholders might wish to know that there was a problem, they may also be upset that a disclosure was made that was not absolutely required, as that will negatively affect their stock value.
Never underestimate the power of stupid people in large groups.
If the client suffers damages that could have be prevented by disclosure of the breach, money may change hands.
Since he consults, he does not set policy. He informed management (best keep a record(s) of that), it's their call.
-- www.globaltics.net
Political discussion for a new world
There is always a danger in being more or less ethical than your employer. If you're more ethical you're a troublemaker and they'll fire you, and if you're less ethical then you're a scumbag. Obviously the ethical thing to do would be to notify the customers. But executives don't really work for the customer--they work for the stockholders, and "doing the right thing" doesn't figure very large in the balance sheet. I don't envy your friend's position, but it's a common one--look at Sibel Edmonds. Employers, public or private, seldom want you to have actual integrity. They want you to do and say whatever makes them look good.
...update everything to windows XP, there an emulation layer that runs 98 software, although I don't know how good it is. If not you could try Linux running wine which would probably work.
Finally there are companies that specialise in moving data from legacy systems to modern systems. You could employ one to move all the data.
Yeah, right. Cause it would never happne to you, would it? ;-)
Swedish plasma phys. PhD student; MSc EE; knows maths, programming, electronics; finance interest; seeks opportunities
I think one of the most important points here is the Operating System. I think it could be an option, if you *really* need to run specific applications on Win98 platform, to install such insecure operating system inside a virtual machine as VMware. I dont care if the operating system is WindwsXP or Linux, but I am sure it will be easier to fix the security hole if you have the OS inside the VM sandbox.
On the other side, it could be the case (it has been in lots of places were I am from) that such machines are not "cutting edge" (say Pentium 3, or even Pentium) and they wont support WindowsXP as the base OS. For that, I would recommend a slim Linux distribution (Xubuntu, or DSL maybe) and then install your licensed copy of Windows98 inside Vmware.
Now, I do not have the specific reasons why virtualizing that will help, but I know it helps. Although maybe I am just trying to solve the problem throwing virtualization (and if it does not works just virtualize the second base OS =op)
Ubuntu is an African word meaning 'I can't configure Debian'
A damn fine product by google & CIA.
The relationship between the client and the client's customers is most likely not what he is being paid to consult about. He is better off pretending that he never thought of the issue at all.
Put on your nerd hat, and treat any non-technical issue as unimportant and uninteresting.
Run them in Virtual Machines. VMWare is just awesome. Not that this fixes the problem after it happened.
Disclosure is required if there was any privacy data stored on those systems (peoples names/numbers/ssn/etc), if you do not know which users data was comprimised, all users need to be notified. This is required when it affects gov agencies, I am not however sure about private and commercial entities, although not notifying your customers if their data was comprimised, is asking for trouble, and when word gets out, people will find alternate solutions to what that company provides.
I came, I conquered, I coredumped
All he can do is give the company his opinion that the clients should be told. What management choose to do after that is entirely up to them. Not informing the customers is the decision of the executives, and any resulting problems this causes are therefore their responsibility.
Informing customers may also cause problems for the company that are disproprortionate to the damage done. If this friend informs the clients himself, he could be held responsible for harm done to the company.
Cover Your Ass.
Document everything. If there where conversations and meetings, send out a follow up email with the notes of what was talked about. Keep copies of everything, make backups and place them in a bank.
The second part comes if the company is publicly traded or not. If so, and these Windows 98 machines hold trade secrets or the accounts logged in had access to trade secrets stored elsewhere on the network, then the company is in some deep doo-doo, otherwise tell him to buckup and carry on.
III.IIVIVIXIIVIVIIIVVIIIIXVIIIXIIIIIIIIVIIIIVVIII
Thieves broke into one of the Corporate offices of the company i work for and stole several thousand dollars worth of computer equipment, including laptops with employee and customer information on them. All customers and employees were notified the next day and advised to post a fraud alert with the credit agencies, complete instructions on how to do so were included in the email. The company was completely transparent in how they dealt with the information loss, and to my knowledge no fraud was committed using that information. That is the only acceptable way of dealing with the possibility of personal information theft. I can easily see the effort to hide the security breach leading to litigation if the information that was stolen gets used. Your friend should tell them that it's a CYA issue. It's better to wind up with a little egg on your face admitting a mistake and telling your customers what happened than to have your customers sue you into oblivion after they trace the information leak back to you.
I work in Security for a very large financial institution. We do not have break ins ;)
Seriously, we have taken a bunch of time to implement several policies. Policies around; passwords, data at rest, data in transit, physical access and two factor authentication (in some areas). We feel very good about what we have in place, but we also realize that it can never totally protect us. We have also done things like phase out any rogue operating systems (win 98) that we may find hidden inside the walls of this company.
In short security is really the assembly of multiple disciplines (at least for us) and we are constantly getting better at it. We have recently become quite interested in being able to handle what happens AFTER some sort of breech occurs. The industry is calling this Resolution Management. This seems like a newer addition to the security pyramid, but a practical one. The idea is that an organization should be able to respond quickly and cost effectively to a breech and provide the same level of system service as before the breech - quickly. One of the innovators in that space is http://www.certalertsoftware.com/. I am sure there are others too, but worth investigating.
I recommend you convience your company to buy Zone Alarm, AVG Anti-virus, Ad-Aware, and use the free program called Hijack This. Now Hijack This requires a third pary web forum for the program to be used correctly, because if you don't know what you're reading and deleting you could very well remove requiered registries. So after you finish scanning each system with Hijack This, upload the scan log to this forum (several other are recommended on CNET but this is the first one I found) http://forums.spywareinfo.com/ These guys know what they're doing and are very helpful from what I've read.
Hope this helps.
you could have a look at
d itorials/dumb/index.html
http://www.ranum.com/security/computer_security/e
2+2 = 5 (for very large values of 2)
As a consultant, your client is the company itself and not that company's customers. You've informed the company, now document it to make sure that's known. Ensure the right bit of the company is informed (ie. compliance, not just your local boss), document and you're done.
Now, if the real question was "should I inform the company's customers because I think this is very important to them?", well you're on an entirely different path and ultimately only you can decide that. Without knowing the details of what might have been disclosed, no-one here can even give you an informed opinion let alone a set of instructions. But as far as what you must do is concerned, then see paragraph one.
Cheers,
Ian
This is a really hard problem, especially given that I don't know what how sensitive the sensitive information might have been, but the bottom line for me (as a client, MD or security guy) would be; disclose.
I come to this conclusion from an evaluation of worst-case scenarios;
possible results:
harmful use of customer data, harms client
disclosure, harms company reputation
I am assuming that the harmed client would not know that company at fault. we shall call this 'harm1'
If the nature of the data means that a harmed client would know that it was this company's fault, this harms both client and reputation; 'harm2'
1) No disclosure + No harmful use = client OK, reputation OK
2) No disclosure + Harm1 = client harmed and reputation OK
3) No disclosure + Harm2 = client harmed and reputation harmed
4) Disclosure + No harmful use = client OK, reputation harmed? *1
5) Disclosure + Harm1 = client harmed, reputation harmed? *2
6) Disclosure + Harm2 = client harmed, reputation harmed *3
From this simplistic analysis we can see that a No Disclosure policy seems best, however:
*1; reputation not necessarily harmed; I would see disclosure before-the-fact as a sign of a very responsible company, and if nothing comes of the data breach, the client will remember the disclosure in a positive light.
*2; reputation only harmed if you own up; harm1 allows plausible deniability (unadvisable).
*3; The client will be pissed off certainly, but "at least they told us"; less damage to reputation than in case (3).
and given this, the new breakdown looks more like this:
1) No disclosure + No harmful use = client OK, reputation OK
2) No disclosure + Harm1 = client harmed and reputation OK
3) No disclosure + Harm2 = client harmed and reputation harmed
4) Disclosure + No harmful use = client OK, reputation better/OK
5a) Disclosure + Harm1 + own up = client harmed, reputation harmed
5b) Disclosure + Harm1 + not own up = client harmed, reputation OK
6) Disclosure + Harm2 = client harmed, reputation harmed less than (3)
Given this; I would disclose, either way, I hope the preceding helps.
And guess who's going to be in the shit if valuable information gets leaked? The execs that covered it up? Noooo.... the poor sap they convinced not to tell anyone about it.
Get everything in writing. If possible get signatures. If you need them for references get then *now* before anything goes wrong.
It's really time to consider that while it may not be easy, it's time to hire some programmers and write that replacement. Really. Win98 support is going to get more and more difficult, to the point where it is no longer reasonable to support it at all. Will it be too late for your company when that time comes?
If you want news from today, you have to come back tomorrow.
is it me?
As a consultant, it's not your place to dictate how another company defines it's business strategy.
You've said your bit to promote disclosure (I assume), make sure that there is a paper trail detailing that, then let them run their business how they see fit. Possibly into the ground.
If you're a third party contractor, and you start letting loose about your clients, thats not a good way to give yourself credibility. Remember that the management team for this company has likely spoken to their lawyers, possibly other security experts. There is the remote possibility that they know what they are doing.
Request that management hire a properly trained security resource.
Doesn't california have a public disclosure law? If the company in querstion has any customers there, they might be required to disclose it.
And once again we see how closed source software leads to big problems. Some "legacy" app, no doubt closed source, and now unmaintained, working on a closed source OS, which no longer gets security updates for that version.
And this is called "smart business" and "shareholder value".
There's a difference between this quarter's bottom line and the bottom line over the company's history. Are businesses only in it for a decade, or forever? That needs to be in their mission statement and SEC disclosures and *moron* shareholders need to understand economics better. If they are in business "forever-as long as possible", they really need to start to get aay from closed source and closed standards, because they are *unsustainable* for the long haul. At least with open source and open standards you at least have a chance of maintaining old apps if you really need them.
just looked, it is called "The California Security Breach Information Act".
the guy who fond out needs to check, get a lawyer, tell him eveything, let the lawyer contact the company and cite this law. After that, up to the company to keep screwing up and being stupid or not.
Again, with the moron shareholders, who are by and large the cause of all these problems all the time, there's-not enough risk anymore, we need MORE shareholders to get hosed, over and over agin with the oversight clue stick, because by and large they are not watching what their money is doing as regards ethics and sound LONG TERM business practices. I think a good start would be to ban mutuals, make shareholders actually be FORCED to go research individual companies.
As a network security engineer of many years, i've learned to LEARN from each experience. If the corporate execs say to keep it under wraps, then you should probably do so, But doing quite a bit of forensic work to find out what was stolen whould be in your best interest. Even though it was windows 98 you should still look for some trail. To prevent this in the future you can implement some inline monitoring systems like an IDS such as SNORT or on system third party firewalls such as Zone Alarm. Never put outdated Operating systems directly online, try to use an Authentication system for external browsing, most firewalls have this capablilty but are rarely used for load and complication purposes. Asking for someone to use a password when connecting to internet should not be viewed as an inconvience, but a security measure to possibly prevent partialy what you have experienced here. If the data was worth any kind of money at all you should have no problem convincing your higher ups to invest into a company wide enterprise monitoring software. They can monitor spyware, viruses, network utilization, patches, and much more across multiple machines enabling you to react quicker. Also I am a big FIRM believer in the PANIC BUTTON when it comes to corporate security. You should be able to enter a command quickly to disable network access until a threat is contained, or you could simply pull a plug if your close enough ;). These measures will save your network, a headache, lots of money and possibly your data at some point in time. But when someone complains that the internet is down and they cant work on their lovelife or get their email at work, you can smile and tell them your under attack so back off they usually go crying to the higher ups and ill bet you get their attention then!
However, if it makes you feel any better, I've found that less than 1% of the corporate entitys (that I have contracted or worked for) in the US actually use reliable monitoring capabilities. Mostly the Military have their act together when it comes to network security but at a price to convience to their employees (its well worth it). The rest just a simple firewall, router and basic patch management with little or no monitoring and are prone to attacks such as yours or even worse.
Good luck, Rember that you control the network, dont let it control you and welcome to the war that hardly no one talks about because they think it will go away with a new firewall....
NETR00T
I dunno... I'd be more embarrassed that the company was still using Windows '98 because it didn't want to replace their legacy software. Oh, I know, I've heard it all before... there's no replacement for it, it would be too costly, blah blah blah.
But there almost certainly IS a replacement for your legacy apps, and your employer is being stupid by continuing to use it. Instead of paying the cost of replacement, they're paying the cost of NOT replacing it... higher IT staffing costs, decreased security, and the potential loss of many customers, not to mention lawsuits.
I've done work for a few companies who tried to hang on to their legacy software forever, usually vertical apps. None were successful with the strategy. Usually it's a lazy CEO/CFO who's afraid of change, or a profound misunderstanding the economics of IT. And, when these companies eventually break down and replace the app, they usually screw it up one way or the other.
Anyway, in your situation, I would recommend full disclosure, not only to customers but also employees (their personal data is at risk). Don't worry about whether they do it, just make sure that you're on record as recommending disclosure. Then look for a new job and get yourself as far from that company as possible. Nothing good is going to happen there.
(Gosh it brings back such horrible memories.... "We need X, Y, and Z, implement them right away." "Okay, here's the PO." "Wow, I didn't know it cost that much. Isn't there some setting you can just change? Tell you what, install the upgrades and we'll buy the licenses next quarter." Uggghh. "We need full daily backups!" "Here's a PO for some blank tapes." "Wow, they're expensive. Find me a cheaper way.")
You should "tell" your "friend" to start looking into another field of work. If your friend is a security consultant, and didn't immediately see the dangers of having Windows 98 PCs on the network, he may not want to continue being a security consultant. Yea, it's a bit harder to convince the company to upgrade their "legacy" software, or migrate to a different solution; but this is, in fact, part his/her job. Stop blaming other people and circumstances, the fault lies on the consultant for not actually consulting on anything. The companies decision to be immoral with their disclosure of the data leak is their decision. Maybe your friend can start working at a warehouse, or floor support at a large corporation if he's that lazy and afraid to actually make any waves.
Why are these machines connected to the Internet?
If they are insecure, sandbox them or cut them off completely.
If they need some kind of network access, use a whole shitload of proxies and firewalls and a carefully-monitored snort install and babysit the hell out of it until they can be secured.
No, forget that. Get them off the net completely.
-- My Weblog.
They should disclose. It will leak out anyways and then it will be both a scandle and a security incident.
I'm not saying you are nasty, but the risk/benefit ratio analysis is certainly psychotic. The ultimate example is that of the air plane manufacturer doing a similar study;
1. If we fix a known fault in all our aircraft, it will cost us 1 Billion Dollars.
2. Over the lifetime of the aircraft, lawsuits due to death and injury resulting from the fault will cost us only 500 million.
3. Don't fix the fault; it's less expensive in the long run.
Money isn't everything. Doing the right thing leads to benefits which cannot be calculated.
-FL
On the matter of disclosure it all depends. IMHO there should be a disclosure. On the otherhand it may not be legally required. If the company is publicly traded then it's required. If not then it may still be required depending on the type of company or the type of data that may have been lost. This will vary from state to state as well. Either way it's not your responsibility to do the disclosing. It's not your data. You're an indirect employee of the company. You can not speak on the company's behalf. I would however recommend documenting the incident with a professional post-mortem on the systems. In that post-mortem report I would indicate the types of data that could have been compromised and recommend that the company have its legal staff review the incident to decide if the company required to take further action. Then leave it at that. You can't force them to do anything. You're a consultant; all you do is consult. If the company does this kind of crap often and is required make a disclosure but doesn't then I would sever ties with them. Eventually this will come back to haunt them. As the local IT contractor you'll feel the heat from it if this ever gets into court. Document everything, make good recommendations and keep your nose clean. If it ever gets to the point where you think the problem is going to become public or end up in court, document the problem and get out. The company can probably withstand the financial blow from a civil court but can you? You are bonded, right?
I can bet with near 100% certainty that I could walk into nearly any enterprise network, jack into the core on a mirrored port and find at least a few owned machines. If you are on a windows network and the clients have access to the internet there are some that are compromised...period. It takes constant monitoring and even then you are performing damage control. Keep your internal secuirty policies tight this will help to reduce the risk slightly.
Got Code?
Mr Worf, security alert. There has been a security breach.
One reason we've seen more disclosures like this lately is because of a recent California law that requires disclosure in such cases if California citizens are affected by the breach. I'm not sure if the law requires actual knowledge of a particular type of data being compromised, but this could be the lever he needs to get the company to DTRT and disclose (you only have to disclose to Californians, but after that, it's pretty much going to get out so you might as well disclose nationally right off). As I understand it the penalties for non-compliance are pretty harsh, and could conceivably fall on him as well, as a person with knowledge of the breach.
-- Old Man Kensey
Your friend should recommend they use virtual machine technology, which will allow
them to test on virtual copies of win 98 instead of real ones. It's much more secure
and will dramatically cut their physical hardware costs. He should then get his resume
ready, and perhaps be ready to change careers completely, because said company might
sue him personally for disclosing this breach and he will certainly end up fired.
If he does the virtual machine thing and says nothing until he can manage his
termination well he'll have done the right thing and looked out for himself too.
He might even go so far as to stage a fake web site break in to divert the storm
away from himself. No, I'm NOT kidding.
-- Programming with boost is like building a house with lego. It's a cool but I wouldn't want to live in it
Your 'friend' has already screwed up. ( sorry to put it that baldly, but he has ) He was hired to deal with security issues, not legal ones. He never should have discussed client notification with them. When he starts expressing opinions about that, he is way outside of what he contracted to do. He may not have recognized this breach of manners, but, I assure you, they have.
Now, if he - or anybody else - leaks this, management will assume that it was him.
If I ever learned that the company responsible for protecting my security covered up a breech, they would be GONE. That day. That shows an incredible lack of integrity on your company's part. There's really nothing you can do to help your situation there. Eventually someone will catch them and that will be their undoing. Anyone around them will be tarnished. The best thing you can do is put the resume to work, DOCUMENT EVERYTHING, and talk with a lawyer that specializes in these things, there are probably a few minor little things you really need to do to make yourself proof against this coming back at you later. An hour's legal advice now may save you thousands and help you keep your next job two years from now. There is the chance that you could be pulled into this, even years from now. A cheap way to document is to send your documentation to yourself in registered mail, and DO NOT open it. If things ever come to a head, this is a very cheap and easy way to document what happened in a way that cannot be acused of being tampered with or fabricated after the fact.
I work for the Department of Redundancy Department.
At least they don't have to worry about nuking the site from orbit.
That window of opportunity has closed.
-- 3 events that reshaped the world in the 20th century: WW1, WW2, and WWW
In addition to all the technical work, your friend should write a letter to his client recommending they seek legal advice on handling notification. He should keep a copy, hand-deliver a copy, and just for backup, send one certified mail restricted delivery.
Your friend should also seek his own legal advice to find out if he has any exposure other than loss of a client and a good reference if things go bad. I doubt it but peace of mind is good for your health.
He should also consider circulating his resume.
Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
Second, all the smoke and mirrors notwithstanding, Windows 9 probably is not much more (or less) insecure than NT based Windows. They both suck as far as I can see. If anything, Windows 9 may be better for two not particularly admirable reasons. First, newer and more sophisicated attacks tend to focus on Windows 2K and XP flaws, which often don't exist in older versions. Or they use API calls and Services not available in the old software. Second, Windows 9 is a lot simpler and therefore easier to work with. The chances of detecting and/or preventing future attacks are probably better.
Third, putting any machine working with sensitive data on the Internet is not a very good idea. In fact, the most sensible practice for any operation in the current or likely future state of computer security is to segregate really important sensitive data and keep it on machines with NO network connection. That isn't always possible of course. But to the extent it can be done, it should be. I don't know what the current military/government policies are, but two decades ago when I last worked with classified data, connecting ANY machine with ANY classified data to a network that wasn't isolated from the outside world would have been a major security violation and would probably have resulted in immediate loss of individual and maybe facility security clearances. I don't see any reason to believe that wasn't a wise and prudent policy.
(Lest anyone protest that there was a provision for connecting to less or un- secure systems even back then. They would be correct. But the hurdles that had to be overcome to do it were so great, that it was hardly ever done)
You can't see ANYTHING from a car, You've got to get out of the goddamned contraption and walk...Edward Abbey
Do not disclose until there is evidence that the information has been used. The people who received the data might not know that they have it and they will not be able to find it without further information. Once you go public, they will start looking for it.
So, until you've got evidence that they already did use the information, you should seriously consider keeping silent. Even mentioning the name of the company could lead to a series of IP-addresses and hence to the data.
Cut that spamming shit out. It's annoying.
I've had enough abrasive sigs. Kittens are cute and fuzzy.
his only choice is to quit working there. He is only a consultant, so he can make recommendations, but the company is free to ignore him. Odds are that he is likely to be bound by a non-disclosure agreement regarding the network and data situation at the company, as well.
Your firm should contact their lawyer, a large majority of states of state disclosure requirements, and these laws vary by jurisdiction. Some have very stiff penalties.
I run it via Win4Lin 9.x over Fedora Core 3. I've never seen ZoneAlarm go off since I put it behind a Linux firewall. To do Windows AV protection, just run F-Prot for Linux, it's got the Windows virus signatures and updates automatically via daily cron job.
Oddly enough, the only legacy Windows apps I run regularly are Eudora and occasionally, Word and Excel. (I have OpenOffice, at what I do, "minor" compatibility problems aren't) I use the Linux host for everything else.
Tech Public Policy stuff
I guess nobody noticed the "gigabytes of data" that was being pumped through the company's Internet pipe? Also, how do you know the server was actually in Eastern Europe?
$nice = $webHosting + $domainNames + $sslCerts
#1 - run the hell away. if the client is not interested in doing what he suggests then he is wasting time. those 98 machines should have been on a secure private network with no internet access for years now. if the company refused to do that he should have said, "then you will have no security, your data can and will be stolen eventually, are you ok with that?", if they say yes, have them sign off on a hold harmless waiver. always end that statement with that question. it delivers ownership of the problem to the exec and allows you to CYA.
when the security breach happened like this you can then say "executive XYZ said he was ok with that, see here is his sign off acknowledging that fact.
Secondly, win98 apps can be ran in a virtual system that would have allowed him to have some security.. why did he not do this? was the client a cheapskate and refused to pay for anything?? if so then once again it's a run away situation.
This could have been avoided, it would not have been cheap, but it could have been avoided. IT consultants need to have the balls to tell a customer "NO! you have to do it this way." because they are paying you to be the expert. If they do not listen to you sugges they hire the "geek squad" from best buy then if all they are looking for is IT people that will do what they are told.
Can you tell I am fed up with incompetent clients that say they want security but refuse to pay for it?
Do not look at laser with remaining good eye.
Make sure that those Win98 machines are isolated from the Internet. Get an old box, fit two NICs and as much RAM you can find, and install Debian. Configure your IPTables to block everything in either direction. Add rules to allow through only whatever you really need and log the most suspicious stuff. If there's e-mail involved, use the Debian box (which will have the excellent Exim MTA installed by default) as an SMTP server -- set your ISP's real SMTP server, or your company's Microsoft Exchange server, as the smarthost (If your connection is on a static IP address, you needn't even bother with the smarthost, just have it send mail normally. But do install your own BIND). Block any other connections on port 25.
.....) Hell, if you're running Windows 98 then there's a chance you caught it from a customer in the first place.
All the above can also be done (slightly differently) on OpenBSD, which is reckoned to be even better for security; but my experience is with Linux.
It's also possible that if you were compromised, anybody you ever e-mailed and who is running Windows will have been compromised. You could just deny everything and let them Find Out The Hard Way (aw shoot, looks like we were just a couple weeks too late with getting the new security kit installed
Je fume. Tu fumes. Nous fûmes!
I'd recommend that your friend talk to the people at the Online Ethics Center for Engineering and Science Helpline. They will give him an answer based on the IEEE Code of Ethics. Although aimed primarily at engineers, they are also able to help IT professionals. The website is also a valueable source of information on topics such as disclosure.
One further note: If any of the leaked information could cause a public saftey concern, disclosure may be even more important to consider. Also, if one of the clients is the federal or a state government, he may want to consult a lawyer to see if diclosure is mandated.
While this doesn't solve the security breach, for future use, Virtual PC from Microsoft may be an option to run virtual Win98 sessions from XP machines. Microsoft made it free and available for download from here: http://www.microsoft.com/windows/virtualpc/default .mspx
If you know there was a breach, some States have laws requiring you to notify the authorities/citizens of the breach. Even as a Consultant, you might have some liability for not disclosing this, and you need to let the company know of their legal obligations. Maybe even get them to sign a document that releases you from any liability from any violations of law or lawsuits resulting from this loss of data.
If the software can handle it, run the entire mess in a virtual environment, in a secure OS. Have the hosting OS take care of opening only the ports necessary for the software to run.
Help! I'm a slashdot refugee.
Who would they disclose it *to*? A press release is pointless. Contacting a state attorney general? Of what state? And then what?
I mean, if they knew the disclosed to particular people, then they should let them know, but in the absence of a known disclosure, they should contact their attorneys and sit tight.
I worked techsupport at a larger radio station over the summer, we have a whole network of Win95 machines (Hey you try running Audiovault Software and soundcards on anything else...). To prevent the havoc and resulting downtime (which would result in us going off the air) these machines are not connected to the rest of the world, just each other. To add something to the machines we use audio cables and record directly in.
Perhaps a future solution for your friend could involve using a similar system (With printers and hard text) to keep the data secure.
Shots: A Populist Parable
Wellll there's yer problem!
--Rob
Towards the Singularity.
In the middle of a security breach? If it's really bad, like publishing nuke secrets in Arabic on the Internet while you're inciting the terrorist world, you should "stay the course". Accuse those disclosing the breach to authorities of "emboldening the enemy" and "disclosing security procedures". Attack, attack, attack. You'll get to keep your job, though your company might go out of business, perhaps in a mushroom cloud. Then you could claim you'd been "right all along", while you burn in hell for eternity.
--
make install -not war
It's the company's customers not his. Make sure he puts his opinion down in writing where it will be visible to any later investigation and that he states his belief strongly and clearly. If they don't want to do it there is nothing productive he can do if he wishes to keep his job.
Hmmm, time to go. Since it happened on your watch and you are a contractor, you will be blamed...
Excuse me, but please get off my Pennisetum Clandestinum, eh!
You should draft an official document stating that you have consulted with experts, curiously named 'Anonymous Coward', and are following their recommendations.
- For the complete works of Shakespeare: cat
The issue is not one of the company's obligation, but that of the consultant to the company. So the first issue is not that the consultant is required to tell anyone outside the company, but weither the company should. Generically, if there is a security issue - the obligation for the security consultant it to inform "the company" of this fact. Write up your recommendation, and send it off to everyone to whom you deal with, who should know. CC yourself.
It is the C-Suite that is responsible for the security of the company. It is the responsibility of the Technical Staff to give the C-Suite the information necessary to make informed security decisions. If your friend is a senior enough professional, it may be his situation to recommend disclosure, but it is in no way his call to actually do it. INAL, but if the company does not wish disclosure, I suspect that the only ethical way to disclose this is with a whistleblower lawsuit.
I have had to deal with similar issues before, and recommend speaking with a Lawyer if you are really concerned. At a minimum you will find out your legal obligations and liabilities.
$.04
you're the consultant who says "nee."
Your friend should ask a lawyer at his company how to handle this situation. If there is a possibility that the "sensitive company data " includes personal data from customers, employees, etc. then the company may be legally required to inform all of the people potentially affected of the situation. The law here has been changing quickly at the state and national level, and I am not a lawyer, so the best advice is to ask a lawyer.
In other words, this is not merely an IT problem, it is a PR problem and probably a legal problem. The company that lost the data probably pays a lot of people to be experts those fields, so your friend should consult with them, not us.
Something else to watch out for: If the exec utterly refuses to take ownership of the problem, then a possibility is that the exec was aware of the potential breach, was unable to get higher-ups to pay to fix it, and wanted the consultant around to pass the buck to and/or sue when it hit the fan.
I am officially gone from
Responses above are "get the MS-Win98 machines off the net".
Think bigger. Get all the company machines behind a firewall.
....
don't ask on slashdot?
Seriously.
If your "friend" thinks he needs legal advise, he should ask a lawyer.
If your "friend" is asking for technical advise, while dosbox and wine are _great_ ways to impose greater restrictions on legacy software, if your "friend" is asking for technical advise by acting like he's looking for legal advise, then your "friend" is an asshat.
Real Digital Forensics: Computer Security and Incident Response available on Amazon. Not an author just a satisfied customer.
-EB
Do you ever walk alone like a drifter in the dark?
Slashdot's internal translator did an excellent job. However, like most things, it's better if you read it in the original Klingon:
Ok slashdotters, a friend...um... yes... a friend didn't do his job, and blamed it on the operating system. The management isn't buying into his excuse and told him to do his job or get sued for neglegence. But instead he wants to get paid so he play WoW and say it isn't his problem.
How do i hack into management's email and holds the good against them so they pay him hush money and call it a job?
Have you read my journal today?
"Shields Up! Red Alert!"
In many states now there are consumer protection acts that require companies to inform those that may have had their information comprimised.
- breaches-law.html?fsrc=rss-security
http://www.networkworld.com/news/2006/010606-data
Of course it may different for your state as it's not nation wide that I'm aware of, but the fact still remains it is illegal in almost half the states in this country to "keep it quiet". More over, he WOULD be implimented in this mess as he knows of the problem and doesn't say anything. Either rate, as professionals...it falls on US to protect clients' and comsumers' data. Most ID theft is caused by poor business practices, not from anything that the individual has done and this is a perfect example of that.
On another note, wtf is this guy thinking having Win98 machines on a business network live on the 'net without firewall(s)? It's one thing to have to use it for legacy software, it's another to make it a juicy target. I hate to even bring this up since it would end up being flamebait or could label me as a Linux Zealot, but have you considered using Wine, dos4lin or anything to run the software? If it works at least then you'd have a current OS to run the software.
Disclaimer: I do forensics and pen testing.
Just because you're running Windows 98 and have an old compromise doesn't mean you're totally out of luck about the audit trail.
If this company has been compromised by a targetted attack and a motivated attacker, this gal may still be "in", having moved on to better jump hosts with better TCP/IP stacks. This can still be traced, and evidence for prosecution may still be gathered.
Otherwise, pray that it was a non-targetted attack (a worm, a botnet sweep, etc). And pray that it is not a California-based company.
I absolutely agree with earlier suggestions that it would be an idea to install the "insecure operating system" inside some sort of virtual machine taking a sandbox approach. That should make it possible to better control how these legacy apps are allowed to interact with their surrounding environment, network connections and the like.
You could consider implementing a thin client system where one or more central servers served the win98 apps to those needing them and the old puters could run a different OS (Nix variant) which alowed screen interaction with the needed apps. while the central server crunched the numbers.
Things to consider:
Stateful inspection - Validate the data content of packages
If your 98 applications require data from the outside - Use Application proxies
Separate Firewall systems from your main system (OS and Hardware wise) so it is a system running by itself.
Make sure the firewall also keeps an eye on internal traffic - Same with the suggested application proxies, which by the way can be run under a totally different OS than the one you run on the inside as their only job is to inspect packages and validate input, making you a harder target.
Never forget - Security is an ongoing process, You got to keep moving to stay ahead in the game.
Pull the contents of every win98 computer. Assume all of it got sent. Assess the damage, and do damage control.
This is a great time to push for a security policy. Had they a security policy, the 98 machines would have been gone since most sane security policies don't allow unsupported software.
Firewall the network properly. Don't allow any access to internet unless it's through a proxy server. Use a whitelist. You'll get gobs of complaints about people not being able to access stuff. Allow sites on an as needed basis. Just allow the stuff people ask for. Allow the search engines, cnn, msnbc and anything else your users need to function.
The only way to control the security of your network is to control the damn network.
The best way to do this is to "allow only the good stuff". Default allow caused this mess. Default deny is the way to fix it. It will be painful at first, but not quite as painful as watching all the data on the network end up in Eastern Europe.
good luck.
-AC
With the new SOX and other recent legislation, companies are now often required to divulge when customer information is leaked. Do you think places like sayign they leaked 20000 people's personal records? They don't, but they also don't have a choice in the matter. If the place you are working for is fairly large then you could potentially be held liable for helping keep this swept under the rug. Read the law and figure out what the right thing to do is and do it!
Ninjas don't carry tic tacs
If I were a consultant (and I am) the first order of business would have been to remove any and all Windows 98 workstations as soon as possible. If they were hesitant they would have to sign a waiver indemnifying me of the almost certain issues they will run into with a network running an outdated OS.
and zeroes are wrong. Wait... or was that ones are wrong and zeroes are right?
Interested in a Flash-based MAME front end? Visit mame.danzbb.com
(disclaimer - I've been doing this WAY too often :-)
;-).
AFAIK you're facing a legal requirement for disclosure, but also a PR nightmare if you mishandle it. If your DR and BCP doesn't say anything about media handling you ought to give its author a bit of a heads up - the disclosure is going to be painful enough, mishandling how you tell the customers this (and the press) can cause serious harm to your customers.
I won't address the legal issues - that's what lawyers are for. Tech stuff you will have covered by the time everyone has had their say as well, so let's pick a less obvious one: media and press.
A couple of things:
- don't lie. When (not if) you'll get caught out it'll destroy the last remnant of trust you're trying to salvage;
- don't estimate anything unless you can back it up with numbers or a method by which you arrived at the estimate. The problem is not the estimate, it's what happens if you got it wrong (+ or -);
- don't duck the truth. Something went wrong. This is IMHO the best route to keep trust: if you have found what happened and have addressed it that's good news. If you're still guessing that's not so good news;
- think as your customer (I know it's 'duh' but you'd be surprised at how often this gets overlooked). If I had data out there I would like you to tell me (1) what the risk is (2) what YOU are doing about it for me and (3) what I can or even must do to protect myself further if so required.
There's a whole set of things you need to do here (besides sorting out the root problem), and be aware that sorting out a crisis is an entitely different skill than running day-to-day ops, but I'm biased as I do this work myself
Good luck, and don't forget to evaluate 3 months on how you did. The lessons you'll learn will then save you a lot of pain the next time...
Insert
III. Professional Obligations
1. Engineers shall be guided in all their relations by the highest standards of honesty and integrity.
a. Engineers shall acknowledge their errors and shall not distort or alter the facts.
b. Engineers shall advise their clients or employers when they believe a project will not be successful.
5. Engineers shall avoid all conduct or practice which deceives the public.
As far as my ethics course was concerned, your obligation here to inform the injured parties was required. Documenting what went wrong but not notifying the injured parties was not acceptable, and to my understanding you would indeed be liable in such a situation.
I notice a lot of the answers avoid the issue of what should be done. His question clearly involves ethical considerations and perhaps he is not looking for a description of the boundaries of the consultant-employer relationship, but whether he should be a moral whistle-blower.
It's always someone's "friend" isn't it? :) Is this friend, by chance, your alter ego?
Find out how likely it is that you can be considered an accomplice to a crime when the break IS discovered and the owners of the company are pointing fingers trying to reduce sentences. It's really easy for them to say they never got the message from you and dump the whole thing in your lap.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
Your friend needs to realize that he consults on technology, not business.
He needs to inform the company that an unknown amount of their data, somewhere between 0% and 100% has been stolen and or changed, and recommend that if they want to stop this, they need to change or upgrade their workstations and network security.
For an immdiate badn-aid, he can recommend that they all be unplugged from the network, but this will not fix the damage or prevent more when they're plugged back in. Then send them a bill and go home and relax.
Whatever the company does with that infomation is up to them.
SOX has nothing to do with this, dipshit. I know it is a popular law to throw around because you've heard about it in the trade press, but it has to do with accuracy of financial reporting, not data breaches of customer data. Closest SOX gets to this is requiring audit of controls on the company's own financial information and the requirement to disclose events that may negatively affect future numbers. You could kind of fit that last bit into SOX if the data disclosure became public and hurt share prices, but SOX is NOT a data breach disclosure law.
I apologize for the flammage ahead of time. Perhaps I was just in a bad mood, but people (especially on slashdot) seem to misrepresent some of the recent corporate confidence laws. I admit I am no expert myself, but I do know SOX does not specifically address data breaches.
Don't just pray that the company isn't California-based - pray that this company does not have a single clientin California. According to Cali's Mandatory Disclosure Law, any company that does business in California is required to notify clients in that state of any breaches in security. If the company does have clients in this state, it is legally obligated to disclose to those clients, and if that news reaches the press, they'll do your disclosure for you.
Inform the clients that their data is preserved on a network of unpatched legacy Microsoft systems and let them connect the dots.
tone
tone
Reading the summary posting here, I'm seriously beyond baffled as to why in the devil's briefcase any business would have a Win98 machine hooked up to the internet these days as anything other than a honeypot. And to have multiple machines? With access to sensitive data? Come *on*, man, wtf were these people thinking? Egad. More to the point, *were* these people thinking?
Because they never should have been there in the first place.
"What in the name of Fats Waller is that?"
"A four-foot prune."
"... He quickly convinced company executives to remove Internet access from all Win98 machines, purchase better firewalls, ..."
Yeah, when your firewall is not strong enough to block IP packets you should buy a better one.
But why these Win98 machines were given Internet access in the first place ?
You shall never give direct Internet access to a Windows machine.
One thing I learned is that when you have Windows machines, firewalls are not for protecting you, but there are for protecting the others from your Windows machines. Giving direct Internet access to a Windows machine is like giving a shutgun to a monkey.
I scream when I see companies with broadband Internet access and using a simple router with the default config that does NAT for all their LAN.
The problem is not solved by getting a better firewall, but by having the balls to deny any forward from LAN to WAN on the firewall, and force everyone to use an authenticated HTTP proxy and SMTP relay. Then, you can seat back, and watch all the PROTO=TCP DPORT=25 or DPORT=80 packets from your Windows machines being dropped by your firewall and filling your logs.
"Corporate Windows machines infected by a worm", "redirector in Eastern Europe" and, of course, the not-so-well-hidden hint that "using Windows potentially leads to serious legal problems"... Appparently, it's been a long time since the previous mandatory made-up Linux-fanboy Slashdot story. This one is pretty lame, meaning that its fakeness is rather obvious, but I'd say that after that robot and the talking pie disaster it's well above the todays Hollywood standards.
Heck, it's even possible to run an instance of Win98 inside qemu.
I've done that to play some of my old games before. If the only reason why the legacy app doesn't run on Win2k/xp is just that it detects the version, this is a free way to virtualize it.
If I was the consultant, I'd document this event, and find a convenient excuse to get the heck out of there.
netr00t's got solid advice for you.
e ction=activescout
l
v ery/
http://slashdot.org/~netr00t
I would add, get a Lawyer, as in, have a Lawyer (anyway).
If you're in the USA, you should know by now, mostly morons make the "rules" of conduct, try not to participate.
Pay the Man:
http://www.forescout.com/index.php?url=products&s
http://www.winternals.com/
Useful:
http://www.sysinternals.com/SecurityUtilities.htm
http://www.porcupine.org/forensics/forensic-disco
http://www.fish2.com/tct/help-when-broken-into
Firewalls and Internet Security
http://www.wilyhacker.com/
First Ed. (online)
http://www.wilyhacker.com/1e/
Practical UNIX and Internet Security
http://www.oreilly.com/catalog/puis3/
FWIW
http://exuberant.ms11.net/index.html
http://exuberant.ms11.net/98sesp.html
http://exuberant.ms11.net/links.html
http://www.oldversion.com/
~hylas
The first line told me everything I needed to know.
Like some people who read forums they automatically respond without reading the whole article and in this case I didn't make an exception and didn't need to read further.
I really don't feel sorry for the company but I bet they were entrusted to safe guard some sort of important data?
First, everyone who says that it's not his decision is right. Second, if he feels that his client should inform the individuals, he could look at the state laws that apply. Figure out which states your customer's customers are from, and then look here. See which states have laws that were in effect at the time of the breach. Depending on what information may have been lost and what states we're talking about, the answer varies. I've handled a couple of these so far, and I'm glad to say that the company involved did the right thing in all cases.
Some Win98 installs I've seen are there for hardware reasons (expensive specialised A/D conversion cards in industrial machines with a few processor boards on the backplane) - but they are not on any networks.
but not oracle's redhat version
no sig = no personality(?)
Do what the client needs but dont go over his head.
The first task is to find out what could have gone out. In my experience, if the customer is using legacy systems, they are still using LAN based stuff (max) and so not likely to have anything but loads of basic raw data (difficult to compile and collate). If so, your friend has to determine whether what could have gone out is really worth "disclosing".
Finally (after covering your ass), try and get the W98 machines on a LAN with a proper firewall. (A reading of the original post seems to suggestr several individual internet connections).
End
What he should do depends in large part on how much he is willing to suffer for people to whom he owes nothing, and who are unlikely to defend him or even thank him. Disclosure could negatively impact (read "seriously screw up") his life for years to come.
He has already fulfilled his basic moral and ethical obligation, which was to disclose the problem to the company's management, tell them everything he knew, and make his best recommendations on how to handle it.
Beyond that, the moral, ethical, and possible legal IANAL) onus to disclose is on the company.
If your friend believes that he has an obligation to go beyond this, things to consider are (IANAL):
1) It may be either civilly or criminally illegal for him to disclose. This can vary widely depending on country and locality, so he needs to consider if he is willing to go be fined, imprisoned, or successfully sued as the price of disclosure;
2) Even if there is no statute explicitly making that disclosure illegal, the company may attempt to sue him for disclosure of trade secrets, etc., anyway and see what sticks. His contract may also contain language directly pertaining to that; if it does, he needs to watch out;
3) They are certain to turn on him for disclosure, even if they don't or can't sue him. Managers know other managers and he will get a ready-made reputation as a consultant who can't keep his mouth shut. Consultants who breach confidentiality have a much harder time finding work;
4) Does he have anyone who depends on him? I have a family to support, and my moral obligation to provide them with food, shelter, and clothing trumps any obligation I may or may not have to companies or individuals whose data may or may not have been compromised. I would do what he has already done: provide a full report to management, along with my recommendations. Then I would shut my pie hole. I owe that to my wife and children.
Finally, whatever path he intends to take - saying nothing further, contacting the customers directly, or spreading it across as much of the Web as possible - it would be a *very* good idea for him to consult with a lawyer, and get answers to these questions:
- In my country/locality, what are my legal obligations?
- In my country/locality, what are my legal risks if I make any kind of disclosure that is not legally required?
- In my country/locality, what are my legal risks from making a disclosure, even if that disclosure is legally required?
Oh, one more thing - if he decides against disclosure, draw up a plan for how that legacy software might be usable on Linux machines running WINE. It may be a lot harder for a Win32 worm to affect one of those than a Windows 98 machine, and even if it's not, the security wrappers you can put around the machine using iptables will likely render the worm unable to function even if it successfully infects the machine. If the WINE approach won't work, then propose running Windows 98 in VM Ware under Linux - the security layer made possible by the host OS still applies, even if the guest OS is totally porous.
No, VMware doesn't support everything, but server applications are unlikely to have fancy hardware requirements (CPU, RAM, Disk, Ethernet, maybe CD-burner, no video or audio.) A new 3 GHz motherboard and CPU with 1GB RAM and a disk will set you back a good $300 these days, and should be plenty to run that K6-400 application. Do whatever firewalling you need to in front of it, and run as much anti-virus as you can fit. There's certainly no need for the antique application to be exposed to the raw Internet or even a semi-cooked intranet.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks