Slashdot Mirror
Security Issues with Windows 2000 Datacenter?
"My company is currently looking to cluster our SQL 7 servers. We're
considering Win2000 advanced server or datacenter. Around a month ago I sat in a meeting with our VP of IT, and the rest of the network admins I work with. Compaq tried to pitch their Windows 2000 Datacenter or Advanced Server solution. Here is the way the compaq people explained it:
You get datacenter only from an OEM. They look at the apps you're running
and customize a solution for you in their lab. Every datacenter implementation is different, and every datacenter CD is different. Since we would be using an EMC SAN as our clustered storage system they said our implementation would take special customization. They would have to contact EMC engineers and work together. Once you deploy it, the OEM monitors it. And you can't install any service packs or anything without getting an OK from your OEM. Any service packs are customized for your enviroment. The SLA guarantees a 99.999% uptime or your money back. Part of your money at least. Datacenter isn't an OS, but a program in their words.
Now here is the problem. With Code Red and Nimda, how do you patch IIS
running on datacenter in a timely manner? The reason IIS servers became
infected was because the admins didn't patch them in the first place. So say
a new worm comes out in a few months and it takes a few days for MS to
create a hotfix. Datacenter admins can't install it until they get their
customized copy from their OEM. And almost every 2000 server runs IIS for
terminal server. It can take a few days and in the meantime your servers
could be down. And I don't see the SLA covering a situation like this. Meanwhile you're explaining to your CEO how this $500K supposedly guaranteed solution is sitting dead in the water and you can't do a thing about.
Is there something I'm missing, or did Microsoft look over something like
this? Especially when they are trying to push Datacenter as 'Big Iron'."
Ask the vendor to modify the SLA to specifically cover the contingency of exploits and how they will be dealt with. Your vendor might try to claim that the 99.9999 uptime would cover this, but I'd counter that a server which is up but exploited is useless.
No one ever had to evacuate a city because the solar panels broke!
"And almost every 2000 server runs IIS for terminal server"
Erm, I work for a Citrix Gold partner and I've never encountered this before. Installing Terminal Server does not require IIS.
In fact, according to M$ recommendations, you should minimise the services running on the TS box.....That means no IIS.
Also, the "smaller but more servers vs fewer 8 way servers" for TS debate has been done and dusted, and the recommendation certainly isn't for having fewer large servers. The "sweet spot" is a farm of dual processor servers with 1.5Gb of RAM, thus you wouldn't need Data Center anyway - normal W2k Server would be more than adequate.
"Mary had a crypto key, she kept it in escrow, and everything that Mary said, the Feds were sure to know."
Keep these SQL apps behind the firewall.. turn off all IIS features on the sql boxes.. and at least Nimda should not be able to get at it. Any web interface would hopefully not use Datacenter, and use standard Advanced Server, which is easily patchable. If sql was available on the front line, well, they almost deserve it.
-=-Ze End-=-
If you aren't allowed to patch your server, then you should isolate it behind a firewall of some sort, so that the chances of infection are minimized. This may not work well for IIS (beyond simply not running it), but it will serve you well in the general case.
Reboot macht Frei.
Is a locked down version of Windows. What happens when you lock it down? Well, intensive testing occurs first to determine what is being done with the box and what possible problems could arrise. Then those problems are solved. Also, only certain applications are certified to run on a datacenter box. The goal here is to achieve five nines. That is have this box up and running for 99.999% of the year. Without thorough testing of applications this level of availability would be impossible.
Part of what you get with a Datacenter purchase is a premier level of support. This includes a named engineer for support, and automatic escalation to the highest level for any support needs. It also includes any updates and or fixes on a priority basis - if you have a Datacenter server you get patches, updates, etc. before anyone else does.
yup, you shouldn't be running IIS and SQL Server one the same machine. Ideally, you'd run SQL Server alone on the big machine and have a cluster of load-balanced inexpensive boxes running stateless ASP/ISAPI pages connecting to the DB over the LAN. You'll be free to patch the IIS boxes as needed and you can put them in a DMZ for extra security.
Then you can negotiate all the details. And remember: 99.999% uptime does not mean that your server stay up that long, but that you have only an unscheduled downtime of 0.001% or less. Applying a patch is, in nearly every case, a scheduled downtime and does not count.
Now imagine you really, really need this patch: you can urge your OEM to install it and keep him free from all responsibility (e. g. a server crash after this does not count to the unscheduled downtime, because it was your decision to apply it). If you trust him to play fair, that's fair for both of you.If the OEM is trustworthy, he'll do what you order him to do, but in that case you will be responsible for the outcome as well.You can't burden someone with responsibility if he can't make the decision (unless you don't play fair).
You found a sword: +4 damage, +5 moderator points
I can see you haven't worked with Microsoft software very much, so I'll give you the solution: Reinstall your machine.
It's *just that simple*, can you believe it? Every time Nimda hits your machine, just wipe out the system drives, reformat and re-install! Easy, right? Sure you may have to reinstall 40 or 50 times a day, but again, if you are familiar with M$ software, you'll know you need tons of backup machines that you can swap out as needed with your infected machines. Make an assembly line of it. Have one guy reformatting, another guy reinstalling and a third guy disconnecting the infected boxes and plugging the fresh machines into the network!
Now, where do you want to go today?
"Your superior intellect is no match for our puny weapons!"
This is commonly refered to as the Mainframe Mentality: these systems are so critical to a business, you don't make any changes to them unless these changes are a. absolutely critical and b. have been tested extensively in the exact configuration you'll be running them.
Now, it may seem that this would cause every Windows 2000 Datacenter server to be instantly infected with Code Red and friends, but in reality this will not be the case, because:
1. You don't expose your Datacenter servers to the Internet -- never. No matter if you're running Microsoft, AIX, Solaris or Linux: only trusted systems should have strict "need to know" access to your server;
2. Datacenter-type servers typically don't run HTTP servers. You would scale out HTTPDs (more boxes), not scale them up (bigger boxes). Also see rule 1;
3. The config of your Datacenter server is the bare minimum. So, in the case of Windows 2000, you would not ever run IIS or Index Server (the true culprit in case Code Red et al...) on it, just your database server and perhaps your business logic (although that, again, tends to scale out better than it scales up).
In summary: security hotfixes and Datacenter-type environments tend to be mutually exclusive. If you need a patch to your Datacenter server, it pretty much needs to be custom-developed for you. Fortunately, since Datacenter setups are not typically designed by the clueless individuals that gave Code Red free reign, this tends not to be an issue in real life.
If anyone out there is running Win2k Datacenter, I've got an important question I've been trying to find the answer to, with no luck so far. Can someone finally give me an answer? The question is this:
Does Windows 2000 Datacenter ship with 3-D Pinball installed by default? If so, is it in the Start menu?
That's all. Thanks.
$x='S24;r)>63/* h@<5+oZ)32"5cz';$me='phroggy'x$];
$x=~y+ -xz+\0-Tx+;print$_^chop$me for split'',$x;
Comment removed based on user account deletion
"I don't know of one bank that uses a non-IIS platform."
You need to look harder then. The first 5 banks I could be bothered to look at:
"Mary had a crypto key, she kept it in escrow, and everything that Mary said, the Feds were sure to know."
Specs are hard to write and all vendors have weasel clauses. Just look at insurance policies - damage due to acts of war are generally excluded. With cracking being described as a "terrorist act" you could end up with exploits not being covered.
A big common exclusion is "unscheduled" downtime. One of our vendors would see a router or firewall machine starting to act funny and then quickly "schedule" some emergency downtime that night to reboot it thus avoiding having to pay.
I have not had good experience with outsourcing - never forget that these are the same bunch of folks who are getting skewered for lousy tech support for poor end-users who have paid extra for support packages. Attitudes don't change much across corporations.
Before I would spend the bucks for any sort of "managed services" I would make sure that the vendor guaranteed 100% availibility without exception. Availibility must be defined as a maximum latency (ie. no end user will wait more than 750ms for a response or whatever is needed).
Rationale? Any app that requires this type of support must be available to the end user without fail. That's why you pay the bucks.
OS is "up" but web server is compromised or down? It's no good to the user. The downtime was scheduled? End user doesn't care.
Why 100%? Why not. They are already guaranteeing less than 316 seconds per year of downtime. Let them work their payments for that downtime into the contract cost. I don't want to have to total up downtime and argue over when the year started. I want the vendor to know that any downtime costs them bucks. No argument, no weasel clauses, no exceptions (better keep those machines maintained, protected and patched).
Been there - been burned. We moved our servers from a "managed solution provider" to a generic server farm and got far better service for one tenth the cost.
Put the datacenter server behind a firewall, preferably with some string matching functionality (ie watchdog).
/default.ida, filtering on global.asa is also a good idea ;-) etc ..
the later iptables have a string-patch included, which allow you to target certain port/string combo's, with this it is easy to block worms from the webserver, as long as you know what request it makes.
exampple to block cmd.exe access (taken from my own internal firewall scripts, this will block nimda)
$IPTABLES -A INPUT -p tcp -i ! $INTERNAL --dport 80 -m limit \
--limit $LIMITLEVEL -m string --string "/cmd.exe" \
-m state --state ESTABLISHED -j LOG \
--log-level $LOGLEVEL \
--log-prefix "MS IIS cmd.exe usage:"
$IPTABLES -A INPUT -p tcp -o ! $INTERNAL --dport 80 -m limit \
--limit $LIMITLEVEL -m string --string "/cmd.exe" \
m state --state ESTABLISHED -j LOG \
--log-level $LOGLEVEL \
--log-prefix "MS IIS cmd.exe usage:"
$IPTABLES -A INPUT -p tcp -i ! $INTERNAL --dport 80 -m string \
--string "/cmd.exe" -m state --state ESTABLISHED\
-j REJECT --reject-with tcp-reset
$IPTABLES -A INPUT -p tcp -o ! $INTERNAL --dport 80 -m string \
--string "/cmd.exe" -m state --state ESTABLISHED\
-j REJECT --reject-with tcp-reset
If you wanted to block codered, filter on
(see iptables docs for more info)
G'luck
I actually posted this question twice, and I'm glad they used this second posting with our actuall situation. The first one was more of a what if scenario.
As far as terminal server and IIS, you need IIS if you want to use the Terminal Server Advanced Client and go in through the web. I was originally taught to use TS through IE and forgot going in through the TS client.
If we do go with Datacenter, the servers will host SQL 2000 Enterprise in a clustered enviroment. We currently use SQL and have a propritery in house written app for it.
And as far as the Code Red holes being found months prior to infection, I just used this as an example. I remember in 1997 and 1998 NT had new security holes every week. Windows 2000 is slightly better. 6 months ago I remember downloading hotfixes that will appear in service pack 3.
My question still remains, if a new flaw in IIS, the kernel or any other part of the OS is found how long are we supposed to wait for a fix? I forgot the specifics, but I'm pretty sure the compaq people said they customize the source code for your enviroment. They will need a copy of our in-house app, get in touch with the EMC engineers because our EMC box will be our clustered storage and analyze everything else. Then we will get a CD with a customized copy of Windows 2000 Datacenter. Like EMC, the servers will be monitored by another company and they will most likely know of any problems before us. Every so often we will get a new CD with updates, service packs, etc customized for us. But if a new worm comes out in a few months that exploits some currently unknown flaw in Win2000 or any other part of the OS, will we be dead in the water while we wait for a patch? After September 11th we were calling EMC for tech support on our Symetrix and we were basically told get in line. They had richer customers to support first.
The issues mentioned in this article are null & void, as a situation like that would most likely never, ever happen. (Then again, you picked Compaq as your OEM, so maybe...*insert rim shot here*)