Slashdot Mirror
The root of all eBay's troubles
UncleRoger writes "A friend pointed me to this article would would appear to explain why eBay has had such troubles with downtime, including the outage since Wednesday evening. " It would appear that MS is tired of having the finger pointed at them - as they point out, it's an Oracle database that's running on Solaris that's causing the troubles.
According to a lot of people, there's no difference.. when Windows dies, the PC is broken. And I have to step in and correct them.. no, it's not broken. It's running Windows. As far as they can understand, that damned wavy-window flag at startup must be burned right into the hardware..
I wonder how much of the blame can actually be placed on the Solaris and not on human error. The eBay announcements board says they've identified the problem, but doesn't detail what the error was. Does anyone have more detailed information?
If eBay uses a single Enterprise 10000 server for the back-end database, they should have had a standby server that they could have switched to in seconds. eBay could also have distributed database operations further.
One thing is clear: NT has no advantage in this area. Sun gives you the option of lots of little servers or one big server with a backup, and depending on the application, one sometimes has to make the latter choice. With NT, however, you are forced to go the lots-of-little-server approach.
On a side note, on the day on which Microsoft's poor security architecture in MS Office has been responsible for shutting down lots of corporate sites (including their own) and caused thousands of users to lose their data, their whining seems very ironic. eBay's problems are eBay's fault; the virus problems are Microsoft's fault.
I nternetWorld Article here.
There's an article from a couple of months ago over at InternetWorld that profiled the EBay server setup and its *two* Enterprise 10000's(Starfires).
Read it and you'll understand just how complex a setup EBay has. One of them performs the searching for the site.. "We had search vendors come in and tell us they had a great product, and we'd point a little of our load at it and it would melt into a puddle of metal on the floor."
And of course the "wills" sprinkled around, prefixed with Windows 2000.. Oh, man, this is just boooooring.
"Ten years from now, they could do it in a few seconds." -- The Racketeer of the Hellfire Club, 1993, Phrack 42
That's because Microsoft's definition of "force" is generally, "We'll give it to you free. We think you should use it." It's how MS has gotten a serious stranglehold on education (where my PHBs and colleagues scoff at me/retreat fearfully from me whenever I mention Linux). While I can certainly understand the discomfort some people have when considering alternatives to MS, both FUD and the lack of cost make it very easy for managers to disregard any advantages to other platforms.
NeXT's WebObjects e-commerce offering to Dell was apparently designed and implemented in less than a week by one or two (granted quite gifted) developers. Microsoft's replacement, however, required a large team working for several months.
"Just fine" may suit your needs now, but you also had a system which likely suited your needs "just fine" which didn't require significant time and energy expenditure for replacement.
When was the last time you had to reboot the SQL Server for problems related to the operating system? When was the last time you had to restart the SQL Server Service? When was the last time you had to restart the server simply because you installed new software on the machine?
"My God...It's full of ads!" -Fry, about the Internet, Futurama
Some of this is just absurd. For example, the six points of failure with the Starfire. Hrmm:
"Applications running in Domains are only as reliable as the instance of the Solaris operating system. For applications to gain enhanced reliability from Domains, users must explicitly set up clustering, just as in standalone systems. Sun does not recommend clustering between Domains, suggesting instead that fail-over occur to either separate, standalone systems or Domains in other Enterprise 10000 systems."
Uhh, duh, isn't that the whole idea? Am I missing something here?
"Daemons that control domain operations and perform monitoring functions run on an unreliable device (Ultra 5 workstation), hardly a desirable situation in the context of a data center."
Excuse me? The Ultra 5 an "unreliable device"?? We have a farm of Ultra 5s that have been running for a year now. Total number of system failures or crashes of any kind: 0. Period. How is the Ultra 5 any less reliable than any other workstation-class system?
"When security is compromised on the System Service Processor, which runs on the Ultra 5 workstation controlling domain operations and performance monitoring, all running domains on the E 10000 can be brought down with a short command sequence."
No kidding. When (or rather, *if*) security is compromised, you could do a whole lot more than bringing down all running domains. Just the same as any other platform. How is this a weakness specific to Solaris or the Starfire?
And besides, these are supposed to *secured* (meaning, physically) control consoles. Meaning, locked in a cabinet in the datacenter.
"System boards that are hosting non-pageable kernel data structures cannot be removed from a domain without interrupting service. The Solaris operating system has to undertake a special "quiesce," or suspend, operation while the critical pages are migrated to another board."
Ummm, yeah. So? How is this any different from any other operating system? Again, I fail to see what the problem is. And besides, how often do you change system boards? Please.
Sure, go ahead... try and remove a CPU card from any NT-based system without first warning the OS. Not only will it hang horribly (ie; you can't do it!), you'll probably fry hardware as well!
The fact that the Starfire can even do this is pretty amazing.
"System boards that are hosting Token Ring adapters, ATM adapters, or non-Sun disk controllers cannot be present in a domain if board-remove operations involving kernel quiescence are to be performed on that domain."
Uh-huh. Sure. I know lots of people with Starfires running Token Ring off of non-Sun hardware that are removing boards with non-pageable data. Happens every day.
I'm not saying it doesn't happen per se, I just think that these arguments are rather ridiculous.
"If you remove a system board from a running domain without enough swap space, Solaris will hang. The administrative tools do not warn you if you do not have enough swap space available."
What kind of idiot doesn't leave enough swap space? What kind of admin would go ripping out system boards without really thinking it through first? What kind of person spends the incredible amount of money the E10000s cost without being informed as to the basics of running a Solaris-based system? Come on.
It's like saying "If you remove a CPU card from an NT-based system while running domains are active, the system will be brought down and all domains brought offline." Ummm, duh. If you remove your legs, you can't walk either. Apparently, M$ thinks that true Unix sysadmins are as stupid and lacking common sense like the server admins that they're used to dealing with.
"Reliable hardware is getting even more reliable. For example, customers can take advantage of 99.9% system-level uptime guarantees for Windows NT-based servers from major systems vendors, such as Compaq, Hewlett-Packard, IBM, and Data General."
These are guarantees on the hardware, not software. I'm sure this looks great for the PR, but hello? I'd love to know what the "major system vendors" think about Windows-based servers being equated with their hardware guarantees.
"Microsoft Windows® 2000 Server builds on these gains. For example, Windows 2000 Server supports COM+ load balancing, which eases customer development of highly available and scalable applications in a multi-tiered environment. On the back-end, Windows 2000 Advanced Server supports two-node fail-over clustering, whereas Windows 2000 Datacenter Server will support four-node clustering. IBM and other vendors will provide support for up to eight nodes."
WOW! I am truly impressed. Two or four-node fail-over. Please.
Finally, at the end:
"Which brings us back to eBay. For those keeping score, eBay relies on Windows NT-based servers running Internet Information Server to provide front-end web services, and a single Enterprise 10000 from Sun Microsystems to host an Oracle database on the back-end. According to published reports, the outages at eBay, which began in February, are due to problems at the back-end."
This is curious. Maybe I'm missing something, but a telnet to port 80 shows that www.ebay.com is using Apache 1.3.6 on Solaris. It doesn't get any more front-end than that, does it?
I did notice that pages.ebay.com and listings.ebay.com are running IIS 3.0, and cgi.ebay.com is running IIS 4.0.
Also notice that their web site is still up and running. Not that that means a whole lot, but hey.
I find a lot of what this article had to say utterly hilarious. The implications that the Starfire is an unreliable and dangerous system is the greatest work of FUD that I've seen in my life.
OK, enough said.
--globalnap.net, product of pure caffeine--
Okay, this is completely off topic, but it's too funny...
-
Yeah! I can see MS's marketing department gearing up right now....
90% of Yugo mechanics recommend MSActiveYugoYoke as the best Yugo yoking solution on the market*.
Studies have shown that the front passenger in the central Yugo in an MSActiveYugoYoke system sustains fewer neck injuries** than in competitors' systems.
Mechanics can take the MSActiveYugoSolutions Certification Test to demonstrate their compliance with a number of stringent guidelines set by Microsoft Customer Relations.
With an MSActiveYugoYoke system, you can go where you want to go today!
With MSActiveYugoYoke Enterprise Customer Satisfaction Enhancement Warranty, you can assure MSActiveYugoYoke functionality decades into the future!
Microsoft is firmly committed to enhancing MSActiveYugoYoke ease of use, particularly in high-speed interfaces with LargeSunTrucks.
Microsoft prides itself on its high degree of MSActiveYugoYoke-VehicleJava compatibility***.
MSActiveYugoYoke-the product line awarded the Gold Consumer Choice Vehicular Safety Award****.
Quotes from satisfied MSActiveYugoYoke:
"Well, it's a big improvement over MSActiveYugoYoke 95, I must say that." -- Dan Smith, Yugo mechanic
"Microsoft's Yugo yoking system is my system of choice for yoking Yugos." -- Steve Jobs, interim Apple CEO
"MSActiveYugoYoke is the supreme Yugo yoking solution." -- Jim Bob, a guy at Microsoft
"I guess if you like Yugos and use other Microsoft products, you might as well go with MSActiveYugoYoke to ensure product interoperability." -- an anonymous guy at C|net.
-----------------------------------------------
* Survey conducted by Mindcraft. Error margin +- 20 percentage points. [Note: only Yugo yoking system on market at the time was MSActiveYugoYoke]
** While traveling between 8.5 and 9.2 MPH on toll bridges during hurricaine conditions. Survey conducted by Mindcraft.
*** Certain procedures for unyoking in emergency conditions bear vague relation to VehicleJava emergency unyoking procedures in other vehicle yoking systems.
**** Award won was January 1996 award, won with MSActiveYugoYoke 95. Current product is the similar MSActiveYugoYoke 99.
If what I heard was correct, the part that croaked on Hotmail was Exchange, not IIS.
(Which shouldn't be a suprise - Exchange only started supporting > 16 GB in it's database last year or so. For those not used to dealing with corporate mail enviornments, 16 GB is not very much.)
--
Business. Numbers. Money. People. Computer World.
"The percentage of users running Windows NT Workstation 4.0 whose PCs stopped working more than once a month was less than half that of Windows 95 users."
More here
Yeah, yeah, it's workstation and not server, totally different operating systems. Not.
Just a couple of choice pieces of FUD from the M$ web page.
--------quote on----------
3.When security is compromised on the System Service Processor, which runs on the Ultra 5 workstation controlling domain operations and performance monitoring, all running domains on the E 10000 can be brought down with a short command sequence.
--------quote off----------
So, let me get this straight. The workstation which is responsible for "controlling" operations can be used to stop operations? What was Sun thinking, including a command that would turn things off! Of course, we all know that one of the main features of NT Server over NT Workstation is that the "Shutdown" command has been removed from the "Start" menu.
--------quote on------------
4.System boards that are hosting non-pageable kernel data structures cannot be removed from a domain without interrupting service. The Solaris operating system has to undertake a special "quiesce," or suspend, operation while the critical pages are migrated to another board.
--------quote off------------
This is supposed to be a problem? Now, I think it's pretty neat that you can migrate kernel memory off of a certain piece of hardware and swap it out at all. We're supposed to believe that under NT you can do this at all? Much less without telling the kernel first? The only conceivable way to allow this to happen without telling the kernel to clear the board out first would be to make sure that all the kernel memory has had a copy paged out to disk. Or perhaps keep multiple copies of all kernel data structures (and hope they don't get out of sync.) Maybe NT does do the last one. That would certainly explain why it's a memory hog.
Pretty amazing if you ask me. This web page is clearly meant for the PHBs of the world, as anybody with any knowledge at all of how computers work is simply going to laugh at this.
Okay, let's try this then:
24*365.25*10000*.001=87660 lost due to downtime.
Let's look at a Sun system, at 99.97% uptime:
24*365.25*10000*.0003=26298 lost due to downtime
So, that's a $60000 savings in one year.
My understanding was that Microsoft paid Dell for the conversion costs. (And that the WebObjects setup was breaking under the load, but who knows if that was hardware of software.)
--
Business. Numbers. Money. People. Computer World.
...two. At least according to an InternetWorld article a month or so ago.
Advice: on VPS providers
The E10000 is a massively parallel computer that has several "system" boards with its own CPU(s), memory and disk controllers. It's somewhat akin to crunching somewhere between 10-64 UltraSparcs into the same chassis.
:-)
Using software tools, it is possible to segment all of the system boards so that they behave as if they were individual physically discrete systems.
And yeah, I'd like to see Microsoft pull off the same thing.
Is it that hard to see? They changed their page layout and tweeked the software, and now they are getting data corruption. It doesn't have anything to do with Solaris or Oracle. If you design a database that can't correctly handle concurrency, doesn't have good constraints, no triggers, etc. ad nauseum, then you are going to get data corruption. Also, the whole 'high availability' arguement is laughable. eBay's buggy software is still has high availability. ;) Just because their software crashes doesn't mean Oracle and Solaris are crashing.
It's just more FUD from the Empire.
--- A Jesus Fish eating a Darwin Fish only proves Darwin's point.
Microsoft has it wrong on the E10k. It sounds like they've been talking to people here and there and haven't actually played with the hardware. The major SPOF is not the SSP workstation, it is the control board. If the control board dies, all your domains will go down. The control board is what, among other things, gives the clock to the entire system. But most E10ks are equipped with two system boards so that you can swap and get up-and-running again quick.
If the SSP (Ultra 5) dies... well, wait. It really doesn't happen. Something like a hard drive crash might do the trick. When you are without and SSP, the domains (virtual hardware systems) on the E10k continue to operate. But you're not going to catch things like record stop dumps (hardware error and warnings... such as persistant ECC memory errors). However, most sites that have purchased E10ks have also purchased two SSPs. They're so cheap in comparison, it makes sense. We have YET to fail over onto the secondary SSP on any of our 10 E10Ks. Since when is an Ultra 5 an "unreliable device"?
Sun complaining that the OS needs to be temporarily quiesced in order to move the kernel from one bank of ram to another? Heck, it's a miracle that it can even happen at all. I'd like to see microsoft write the code to move the kernel on the fly. Not a project I would want to be on.
Poo-poo on the adaptors that don't do DR? Hardly even an issue. Look at them... token ring, ATM, third party. I wouldn't even run a third party SBUS card on my E10k. The translation is that "a minority of SBUS cards are not a good choice for the E10k." Big deal, Bill.
About the swap space issue... they might actually have an issue there. I'm sure Sun is working on a warning now, if it is a problem. BTW... at that point you haven't actually REMOVED the system board. You are doing an operation called a "DR Drain" which moves all the pages of memory from the RAM in that system board to another. Once successful, you are able to remove the system board from the configuration, or abort the change.
Let me translate Microsoft's position:
1. Sun Enterprise 10000 systems have single points of failure. You can't hot-swap CPU boards arbitrarily, and the Ultra-5 front-end is a critical component.
2. Sun recommends that for high availability you cluster between multiple 10000 systems. This is bad.
3. Microsoft's commodity hardware platforms do not offer any of the scalability or reliability features of the Enterprise 10000, so clustering is the only option. This is good.
4. Microsoft's current clustering offering is primitive. In a survey, a majority of people said it was adequate.
5. Microsoft promises that Windows 2000 will have better clustering than NT.
6. eBay is not following Sun's recomendations that high-availability requires multiple systems. They have experienced outages.
BTW, it is shocking to me that eBay could have only a single server. This is at best incredibly naive; at worst blatant incompetence. Therefore I suspect it is false.
When people can't design a reliable system with budgets that allow the purchase of Sun 10000's!
I run a Sun 10000 with two SSP's. 10000's are connected to their SSP's via private ethernets. I have three private networks; two to allow redundant interconnects between the SSP's and the two 10K control boards and a third for general use, NFS mounting CDROM's and the like. Most people will have no reason to put the SSP's on a public network at all -- I certainly don't. In order to hack the SSP, one must first hack the 10000. Once they've done that, the ability to reach the SSP's by network is irrelevant. The point about the "problem" of the SSP having control is as silly as claiming that EMC Symmetrix disk arrays (heavily used in IBM mainframe shops) can be crashed by the single laptop each array contains.
I would love to know the details of their failures -- I suspect the article is hinting at issues that have nothing to do with their real problems. Further, I'd bet that the main vulnerability that people cluster Sun's against is hardware failure -- and I'd also bet that the main reason people cluster NT boxes is software unreliability!
Geeky modern art T-shirts
It's funny that MS holds up Dell as an example of a reliable, scalable NT-based site. At least their WebBoard support area is frequently inaccessible, and always incredibly slow.
MS also touts 99.9% uptime guarantees from Compaq, etc., but fails to mention that Sun claims 99.95% for the Enterprise 10000.
Nonetheless, my intuition (totally unsupported by any concrete info, other than their poor response to the eBayla exploit) is that eBay is a mickey mouse operation that got really lucky and rich, but does not have the technical expertise commensurate with a multi-billion dollar company. I wouldn't blame any of their vendors, MS or otherwise, for their troubles.
Much as I'd like to read the linked-to article, Microsoft's web site seems incapable of serving up a version of it that I can read in Netscape 3.01.
I get one window of text (along with the usual decorations) which is empty if I scroll down, and has vanished if I scroll back up. Fascinating. "View source" shows more JavaScript than actual document text...
No doubt it works just peachy in Internet Exploiter. But MS misses the first point of communication, which is to convey the message.
No wonder MS is losing.
-- Alastair
Hardware: I think that Sun really made a mistake here. I'm not too unhappy that they threw out SBus and went to PCI, that really does strike me as a good idea, but dropping onboard SCSI in favour of onboard IDE, well, that was just plain stupid. As it is, every time we buy and Ultra 5 we have to burn a slot to get SCSI into the thing. I notice performance problems with my IDE disks on my workstation, I'd hate to imagine them in any kind of server. Likewise, they seem to have redesigned the case with inconvienience in mind. You have to eviscerate the damn thing every time you want to change anything (memory being the worst) and all the little bits and pieces seem to be fairly low quality.
In the end, the only reason that I upgraded from an Ultra 1 was the frambuffer. 24 (or 32) bit graphics are nice, especially when compared to the measley 8 bits I had before. I don't really have any one application that takes advantage of the extra colors, but color map conflicts (and thus epileptic flickering as maps are switched) are a thing of the past.
And, of course, there's the Mystery Bay. On the front of Ultra 5s is a little flip door that looks just about the right size to admit a 4mm tape. Of course, it isn't the right size, and no tape drive would fit inside anyway. When asked Sun said (after _much_ internal research and many days of not calling me back) that it was for a PCMCIA card reader. Great! I said, and where can I find this reader? "Well," they said, "we don't know. I actually don't think there is one. But when there is, you'll have a bay for it." -- wonderful.
OS: Solaris 7 is the standard these days, ships preinstalled on the Ultra 5. I ran Solaris 7 for two whole weeks on my Ultra 5 before purging it from the disks in a fit of retribution. To say that it's slow is an understatement. My TI-85 can serve web pages faster! I don't know if Solaris 7 is just broken (note the short time between Solaris 2.6 and Solaris 7 releases) or if it's only broken when it runs on Sun's new hardware. In any case, I dropped back to Solaris 2.6 and am much happier.
On an Ultra 1 Solaris 2.6 shows significant speed increases over 2.5.1, but all of these speed increases seems to have been effectively countered by the hardware in the Ultra 5. The end result: my Ultra 5 running Solaris 2.6 is now just about as fast as my old Ultra 1 running 2.5.1.
Marketing: Given all of these experiences I decided to go check out Sun's site to see what they had to say about Ultra 5s and new Solaris versions. I was somewhat amazed to find that they seemed to be marketing the thing as a desktop machine, trying (or so it seems) to compete with PC manufacturers. Now I'll admit that I like having a Sparc on my desktop, but a PC it ain't! The complete lack of emphasis on marketing the machines as servers was simply amazing. And this pattern seemed to be repeated for the other new Ultra machines.
It's really not clear to me what the heck Sun is up to, but I think that they have some serious thinking to do about their direction in the market. Presumably there's a reason to be (seemingly) ignoring their strengths, but I sure don't know what it is.
--
Behold the Power of Cheese!
According to netcraft (http://www.netcraft.com/cgi-bin/Survey/whats), they are running on MS-IIS/3.0.
------------------------------------------
www.ebay.com is running Microsoft-IIS/3.0
Microsoft-IIS is also being used by Walt Disney, Compaq, Nasdaq, and The National Football League.
www.ebay.com is hosted by ebay.com.
------------------------------------------
Mario.
A single Starfire is rated as being able to deliver 99.95% availability with one - ie no clusters, and without all those caveats above - though it does need to be setup with reliability in mind for this - there's plenty of options.... Starfires aren't simple either - up to 64 CPUs, many more PCI and similar slots, memory slots, etc, etc. So, plenty of things to go wrong. Similar sized computers (from everyone) are really hard to transport without something going wrong. The only people more nutso on reliability on 'big iron' computers are IBM (from the companies I know a fair bit about anyway). Not only do they have backup CPUs, in their CPUs they do the same operation twice (in parallel, with checking at the end) to trap the ultra-freak chance of cosmic radiation or something casuing a flipped bit, or worse. (yes, they do seriously actually worry about such things... I remember an IBM proposal about how to design memory that can handle a once-a-month chance, for when you have a huge about of RAM, for some particular kind of radiation....)
The only complaint I've ever heard about Starfires in general is that if a PCI card (though not SBus card) breaks down it can hang the entire system until an operator manually flicks a switch to say that that particular card is defunct. Though this is really because of how PCI works - Sun's 99.999% reliable Netra 1800 has some highly specialised custom hardware to get around this problem with PCI cards, as well as backup CPUs and plenty of other stuff... ridiculously expensive too, they are... though apparantly more cost effective than anything in the same class. The Netra 1800s are a few months old, while the Starfire design is over 2 years old, btw.
I dunno about all of MS's claims, but I'm pretty damn sure that you can have hardware redundancy for just about everything, if you want, including the Ultra-5 controller. Most of the other claims seem to be related to the fact that you can hot-swap PCI cards, memory, CPUs and even mother-boards in a Starfire...
EBay do seem to have had more than their fair share of problems though... quite a few hardware problems it seems - I vaguely remember a problem earlier in the year was due to some controller card or something. As far as I know, nobody has had anything close to the problems EBay are having with their Starfire(s)...
Another little point... MS's idea of expensive downtime is $10,000/hour. I remember reading something on Sun's site a while back about high end availability systems. Sun's idea of expensive downtime is $10,000,000/hour - ie stockbrokers. They also had a list of most common causes for 'unplanned' downtime on their HA systems - first was 'operator error' (or lack of training, etc). I can't remember was second was, but third was 'fire'! (I'm pretty sure Sun's computers don't have a reputation for spontaneous combustion!)
Yeah. A while back I did some work for a large cable company -- a different sort of e-business, but still an e-business. Their system (customer info, billing, and digital settop box authorization) was based on Sun Enterprise servers, and the whole system was replicated in both Denver and Dallas via a T-3. You could nuke one data center and the overall system would stay up.
-- Alastair
The problem with your assertion is that Microsoft is using eBay's problems, which may or may not have anything to do with the reliability of Sun hardware or software (more likely it's human error) to further their outrageous FUD campaign.
It has nothing to do with a Microsoft system crashing, but rather trying to turn the eBay problem in into a FUD event that has people here upset.
This web page is the most outrageous piece of crap I have ever seen. Advising customers to rely of some piece of untested software still in beta to handle a massive mission-critical load. If I was eBay's CTO I would be looking to go upscale into some really heavy iron like the Himalayas or mainframes that operations that need REAL reliability use. The idea of going to a MS operating system for this sort of application is purely ludicrous.
The same day this is going on we have another round of word macro viruses terrorizing MS users everywhere. Why don't you see Corel and Lotus touting the fact that Word Macro virii don't trash their systems? Because they aren't low-life like Microsoft. Do you see slashdot trashing MS over this? No, even though they richly deserve it.
Microsoft deserves to be roundly excoriated on this one.
The one, let's call it Site A, uses a $20,000 Dell NT Server 4.0 SP3 (dual PII-300) with 50 win95 clients; also runs MS Proxy Server 2.0.
Site B uses a Sun Ultra 2 Model 2300, dual sparc 300mhz. It supports tin, pine, lynx, gcc, filesharing, and ftp with 300 concurrent users across a 2 mile radius WAN with .2 cpu usage.
Is it fair to note that the last "restart" of the Sun was 67 days ago--the last "restart" of the NT...well, with all seriousness, it's been at an average 2 crashes per day (that's an 8 hour day).
eBay may have DB problems; let's not forget Oracle has all of its products availible for Linux and Oracle products are sold OEM thru Dell and their Online Store. I'm afraid Microsoft is trying to bolster their image. Don't believe NT needs PR help? See http://www.ntsecurity.net
Adios.
It should be noted that Oracle is also used by these top sites.... and they don't have problems....
www.amazon.com
www.1800flowers.com
www.cdnow.com
www.charlesschwab.com
www.cisco.com
www.dell.com
www.etrade.com
www.onsale.com
www.rei.com
I fear that it is the ebay ppl who are at fault....
9 times out of 10 eBay is hosed due to the ASP/ODBC/IIS front end. Today it's the database backend (Sun + Oracle).
--
Business. Numbers. Money. People. Computer World.
Sometimes you have to wonder how things ever get approved to be on their website. Let's look at a few of the more imflammatory claims, which is really quite a kettle o' fish:
RED HERRING: Daemons that control domain operations and perform monitoring functions run on an unreliable device (Ultra 5 workstation), hardly a desirable situation in the context of a data center.
So what? The E10000 will continue to truck on as before without it. This is a complete red herring. The SSP is a really just a console station, nothing more. If it dies, you reboot it, or in the worst case, replace it with another one from the closet, which with Sun's AutoClient technology, can take on the entire identity of the failed box in a couple of minutes. (AutoClient allows Wall St. traders to replace their workstations and be working again with NO IMPACT in 5 minutes. Let's see NT do that.)
FUD SHARK: When security is compromised on the System Service Processor, which runs on the Ultra 5 workstation controlling domain operations and performance monitoring, all running domains on the E 10000 can be brought down with a short command sequence.
No one in their right mind would put the SSP on a network that extends beyond the glass house!! It's a *console*, designed to be locked up securely, like all other mission-critical control consoles. MS still doesn't get the data center, do they?
DUH WHALE: System boards that are hosting non-pageable kernel data structures cannot be removed from a domain without interrupting service. The Solaris operating system has to undertake a special "quiesce," or suspend, operation while the critical pages are migrated to another board.
This is incredible. They're knocking the E10K because you can't walk up to it and pull a CPU card at random without telling the machine first that you plan to do this. These cards contain memory, too, folks, which is why it's pretty reasonable to let the system move things to a safe place before the card goes bye-bye. Pretty much only Tandems can accept this sort of things (because they've got at least two of everything all the time, and they cost like they have even more), and if you're after real fault tolerance, you won't be running NT on them, even though you could...
STING RAID: If you remove a system board from a running domain without enough swap space, Solaris will hang. The administrative tools do not warn you if you do not have enough swap space available.
This is pretty low. Yeah, it can happen - what else is an OS supposed to do when it has more processes than now remains as memory? Although a warning would be nice, E10K admins aren't stupid (we hope), and they understand that there are easy workarounds to this - the E10K makes it very easy to move enough resources into the OS domain in quesiton on a temporary basis. If you don't have enough hardware to do that, you misconfigured the machine in the first place. This is hardly a weakness.
On the whole, the incredible thing about this is that MS is throwing rocks at a really good system with availability features far in excess of that for any practical NT box. You've gotta admire their guts, though - some people will read this and think the E10K is a really expensive, dangerous computer. Funny how they neglect to mention that there's not an NT box on the planet that can provide the performance of an E10K, regardless of how much you spend. This may change eventually, but it's pretty cheeky now.
If you need real fault-tolerance, get a Tandem/Compaq - but after you've paid all that money, I bet the Compaq folks would be the first to advise against using NT on it if you really want fault tolerance.
"The future's good and the present is nothing to sneeze at." - Roblimo's last
Well, eBay is constantly slow or being interrupted due to IIS or MS-ODBC flakeyness. When I heard eBay was down for 19 hours (on the radio), I assumed it was the Microsoft side. If I was them, I'd have a press release washing their hands too.
By the way - has anyone tried to buy anything at buy.com? I have on a couple occassions, and the damn thing is so flakey and defective that it won't let me. It also appears to be all MS Tech.
--
Business. Numbers. Money. People. Computer World.
I don't know about EBay, but I know that E-10000's are extremely tricky to configure correctly.
Sun markets them as ultra-reliable and hardware-level redundant, but the truth is that configuring them is so complicated that even a team of experienced sysadmins is bound to screw something up sooner or later. If you bet the store on a single E-10000, then sooner or later the machine will crash hard and your store is hosed.
Given their size, expense and complexity, they are not appropriate for use as the main server in an internet commerce company. Sun should not sell these machines to companies like EBay.
In defense of Sun, I should point out that their "smaller" systems, namely the E-4000, E-6000, E-3000 etc., are rock solid and just as easy to configure as a small server. But no one would dream of running a whole store on a single one of them -- for reliability, you need to run several of them redundantly.
And Windows NT is far less reliable than any Sun machine. NT is the opposite of reliability. Production Solaris machines routinely stay up and running for months or years at a time. Show me an NT server which can do that.
You know what, I'm TOTALLY sick of seeing /. commenters correct the lies that Micro$oft proports on their pages. You need not look at 5 comments to this article alone to get the idea. Instead, let's do something CONSTRUCTIVE with all this criticism! :)
/. did this, they'd get hundreds upon hundreds of emails.
I'm declaring tomorrow Constructive M$ Bashing Day!
(Why do I have the power to do this? Because Barney says everyone's special in his or her own special way, and I'm invoking my privledge as a Special Person. That'll teach you to ask why. Feh!)
The next time you feel like correcting something that M$ claims and that's blatantly false, do so. THEN, email it directly to M$! If we had one day where EVERYONE from
Then, everything would start changing. The wheels would be in motion. M$ would realize the error of their ways and become Tibetian monks to pray for forgiveness!
We can make a difference, dammit! Can't we??
-
I refuse to let 95 or 98 in my office. Not even a CD-ROM. NT 3.51 is a bit more stable than 4.0 in my experience, but doesn't handle big hard drives, so I moved my data crunching system up to 4.0 and it has basically been fine.
What really sucks is having to reboot whenever you install a new app with one of Micro~1's crappy shared DLLs. The latest offender was Foxpro 6, which I only need for some light file management, and which had me reboot three times (twice for the mandatory install of evilbad IE which I never use). Rebooting for network changes is bad enough, but for mere mortal applications??
This reminds me yet again why
Bill Gates Is My Evil Twin.