Slashdot Mirror
The root of all eBay's troubles
UncleRoger writes "A friend pointed me to this article would would appear to explain why eBay has had such troubles with downtime, including the outage since Wednesday evening. " It would appear that MS is tired of having the finger pointed at them - as they point out, it's an Oracle database that's running on Solaris that's causing the troubles.
You may suspect that it is false, but it seems to be ebay's story and they are sticking to it.
From the open letter from the founders now on Ebay's home page
To help ensure this, we are working diligently on a hot backup system that should automatically limit the length of potential outages to less than an hour or so. We have been working on this system for many months, and it is almost ready. Sadly, if we had this system in place a few days ago, we might have avoided this outage.
More interesting is that when I view Page Info with my Netscape browser, I get a June 9th date.
If you check Hotmail.com (now a product of M$) you will notice it is running Apache on a Unix box.
(Server: Apache/1.3.6 (Unix) mod_ssl/2.2.8 SSLeay/0.9.0b)
I guess even when M$ considers "The Importance of Reliability in an e-Commerce World" they choose _NOT_ to use NT, like everyone else.
How do they expect other people to use their products when they don't?
I don't get it.
My studio - www.graylands.ca
No, as you noted, clearly 356.25 was the design goal.
---------
Bill Gates Is My Evil Twin.
My fault,.. I was reading another webpage while writing that,. hehh heh
My studio - www.graylands.ca
According to a lot of people, there's no difference.. when Windows dies, the PC is broken. And I have to step in and correct them.. no, it's not broken. It's running Windows. As far as they can understand, that damned wavy-window flag at startup must be burned right into the hardware..
Posted by d106ene5:
If your website is not high capacity, you should be okay. By high capacity I mean over 10 million hits a day. Hence Slashdot is not high capacity. If Slashdot got 10 million hits a day you would watch Rob's little world melt down into a puddle of pee - MySQL and mod_perl would meltdown big time.
If you are exceeding 10 million hits a day, avoid hitting a database. You're going to slam the machine in no time. Go for static pages or Apache SSI unless you truly need complex pages built out of a DB. Even then you can usually fudge it with static pages.
Your DBA who wants everything in Oracle should ask himself how much he wants Larry Ellison to control his destiny. Most old timers I know would like to minimize their debt to Oracle.
Posted by d106ene5:
Sorry, those sites are small potatoes. You could run them with an abacus and smoke signals.
I'm getting more than a little annoyed at the slander being casually thrown ebay's way. I know Mike Wilson, their VP of engineering. He is a solid guy and really knows his stuff. Anyone in the database business knows that things go wrong. ebay scaled up from nothing to huge in record time. Is that for nought because they have had problems recently? They moved the bar up a whole order of magnitude on what was considered possible for realtime e-commerce on the net.
So lay the fuck off Mike and his engineering operation until you understand the actual details of what was happening there. People seem to believe that "state of the art" comes about through random acts of kindness or something. No. It comes from learning from mistakes and accidents, and the bigger the flaw, the bigger the step forward that those on the leading edge make to put those things behind.
I agree that Microsoft's leap to take advantage of this in marketing and PR was uncalled for. On the other hand, McNealy's crew is famous for that kind of stuff too.
But the spinmeisters at Sun have nothing on Redmond, and so I still have to say
Bill Gates Is My Evil Twin.
Is it just me, or does MS tend to try and justify its problems by pointing out flaws in other systems? Does it ever occur to them that when problems are pointed out, it's generally a good idea to take the oppurtunity to fix them, rathar than saying "but everyone else is doing it!"?
--
"Insert witty quote here."
After a quick pass through an optimizing compiler:
if (nt == unstable) { switchTo.linux() }
becomes
switchTo.linux();
Slashdot runs on Linux with mysql as the database, and it crashes too often. For a while mysql was crashing. Then the Linux kernel was crashing. All in all it's a big pile of unstable mess.
10 PRINT CHR$(205.5+RND(1)); : GOTO 10
I wouldn't mind one of those "unreliable" Ultra Sparc 5's :)
Of course the thing can crash its a computer.. Live w/ it Microsoft..
--Mark
I'm an oracle dba and I'd have to say that ebay's problems practically *have* to be human fsck-ups! I've never had big problems recovering oracle and even fewer problems keeping it going. What's this crap about failing hard disks? Where's the mirroring? I've had 3 disk mirror sets when it was very important to have 99.999% up time. One of my clients has oracle running on a 2 process ultra (unclustered) with NO disk mirroring and their system and database have been up for over 10 months now.
based on Microsoft's guaranteed 99.9% uptime, and $10,000 per hour downtime average cost, 24*7*365.25*.001*10,000=598,500 dollars per year in downtime costs.
- None can love freedom heartily, but good men; the rest love not freedom, but license. -- John Milton
I wonder how much of the blame can actually be placed on the Solaris and not on human error. The eBay announcements board says they've identified the problem, but doesn't detail what the error was. Does anyone have more detailed information?
if (nt == unstable) { switchTo.linux() }
becomes
switchTo.linux();
that's after a -O. after a -O2 you get FreeBSD using SMP
-
ping -f 255.255.255.255 # if only
But I understand that MS is moving WebTV into the same building, so they may move to Magnavox and and Curtis Mathis any day now...
A few days ago, they roll out their new website design, featuring more graphics and more dynamic content. What happens? They crash and burn during their high-load times...
I think they're just overloading their servers... Again.
If eBay uses a single Enterprise 10000 server for the back-end database, they should have had a standby server that they could have switched to in seconds. eBay could also have distributed database operations further.
One thing is clear: NT has no advantage in this area. Sun gives you the option of lots of little servers or one big server with a backup, and depending on the application, one sometimes has to make the latter choice. With NT, however, you are forced to go the lots-of-little-server approach.
On a side note, on the day on which Microsoft's poor security architecture in MS Office has been responsible for shutting down lots of corporate sites (including their own) and caused thousands of users to lose their data, their whining seems very ironic. eBay's problems are eBay's fault; the virus problems are Microsoft's fault.
I nternetWorld Article here.
There's an article from a couple of months ago over at InternetWorld that profiled the EBay server setup and its *two* Enterprise 10000's(Starfires).
Read it and you'll understand just how complex a setup EBay has. One of them performs the searching for the site.. "We had search vendors come in and tell us they had a great product, and we'd point a little of our load at it and it would melt into a puddle of metal on the floor."
Man, the balls on those guys at MS get bigger every day.
Well, I noticed they started crashing (repeatedly)
right after they changed the lay-out of their
pages. Maybe they should take a look at how
Slashdot orgainzes itself (or re-organizes itself)....
Anyhow I am mildly annoyed, I'm in the middle of
an auction. Rather Reminds me of the UO beta...
What do you mean Penguins don't fly, they fly perfectly well. Under water that is.
Sun is one of the best super duper high end dynamic made for the web serving thousands of happy users a day operating systems ever made. cdrom.com does jack to prove FreeBSD reliable for big tasks, it's a bunch of static FTP requests and sends, big whoop. If you want to point to FreeBSD reliability (I like FreeBSD dont get me wrong) then point out the special effects for Matrix were done with a big prallel (29 nodes I think) system running on FreeBSD.
I'm a loner Dottie, a Rebel.
That's because Microsoft's definition of "force" is generally, "We'll give it to you free. We think you should use it." It's how MS has gotten a serious stranglehold on education (where my PHBs and colleagues scoff at me/retreat fearfully from me whenever I mention Linux). While I can certainly understand the discomfort some people have when considering alternatives to MS, both FUD and the lack of cost make it very easy for managers to disregard any advantages to other platforms.
NeXT's WebObjects e-commerce offering to Dell was apparently designed and implemented in less than a week by one or two (granted quite gifted) developers. Microsoft's replacement, however, required a large team working for several months.
"Just fine" may suit your needs now, but you also had a system which likely suited your needs "just fine" which didn't require significant time and energy expenditure for replacement.
When was the last time you had to reboot the SQL Server for problems related to the operating system? When was the last time you had to restart the SQL Server Service? When was the last time you had to restart the server simply because you installed new software on the machine?
"My God...It's full of ads!" -Fry, about the Internet, Futurama
I have found Access to be a pretty good solution for what we do. We run the standard MS desktop app's, and Word, Excell and Access are company standards, so we can get some of our clients to do a lot of the inputing. . . it is compatible, and for lightly loaded, small db's it is really a good solution. The interface _is_ really nice. Not perfect, but for development, it allows for a quick turn-around and gives us a way to talk with our clients about the design. But for a "production quality enterprise app" I wouldn't fool myself for a second that it is suitable. Not even close. The ODBC drivers leak memory. It crashes far too easliy under any kind of real load. But for development, where you are going to have a few iterations of a design, it is a good lightweight solution.
My experience has shown Oracle to be really fast, really solid and _really_ expensive. As long as you are willing to put a lot of time into the set-up and initial tuning, run it on a really expensive machine, and put a little time into maintenance it is a rock-solid solution. But it is rather ugly if you need to make any changes to the schema or data types. If we want to make any changes, we have to go through the DBA, which is a real pain in the ass. He's a nice guy and all, but extremely overworked.
Actually, we just started using Oracle 8, Personal Edition on our desktops. That might be a good solution, too. If anyone is interested in how well it works, let me know and I'll keep you updated updated
Jeff
nrrd@earthlink.net
"Eye halve a spelling chequer, It came with my pea sea, It plainly marques four my revue, Miss steaks eye kin knot sea"
Posted by generic kewl tech reference:
Exactly. See #2.
I don't know about anyone else, but if a bad CD will bring down NT server I can't buy their line about HA servers. Maybe a bad CD would bring down Linux, too, but it's never happened to me. I can't think I've ever had a severe problem with a bad CD on any os other than MS's.
Just a thougt. . .
"Eye halve a spelling chequer, It came with my pea sea, It plainly marques four my revue, Miss steaks eye kin knot sea"
You are right about privileged access being an NT weakness due to most users being setup like this. However this has nothing to do with Worm.Explore, which simply deletes user files (documents and source code) and emails itself to other users - both possible with a non-privileged user.
The only thing that Linux would prevent is corrupting the equivalent of win.ini to launch the worm on other machines, or for other users of the same machine.
I agree witht he folks blaming the problem on poorly engineered backend on the part of eBay with no failover. Also, I'm no big fan of Solaris, nor of Microsoft systems. My personal favorite is linux, but if I needed to set up something that would be the best performance and most reliable, I would go with FreeBSD... I think ftp.cdrom.com alone should be enough to convince people of FreeBSD's perfomance/reliability. I nearly worked with a company who did all compiles on clusters of FreeBSD machines, uptime was over a year when I got there... I think more people should be singing FreeBSD's praise, but I'll still be running linux on my personal system for a while to come, maybe one day I'll install FreeBSD again..
XML is like violence. If it doesn't solve the problem, use more.
There is - it's called the "Disable Macros" button that pops up when you open a document that contains macros.
As I said before though, this most recent virus was not a macro virus.
Cool. It's good to see that MS is finally offering options of this caliber... which goes without saying that this is a new policy.. I'd love to figure out how to disable that stupid HTML replacement for a "host not found" error in IE4 (i hope this was fixed in IE5, it has to be the most annoying thing in the world).
Enough bickering though. Even though this button exists, and despite the fact I don't know crap about this "virus" (which 90% of media-labeled "viruses" are about as complex as a batch file -- a trojan), the admins at these places should not be allowing any form of automatically-executing or manually-executing attachments arriving in people's mailboxes.
The standardly traded mail format is just text - these guys should be processing this crap before it gets to the office. I'm thoroghly amazed it's not, especially in gov't offices where mission critical has just a little extra dash of critical sugar sprinkled onto it.
Caffiene is good.
-Erik-
All the recent wibble on E-Bay's site is about CGI servers and problems thereof - are these not running on En-Tee ?
Yep, they have two E10K's, plus numerous sun and PC boxes.
Question (I am putting together a web server myself): What are the specs of the machine? What server (Apache)? What dynamic content language (Perl? PHP?)
Wow, over 40% of Win95 users reported that their PC quit working at least once a month... 15% on NT Workstation.
The SPARC Solaris machine I use at work has been running for 2 1/2 months straight (and that reboot was because of a power outage). On the other hand, if our NT server doesn't BSOD at least twice a day, it's a red-letter day.
Hemm. That's a new one - I could tell when B&N Online went from basically rock solid to an SQL cough-out error every other day I browsed that something had changed for the worst.
The TechNet article is written in a dull, prejudiced way that's more than a bit obvious in its selectiveness. The other points have been pretty well attacked, so I won't touch 'em.
Some of this is just absurd. For example, the six points of failure with the Starfire. Hrmm:
"Applications running in Domains are only as reliable as the instance of the Solaris operating system. For applications to gain enhanced reliability from Domains, users must explicitly set up clustering, just as in standalone systems. Sun does not recommend clustering between Domains, suggesting instead that fail-over occur to either separate, standalone systems or Domains in other Enterprise 10000 systems."
Uhh, duh, isn't that the whole idea? Am I missing something here?
"Daemons that control domain operations and perform monitoring functions run on an unreliable device (Ultra 5 workstation), hardly a desirable situation in the context of a data center."
Excuse me? The Ultra 5 an "unreliable device"?? We have a farm of Ultra 5s that have been running for a year now. Total number of system failures or crashes of any kind: 0. Period. How is the Ultra 5 any less reliable than any other workstation-class system?
"When security is compromised on the System Service Processor, which runs on the Ultra 5 workstation controlling domain operations and performance monitoring, all running domains on the E 10000 can be brought down with a short command sequence."
No kidding. When (or rather, *if*) security is compromised, you could do a whole lot more than bringing down all running domains. Just the same as any other platform. How is this a weakness specific to Solaris or the Starfire?
And besides, these are supposed to *secured* (meaning, physically) control consoles. Meaning, locked in a cabinet in the datacenter.
"System boards that are hosting non-pageable kernel data structures cannot be removed from a domain without interrupting service. The Solaris operating system has to undertake a special "quiesce," or suspend, operation while the critical pages are migrated to another board."
Ummm, yeah. So? How is this any different from any other operating system? Again, I fail to see what the problem is. And besides, how often do you change system boards? Please.
Sure, go ahead... try and remove a CPU card from any NT-based system without first warning the OS. Not only will it hang horribly (ie; you can't do it!), you'll probably fry hardware as well!
The fact that the Starfire can even do this is pretty amazing.
"System boards that are hosting Token Ring adapters, ATM adapters, or non-Sun disk controllers cannot be present in a domain if board-remove operations involving kernel quiescence are to be performed on that domain."
Uh-huh. Sure. I know lots of people with Starfires running Token Ring off of non-Sun hardware that are removing boards with non-pageable data. Happens every day.
I'm not saying it doesn't happen per se, I just think that these arguments are rather ridiculous.
"If you remove a system board from a running domain without enough swap space, Solaris will hang. The administrative tools do not warn you if you do not have enough swap space available."
What kind of idiot doesn't leave enough swap space? What kind of admin would go ripping out system boards without really thinking it through first? What kind of person spends the incredible amount of money the E10000s cost without being informed as to the basics of running a Solaris-based system? Come on.
It's like saying "If you remove a CPU card from an NT-based system while running domains are active, the system will be brought down and all domains brought offline." Ummm, duh. If you remove your legs, you can't walk either. Apparently, M$ thinks that true Unix sysadmins are as stupid and lacking common sense like the server admins that they're used to dealing with.
"Reliable hardware is getting even more reliable. For example, customers can take advantage of 99.9% system-level uptime guarantees for Windows NT-based servers from major systems vendors, such as Compaq, Hewlett-Packard, IBM, and Data General."
These are guarantees on the hardware, not software. I'm sure this looks great for the PR, but hello? I'd love to know what the "major system vendors" think about Windows-based servers being equated with their hardware guarantees.
"Microsoft Windows® 2000 Server builds on these gains. For example, Windows 2000 Server supports COM+ load balancing, which eases customer development of highly available and scalable applications in a multi-tiered environment. On the back-end, Windows 2000 Advanced Server supports two-node fail-over clustering, whereas Windows 2000 Datacenter Server will support four-node clustering. IBM and other vendors will provide support for up to eight nodes."
WOW! I am truly impressed. Two or four-node fail-over. Please.
Finally, at the end:
"Which brings us back to eBay. For those keeping score, eBay relies on Windows NT-based servers running Internet Information Server to provide front-end web services, and a single Enterprise 10000 from Sun Microsystems to host an Oracle database on the back-end. According to published reports, the outages at eBay, which began in February, are due to problems at the back-end."
This is curious. Maybe I'm missing something, but a telnet to port 80 shows that www.ebay.com is using Apache 1.3.6 on Solaris. It doesn't get any more front-end than that, does it?
I did notice that pages.ebay.com and listings.ebay.com are running IIS 3.0, and cgi.ebay.com is running IIS 4.0.
Also notice that their web site is still up and running. Not that that means a whole lot, but hey.
I find a lot of what this article had to say utterly hilarious. The implications that the Starfire is an unreliable and dangerous system is the greatest work of FUD that I've seen in my life.
OK, enough said.
--globalnap.net, product of pure caffeine--
Just get a trained monkey to reboot your NT as soon as it goes down. A typical NT server will not go down more than once every 4 days. An NT email server fell over this often, because of a memory leak, usually NT is even more reliable than that.
:)
So let see 24 hours * 60 Minutes * 4 days
= 5760 minutes
At 99.9% uptime, this gives you over 5 minutes to reboot the system - should be OK if you have a fast trained monkey.
Lets face it, Linux cannot reach that level of reliability, unless you also hire a trained monkey to pull out the power cord at regular intervals.
We use GNU/SunOS.
What sucks is having to reboot when making the msot trivial of changes to the system, like modifying the DNS info or something.
If only "common" sense was actually that common...
Some of this is just absurd. For example, the six points of failure with the Starfire. Hrmm:
"Applications running in Domains are only as reliable as the instance of the Solaris operating system. For applications to gain enhanced reliability from Domains, users must explicitly set up clustering, just as in standalone systems. Sun does not recommend clustering between Domains, suggesting instead that fail-over occur to either separate, standalone systems or Domains in other Enterprise 10000 systems."
Uhh, duh, isn't that the whole idea? Am I missing something here?
"Daemons that control domain operations and perform monitoring functions run on an unreliable device (Ultra 5 workstation), hardly a desirable situation in the context of a data center."
Excuse me? The Ultra 5 an "unreliable device"?? We have a farm of Ultra 5s that have been running for a year now. Total number of system failures or crashes of any kind: 0. Period. How is the Ultra 5 any less reliable than any other workstation-class system?
"When security is compromised on the System Service Processor, which runs on the Ultra 5 workstation controlling domain operations and performance monitoring, all running domains on the E 10000 can be brought down with a short command sequence."
No kidding. When (or rather, *if*) security is compromised, you could do a whole lot more than bringing down all running domains. Just the same as any other platform. How is this a weakness specific to Solaris or the Starfire?
And besides, these are supposed to *secured* (meaning, physically) control consoles. Meaning, locked in a cabinet in the datacenter.
"System boards that are hosting non-pageable kernel data structures cannot be removed from a domain without interrupting service. The Solaris operating system has to undertake a special "quiesce," or suspend, operation while the critical pages are migrated to another board."
Ummm, yeah. So? How is this any different from any other operating system? Again, I fail to see what the problem is. And besides, how often do you change system boards? Please.
Sure, go ahead... try and remove a CPU card from any NT-based system without first warning the OS. Not only will it hang horribly (ie; you can't do it!), you'll probably fry hardware as well!
The fact that the Starfire can even do this is pretty amazing.
"System boards that are hosting Token Ring adapters, ATM adapters, or non-Sun disk controllers cannot be present in a domain if board-remove operations involving kernel quiescence are to be performed on that domain."
Uh-huh. Sure. I know lots of people with Starfires running Token Ring off of non-Sun hardware that are removing boards with non-pageable data. Happens every day.
I'm not saying it doesn't happen per se, I just think that these arguments are rather ridiculous.
"If you remove a system board from a running domain without enough swap space, Solaris will hang. The administrative tools do not warn you if you do not have enough swap space available."
What kind of idiot doesn't leave enough swap space? What kind of admin would go ripping out system boards without really thinking it through first? What kind of person spends the incredible amount of money the E10000s cost without being informed as to the basics of running a Solaris-based system? Come on.
It's like saying "If you remove a CPU card from an NT-based system while running domains are active, the system will be brought down and all domains brought offline." Ummm, duh. If you remove your legs, you can't walk either. Apparently, M$ thinks that true Unix sysadmins are as stupid and lacking common sense like the server admins that they're used to dealing with.
"Reliable hardware is getting even more reliable. For example, customers can take advantage of 99.9% system-level uptime guarantees for Windows NT-based servers from major systems vendors, such as Compaq, Hewlett-Packard, IBM, and Data General."
These are guarantees on the hardware, not software. I'm sure this looks great for the PR, but hello? I'd love to know what the "major system vendors" think about Windows-based servers being equated with their hardware guarantees.
"Microsoft Windows® 2000 Server builds on these gains. For example, Windows 2000 Server supports COM+ load balancing, which eases customer development of highly available and scalable applications in a multi-tiered environment. On the back-end, Windows 2000 Advanced Server supports two-node fail-over clustering, whereas Windows 2000 Datacenter Server will support four-node clustering. IBM and other vendors will provide support for up to eight nodes."
WOW! I am truly impressed. Two or four-node fail-over. Please.
Finally, at the end:
"Which brings us back to eBay. For those keeping score, eBay relies on Windows NT-based servers running Internet Information Server to provide front-end web services, and a single Enterprise 10000 from Sun Microsystems to host an Oracle database on the back-end. According to published reports, the outages at eBay, which began in February, are due to problems at the back-end."
This is curious. Maybe I'm missing something, but a telnet to port 80 shows that www.ebay.com is using Apache 1.3.6 on Solaris. It doesn't get any more front-end than that, does it?
I did notice that pages.ebay.com and listings.ebay.com are running IIS 3.0, and cgi.ebay.com is running IIS 4.0.
Also notice that their web site is still up and running. Not that that means a whole lot, but hey.
I find a lot of what this article had to say utterly hilarious. The implications that the Starfire is an unreliable and dangerous system is the greatest work of FUD that I've seen in my life.
OK, enough said.
--globalnap.net, product of pure caffeine--
...that microsoft's marketing department is like a midget boxer. They're harmless and easy to keep in check; however, if you let them get to close, they'll use your testicals for a punching bag.
All said and done, this is yet another good reason to not have ANY microsoft products in YOUR company's final solution.
Microsoft was giving ebay a firm kick in the teeth while they were down. Sun got splattered with blood, spittle and ebay's missing teeth. You know, if I were in EBay's position, I would really resent being used as market leverage. Yeah, fscking microsoft... that's the kind of people I want to do business with. It was about a brilliant a maneuver as Cabletron's bashing of cisco several years ago... cisco yanked their licensing agreement with Cabletron and someone in Cabletron's marketing department got fired. I suppose if you're stuck under some power hogging motherfscker in your company's marketing department, you have to use shock, stormtrooper tactics to get recognized, but jesus folks... there are much more creative ways to be fired or quit.
Still, we all know that nothing ventured is nothing gained. Sometimes it is better to not "venture" for fear of what you might "gain". If microsoft gains stupid customers from this venture, it will only make the final outcome darwinian.
MicroSquish lecturing Oracle on reliability?
That's like Bill Clinton lecturing the Dalia Llama on self-control.
-jcr
The only title of honor that a tyrant can grant is "Enemy of the State."
"Windows NT Workstation is the most reliable Windows operating system yet, resulting in significantly lower downtime for users. "
Talk about daming with faint praise!
Don't they have anyone to give this tripe the "giggle test" before they put it up on their web site?
-jcr
The only title of honor that a tyrant can grant is "Enemy of the State."
Hmpf, good think MS left some competitors to blame things on - I'd like to see Ellison and McNealy come out slamming NT/IIS3 - In fact, I've strong reason to think, from what little I've tried, that an old copy of IIS3 or ASP is what broke dragon-drop on my nt box.
Chuck
try { do() || do_not(); } catch (JediException err) { yoda(err); }
Okay, this is completely off topic, but it's too funny...
-
Yeah! I can see MS's marketing department gearing up right now....
90% of Yugo mechanics recommend MSActiveYugoYoke as the best Yugo yoking solution on the market*.
Studies have shown that the front passenger in the central Yugo in an MSActiveYugoYoke system sustains fewer neck injuries** than in competitors' systems.
Mechanics can take the MSActiveYugoSolutions Certification Test to demonstrate their compliance with a number of stringent guidelines set by Microsoft Customer Relations.
With an MSActiveYugoYoke system, you can go where you want to go today!
With MSActiveYugoYoke Enterprise Customer Satisfaction Enhancement Warranty, you can assure MSActiveYugoYoke functionality decades into the future!
Microsoft is firmly committed to enhancing MSActiveYugoYoke ease of use, particularly in high-speed interfaces with LargeSunTrucks.
Microsoft prides itself on its high degree of MSActiveYugoYoke-VehicleJava compatibility***.
MSActiveYugoYoke-the product line awarded the Gold Consumer Choice Vehicular Safety Award****.
Quotes from satisfied MSActiveYugoYoke:
"Well, it's a big improvement over MSActiveYugoYoke 95, I must say that." -- Dan Smith, Yugo mechanic
"Microsoft's Yugo yoking system is my system of choice for yoking Yugos." -- Steve Jobs, interim Apple CEO
"MSActiveYugoYoke is the supreme Yugo yoking solution." -- Jim Bob, a guy at Microsoft
"I guess if you like Yugos and use other Microsoft products, you might as well go with MSActiveYugoYoke to ensure product interoperability." -- an anonymous guy at C|net.
-----------------------------------------------
* Survey conducted by Mindcraft. Error margin +- 20 percentage points. [Note: only Yugo yoking system on market at the time was MSActiveYugoYoke]
** While traveling between 8.5 and 9.2 MPH on toll bridges during hurricaine conditions. Survey conducted by Mindcraft.
*** Certain procedures for unyoking in emergency conditions bear vague relation to VehicleJava emergency unyoking procedures in other vehicle yoking systems.
**** Award won was January 1996 award, won with MSActiveYugoYoke 95. Current product is the similar MSActiveYugoYoke 99.
I can second this. When I was at E10K training
last year, they had upgraded the SSP software to
a buggy level. As a result, the E10K hung, and
had to be phyiscally powered down. All during my
test.
----------------------------
Dammit Jim...It's "U-N-I-X",
Maybe investors think crashes are indicative of the market growing faster than their technical planners expected....
I think it would be quite possible to write a Worm.Explorer type worm for Linux - all you need is to be able to find Samba or NFS mounted drives (mount(1)), find the user's email address book (mail UI dependent), and (optionally) find a real message in their Inbox to fake a reply to (not too hard if they are using mail(1) format mailboxes, or Netscape Messenger.) Then pipe a message into sendmail or mail(1).
I'm not advocating anyone writing a worm like this, but Linux is going to be quite susceptible to this sort of problem. If sending mail requires authentication beyond just being logged in can you prevent this, but that's not very realistic.
In fact, all this could easily have been done on the mid-80s UNIX systems that I used to run.
I agree that macro viruses are based on the absence of any security controls for Office macros, but this sort of worm is not dependent on that - in fact you could write it to attack Netscape on Windows or Linux.
Ok, i'll give you a factor of 10, the rest of the machines had local drives for compilers and object files. I think that this is also offset by the fact that the Sun is a heavily used multi-user machine and the others were entirely stand-alone dedictated to the build.
The difference between Canada and the USA is that in Canada healthcare is a right and gun ownership is a privilege.
It just ain't so. Microsoft is lying again. Nothing new, eh?
--
Get your fresh, hot kernels right here!
The Truth is that the Oracle Database got corrupted but this was because of an NT Server fronting the pages. The ThunderStone (Texis) peice was still running..and you can test this by doing a search on Ebay. Texis RDBMS Runs on the same Solaris Box that Oracle runs on. Solaris is running fine and still keeps running flawless. The Oracle/NT RPC via HTTP calls to the database is what caused the problem..not SOLARIS
That leaves them with one reasonably uncompelling point about dependence on a service processor when they start plugging their own high-availability "story" - and what a piece of fiction that is. I've worked professionally in the HA field, and Wolfpack is the laughingstock of the industry. It's the most unbearably pathetic HA package I've ever seen, only surviving because of its parentage, and even then it amazes me somewhat that anyone uses it.
I know marketing material is not intended to be objective, but this piece is the most blatantly offensive piece of misinformation I've seen in a long time. While no individual claim in it is untrue, the overall result is incredibly misleading. Whoever wrote it is a master of their craft, but some forms of mastery don't deserve to be acknowledged.
Slashdot - News for Herds. Stuff that Splatters.
CNet implies that it's Sun, but doesn't come out and say it.
By the way, their IIS is version 3, which very few people are still running. Hard to see why they stick with it.
Of course you can do a search on eBay now, it's back up. And are you seriously saying the web server can cause the back-end database to crash? Explain this please.
All respect to Hemos, but Slashdot is a much simpler proposition than Ebay, even is it were getting 10 million hits a day. The forums could be divided among different servers on /., but on Ebay the need for cross-communication between different parts of the database is much greater.
Whoever is in charge of maintaining Ebay has a herculean task, and I respect them.
However:
I don't like the way Ebay handled the Ebayla fiasco, I didn't like the way they dealt (er, didn't deal with) the earlier allegations of security problems, and I'm just plain concerned about Ebay's growth outstripping their ability to keep up with it.
But we have been having power problems, the jokers who wired our new server room screwed up big time. We've also lost 4 power modules in an APC Symmetra that was powering it. Can't fault Sun._ ___________________________
___________________________________________
Auditing and dentistry are excellent career choices for people who don't like other people but aren't coordinated enough
If what I heard was correct, the part that croaked on Hotmail was Exchange, not IIS.
(Which shouldn't be a suprise - Exchange only started supporting > 16 GB in it's database last year or so. For those not used to dealing with corporate mail enviornments, 16 GB is not very much.)
--
Business. Numbers. Money. People. Computer World.
Irrelevant.
Even if this PARTICULAR virus isn't an Office Macro it sure does depend on hooks into the email system to spread. Not to mention that a quick look at any virus def file these days shows that 90% of recent virii are based on attacking Office security holes. Mcrosoft's Office is a nice agar plate Just Waiting to be innoculated.
Only Microsoft is bozo enough to develop a system enabling the rapid spread of portable, CROSS PLATFORM virii. If you are a Mac user, something like 98% of your virus def file is made up of Office macro virii.
Office macro virii are a huge cost for IT organizations. If they are not careful somebody is going to realize this and start publishing a cost analysis....
But how on *earth* can *anyone* get away with charging a third of a million dollars for a computer *case*?
I think it's the chassis, powersupply, and (most importantly) the backplanes for the system cards.
Posted by generic kewl tech reference:
I guess it is possible that another OS can, under certain conditions, fail. Of course, given my experiences with NT Server, I think it would be insane to put an NT server under those types of load conditions.
Any of y'all out there running Linux with the types of loads eBay has been experiencing?
And two other observations:
1) One wishes that Microsoft was as perceptive about their OS' flaws as they are about Solaris.
2) I'm just a wannabe anyway, what the hell do I know?
"The percentage of users running Windows NT Workstation 4.0 whose PCs stopped working more than once a month was less than half that of Windows 95 users."
More here
Yeah, yeah, it's workstation and not server, totally different operating systems. Not.
I use Linux at home.
I use NT at work.
From those two facts, guess which one I have found more reliable, more useful, and enjoy the most?
Thad
The Bolachek Journals
Yeh, I've done it w/ Tandem too. I used to work in the Hardware labs of the Austin, Tx. site. Worked on the PUMA project.
Do you have any links for MS cooperating w/ Tandem on this stuff?
Even with Tandem's help, I don't see it doing them much good. The reason you were able to hot-swap the CRU's with the Motherboards and stuff in them, was because the hardware switched things over to one of the many backup boards. Unless MS gets someone to develop this kind of hardware for x86, I can't see it doing them much good.
Your right that eBay should have switched over to a Tandem. Even though this is a software and not a hardware or OS problem, Non-stop UX does a hell of a lot to help you when something goes wrong. They probably would have been able to detect the problems with their software by now.
Well, before I blame eBay to much, I'm off to go see if Oracle has recent ports to tandem hardware.
Just a couple of choice pieces of FUD from the M$ web page.
--------quote on----------
3.When security is compromised on the System Service Processor, which runs on the Ultra 5 workstation controlling domain operations and performance monitoring, all running domains on the E 10000 can be brought down with a short command sequence.
--------quote off----------
So, let me get this straight. The workstation which is responsible for "controlling" operations can be used to stop operations? What was Sun thinking, including a command that would turn things off! Of course, we all know that one of the main features of NT Server over NT Workstation is that the "Shutdown" command has been removed from the "Start" menu.
--------quote on------------
4.System boards that are hosting non-pageable kernel data structures cannot be removed from a domain without interrupting service. The Solaris operating system has to undertake a special "quiesce," or suspend, operation while the critical pages are migrated to another board.
--------quote off------------
This is supposed to be a problem? Now, I think it's pretty neat that you can migrate kernel memory off of a certain piece of hardware and swap it out at all. We're supposed to believe that under NT you can do this at all? Much less without telling the kernel first? The only conceivable way to allow this to happen without telling the kernel to clear the board out first would be to make sure that all the kernel memory has had a copy paged out to disk. Or perhaps keep multiple copies of all kernel data structures (and hope they don't get out of sync.) Maybe NT does do the last one. That would certainly explain why it's a memory hog.
Pretty amazing if you ask me. This web page is clearly meant for the PHBs of the world, as anybody with any knowledge at all of how computers work is simply going to laugh at this.
Seems like M$ knows a lot about Sun's "weaknesses". Maybe they should spend less time finding faults with other companies and more time fixing their own....crazy idea right?
STING RAID: If you remove a system board from a running domain without enough swap space, Solaris will hang. The administrative tools do not warn you if you do not have enough swap space available.
This is pretty low. Yeah, it can happen - what else is an OS supposed to do when it has more processes than now remains as memory?
Come on now, NT's "You are running low on virtual memory" error messages is one of the most beautiful parts of the OS. It is perhaps the single most profound statements bestowed upon us. If Solaris (or Linux, what the hey) cannot provide the most highly trained administrators (I'm talkin top notch MCSErs here) with this sort of insight, well you get what you deserve then.
My understanding was that Microsoft paid Dell for the conversion costs. (And that the WebObjects setup was breaking under the load, but who knows if that was hardware of software.)
--
Business. Numbers. Money. People. Computer World.
This seems to me to be an operations problem.
1) No hot swap for the back end database
2) According to ABC news they are restoring from tape.
I mean, even if your back-end database bursts into flames, you should have a path to recovery that doesn't take 12+ hours, right?
Perhaps the problem is data that didn't reach that AC's computer. I had the same thing happen apprx. 3.5 hours later, clicked reload, still just the part of the page that shows initially, scroll down and there's nothing else there. I finally clicked Help, About Netscape cause I couldn't remember which 3.x I'm running. This repainted the screen with the info instead of popping up a dialog box. When I clicked Back the full MS page loaded with the middle of the page showing in my browser window so maybe all those scroll bar clicks got caought in a loop somewhere and didn't execute until the MS page reloaded. Maybe it's my browser or hardware. But how come it's MS pages that always seem to have or cause the most trouble?
I see even classic Slashdot is now pretty much unusable on dial up anymore.
...two. At least according to an InternetWorld article a month or so ago.
Advice: on VPS providers
We have a couple of these at work. It was the only thing that could handle the level of data warehousing we wanted to throw at it. You don't buy a new one of these every two years.. And individuals certainly don't buy them.
maybe try redbrick? we use this where i work (sorry, I'd like to say where, but i can't) and me like it a lot. me like a whole lot.
It didn't bring down the server. The companies made the operating decision to bring down the server because that is far easier than telling 10Kpeople not to open attachments to their e-mail. Have you ever tried to tell everyone in a corporation the same thing at the same time while using terminology that most do not understand? ("What's an attachment? What's he talking about?")
I'm sorry, that's a very poor excuse. Either MS's software should have a option to be able to keep these things from happening, or the users of these systems should be educated in these things.
It's no different than the idiot who wires up a chain of servers to a light switch, forgets to label it, and the janitor comes in and takes the servers down for a few hours.
Either there should be a method to disable these macros, or the admins at these systems should be processing all incoming/outgoing anything with something to remove the attachments (procmail does this I believe).
Ignorance is only an excuse for children and alzheimers patients.
-Erik-
Wow, your Microsoft sponsored brain cells must have worked overtime to think up that post.
You'd have to be pretty damn clueless to suggest that someone would come into a forum wtih this topic, announce by their own free will they work for microsoft, and not be sincere.
Perhaps you are looking for the poster who was talking about that beachfront property in nevada....
-Erik "Chronic Caffiene Deficiency" Hollensbe-
"6.If you remove a system board from a running
domain without enough swap space, Solaris will
hang. The administrative tools do not warn you if
you do not have enough swap space
available. "
I may be off but is system-board == motherboard? In that case I
wanna see a normal PC remain functional while you
remove the motherboard while its running.
I know not what course others may take; but as for me, give me liberty or give me death!
Their stock dropped 9.2% after they suffered 21 hours of downtime.
Estimated cost of downtime: $87,000,000 per hour.
Jamie McCarthy
Jamie McCarthy
jamie.mccarthy.vg
From looking at their announcements about the ongoing problem, looks like they've got database corruption issues. I'm not an Oracle guy (Sybase..), but I have to wonder how good, or bad, their disaster recovery planning is - what they do for failovers, that sort of thing. You really shouldn't be able to knock an entire "e-business" out like that, if things are set up properly. I wonder if this is more a case of too fast growth, not enough real hardcore planning and work on the robustness of their back-end.
Any server is only as reliable as the people who
run it, whether it's your own Linux box, or an
E10000 running the Oracle backend at Ebay.
I would stack up the reliability of any Sun Sparc running Solaris 7 against a Wintel box running NT any day.
Either there should be a method to disable these macros, or the admins at these systems should be processing all incoming/outgoing anything with something to remove the attachments (procmail does this I believe).
There is - it's called the "Disable Macros" button that pops up when you open a document that contains macros.
As I said before though, this most recent virus was not a macro virus.
Coming soon - pyrogyra
your dba is "overworked" not because oracle databases are inherently difficult to modify; there are plenty of 3rd party applications for graphically manipulating tables and other objects for Oracle and every other major database out there. The reason the changes go through him/her is because on a highly loaded system, MS Access-style datatypes like "Text" and "Number" just don't cut it; data structures have to be highly tuned to fit the typcial usage as closely as possible without adding any unnecessary overhead to storage volumes or lookup times. You might hand him a table to store name and address info, and his job would be to add a primary key to it, some indexes, possibly normalizing it with more than one table and some foreign key constraints, stuff like that. If these things are not done, with any substantial load your database will grind to a snail's pace if not deadlock or completely crash, no matter *what* database vendor you are using...Oracle in fact will show the symptoms of these problems much more slowly than MS-SQL in my experience. There is no tool out there that can do these things without a capable database programmer - just like every other "wizard" tool MS gives you that allows unqualified people to throw up unstable code, the Access tools are helpful for design-time "fooling around" but have no place in a production environment.
Is it that hard to see? They changed their page layout and tweeked the software, and now they are getting data corruption. It doesn't have anything to do with Solaris or Oracle. If you design a database that can't correctly handle concurrency, doesn't have good constraints, no triggers, etc. ad nauseum, then you are going to get data corruption. Also, the whole 'high availability' arguement is laughable. eBay's buggy software is still has high availability. ;) Just because their software crashes doesn't mean Oracle and Solaris are crashing.
It's just more FUD from the Empire.
--- A Jesus Fish eating a Darwin Fish only proves Darwin's point.
The eBay crash made the front page of today's Orange County Register:
EBay dark for 2nd day
ONLINE: A software problem leaves the site's auctions blocked.
The only techinical detail, FWIW, is the quote ``Company officials said Friday that the outage apparently stemmed from a problem with software provided by Sun Microsystems Inc.''
The article mostly deals with the missed selling opportunities, including a quote from someone who had quit her job to sell stuff on eBay and was losing a fortune.
IIS4 is better than IIS3 for dynamic content. The introduction of MTS, and improved data access objects and such is a great boon for programmers.
However, the move from 3 to 4 is not uneventful, especially with dynamic content. It would require considerable testing and several programmers to fix the problems as they arise in testing.
The old adage likely applies... if it ain't broke, don't fix it.
At work right now we're upgraing our intranet web servers using IIS4 by applying SP4, and beginning to use VStudio6.0 stuff. We've encountered a number of issues. Although mostly it's been that code which was poorly written will no longer run as it now throws an error.
I recall a certain Exchange server at my last job which lost a disk out of a RAID and we lost 67% of the store. Granted the data loss would've been pretty much cancelled if we had a backup and HARDWARE RAID, but still... 15hr straight and 33% recovery with the stuff I had to work with. . .
--
--
Me spell chucker work grate. Need grandma chicken.
Not only is Microsoft's website often returning database error when I try to utilize it (mostly the msdn sections), but it's frequently rearranged so that there are many broken links, and I can't easily find the same page there twice.
The latest insult is that a few days ago when I tried to reach http://www.microsoft.com/Data/ in order to download the latest MDAC, I kept getting what appeared to be very wordy 404 errors that indicated that that page did not exist. This was very discouraging, since I needed MDAC 2.1 for a consulting project. Figuring that they'd rearranged the site yet again, I did a search of Microsoft's site for "MDAC" only to find that all the hits were under the previously attempted link, and all of these pages returned the same error. All of this occurred on a Win98 box with all the latest updates (except IE 5) running IE 4.01SP2.
Unwilling to believe that they'd taken down the data access site, I tried to reach http://www.microsoft.com/Data/ from a WinNT 4.0SP5 box running IE 5. Guess what? It worked. It appears that Microsoft has effectively "IE 5 only"ed all of the developer's content on their site. Considering that IE 5 exhibits big memory/handle leaks on the one machine I have it installed on, I'm not eager to promote its use (however, I do like the updates to Outlook Express, and have had no troubles with that part of the product.)
Anyone care to check if http://www.microsoft.com/Data/ is accessible from Netscape, Opera, or other browser?
everything, perhaps they should start writing decent software ... just a thought
-
if knowledge is power, the internet is god - me again
Microsoft has it wrong on the E10k. It sounds like they've been talking to people here and there and haven't actually played with the hardware. The major SPOF is not the SSP workstation, it is the control board. If the control board dies, all your domains will go down. The control board is what, among other things, gives the clock to the entire system. But most E10ks are equipped with two system boards so that you can swap and get up-and-running again quick.
If the SSP (Ultra 5) dies... well, wait. It really doesn't happen. Something like a hard drive crash might do the trick. When you are without and SSP, the domains (virtual hardware systems) on the E10k continue to operate. But you're not going to catch things like record stop dumps (hardware error and warnings... such as persistant ECC memory errors). However, most sites that have purchased E10ks have also purchased two SSPs. They're so cheap in comparison, it makes sense. We have YET to fail over onto the secondary SSP on any of our 10 E10Ks. Since when is an Ultra 5 an "unreliable device"?
Sun complaining that the OS needs to be temporarily quiesced in order to move the kernel from one bank of ram to another? Heck, it's a miracle that it can even happen at all. I'd like to see microsoft write the code to move the kernel on the fly. Not a project I would want to be on.
Poo-poo on the adaptors that don't do DR? Hardly even an issue. Look at them... token ring, ATM, third party. I wouldn't even run a third party SBUS card on my E10k. The translation is that "a minority of SBUS cards are not a good choice for the E10k." Big deal, Bill.
About the swap space issue... they might actually have an issue there. I'm sure Sun is working on a warning now, if it is a problem. BTW... at that point you haven't actually REMOVED the system board. You are doing an operation called a "DR Drain" which moves all the pages of memory from the RAM in that system board to another. Once successful, you are able to remove the system board from the configuration, or abort the change.
Let me translate Microsoft's position:
1. Sun Enterprise 10000 systems have single points of failure. You can't hot-swap CPU boards arbitrarily, and the Ultra-5 front-end is a critical component.
2. Sun recommends that for high availability you cluster between multiple 10000 systems. This is bad.
3. Microsoft's commodity hardware platforms do not offer any of the scalability or reliability features of the Enterprise 10000, so clustering is the only option. This is good.
4. Microsoft's current clustering offering is primitive. In a survey, a majority of people said it was adequate.
5. Microsoft promises that Windows 2000 will have better clustering than NT.
6. eBay is not following Sun's recomendations that high-availability requires multiple systems. They have experienced outages.
BTW, it is shocking to me that eBay could have only a single server. This is at best incredibly naive; at worst blatant incompetence. Therefore I suspect it is false.
Buddy, I've done the same. I tried Win2k. So did my best friend. We couldn't even get the thing running. The driver support was flaky (no backwards compatibility with ANY drivers), it crashed routinely running Maya, Lightwave, etc.. Sure it played games, but without full driver support, we were screwed outta our A3D sound.
As far as I'm concerned, MS is in a LOT of trouble with this product, claiming it's all that and a bag of chips.. And it doesn't even run.
Your plan still wouldn't cause the massive damage that it is causing for one main reason: priviliged accounts. MS is an OS where a standard user (level program) can take down the whole OS. I'd never run unknown attachments as root. As a matter of fact, most Linux distros set it up to automatically foreward all mail to root to another user. So, _maybe_ one users data is trashed. The system is running and backup tapes are at the ready. This virus didn't affect most NT systems, but there were still people out there who felt like reading email while logged on as an administrator. It's stupid users, not stupid OSes that cause the most damage. So at least use an OS that protects a user from himself in production environments.
Reid G. Ormseth, Esq.
Incendiary asked:
Very interesting.
I checked it out on Netscape 3.04 running on WinNT4. I should add that I have Java and JavaScript disabled, and go through 2 proxies (Junkbuster and LPWA). So the first proxy lies about my browser and OS, and the second proxy strips out the reference to OS. So as far as Micros~1 knows, I'm using Netscape 3.something Gold, but they don't have an OS mentioned.
The first time I went there, I got about 19K of the page, but blank page below all the images at the top. I thought this was FrontPlague's infamous omission of the closing table tag. But when I viewed source, I found that transmission of data had been cut off mid-sentence. Hmm. So I hit "reload", and promptly got a (bogus) 404 message.
I'm an Oracle DBA for a living, and to me it smells like a *major* hardware fault (i.e. a pipe fell on the EMC array), or some idiot SysAdmin (or DBA with too much access) did something amazingly bad, like rm -R * from /, and the machine let them.
I've had Oracle instances on Solaris running in Production, handling millions of transactions a day, with uptimes in the 6-month to one-year range (depending on what the maint. schedule is. Generally, Oracle/Sun is a good combination.
There is another possibility, though...
I've found Oracle (and Sun for that matter) to be extremely reliable -- as long as you are about one version behind the "latest and greatest", when all the (usually minor) bugs are resolved. If they tried to run 8i in production, for instance, Lord help them. Again, that would be an Admin fault...
Amazon uses Oracle...they don't go down. What's the problem at Ebay? Hmmm....
When people can't design a reliable system with budgets that allow the purchase of Sun 10000's!
I run a Sun 10000 with two SSP's. 10000's are connected to their SSP's via private ethernets. I have three private networks; two to allow redundant interconnects between the SSP's and the two 10K control boards and a third for general use, NFS mounting CDROM's and the like. Most people will have no reason to put the SSP's on a public network at all -- I certainly don't. In order to hack the SSP, one must first hack the 10000. Once they've done that, the ability to reach the SSP's by network is irrelevant. The point about the "problem" of the SSP having control is as silly as claiming that EMC Symmetrix disk arrays (heavily used in IBM mainframe shops) can be crashed by the single laptop each array contains.
I would love to know the details of their failures -- I suspect the article is hinting at issues that have nothing to do with their real problems. Further, I'd bet that the main vulnerability that people cluster Sun's against is hardware failure -- and I'd also bet that the main reason people cluster NT boxes is software unreliability!
Geeky modern art T-shirts
An excellent article from Performance Computing on this very subject and the cluelessness of /all/ the vendors in the market as these beasts become more common:
9 05f1.shtml
http://www.performance-computing.com/features/9
Also check out http://www.wintercorp.com. An interesting note from that site is that, even though MicroSoft participated in and helped sponsor the survey, they did not place in the top 10 in any of the important, published categories. Maybe some (all?) of the interested slashdot readers could send the Redmondites a missive, asking for comment on this survey.
BTW, the Performance Computing site is maddog's publishing home and an excellent source of well-thought, sane information for both the erudite Linux/Open Source professional and his PHB.
It's funny that MS holds up Dell as an example of a reliable, scalable NT-based site. At least their WebBoard support area is frequently inaccessible, and always incredibly slow.
MS also touts 99.9% uptime guarantees from Compaq, etc., but fails to mention that Sun claims 99.95% for the Enterprise 10000.
Nonetheless, my intuition (totally unsupported by any concrete info, other than their poor response to the eBayla exploit) is that eBay is a mickey mouse operation that got really lucky and rich, but does not have the technical expertise commensurate with a multi-billion dollar company. I wouldn't blame any of their vendors, MS or otherwise, for their troubles.
Much as I'd like to read the linked-to article, Microsoft's web site seems incapable of serving up a version of it that I can read in Netscape 3.01.
I get one window of text (along with the usual decorations) which is empty if I scroll down, and has vanished if I scroll back up. Fascinating. "View source" shows more JavaScript than actual document text...
No doubt it works just peachy in Internet Exploiter. But MS misses the first point of communication, which is to convey the message.
No wonder MS is losing.
-- Alastair
hey, don't forget the trailing / :
s/Suing/Using/
:)
According to netcraft (http://www.netcraft.com/cgi-bin/Survey/whats), they are running on MS-IIS/3.0.
------------------------------------------
www.ebay.com is running Microsoft-IIS/3.0
Microsoft-IIS is also being used by Walt Disney, Compaq, Nasdaq, and The National Football League.
www.ebay.com is hosted by ebay.com.
------------------------------------------
Mario.
The above is not a rhetorical question--I really would like to know! What product from MS and its hardware partners is supposed to be equivalent to a Sun E10000? I guess I haven't been following MS well enough, because the last I heard, NT scaled to no more than eight processors and loses any sign of linear scalability after four.
A Sun E10000 maxes out at 64 UltraSPARC processors. NT may (try to) compete with Solaris on low end machines but are people really using NT servers for tasks that, in the Sun world, would typically be assigned to a fully loaded E4500 or higher?
Even on Alpha, NT isn't a 64-bit OS. That matters at the level we're talking about. I could understand Microsoft taking on Sun's workstations (where NT is eating everyone alive except Linux and the BSDs) and low end servers, but unless the game has changed dramatically there isn't anything Microsoft that competes in this market.
Microsoft's claims of the E10000's faults sound ridiculous considering that, to my knowledge, no MS-based system has any of the functionality they discuss AT ALL, much less better. I find it hard to believe even MS could make such statements, so what product do that have on that level? Clearly I missed something!
As for eBay, I used it for a few months a year or so ago and it was slow and extremely unreliable even then. I have to believe that poor configuration and administration are more to blame than any OS, even NT.
I have a lot of experience with the E10ks. I manage 4 of them (fully loaded). As far as the SSPs being a security risk. That is true *IF* you do not secure them. But then again if you login as root you can do just about anything on that machine. MS is trying to point out that the SSP is a different machine than the E10k. Physically it is, but logically it is not. I have secured my SSPs and I have not had any troubles. Period. MS talks about how you can not dynamicaly pull out the system board that the kernel resides on. Well, when was the last time you saw a NT box that you could pull a system board out of? Never comes to mind. For pure processing power, like a huge Oracle dbase that is a backend for a website, the E10k is very hard to beat. Sure it has it's flaws but doesn't all hardware platforms? Pointing out the flaws to diverte the attention from your flaws is just plain bogus.
Can we all just get along?
Scott
Scott
C{E,F,O,T}O
sboss dot net
email: scott@sboss.net
Scott
janitor
sdn website family
email: scott at sboss dot net
A single Starfire is rated as being able to deliver 99.95% availability with one - ie no clusters, and without all those caveats above - though it does need to be setup with reliability in mind for this - there's plenty of options.... Starfires aren't simple either - up to 64 CPUs, many more PCI and similar slots, memory slots, etc, etc. So, plenty of things to go wrong. Similar sized computers (from everyone) are really hard to transport without something going wrong. The only people more nutso on reliability on 'big iron' computers are IBM (from the companies I know a fair bit about anyway). Not only do they have backup CPUs, in their CPUs they do the same operation twice (in parallel, with checking at the end) to trap the ultra-freak chance of cosmic radiation or something casuing a flipped bit, or worse. (yes, they do seriously actually worry about such things... I remember an IBM proposal about how to design memory that can handle a once-a-month chance, for when you have a huge about of RAM, for some particular kind of radiation....)
The only complaint I've ever heard about Starfires in general is that if a PCI card (though not SBus card) breaks down it can hang the entire system until an operator manually flicks a switch to say that that particular card is defunct. Though this is really because of how PCI works - Sun's 99.999% reliable Netra 1800 has some highly specialised custom hardware to get around this problem with PCI cards, as well as backup CPUs and plenty of other stuff... ridiculously expensive too, they are... though apparantly more cost effective than anything in the same class. The Netra 1800s are a few months old, while the Starfire design is over 2 years old, btw.
I dunno about all of MS's claims, but I'm pretty damn sure that you can have hardware redundancy for just about everything, if you want, including the Ultra-5 controller. Most of the other claims seem to be related to the fact that you can hot-swap PCI cards, memory, CPUs and even mother-boards in a Starfire...
EBay do seem to have had more than their fair share of problems though... quite a few hardware problems it seems - I vaguely remember a problem earlier in the year was due to some controller card or something. As far as I know, nobody has had anything close to the problems EBay are having with their Starfire(s)...
Another little point... MS's idea of expensive downtime is $10,000/hour. I remember reading something on Sun's site a while back about high end availability systems. Sun's idea of expensive downtime is $10,000,000/hour - ie stockbrokers. They also had a list of most common causes for 'unplanned' downtime on their HA systems - first was 'operator error' (or lack of training, etc). I can't remember was second was, but third was 'fire'! (I'm pretty sure Sun's computers don't have a reputation for spontaneous combustion!)
Was ebay really running it's backend on one system, with no fail-over? That's pretty stupid...
But, to make use of this ability, you've pretty much gotta be a Sun outfit. I'm an old mac-head, so I know how much it sucks to be beholden to a single company hardware-wise. My PCI powermac represents a breakthrough for Apple just to use an industry-standard expansion bus. Then it took forever to get to the point where I really can use commodity goods, because the Mac third party manufacturers really didn't like competing with PeeCee suppliers and their down-to-earth prices.
In sum, the article is a piece of marketing crapola, spinning the problem away from Seattle like a top. It also makes its point in response to a pretty serious problem that reflects quite poorly on M$ products (not that I *knew* eBay was down; is there a slashbox I can set up to keep me appraised?). After reading this, I'm eagerly anticipate what they can come up with in response to ExploreZip/ZippedFiles! :P
Again, thanks to Hemos for posting this.
jaz
Death to Argument by Slogan!! (This post twice-encrypted with ROT-13. Replies not using same will be ignored)
The problem with your assertion is that Microsoft is using eBay's problems, which may or may not have anything to do with the reliability of Sun hardware or software (more likely it's human error) to further their outrageous FUD campaign.
It has nothing to do with a Microsoft system crashing, but rather trying to turn the eBay problem in into a FUD event that has people here upset.
This web page is the most outrageous piece of crap I have ever seen. Advising customers to rely of some piece of untested software still in beta to handle a massive mission-critical load. If I was eBay's CTO I would be looking to go upscale into some really heavy iron like the Himalayas or mainframes that operations that need REAL reliability use. The idea of going to a MS operating system for this sort of application is purely ludicrous.
The same day this is going on we have another round of word macro viruses terrorizing MS users everywhere. Why don't you see Corel and Lotus touting the fact that Word Macro virii don't trash their systems? Because they aren't low-life like Microsoft. Do you see slashdot trashing MS over this? No, even though they richly deserve it.
Microsoft deserves to be roundly excoriated on this one.
As we can see from this example, hiring administrators who only know how to point and drool may be less expensive in the short run but you'll get burned and burned badly in the long run.
Anyone with half a brain would have a failover (or two) for such an important system. Again, it's cheaper in the long run. Hardware does fail. Systems go down. It happens. If you got your administrators out of a cereal box, your company goes down because of it. If you can afford that $10,000 an hour down time (Or that $1,000,000 an hour down time) go ahead and get the point and drool admins at $40K to $60K a year. If you can't, get the $120K+ ones. It'll be less expensive in the long run.
Posted by d106ene5:
Everyone who has tried tying Oracle to a live, high-capacity system on the web ultimately has learned this lesson.
You can squack all you want about what you think you know about Oracle, but the proof is in the pudding. Remember when Excite's home page used to give you an Oracle error??
Lycos is going down this road now too - trust me, they'll be sorry.
I am not talking out my ass, I work at a company that has learned this lesson on a high capacity site.
I'm seeing this story everywhere. If the systems at your bank go down, and all the ATMs are out of service for 6 or 12 hours.. nobody even really notices... and those are considered "mission critical applications."
:-)
What I think we might be seeing here.. is the creation of an even *higher* category of important systems. Some sort of world-stopping problems that result from a downed server/cluster. Maybe the PHBs will start talking about ultra-critical systems.. or some buzzword like that.
Previously, a system goes down..and you can expect to catch hell from everybody in your company, and maybe even a few outsiders... but now.. you catch hell from CNN and the likes..
http://www.oz.net/~rcw/deleted.the.data base
Did anyone notice that they say eBay has problems in the back end? They say in another sentence that they use Oracle on the backend. What they don't say is that the two sentences are related! Perhaps they are having malfunctioning switch (which now can run NT!) on the backend.
The story is that they only implied that Oracle on Solaris was the problem. Perhaps a programmer forgot to unlock a row in Oracle. Does that mean Solaris' reliability is bad? They never said.
Microsoft and its enterprise associates believe that customers benefit most when they combine value priced, high performance Windows NT® server-based systems with the high availability advantages that distributed computing offers.
If you believe this sentence coming from Microsoft, then let me sell you some beach property in Nevada please. One rule in reading articles from any corporation, is to question the origin. Was the article written by an independent observer, or someone paid to write that line. In this case, I think someone was paid to write that line, whether or not he/she believed it.
-Ben
>>given my experiences with NT Server, I think it would be insane to put an NT server under those types of load conditions.
/.
Any of y'all out there running Linux with the types of loads eBay has been experiencing?
There's a long list of sites on the web at least as big as eBay running NT (Dell, Barnes&Noble, 1800flowers, ESPN, etc.), and they don't have eBay's persistant reliability problems. I suspect eBay programmers are at fault.
What's the biggest Linux site on the net you can think of? Dejanews? Hardly a bastion of reliability, and neither is
I think Microsoft's comment would be that for the same price, you could probably have duplicate fall over servers waiting...
Posted by FascDot Killed My Previous Use:
MSSQL "rox" at what? Not providing basic features like large page sizes and query-embeddable user-defined functions?
I'm researching what to use for a midsize datawarehouse-esque DB and MS got the axe immediately in a totally objective feature comparison. That's before we talk about uptimes and speed.
--
"Please remember that how you say something is often more important than what you say." - Rob Malda
Sun Microsystems Inc. SunOS 5.5.1 [...]
# uptime
4:26pm up 278 day(s) [...]
I wish I could say THAT about even ONE of our
NT servers.
--
Chris
Where I work we have been Suing Oracle for a lot of internal apps, but nothing realy high-load. In fact 80% of out apps are simple enough and low-use enough that they work on M$ Access. Of course it's a really big bummer if there is a lot of demand on an Access DB, as it'll take down the server. . .
We have this Oracle DBA who is pushing for _everything_ to be done on Oracle. I would love to hear more about the problems you've had with Oracle and web apps. What is the "best" DB you've used for the web? What kind of problems have you had with Oracle?
"Eye halve a spelling chequer, It came with my pea sea, It plainly marques four my revue, Miss steaks eye kin knot sea"
The one, let's call it Site A, uses a $20,000 Dell NT Server 4.0 SP3 (dual PII-300) with 50 win95 clients; also runs MS Proxy Server 2.0.
Site B uses a Sun Ultra 2 Model 2300, dual sparc 300mhz. It supports tin, pine, lynx, gcc, filesharing, and ftp with 300 concurrent users across a 2 mile radius WAN with .2 cpu usage.
Is it fair to note that the last "restart" of the Sun was 67 days ago--the last "restart" of the NT...well, with all seriousness, it's been at an average 2 crashes per day (that's an 8 hour day).
eBay may have DB problems; let's not forget Oracle has all of its products availible for Linux and Oracle products are sold OEM thru Dell and their Online Store. I'm afraid Microsoft is trying to bolster their image. Don't believe NT needs PR help? See http://www.ntsecurity.net
Adios.
Just exploring the oracle site after looking over this article and saw this great message after clicking on the links on the left menubar:
"Can not service this request, please try again later"
It should be noted that Oracle is also used by these top sites.... and they don't have problems....
www.amazon.com
www.1800flowers.com
www.cdnow.com
www.charlesschwab.com
www.cisco.com
www.dell.com
www.etrade.com
www.onsale.com
www.rei.com
I fear that it is the ebay ppl who are at fault....
You have to wonder about the relative competence of an IT department that chooses to run IIS and ASP in lieu of the more reliable alternatives.
The worst thing is the PHBs and advocates will now use a dumn article like that to avoid acknowledging the alternatives.
"My God...It's full of ads!" -Fry, about the Internet, Futurama
Sometimes you have to wonder how things ever get approved to be on their website. Let's look at a few of the more imflammatory claims, which is really quite a kettle o' fish:
RED HERRING: Daemons that control domain operations and perform monitoring functions run on an unreliable device (Ultra 5 workstation), hardly a desirable situation in the context of a data center.
So what? The E10000 will continue to truck on as before without it. This is a complete red herring. The SSP is a really just a console station, nothing more. If it dies, you reboot it, or in the worst case, replace it with another one from the closet, which with Sun's AutoClient technology, can take on the entire identity of the failed box in a couple of minutes. (AutoClient allows Wall St. traders to replace their workstations and be working again with NO IMPACT in 5 minutes. Let's see NT do that.)
FUD SHARK: When security is compromised on the System Service Processor, which runs on the Ultra 5 workstation controlling domain operations and performance monitoring, all running domains on the E 10000 can be brought down with a short command sequence.
No one in their right mind would put the SSP on a network that extends beyond the glass house!! It's a *console*, designed to be locked up securely, like all other mission-critical control consoles. MS still doesn't get the data center, do they?
DUH WHALE: System boards that are hosting non-pageable kernel data structures cannot be removed from a domain without interrupting service. The Solaris operating system has to undertake a special "quiesce," or suspend, operation while the critical pages are migrated to another board.
This is incredible. They're knocking the E10K because you can't walk up to it and pull a CPU card at random without telling the machine first that you plan to do this. These cards contain memory, too, folks, which is why it's pretty reasonable to let the system move things to a safe place before the card goes bye-bye. Pretty much only Tandems can accept this sort of things (because they've got at least two of everything all the time, and they cost like they have even more), and if you're after real fault tolerance, you won't be running NT on them, even though you could...
STING RAID: If you remove a system board from a running domain without enough swap space, Solaris will hang. The administrative tools do not warn you if you do not have enough swap space available.
This is pretty low. Yeah, it can happen - what else is an OS supposed to do when it has more processes than now remains as memory? Although a warning would be nice, E10K admins aren't stupid (we hope), and they understand that there are easy workarounds to this - the E10K makes it very easy to move enough resources into the OS domain in quesiton on a temporary basis. If you don't have enough hardware to do that, you misconfigured the machine in the first place. This is hardly a weakness.
On the whole, the incredible thing about this is that MS is throwing rocks at a really good system with availability features far in excess of that for any practical NT box. You've gotta admire their guts, though - some people will read this and think the E10K is a really expensive, dangerous computer. Funny how they neglect to mention that there's not an NT box on the planet that can provide the performance of an E10K, regardless of how much you spend. This may change eventually, but it's pretty cheeky now.
If you need real fault-tolerance, get a Tandem/Compaq - but after you've paid all that money, I bet the Compaq folks would be the first to advise against using NT on it if you really want fault tolerance.
"The future's good and the present is nothing to sneeze at." - Roblimo's last
More like...
It's about time that there's some media coverage.
eBay has had outages all over the place, and every time they do, it seems that their stock just goes UP another dozen points or so. Maybe somebody might actually figure out that even though the company is worth $billions on paper, their infrastructure still seems as unreliable as a garage operation. (Apologies to all the well-run garage operations out there.)
.
MrCreosote Meow!Thump!Meow!Thump!Meow!Thump! "You're right! There isn't enough room to swing a cat in here!"
A couple of years ago, Dell used to run their ecommerce site on NeXT's WebObjects, but Microsoft basically forced them to switch to MS products. The old WebObjects site ran great and it took the engineers many many months to build an equivalent version using MS's tools. It's pretty sad to see Microsoft trumpeting that Dell uses the MS platform when it clearly wasn't the first choice and after they had so much pain in switching.
Yea, yea. We? We who? Having seen Microsoft's server (server?) software at work, I certainly have a hard time believing that. And I don't think "a few" NT servers are gonna be able to beat a Starfire on database serving. A Starfire is just plain huge.
Sam: "That was needlessly cryptic."
Max: "I'd be peeing my pants if I wore any!"
I work for a software company and did time as build captain for a while on our product. I was amazed at the difference in speed between various systems. The product is a fair size with a mix of C and Java and here are the relative speeds of a build on the various machines. (this was over a year ago and from memory :)
- nt on a 200 mh intel 128MB standalone - 140 minutes
- nt on a 300 mh alpha 256MB standalone - 130 minutes
- Unix on some sort of AIX ~40 minutes
- Sun Ultra with 2 160 mh CPUs 500 MB - 50 some odd users developers pounding on it, over 400 processes - 6 minutes
One major point in the Sun machine's favour was that the code was on local file systems, the rest of the machines had to go through a 100 mb ethernet to get it.
It was unbelieveable the difference in speed. Next look at the absolute pain it is trying to set up automated builds on an nt machine, not to mention the fact that both the nt machines would be dead half the time when you try to launch the builds. Microsoft has nothing to brag about.
The difference between Canada and the USA is that in Canada healthcare is a right and gun ownership is a privilege.
I don't know about EBay, but I know that E-10000's are extremely tricky to configure correctly.
Sun markets them as ultra-reliable and hardware-level redundant, but the truth is that configuring them is so complicated that even a team of experienced sysadmins is bound to screw something up sooner or later. If you bet the store on a single E-10000, then sooner or later the machine will crash hard and your store is hosed.
Given their size, expense and complexity, they are not appropriate for use as the main server in an internet commerce company. Sun should not sell these machines to companies like EBay.
In defense of Sun, I should point out that their "smaller" systems, namely the E-4000, E-6000, E-3000 etc., are rock solid and just as easy to configure as a small server. But no one would dream of running a whole store on a single one of them -- for reliability, you need to run several of them redundantly.
And Windows NT is far less reliable than any Sun machine. NT is the opposite of reliability. Production Solaris machines routinely stay up and running for months or years at a time. Show me an NT server which can do that.
You know what, I'm TOTALLY sick of seeing /. commenters correct the lies that Micro$oft proports on their pages. You need not look at 5 comments to this article alone to get the idea. Instead, let's do something CONSTRUCTIVE with all this criticism! :)
/. did this, they'd get hundreds upon hundreds of emails.
I'm declaring tomorrow Constructive M$ Bashing Day!
(Why do I have the power to do this? Because Barney says everyone's special in his or her own special way, and I'm invoking my privledge as a Special Person. That'll teach you to ask why. Feh!)
The next time you feel like correcting something that M$ claims and that's blatantly false, do so. THEN, email it directly to M$! If we had one day where EVERYONE from
Then, everything would start changing. The wheels would be in motion. M$ would realize the error of their ways and become Tibetian monks to pray for forgiveness!
We can make a difference, dammit! Can't we??
-
Yeah, yeah, we've seen it before! :) (Well, I have. You sent it to me once. How's it goin'? :) -matty
s/Suing/Using :)
john
I have been a ebay user for a couple of years now. They have been crashing so often I would swear that they were using a string of windoze 95 boxes. When they weren't crashing they were really Sslooooooooowww.
I got to be honest here. I do not understand ebay's high stock price. Shareholders must never use the site.
Maybe if they did, they would bitch slap ebay with a stock sell off. Maybe then ebay would deploy a load balancing MOSIX cluster and get some serious uptime for a change.
BTW, I really don't understand Microshaft's take when they can't stop that file loader called windows from crashing my box after two hours.
"ebay.com" return 'Solaris' and
"www.ebay.com" returns 'Microsoft-IIS/3.0'
Note: 'ebay.com' gets redirected to 'www.ebay.com'. So, they are probably using MS products for the internet part of their software and Sun/Solaris software for the back processing. Maybe the problem lies in the communication between the 2 systems... A buffer might have the probability of getting corrupted during the transfer (bad use of semaphores resulting in overwriting part of the buffer or such). That might be what caused bad data in the database. Their software may have problem to handle these corrupted data properly.
Mario.
I refuse to let 95 or 98 in my office. Not even a CD-ROM. NT 3.51 is a bit more stable than 4.0 in my experience, but doesn't handle big hard drives, so I moved my data crunching system up to 4.0 and it has basically been fine.
What really sucks is having to reboot whenever you install a new app with one of Micro~1's crappy shared DLLs. The latest offender was Foxpro 6, which I only need for some light file management, and which had me reboot three times (twice for the mandatory install of evilbad IE which I never use). Rebooting for network changes is bad enough, but for mere mortal applications??
This reminds me yet again why
Bill Gates Is My Evil Twin.
Interesting that this is an almost-identical repeat of a previous post. Trying to get as much attention as possible, eh?
I can't decide if this is just flamebait or genuine astroturf. (When an MS person writes something like this, posing as an ordinary person, he is trying to create the appearance of grass-roots support for MS. Astroturf is fake grass. ESR seems to thinks it's a verb; I think it's a noun.)
I suspect the latter, but do they really think anyone here is dumb enough to take this seriously? Maybe they're just doing it so they can elsewhere claim to have the support of some
It's pathetically transparent. Why won't they just go away?
David Gould
David Gould
main(i){putchar(340056100>>(i-1)*5&31|!!(i<6)<< 6)&&main(++i);}
www.netcraft.com, toss in www.hotmail.com.... the result is
www.hotmail.com is running Apache/1.3.6 (Unix) mod_ssl/2.2.8 SSLeay/0.9.0b on FreeBSD