Re:Keywords: Government. Health Care. Disaster
on
Biggest IT Disaster Ever?
·
· Score: 2, Informative
One of my lady's best friends runs a huge network for a hospital chain that is in the process of combining with another hospital. She's told me repeatedly that the biggest costs for her MIS department is integrating all the bureacratic changes that the government requires -- paperwork, forms, etc
When I used to do LIM systems, I once did one for a UK-based pharmaceutical R&D company. The work broke down to roughly 40% effort to make the software function correctly, with the remaining 60% going towards writing code to generate the reports required by the Home Office.
My boss was actually a little upset that I used a template-based approach, as he felt it would permit the users to modify the reports on their own (denying us a revenue stream). Then he found that the users would gladly pay to have someone else keep those reports up-to-date.
I wonder if Microsoft would be better off if they disregarded at least the incorrect coding by outside developers.
Better off from a development and maintenance standpoint, yes. But I've heard that one of the biggest offenders among the "buggy third-party apps" crowd is Lotus Notes. Which is still insanely popular, even among people who have actually used it. I wouldn't be surprised if several of the most popular Windows apps (modulo Office, although who knows?) had some kind of legacy support within Vista. All it takes is one corporation like Exxon-Mobil or GM who has a business-critical app written in Frameworks or something to lean on MS and there you have it.
With all this virtualization support available now, I wonder if MS is considering just encapsulating entire installations of Windows 3.11/98/NT and running legacy apps on those. That way they could at least clean up their current codebase.
Here's the question that should really determine your answer: are you in a position to move? If not, then take the.NET job. Four hours of commuting each day will suck whatever pleasure advantage the Perl job might give right out. Unless they'll let you work from home at least three days a week, you'll burn out from commuting.
If you can move, though, then I'd take the Perl gig. Sure, maybe Perl won't lead to any great career advances, but I went from C to a 4GL then back to C then to a different 4GL then back to C then to VB then to Java then to PHP and back to Java (with brief stops at C++, Perl and ASP along the way). If you think the prospects are better using another technology, then pick up the tech and learn it on your own. One of the hardest things about learning web development (for me, anyway) was wrapping my head around the whole stateless-session thing, picking up Java and the DOM and HTML was pretty easy. So if you're already a web developer, learning the specifics of.NET (or whatever) shouldn't be that much of a challenge.
Porsche AG announced it will be sharing part of the profits of its sports-car sales with local police forces everywhere. "It's well-known them por-shees is just too damn fast for us to ketch 'em, so anybody drivin' one is a CRIMINAL!", Sherrif Roscoe P. Coltrane explained.
I doubt the JVM would be really incorporated into Windows. It would probably be just like it is now, an additional execution environment. If MS does take Sun's implementation, they would be free to make the changes the made previously (the ability to work with COM/DCOM/COM+, mostly), they would just have to release those to the community. Which would probably be a good idea, as I'm sure the bulk of their changes would just be JNI calls to Windows, which wouldn't really help anyone on a non-Windows platform, but people who used them would tie their apps to Windows.
Then again, if it was that easy they could have just released a package with a jar file and a DLL to do the same thing. So either they were really paranoid because of the lawsuit, or they were so miffed over Sun's protests that they vowed to do.NET and never do anything to promote Java. Hard to say...
SQL Server is a very competitive database these days
Yeah, these days. Actually, I started using it in the 4.2 days, and liked the nice clean implementation (props to Sybase!). It's gotten even better since, I especially like the XML integration, which seems to be better thought-out than most others.
That said, it's hard to find many large, well-established businesses that haven't needed a database until now. Big companies have been using either DB2 or Oracle on UNIX for twenty years, and have a pretty entrenched infrastructure to support it. If the choice was.NET or COBOL, then it'd be a slam-dunk, but as it is, Java is less disruptive. And with both IBM and Oracle championing Java, it'll be a long time until MS is going to get much database traction in serious enterprises. They'll have to come up with something compelling that the others can't match, and an incremental improvement on Java isn't it.
Oh, and the CRM project I was referring to occurred during the decline of PeopleSoft, around 3-4 years ago. Nowadays, most companies I work with are just using salesforce.com...
it lets vendors write OS agnostic server platforms
And this is the thing that blows me away to this day. I don't know how many good-sized projects I've seen[1] (e.g., division-wide CRM) that get originally proposed for.NET implementation, only to founder when someone points to the hardware requirements and says "I have to buy $50K worth of new Windows servers? Why can't we run it on the Oracle server? That thing's huge and cost us a mint." Said Oracle server (naturally) running on either HP-UX, Solaris or AIX. So, rather than throw all their analysis and high-level design away, the team decides to write it in Java. I can't believe MS hasn't moved more aggressively to port.NET to Unix. Oh, I'm sure they've thrown the Mono project a bone or two, but without a branded version, they're looking at a very uphill battle to get into a lot of big shops. I'm guessing they're trying to go the other direction and get Windows into more big shops, but I don't think that's the quickest way to do it.
[1] More than one, probably less than five. Still...
I found the viewfinder the most convincing reason to get a DSLR
Heh, I found just the opposite. My wife wanted a new camera (our old Lumix's shutter delay is horrendous), so I scoured the web for reviews and prices, agonized over the minutiae of color balance and lens astigmatism, and finally almost settled on the Nikon D50. Then the I did the thing that got me suspended from the Suburban Husband Association — I actually listened and paid attention when she took some pictures and talked about what she hated about our camera and what she wanted from a new one. It boiled down to:
quicker start-up time
bigger viewfinder
C'est tout. She didn't care about megapixels, picture quality (the Leica lens on our Lumix is actually pretty good), macro focus distance or any of the other feature the reviews were going on about. All she wanted to do was have the ability to snap off a shot of our daughter as she leaped off the arm of the sofa to try the wings on her new Tinkerbell costume, without having to wait five seconds for the camera to boot and another three for it to focus and meter the exposure (or whatever the hell takes it so long). So I got her a Canon SD630. Near-instantaneous power-up, and a 3" LCD. She's thrilled, and loves the fact that I also got her a 512M SD card so she can shoot video clips.
I was worried that the IT worker shortage was going to drag on for months, but now that it's all over I feel better. I just hope that my salary doesn't get cut, what with supply finally meeting demand and all...
I could more convincingly argue that the failure is due to the software not being tested
Actually, one of the tenets of software reuse is that a reused module should not need to be tested. Testing an already-verified module wastes close to 40% of the cost savings associated with (or at least "claimed for") reuse. Of course, another tenet is that reuse consists of using a module for its specified purpose (which includes, as you mentioned, verifying that input values will be valid). What was done with the Ariane 5 software is exactly how software reuse is not supposed to be done.
Do you realize you're accusing an author of being wrong about what happens in the world he created?
No he's not, he's accusing the author of basing one of the cornerstones of his world on a faulty or falsely-justified premise. Usually, this just means that the particular reader isn't interested in willingly suspending their disbelief on the point in question. It's more akin to someone telling Tolkien "There's no such thing as elves, this whole story is bogus!"
in real life and real work I really really seldom needed tool like [dtrace]
Well, I don't know what kind of "real work" you do, but I develop software and dtrace is like a gift from above when you really need it. Even if you do have the source to everything, you can't just jump on a mailing list when the software that isn't working is stuff you wrote yourself. And even with source, sometimes it's easier to trace through some system calls to get a feel for what's going wrong than it is it meditate over the code.
I wasn't suggesting that OpenBSD needed to adopt ZFS, I was just commenting that I didn't think the OpenBSD team would engage in something as blue-sky as developing an advanced file system. Like you, I think they're more motivated by pragmatism, and would be much more likely to adopt an existing file system than create a new one from scratch.
Can you hire the current IT staff at your new clients? Or are you taking them away from a competitor? Do you guys specialize (somebody's the network guru, somebody's the server wrangler, the third guy's the application specialist)? If so, then maybe each of you should hire an "apprentice" and train them in your specialty. Consider trolling around the local community college with a decent IT program (if you have one).
One thing to do is compile a list of best practices for your shop. This will help communication among the staff (by establishing both common procedures and a common vocabulary), which is very helpful when you're trying to bring a lot of new guys up to speed. Even if it's something as simple as installing an OS on a fresh machine, document it. If you've got any common tools you use or wrote for fault isolation or troubleshooting, document them. Plug it all into a wiki or similar so everyone can access them. Communication is the key for keeping everyone in sync when you're in a rapidly-expanding environment.
But offshore companies are the only way that state agencies can get someone who had the expertise and the ability to get the job done on time, on budget and provide the quality product that we were looking for... (scroll down to "Kitty Pilgrim reports", about halfway down the page). The Nebraska Dept of Labor outsourced the development of their new unemployment insurance system to TCS America, one of the largest consulting firms in India. TCS, hot off botching a similar job in New Mexico, was supposed to write the software on-site in Nebraska, and hire 25% of their work force locally. I think they were later released from that requirement because of the "difficulty finding suitable workers". What that means is they couldn't find enough Indians in the area to hire, since of course American workers aren't good enough to meet their high standards...
TFS cites this as an exception, because it's not mudslinging. I'm sure it woudn't take more than two fingers and three minutes to find plenty of business-as-usual campaign rhetoric. Not that I'm going to go look — I get quite enough of that already.
Actually, I wouldn't have been surprised if they had come out with a blanket "no benchmarking" clause (except that it's difficult to benchmark an entire O/S). That's pretty common with high-end software (databases in particular), so MS would just be applying it to something more common.
I don't remember exactly what the restriction is, but I think that nobody can publish TPC (the Transaction Processing Council) benchmarks, they can only submit them to the TPC, which will then make them available. So if your new system breaks the old record, you can't tell anyone until the TPC releases the information, which I'm sure just twists the guts of many a marketroid in the RDBMS and high-end server segments....
My opinion is that they are probably on a level close to Sun and its multi-million dollar R&D in pumping out Unix inovations.
Hmmmm, not sure I'd go that far. The OpenBSD group is very good at taking current (or legacy) software and improving it (often by an order of magnitude, however you'd measure that). However, I don't expect to see anything like ZFS coming from them anytime soon.
it would require all name-resolving equipment that communicates via the Internet to support both the existing Domain Name System and whatever technologies the rest of the world comes up with to replace it, in order for said devices to access the entire Internet. That's just stupid from a technological and functional perspective
Oh I don't know, doesn't sound much harder than supporting both IPv4 and IPv6. Remember, DNS is just a prorocol that works over TCP. As long as you had some kind of resolver that could translate names to IP addresses, you could use the existing network infrastructure.
Presumably a new DNS-type system would have addresses in a different format, so it would be pretty simple to look at an address and figure out which resolver to send it to. "foo.bar.com" gets sent to DNS, "com::bar#foo" goes to WNR (wack-ass name resolver). You'd probably quickly get RIAF (resolver-independent address format) so you could publish an address on your site that would get correctly translated if people didn't have access to one or the other networks (e.g., people in the EU might not have DNS available, but instead of publishing "www.google.com" and "com::google#", they could just refer to "riaf[d=com+e=google]". Not something you'd want to type in if you didn't have to, but probably OK for links.
I think that we could do or try for 1 - 10 second boot times, but it would involve "cheating." It would have to be a suspend state with something like a flash boot drive just so computer instant ons to it's previous state.
I've wondered about this, specifically how to isolate "stale" state (things like network buffers that refer to sessions that are no longer active), and if it's possible to lazy-initialize/restart hardware. You could certainly bring the system back up using a memory snapshot, but you'll also have to restart the disk, and do you really want to do that with no consistency checks? In the end, it all seems to fall apart due to multiple interdependencies among all the pieces of the system. I'm sure it wouldn't be especially difficult to design a system to do this efficiently, but I doubt it'd be easy to retrofit onto any current system.
Interestingly, this article (linked from the IIL website), reports that IndiaMART InterMESH has received mixed results from applying the vastu shastra principles to web sites. Three of five sites has reported a 60% increase in the number of hits, but two other sites saw little or no change. I'd guess it wasn't the actual visual redesign that improved hit counts, more likely it was some content updates that improved the search engine rankings of the sites in question, but who knows?
Slashdot Mirror
When I used to do LIM systems, I once did one for a UK-based pharmaceutical R&D company. The work broke down to roughly 40% effort to make the software function correctly, with the remaining 60% going towards writing code to generate the reports required by the Home Office.
My boss was actually a little upset that I used a template-based approach, as he felt it would permit the users to modify the reports on their own (denying us a revenue stream). Then he found that the users would gladly pay to have someone else keep those reports up-to-date.
Better off from a development and maintenance standpoint, yes. But I've heard that one of the biggest offenders among the "buggy third-party apps" crowd is Lotus Notes. Which is still insanely popular, even among people who have actually used it. I wouldn't be surprised if several of the most popular Windows apps (modulo Office, although who knows?) had some kind of legacy support within Vista. All it takes is one corporation like Exxon-Mobil or GM who has a business-critical app written in Frameworks or something to lean on MS and there you have it.
With all this virtualization support available now, I wonder if MS is considering just encapsulating entire installations of Windows 3.11/98/NT and running legacy apps on those. That way they could at least clean up their current codebase.
Hmmmm. So Apache, BEA and Oracle are making "huge servers" now? I wasn't aware that any of them were in the hardware business...
Here's the question that should really determine your answer: are you in a position to move? If not, then take the .NET job. Four hours of commuting each day will suck whatever pleasure advantage the Perl job might give right out. Unless they'll let you work from home at least three days a week, you'll burn out from commuting.
.NET (or whatever) shouldn't be that much of a challenge.
If you can move, though, then I'd take the Perl gig. Sure, maybe Perl won't lead to any great career advances, but I went from C to a 4GL then back to C then to a different 4GL then back to C then to VB then to Java then to PHP and back to Java (with brief stops at C++, Perl and ASP along the way). If you think the prospects are better using another technology, then pick up the tech and learn it on your own. One of the hardest things about learning web development (for me, anyway) was wrapping my head around the whole stateless-session thing, picking up Java and the DOM and HTML was pretty easy. So if you're already a web developer, learning the specifics of
Porsche AG announced it will be sharing part of the profits of its sports-car sales with local police forces everywhere. "It's well-known them por-shees is just too damn fast for us to ketch 'em, so anybody drivin' one is a CRIMINAL!", Sherrif Roscoe P. Coltrane explained.
I doubt the JVM would be really incorporated into Windows. It would probably be just like it is now, an additional execution environment. If MS does take Sun's implementation, they would be free to make the changes the made previously (the ability to work with COM/DCOM/COM+, mostly), they would just have to release those to the community. Which would probably be a good idea, as I'm sure the bulk of their changes would just be JNI calls to Windows, which wouldn't really help anyone on a non-Windows platform, but people who used them would tie their apps to Windows.
.NET and never do anything to promote Java. Hard to say...
Then again, if it was that easy they could have just released a package with a jar file and a DLL to do the same thing. So either they were really paranoid because of the lawsuit, or they were so miffed over Sun's protests that they vowed to do
Well, according to trickle-down theory, I should be getting my own loose-moral intern any day now!
Yeah, these days. Actually, I started using it in the 4.2 days, and liked the nice clean implementation (props to Sybase!). It's gotten even better since, I especially like the XML integration, which seems to be better thought-out than most others.
That said, it's hard to find many large, well-established businesses that haven't needed a database until now. Big companies have been using either DB2 or Oracle on UNIX for twenty years, and have a pretty entrenched infrastructure to support it. If the choice was
Oh, and the CRM project I was referring to occurred during the decline of PeopleSoft, around 3-4 years ago. Nowadays, most companies I work with are just using salesforce.com...
And this is the thing that blows me away to this day. I don't know how many good-sized projects I've seen[1] (e.g., division-wide CRM) that get originally proposed for
[1] More than one, probably less than five. Still...
Heh, I found just the opposite. My wife wanted a new camera (our old Lumix's shutter delay is horrendous), so I scoured the web for reviews and prices, agonized over the minutiae of color balance and lens astigmatism, and finally almost settled on the Nikon D50. Then the I did the thing that got me suspended from the Suburban Husband Association — I actually listened and paid attention when she took some pictures and talked about what she hated about our camera and what she wanted from a new one. It boiled down to:
C'est tout. She didn't care about megapixels, picture quality (the Leica lens on our Lumix is actually pretty good), macro focus distance or any of the other feature the reviews were going on about. All she wanted to do was have the ability to snap off a shot of our daughter as she leaped off the arm of the sofa to try the wings on her new Tinkerbell costume, without having to wait five seconds for the camera to boot and another three for it to focus and meter the exposure (or whatever the hell takes it so long). So I got her a Canon SD630. Near-instantaneous power-up, and a 3" LCD. She's thrilled, and loves the fact that I also got her a 512M SD card so she can shoot video clips.
Note: the article was originally titled "IT Worker Shortages All Over".
Honestly, some days it's not worth poking your head outside the firewall...
I was worried that the IT worker shortage was going to drag on for months, but now that it's all over I feel better. I just hope that my salary doesn't get cut, what with supply finally meeting demand and all...
Actually, one of the tenets of software reuse is that a reused module should not need to be tested. Testing an already-verified module wastes close to 40% of the cost savings associated with (or at least "claimed for") reuse. Of course, another tenet is that reuse consists of using a module for its specified purpose (which includes, as you mentioned, verifying that input values will be valid). What was done with the Ariane 5 software is exactly how software reuse is not supposed to be done.
No he's not, he's accusing the author of basing one of the cornerstones of his world on a faulty or falsely-justified premise. Usually, this just means that the particular reader isn't interested in willingly suspending their disbelief on the point in question. It's more akin to someone telling Tolkien "There's no such thing as elves, this whole story is bogus!"
Well, I don't know what kind of "real work" you do, but I develop software and dtrace is like a gift from above when you really need it. Even if you do have the source to everything, you can't just jump on a mailing list when the software that isn't working is stuff you wrote yourself. And even with source, sometimes it's easier to trace through some system calls to get a feel for what's going wrong than it is it meditate over the code.
I wasn't suggesting that OpenBSD needed to adopt ZFS, I was just commenting that I didn't think the OpenBSD team would engage in something as blue-sky as developing an advanced file system. Like you, I think they're more motivated by pragmatism, and would be much more likely to adopt an existing file system than create a new one from scratch.
Can you hire the current IT staff at your new clients? Or are you taking them away from a competitor? Do you guys specialize (somebody's the network guru, somebody's the server wrangler, the third guy's the application specialist)? If so, then maybe each of you should hire an "apprentice" and train them in your specialty. Consider trolling around the local community college with a decent IT program (if you have one).
One thing to do is compile a list of best practices for your shop. This will help communication among the staff (by establishing both common procedures and a common vocabulary), which is very helpful when you're trying to bring a lot of new guys up to speed. Even if it's something as simple as installing an OS on a fresh machine, document it. If you've got any common tools you use or wrote for fault isolation or troubleshooting, document them. Plug it all into a wiki or similar so everyone can access them. Communication is the key for keeping everyone in sync when you're in a rapidly-expanding environment.
Tow chimps? Are those like junkyard dogs? They sound nasty...
Heh, that was my first thought: "Hey, where are my skis? There's sure to be some champagne powder at Satan's place tonight!"
But offshore companies are the only way that state agencies can get someone who had the expertise and the ability to get the job done on time, on budget and provide the quality product that we were looking for... (scroll down to "Kitty Pilgrim reports", about halfway down the page). The Nebraska Dept of Labor outsourced the development of their new unemployment insurance system to TCS America, one of the largest consulting firms in India. TCS, hot off botching a similar job in New Mexico, was supposed to write the software on-site in Nebraska, and hire 25% of their work force locally. I think they were later released from that requirement because of the "difficulty finding suitable workers". What that means is they couldn't find enough Indians in the area to hire, since of course American workers aren't good enough to meet their high standards...
TFS cites this as an exception, because it's not mudslinging. I'm sure it woudn't take more than two fingers and three minutes to find plenty of business-as-usual campaign rhetoric. Not that I'm going to go look — I get quite enough of that already.
Actually, I wouldn't have been surprised if they had come out with a blanket "no benchmarking" clause (except that it's difficult to benchmark an entire O/S). That's pretty common with high-end software (databases in particular), so MS would just be applying it to something more common.
I don't remember exactly what the restriction is, but I think that nobody can publish TPC (the Transaction Processing Council) benchmarks, they can only submit them to the TPC, which will then make them available. So if your new system breaks the old record, you can't tell anyone until the TPC releases the information, which I'm sure just twists the guts of many a marketroid in the RDBMS and high-end server segments....
Hmmmm, not sure I'd go that far. The OpenBSD group is very good at taking current (or legacy) software and improving it (often by an order of magnitude, however you'd measure that). However, I don't expect to see anything like ZFS coming from them anytime soon.
DTrace, though, hmmmm, maybe...
Oh I don't know, doesn't sound much harder than supporting both IPv4 and IPv6. Remember, DNS is just a prorocol that works over TCP. As long as you had some kind of resolver that could translate names to IP addresses, you could use the existing network infrastructure.
Presumably a new DNS-type system would have addresses in a different format, so it would be pretty simple to look at an address and figure out which resolver to send it to. "foo.bar.com" gets sent to DNS, "com::bar#foo" goes to WNR (wack-ass name resolver). You'd probably quickly get RIAF (resolver-independent address format) so you could publish an address on your site that would get correctly translated if people didn't have access to one or the other networks (e.g., people in the EU might not have DNS available, but instead of publishing "www.google.com" and "com::google#", they could just refer to "riaf[d=com+e=google]". Not something you'd want to type in if you didn't have to, but probably OK for links.
I've wondered about this, specifically how to isolate "stale" state (things like network buffers that refer to sessions that are no longer active), and if it's possible to lazy-initialize/restart hardware. You could certainly bring the system back up using a memory snapshot, but you'll also have to restart the disk, and do you really want to do that with no consistency checks? In the end, it all seems to fall apart due to multiple interdependencies among all the pieces of the system. I'm sure it wouldn't be especially difficult to design a system to do this efficiently, but I doubt it'd be easy to retrofit onto any current system.
Interestingly, this article (linked from the IIL website), reports that IndiaMART InterMESH has received mixed results from applying the vastu shastra principles to web sites. Three of five sites has reported a 60% increase in the number of hits, but two other sites saw little or no change. I'd guess it wasn't the actual visual redesign that improved hit counts, more likely it was some content updates that improved the search engine rankings of the sites in question, but who knows?