Slashdot Mirror
The Economist Tackles Complexity in IT
yfnET writes "In recent weeks, The Economist has run a number of articles addressing the ever-increasing complexity of software systems. The magazine, with typical Economist wisdom, casts an eye towards past human endeavors for lessons on how today's IT industry can succeed in dealing with complexity. As part of last month's extensive survey of information technology (see Related Items sidebar), the magazine offers insight on the limits of real-world metaphors, the perils of managing a rat's nest of obsolescent systems, and the need for 'disappearing' technology. And hitting newsstands just today is an overview of development models for increasingly large and unwieldy software projects. Among other things, this article compares the open source model to Microsoft's efforts using a quasi-open license. It also describes the 'agile' programming movement and its potential to keep even the most gigantic of projects under control."
Powerful Languages Like Smalltalk, and Lisp help one handle complexity.
Aren't these the same people who tried to tell us that "comparative advantage" meant we should give up our manufacturing and all get degrees in high tech? Why should we believe ANYTHING they have to say after they've lied to us for so long?
Economics is a religion- and a failed mythology at that. Economists need to learn to examine and reconfigure their basic axioms before ANYBODY should ever listen to them again.
SJW: a person who perceives an injustice, and while correcting it, commits a greater injustice.
There's a serious problem with agile methodology and outsourcing (I didn't see any articles on Economist.com related to outsourcing, but may have missed it as I gave it only a cursory look).
Large and unwieldy projects benefiting from agile methodologies? Yeah... when you have easy communication between the "customers" (business partners) and your IT staff.
How does that happen when your developers are thousands of miles away, in a different timezone, with a totally different culture, and don't speak your native language?
Diplomacy is the art of saying, "Nice doggie!" until you can find a rock.
Translated from a a french newpaper: :-)
I'm not sur it will help...
"The historical operator, his Orange subsidiary company and Bouygues Telecom have all three known, in the last seven months, of the significant breakdowns, depriving their customers of telephone service for durations varying from a few hours at almost two days.The succession of these events clarifies the increasing complexity of the systems of telecommunications and the difficulties raised by the needs, news, of interconnection of many heteroclite networks". A minister in the french government as asked for a study about all these failures
le souvenir d'une certaine image n'est que le regret d'un certain instant (M.Proust)
Hardware complexity can be reduced with homogenous environments.
Good planning, documentation and standards reduce complexity in software.
How much more do they have to complicate the issue?
That last link is premium content. Anyone have a bypass? Apparently I can't even sell my soul for it. (Just as well, they'll have to timeshare with the NY Times.)
One line blog. I hear that they're called Twitters now.
People tend to make things over complex if you let them. Bosses want that technology they read about in whatever PHB read these days. Such and such wants some TLA to do some other TLA.
I know it's hard but you have to tell them that these things don't add any value in and of themselves. You want the simplest possible system that will solve the problem at hand. Really, nothing more, don't implement something because you may want/need it tomorrow 'cos when tommorrow comes it won't be right (and if it is, hey, you can implement it then).
----
I'd say just hire good people and keep the managers focussed on just a few projects, but this seems to be beyond the capabilities of most companies. So you end up with programmers who write Java like it was fortran and managers who juggle so many projects that they barely know your name much less what you do for the company. There doesn't seem to be a fix for this which companies will be willing to accept.
I'm trying to teach myself to set people on fire with my mind... Is it hot in here?
This push to make [for the user] simple what is after all increasingly complex, can only hide, not eliminate the role of the nerd class, a role the article disparages because nerds are presumed, as the inventors, to have foisted off complexity on the unwitting public. Was it Heinlein who said that "any sufficiently advanced [or was his word complex] technology is indistinguishable from magic"? The wish, on the part of typical users and marketers, that all the wonders of our age and those ages coming next should all just work like magic will in fact only ADD the complexity of UI technologies that are good at hiding the guts of the systems we depend upon. The the engineers and technicians will be as needed as ever and get even less sympathy from a public that never sees directly what it is that the "nerds" are doing for them.
SLASHDOT: news for people who can't concentrate on work or have no life at all and got tired of yelling back at the TV.
Complexity in IT isn't going to go away. In fact, I'd argue it is a necessity. There are some tasks that simply require complex systems and those complex systems require complex data and/or complex user interfaces.
"The market alone cannot provide sufficient constraints on corporation's penchant to cause harm." -- Joel Bakan
So a lot of this space was spent explaining to Joseph P. Siquespack, Esq. what a "protocol" was and the like, but there were two points in here that I'm really glad my great-grandboss might be reading:
Neither of the above are impossible goals! They can be done with a little thought and elbow grease. And the great part is, they're probably already being done! Next time you're reading over your IT department head's recommendations for a project, call them up and ask WHY. You might be amazed at how awesome the answer is, and it might even persuade you to put away the "my way or the highway" stamp.
adam b.
Sure, information technology is becoming ever increasingly complex; but it is also a great opportunity for companies that have the management able to deal with that complexity to excel and outperform their competition.
What we should do is to terminate all trade (including goods and services -- which includes labor) with mainland China. Ditto for India and Mexico. Until the Chinese, the Indians, and the Mexicans commit to free trade and Westernization, we do not trade with them.
Further, we deport all the H-1B workers.
The links I tried resulted in a page that announced I was trying to access "premium" content, and then asked if I wanted to subscribe for a month (~$20) or a year (~$90). In other words, that was a dumb thing for you to do, wasn't it?
As long as corporate politics, and the associated middle management war games, as well as managers that can't see beyond a programs graphical user interface (especially the details..), keep on dominating it development, the software systems mess will be uncontrolable. The more complex these systems become, the more problems will be created regardless of the technology and/or development methodologies that are being used. Untill the day that the people in charge finally start facing reality instead of living in a dream world nothing will change. Of course the fact that there are a lot of cheap and worthless analysts and developers out there promising these people heaven with any new proposal (and eventually delivering hell) doesn't help either.
Engineers love to break things into smaller parts, each part serving one and only one function, like pulleys, shafts, rotors, etc.
For really effective design each part has to serve multiple functions, like evolution is able to do: The human mouth can be used to eat, breathe, talk, etc etc.
That's why a robot can't compete with an animal- In a robot each part usually severs only one function, making the machine inefficient as a whole.
This problem is just magnified in computer software and will only get worse unless engineers start changing their tune. I think the worst offender of this philosophy is object oriented programming: It's the ultimate embodyment of this philosophy- Most big object-oriented software have only about 2-3% of code that performs any real work, with the rest only is window dressing to fulfill the engineer's urge to "modularize".
The best software I see seems to be written either in more pragmatic procedural styles, or uses better mathematical underpinnings for its structure, like you'll find in functional programming languages (Haskell, Lisp, etc.)
My apologies for living up to my user name!
Conrad Barski
The problem with trying to compare the software industry to other endeavors of human experience is that in the realm of the computer/electronics industry that most software developers are dealing with, you are near the pinacle of dealing with abstraction of complex systems. While there may be other very complex systems that on a large scale can compare to some computer networks or CPU designs, computer science is the practice of dealing with abstractions on many levels, sometimes simultaneously.
Indeed, electronic state machine digitial computing devices (also called computers) have proven so successful, with the software abstraction dealing with the various levels of abstraction, that they are used in the controlling of other complex systems, from air traffic management, urban water system management, freeway traffic monitoring, and law enforcement dispatching. You've seen them, and they are out there.
Some very talented engineers have done a surprisingly good job of simplifying the tasks and reducing the abstractions to the point that all you need to do for the most part is plug it in and watch the gizmo do its thing. What this article in the Economist seems to be doing is complaining that the job isn't finished, and that complexities in setting up a computer system for some project is more difficult than it should be. That is primarily due to the fact that the author is using products that don't comply with standards (a real problem if standards don't exist yet for a certain concept or technology) or they are using the wrong tool for the job, like using a hammer to put in a few screws. Sure, it will work, but it is aggrivating and sometimes takes quite a bit longer to get the job done, and can damage things around it as a side effect. How many software/electronic gizmos out there do you know get used like that?
While I'm willing to acknowledge that I don't know everything there is to know about the management and organization of complex systems, I would be more inclined to get the opinion on such a subject from a computer programmer than from a plumber.
Anyone who claims that it's the engineers who are responsible for "feeping creatureism" evidently has no real knowledge of product development, or is a self-serving marketing drone.
All my programs would have a great interfaces if they only had 1 boolean variable in it. A light switch is very simple. But try connecting it to the internet, email a friend and do your accounting with a light switch.
Ever heard of BugMeNot?
To paraphrase oo doesn't make messy code, messy people who mis-use oo make messy code.
I think the main problem with what, aparently all of us, have seen with oo code, is the universities. The coding style that is taught in universities makes for really ugly, unmaintainable code.
If you cram 100 abstractions and modularization into a project in university, you get an A and every one says how clever you are to have used all of these features in your project. Do the same in the real world and you are left with unmaintainable blech.
People have to learn that the various oo features are there to help them simplify their code, not to make it more complex. If using a particular modularization technique, say an interface, doesn't remove complexity then don't add it.
Another really bad thing that people do is add code for some unspecified future purpose. Maybe they are creating a class that does some math, they need an add method and a subtract method, so they think, what the hell, I'll add a multiply and divide too. Why? All this does is make the code less readable. Never implement anything that you don't need right now.
I find I can write more (internally) complicated programs because of OO. OO-based design patterns, such as Model-View-Control for interactive programs, are a God send. I can pull off computational tricks that, without OO techniques, would make an incomprehensible and unmaintainable mess of the code.
And by making things more powerful internally, I can write applications that are easier to use. The Economist articles are correct on that point.
The fact that you even have it out for modularization, which predated OO by decades, makes me glad I don't have to deal with your code.
Or maybe I do. Now there's an ugly thought.
I don't know what you are talking about.
You very easily can have many different input fields for a web page.
All interfaces are not web pages.
"the advantage Lisp and Smalltalk have that other languages lack is that their syntax is so simple that extensions meld in as if they were part of the language."
The same could be said for Forth.
You don't understand modularity.
If you have layered software that means that you leave hooks in to allow for easy expandability later. That doesn't mean that all layers are actual code,but can be just proxies for a possibility.
Have you done any real development with complex systems? If so then you would understand that an engineer makes things as modular as they need to be and no more. But they design so that they can replace with better technology later.
Could you really make a motor out of a solid block and machine it? No. So you have pieces like the carberator and the oil pump.
Software is the same.
You don't know what you are talking about
>All my programs would have a great interfaces
.jpg. Which one do you use without thinking in the slightest? Which one does your grandmother feel more comfortable using?
>if they only had 1 boolean variable in it.
You're setting up a straw man.
Think clicking an image in Firefox vs. running command line FTP to get a
Do you think the user cares if the tool is written in Smalltalk, C or VB?
Wow, how about moderating flamebait. All the world's software SHOULD be written in PERL.
*sigh*
This is why your job is being exported to India. It's easier to stomach incompetent development for pennies on the dollar.
I hear this every damn day. 'We need to make it simple', 'It is a simple service' or 'It is a low option service'. This may work fine for the sales and marketing drones that make their commission off selling unnecessary services to uninformed customers, but as long as there is _choice_ out there, the backend is going to be complicated, and somebody, somewhere is going to have to known how all of it (or at least a major part of it) works.
Try all you want, but unless the entire IT industry decides to switch to one massive global device that we all plug _directly_ into, you can't make video conferencing/VOIP/disaster recovery/etc through 2 LANS, 3 Service Providers and 10 different security layers a 'green/red' push button operation.
I've gotta go get drunk now....
-- I care not for your foolish signatures.
"If an application uses several layers and it screws up there has to be a way to trace and find out what happened.
Perhaps a new opensource protocal could help? I like that idea."
I dub thee, XML.
Just noticed they got an IE extension also.
> This is why your job is being exported to India. It's easier to stomach incompetent development for pennies on the dollar.
I think my statements about parts serving only one function versus multiple functions was arguably, legitimately, interesting and on-topic for this discussion, Anonymous Coward.
--Conrad Barski
> That doesn't mean that all layers are actual code,but can be just proxies for a possibility.
.NET, it has 18 levels of inheritance above it, all designed for these kinds of hooks. Are you able to break the concept of a button into 18 separate levels? I can't. Do you think this is a good design? I don't.
If you look at a button control in the latest iteration of
> an engineer makes things as modular as they need to be and no more. But they design so that they can replace with better technology later.
If that was really the case, I think that line of reasoning would be justifiable. Personally, though, I find a lot of the complex software written that I have to deal with is filled with good modularization intentions that would have been a lot less problematic if people had thought to themselves "Don't write what I don't need right now, today."
> You don't know what you are talking about
You make some good points. Why follow them with attacks?
good points.
I like how this post instntly soawned a flmewar pn economy, mostly fueled by people that either don't understand
1. that The Economist is a magazine, not the people referred to in the sentence "economists say".
2. anything about economics.
3. both.
The Economist is a very good news magazine full of reasonable articles and opinions, in all senses of the word "reasonable". There is not enough praise on Slashdot to make it justice. You should all subscribe, assuming that you are interested of knowing what happens in the world from a political, economical and yes, technical standpoint.
Code for rm could be implemented in C with handfull of lines. Todays alternatives take thousands of lines of code, but to an end user the second alternative is simple. User doesn't have to know what the commands are, just toss the file away, as you would with solid objects.
So there we have it, simple problem becomes complex from implementation point of view. I once had a customer who joked when I delivered them a new system that calculated the price and basic layout of the systems they were manufacturing, that inspite the fact that now it took less than tenth of the time to do the calculations, that we could still improve it so it would do the calculations when he pressed a button while thinking of something else.
It should also be noted that what was impossible few years ago is now possible, because of improvement to hardware. This adds to the layers of complexity, because implementers can actually use modular approach instead of optimizing at lowest possible level.
Main problems in IT in the US:
1. Companies have gotten to big
2. Companies try to centralize everything, instead of delegating duties to competent people (this practice also encourages hiring incompetent people). Competent IT workers are also very unhappy when they hands are tied by bureaucratic BS
3. relating to (2), companies don't give raises or benefits anymore, which causes competent / motivated workers to hop jobs to get increases in pay
4. People doing all the "centralizing" are ignorant of standards
-
Interface specifications dominate If it doesn't work the way the spec says it does, fix the box, not the spec. If A won't talk to B, run the tests to check compliance with the spec. If you can't tell who's at fault, the spec is broken.
This is why you can swap a Pratt and Whitney engine for a Rolls Royce engine.
-
The buyer, not the vendor, decides what is a "defect". One of the fundamental problems in IT is that vendors have sole discretion to decide what is a defect and what isn't. That doesn't fly in aerospace.
-
Fix blame. In aerospace, people get blamed for screwing up. You do not want your name or the name of your company to appear in an NTSB crash report. If you screw up big time, it will.
Mistakes in aerospace are publicized. There's an NTSB database of 140,000 crashes. If it was a hardware failure, the vendor is named.
-
Warranties have real meaning Airplanes come with good warranties, and so do all the parts that go into them. Commercial software doesn't.
This runs engineering costs way up, and the life cycles are longer, but in IT, most of the commercial products are sold in large numbers, so you get to spread that engineering cost over a large number of items.It's time for computing to grow up and accept this kind of discipline. The automotive industry had to accept it in the 1960s, and cars got much better within a decade.
Eh... unconvincing. Organic systems are great if you don't mind a little fudging of the numbers. For example, human memory is notoriously unreliable. Combining features tends to create single points of failure and organic systems have a very narrow optimal operating environment. Try dropping the temperature by a hundred degrees or so and see how you fare. The lack of validation, if you will, on inputs, creates a significant susceptibility to viruses, bacteria, food poisoning and the like.
There are plenty of successful organic systems, but there's a whole lot more failures, which you don't see, because they are all dead. And failure is pretty expensive.
Also, you aren't taking into account the fact that complex computer systems perform sophisticated analysis on other complex information systems that organic systems are ill-equipped to handle. They are really best at operating on other organic systems. So if you were to design organic financial software, it would work best on an organic financial system. When you want near-instant, reliable, repeatable and accurate information, an organic system won't work for you at all.
Its like the difference between designing a bridge over a river, and waiting for a rockslide to fill in the river.
"It's Dot Com!"
From the metaphor FA:
Really? I heard that 87.4% of all statistics are made up on the spot ... I just have to wonder if this one is part of that majority...
Robots haven't been competetive with humans (on complex tasks that require closed-loop control) because we lacked manufacturing capabilities at the scales required, small and dense power supplies, and computer power (both for 3D sensing and for path planning). For 3D sensing (such as image processing and inverse kinematics), the theory is complex but the code is not. For path planning, the code is currently complex, and will remain so as long as we are trying to emulate humans. But if you believe that low-efficiency, infrequently-used code is a hindrance to complex tasks, don't look at human DNA! Less than a percent of it appears to be functional. Much of it appears to be present by chance or to have survived because of some rare selection event in the distant past.
"It is very frustrating to have both automagical type and have operators that do radically different things depending upon type."
The HP48 series calculators did that: number + number = number. string1 + string2 = string1string2
It's called operator overloading, and used judiciously it can be a big help. It can also help in keeping bad things from happenning that a strongly typed language might not. Remember that Slashdot story about the airport that had it's communications knocked out because one of the computers wasn't regularly rebooted? The problem basically was of an overflow nature. Smalltalk would have been able to handle it because instead of overflowing, it graduated to a type that could handle the larger values. That kind of behaviour is part of being "robust", in a world that's not "strongly typed."
http://shit.slashdot.org/article.pl?sid=04/11/26/1 853231
Your billing, CSR and provisioning apps are each different because they serve different functions.
But you can have two billing apps that serve the same function, yet have completely different interfaces and data structures. That's because the programmers had different approaches and priorities when writting them.
It's the vi vs emacs debate all over again. We can't even settle on one "standard" for an editor.
From "The Zen of Programming":
Hearing a disturbance, The master programmer went into the novices cubicle.
"Curse these personal computers!" cried the novice in anger, "To make them do anything I must use three or even four editing programs. Sometimes I get so confused that I erase entire files. This is truly intolerable!"
The master programmer stared at the novice. "And what would you do to remedy this state of affairs?" he asked.
The novice thought for a moment. "I will design a new editing program," he said, "a program that will replace all these others."
Suddenly the master struck the novice on the side of his head. It was not a heavy blow, but the novice was nonetheless surprised.
"What did you do that for?" exclaimed the novice.
"I have no wish to learn another editing program," said the master.
And suddenly the novice was enlightened.
Oh, I'm with you on the operator overloading bit. I think it's a small issue of appropriate overloading.
For instance:
Doesn't make quite as much sense to have string + string = concatenation. Just something I've noticed.
Good point about robustness, though.
you can't slam modularity w/o being self-inconsistent.
why? because to slam something means to put yourself outside of it in the first place (and pour invective on it in the second). putting yourself outside of something (drawing a boundary around it) is the essence of encapsulation, which is one of the techniques of modularity.
so, let's be charitable and assume you don't want to slam modularity full-force, but rather some of the other techniques for modularity that often find themselves badly understood and/or badly implemented.
btw, lisp is modular, object-oriented, etc etc, too.
as for the invective you poured, not bad. just don't waste it in the gutter the next time 'round.
PHP uses a period rather than a plus to concat strings. That is the smart way to do it so you always know if you use a plus, you are adding numbers.
1. Scrum, as an example of an "Agile project shell" can be quite adept at large projects. Largest one I've heard of had about twenty teams. At my recent Scrum training, one organization were finding creative ways of implementing agile processes across two continents and three major centres. You still get value from "Agile" at a distance, but it's less valuable. However, large, monolithic defined processes, such as RUP, waterfall, etc., don't work, imperically. They come in late, with less functionality than planned for, and over budget. Can they work? Yes. Where there is increasing complexity, agile approaches are better.
2. Name me a method that works when your developers are thousands of miles away, in different time zones, with a totally different culture, and who don't speak your native language. What's that? They all suffer? In that case, why don't you go for the ones that manage complexity the best. Impirical processes (short cycles, lots of measurement, feedback cycles) work better than defined processes (requiremenents fully articulated and not changing, can be completed before the requirements are out of date, all your developers are of the same (expected) quality, all your communications are relatively noiseless, misunderstandings from documentation are within small error tolerances, etc.)
Guess what. You might complain about agile not working at a distance, but by normal success metrics, (time, cost, quality, function) traditional methods don't work either.
Having delivered software with both kinds of processes, I can tell you, you get far more control and corrective ability with an agile process like Scrum than with RUP on a two-year project... and you deliver business value faster.
3. You can mix and match. Agile approaches are approaches. You can, though you incur a cost, take some agile practices and use them in a non-agile context. You can take RUP practices and use them in an Agile context. You don't get all the benefits of either, but in your specific context, you can get the benefits you choose. Each project is different, and no method properly addresses every project's particularities. But the more adaptable the process, the better. This can be either the way RUP does, by having so many steps and layers that you can choose what you need, and "lighten" it by scraping out the rest. The other approach is to have a lightweight agile management framework that is powerful because of it's feedback systems, and doesn't make presumptions about steps in the same way as a larger process. The power from the latter comes because you get constant feedback of progress in a way that is impossible to relate to business value in a more monolithic process. Because you can relate it to business value, you can constantly be tweaking the project to match the business interests.
Pick your poison, but don't toss out the baby with the propaganda.
i - This sig provided by
... which is a magazine, and not to be confused with the profession. In fact, I know of a few newspapers with that sort of name: The Economist and Sun, the Economist-Tribune.
Newspaper and magazine names are usually historical, and the words they contain have often changed meaning. For example, there is a local newspaper north of Toronto called "The Liberal". This newspaper has, for example, no ties to the Liberal Party, is not particularly philosophically Liberal (whatever that means), and is basically a local community newspaper that was named almost a century ago {more or less). What's in a name? Often, when it comes to media titles and tag-lines... nothing.
Just rememeber, "fair and balanced" is the tagline of Fox News.
i - This sig provided by
bug me will get you in
I think the problem is usually implicit type conversions rather than operator overloading. In particular, allowing almost-random conversions between a concrete type such as a number and a string form is just asking for trouble: it's great for quick 'n' dirty scripts, but a child's toy in a grown-up world for serious projects.
If you disagree, post your argument. (-1, Overrated) isn't your personal censorship tool for views you don't like.
Then maybe you should have prototyped the argument types so they would be checked or converted as appropriate. Dynamic typing does work (in some languages better than others); you just have to learn to use it right.
now we need to go OSS in diesel cars
you're right- I'm regretting having used the word "modularity", because it is almost impossible to slam such a basic concept... :)
Admittedly, very eloquent retort.
"In particular, allowing almost-random conversions between a concrete type such as a number and a string form is just asking for trouble: it's great for quick 'n' dirty scripts, but a child's toy in a grown-up world for serious projects."
The thing is, proper operator overloading isn't almost-random. There are clear rules as to what's allowed, and what isn't. For example:
number +number = number is OK.
number + GROB (Graphics Object 'image') would not be OK, and generate an error condition.
Also one needs to keep in mind that strong typing is to protect the programmer against themselves. However the world we live in isn't stronly typed. Making our languages more flexible in dealing with that world while hiding the complexity of implimentation (complexity doesn't really "go away". It's always there, somewere) is the best compromise (until the underlying machine changes to a more organic model) we presently have.
All right, all right...
I admit my post was a bit rantish and clearly there are good uses for OOP and modularity in many instances. My point is that the underlying philosophy of deconstructionism in engineering needs to be kept in check sometimes, because it can run counter to the overall goal if applied to blindly. -Conrad
"But businesses have standards. All you are doing is adding another language that needs to be supported which would make matters worse."
And yet Fedex handle it quite well. And apparently others likewise have no trouble.
Just because your having difficulties dealing with Lisp and Smalltalk doesn't mean others are.
Complexity may be inevitable, but for a moment lets put on our economist hat and look at what motivates companies to tolerate complexity. While some complexity in inherent, I believe that a lot of this complexity is often accidental, sometimes born out of naivete and occasionally deliberate. However, at the end of the day companies are not in a big hurry because the added cost of complexity often doesn't cost them anything, it is merely added to the cost of doing business and paid for by the consumer.
Furthermore, in some cases complexity can be quite convenient. For example, in my city the gas company outsources its billing so that one company provides gas and the other bills for it. Sounds great, but when ever there is a discrepancy between the bill and the amount of gas you've used, good luck trying to get something resolved. In my case the gas had long been cut off but the billing continued. After fighting through automated telephone systems to speak to real live people they quite happily point the finger at the other guy and leave it at that. In effect the complexity that they have engineered into their business model provides them with a very handy excuse for inaction.
Perhaps you expect capitalism and free markets to address this problem? I think that the much of our technical infrastructure is so monstrous and well entrenched that, just like your local utilities, there no room for upstart firms to enter the market. For the consumer (end consumer or other business entities) there are very few legitimate choices. More often than not I think that complexity is often just another excuse to maintain the status quo and an excuse that is held up to 'educate' consumers not to expect any better.
The feeling of religious awe when looking at something with the beauty and complexity of DNA is the same regadless of your belief in God or lack thereof. Assigning "miricale status" to DNA gets logically messy since if DNA is so "special" that God must have developed it, then who developed the "special" God. The usual answer is that God's simply states "I am", but if God "just is" then why can't the Universe "just be"? Scientific dogma says nothing about the existence of God. It's just that as far as explaining how the Universe works, God is considered irrelevant.
And did you exchange a walk on part in the war for a lead role in a cage? - Pink Floyd.
It's so much about modularizing things as it is to abstract them. I don't want to code a data access layer for each and every projects. And if I need to change from one DBMS to another, I don't want to adapt large chunks of code. Same thing for transaction management, concurrency, physical-conceptual mapping, exception handling, security and authentication, etc. etc. etc.
.NET, Mono, whatever. And actually DO it and do it RIGHT.
The big problem here is that most people fail miserably when trying to make those abstractions. It's not the kind of thing that you design and code in one night, one week or even on month. It takes time, experience and lots of thinking before choosing between different avenues.
Granted, for small projects and for specialized applications, that might not be necessary and you could probably do away without abstracting stuff because you don't have that kind of constraints. But the vast majority of code written out there do need it. And even in my limited experience, I have seen that kind of design/code written over and over and over.
Maybe someday someone somewhere will get the idea that it would be very useful to have a full-fledged n-tier/service oriented architecture build right into the framework, be it J2EE,
Intelligence shared is intelligence squared.
I have worked in a Telco on a large job dispatch system and what the parent says is spot on. I found that the middleware was indespensible since various changes would impact the transaction formats. The middleware was used to translate transactions into various versions so that upgrading the whole system could be managed one app/sub-system at a time. I don't know of any simpler way that gives you the flexiblity to "evolve" a system regardless of project deadlines being met or not. However the "evolution" by default gets more complex since you can easily add a field to the end of a transaction but it is difficult to remove them. Instead people tend to recycle old fields, ignore them or do something "clever" like variable length field with sub fields, all sorts of arcane rules and band-aids some going back 20 years! A Telco system keeps it's history in its transaction formats (kinda like DNA). Even though the original app may have died 3 generations ago the ability to "talk" to the system is dependent on the original transaction formats. This is because replacement systems are required to "fit in" with the existing formats either directly or by using middleware. In effect a new app (or the middleware) inherits the DNA from the app's it replaces or talks to. It then starts adding some of it's own mutations.
And did you exchange a walk on part in the war for a lead role in a cage? - Pink Floyd.
I thought "You don't know what you are talking about" was a good point.
And did you exchange a walk on part in the war for a lead role in a cage? - Pink Floyd.
please advise!
"Cell Phone UI Problem #1: Do you type in the phone number, THEN push the call/green/phone/whatever button ... or do you push the call/green/phone/whatever button, THEN type in the phone number? "
The first is better. Why? Well entering the phone number first gives this advantage. You can take your time entering long phone numbers, make changes, and so forth before an inconvienent (and possibly costly) commitment. Remember the little things add up over time.
The link to large and unwieldy software projects is classified as PREMIUM CONTENT which, unfortunately, I, and many other people, do not have access to.
Can anyone please share it with the rest of us ?
Thank you !
Muchas Gracias, Señor Edward Snowden !
Your overall point is good - engineers and phsyical scientists prefer reductionist modes of thinking, which have their limitations (see The Hedgehog, the Fox, and the Magister's Pox).
But your example actually shows how overloaded design can introduce new problems. Having one mouth (throat, actually) resposible for all three functions is why we can't talk while we eat and why we can choke to death. Surely a better design is to have separate GI and air tracts.
Suboptimal design like this is actually the strongest evidence for evolution. No sane designer would overload functions so foolhardily. But evolution has to adapt what's already available; there is no clean slate to start from.
Democracy is two wolves and a sheep voting on lunch.
You are talking about quality, safety and accountability not complexity. Planes could not do all the things they do today without robust software and computer systems to handle the complexity. Are you seriously suggesting that general purpose software should be developed with the same rigour (and associated cost) as general purpose aircraft? There are more planes than there are air traffic contol systems so the argument about "sold in large numbers" is rubbish unless maybe you were thinking you could use Windows to manage the traffic at Heathrow. In the area of nuclear reators some installations of QNX have been "up" continuosly for over 10yrs, can you say the same about one of your planes? But the biggest mystery of all is why banks and governments would trust the western worlds economy to the undisiplined IT department. I can't begin to count the number of times "management" or "engineering" tries to tell "IT" how to deal with "complexity" when very few of them have a fucking clue as to what complex means. Complexity is not about the amount of paperwork and it cannot be solved by stringent testing, regulation and accountability. One very complex but common problem is "assignment". How do I allocate resources? The problem is relevant to such diverse applications as airline reservation systems, dispatch systems, economic modeling and University timetables. In many cases the magnitude of the complexity is such that the optimal answer can not be found within an acceptable time-frame (say within this century) so there are various methods used to get a "good enough" answer. It is also interesting that for smallish systems this sort of complex problem is a snap for the human brain as is another complex class of problems called the "the travelling salesman problem". I am not saying it is impossible to find a better algorithm for either problem but it will be an exceptional maths genius not an engineer who finds it. So before we waste anymore of each others time I suggest you read up on the beautifull simplicity of Alan Turing's genius and get back to us.
And did you exchange a walk on part in the war for a lead role in a cage? - Pink Floyd.
Managing complexity
Nov 25th 2004
From The Economist print edition
Most software projects fail to meet their goals. Can this be fixed by giving developers better tools?
ON SEPTEMBER 14th, the radios in an air-traffic control centre in Palmdale, California shut down, grounding hundreds of flights in southern California and Nevada, and leading to five mid-air encounters between aircraft unable to talk to the ground controllers. Disaster was averted because aircraft managed to communicate with more distant back-up facilities. But why did Palmdale's radios fail? A glitch in the software running the system meant the computers had to be re-booted every 30 days, and somebody forgot to do so. But software running a mission-critical system should not have to be restarted every month. The culprit: poor design.
At least Palmdale's software worked some of the time. The same cannot be said of an $4 billion write-off that America's Internal Revenue Service (IRS) had to swallow when a multi-year effort to overhaul its computer system failed completely in 1997. And such problems are confined neither to governments nor to America. A £456m ($844m) project for Britain's Child Support Agency came in over a year late, and has failed to deliver payments to more than half of eligible applicants.
As software has become more and more pervasive in business and government, and more complicated, the impact of poor software design has been steadily growing. A study earlier this year by the Standish Group, a technology consultancy, estimated that 30% of all software projects are cancelled, nearly half come in over budget, 60% are considered failures by the organisations that initiated them, and nine out of ten come in late. A 2002 study by America's National Institute of Standards (NIST), a government research body, found that software errors cost the American economy $59.5 billion annually. Worldwide, it would be safe to multiply this figure by a factor of two. So who is to blame for such systematic incompetence?
Cost overruns and delays are common in numerous industries--few large infrastructure projects, for instance, are completed either on time or on budget. But it is peculiar to software that billions of dollars can be spent only for nothing useful to result. At a very basic level, it is the fault of the software engineers who are writing the programs, and of their bosses. Even companies that specialise in software development suffer from delays and overruns. An obvious example is Microsoft: its "Longhorn", the long-heralded successor to its Windows XP operating system, was originally scheduled for launch this year. Longhorn is now not expected before mid-2006, and many of its key features have been put off until 2007.
The prevalence of such failures can be explained by one startling weakness: the tools available to software developers. As software projects have become more and more complicated, it has become impossible for even the most talented team of programmers to keep track of the millions of lines of "code" required. As long ago as the 1980s the industry began to rely heavily on software-development applications--basically, software that helps write software, for example by creating reusable modules that form part of broader processes. The problem is that these have simply not been up to the task. As a report in May by Forrester Research, another consultancy, succinctly put it: "Corporate software development is broken."
Dale Fuller, the boss of Borland, a software-development company, agrees. He also thinks he can fix the problem of weak tools. So does John Swainson, long in charge of software development at IBM and now bound for the top job at Computer Associates. John Montgomery, who runs such things for Microsoft, does not think the situation is quite so bad. However, he believes Microsoft has what it takes to "commoditise common problems" and so enable average software developers to write above-average programs.
If they really are so powerful why haven't we seen more free/OSS code, modules, programs written in these? If one Lisp programmer = 5 to 50 programmers in other languages[1], we should be seeing more stuff from them eh? If one Lisp programmer is only 2 x other programmers in output, then erm thanks but no thanks...
Look at the "software ecosystems" of Perl (e.g. CPAN), Python, PHP, Java etc.
Compare them with Lisp/Smalltalk. Even Ruby (a newcomer) appears to be doing better.
You have the top perl coders whipping out weird and funny modules/programs (and sometimes even weird AND useful modules/programs) in their _spare/idle_ cycles.
You have the top python coders whipping out free code (for fun?) too. Examples: spambayes, bittorrent, Zope, Plone.
What happened to the Lisp programmers? No spare cycles? Where's all that alleged power I keep hearing about?
Maybe languages like Smalltalk and Lisp aren't that powerful - they just attract/keep a certain group of programmers (more likely to be decent/top programmers).
Most people can only keep/visualize/instantiate about 7 distinct objects in short term memory at any one time. You might want to stick to a max of 5 to cater the slightly below average. (Maybe someone might want to be a hero and try increase the population average to 10 items or something ;) ).
There are workarounds if you can easily get people to group a bunch of items as one object.
If people have to remember 7 or more important things during the learning period where short term memory is used then the thing is hard.
Given the amount of choices and options possible with software, it's going to be hard.
So the workaround is to split the learning phases to small absorbable chunks and give time between these phases for people to commit them to long term memory.
If you use common/defacto UI standards, many of the users would be familiar them and so the effective number of learning phases required drops.
Yeah, but think ahead of your immediate complexity. Imagine you could have a button that says "do just what you want me to do", that figures out that you want to connect to the internet, email your friend and fiddle your taxes.
(and I don't mean some goddam paper clip that buggers up everything you're trying to do)
Pretty hard to do because it requires... complexity.
'agile' programming movement and its potential to keep even the most gigantic of projects under control." Sounds like marketing. It always goes smoother when the business owners actually think about the business needs and define them well including functionality, process and who, what, when. Follow this up with a project plan that includes a rational look on how to get it done. Note the absence of technology so far! I/T as part of the project plan you engineer and integrate a solution where appropriate. Unfortunately too many business types think because they have installed windows XP from vendors disks it makes them a Sr Systems Engineer. But this does not work.
Suboptimal design like this is actually the strongest evidence for evolution.
It's a minor point but there is stronger evidence for evolution than suboptimal design.
I've often thought that the proponents of Intelligent Design would have a better case if they argued that suboptimal design was evidence of a conscious intelligence. If one looks at other products of engineering like cars designed without seatbuilts, padded dashboards or protected gas tanks, one could plausibly argue that organisms were designed with the same carelessness. Surely, organisms that combined breathing and eating in the same orifice would have died out by the forces of natural selection.
It is interesting to note that organic systems have solved the problem of complexity by tolerating a high failure rate. It's evidence that if intelligent design is at work, the intelligent designer is not a lawyer.
Agile works when you are local timezones. Far-flung teams can be more productive than co-located groups. Which would you rather do, share a monitor or VNC? Stretching across timezones has led offshore development to develop rich competencies in more traditional software project management. When outsourcing you can't embrace change, you have to specify against it, and contracted companies then focus on Taylorist execution. I'd suggest that developing competence in agile methodologies is the greatest advantage US teams can gain in light of offshoring. But that does not mean the team has to be in one place.
I used to agree that 2-3% thing and I did hate object oriented programming but that modularized abstraction can be made use off... ;) ;) ;)
Its when you start using the class structure, so that you won't have to write lots of code again and again in the structure. Too bad those things are not supported well in the language. Or it looks more or less cludge when I did so, but still ended up getting LESS lines of codes than doing it the way we where taught to do. One of tricks, is that in a interpreter I was writing to there was a class to handle the namespace, that class constructor simply looked all the classes inherited from operator class so that the parser, and evaluator code didn't need to know what operators there where nor how they looked for. They simply got them from the class structure and used what was in the operator class, instead of hardcoding them in the interpreter or parser
I extensively abused the class structure and the assistant loved that. BTW: There is function pointers in java so that you COULD use them but language is made it HARD to use them
I hated object orientation BEFORE I started abusing class structure as much as possible
Now I must tell you object orientation is really usefull abstraction if done right. If done like we are taught to do in universities its terrible.
Of course that was mostly optimized for impressing the assistant who is religious about object orientation, not optimized for speed and TRUE object orientation is not for code where performance is nr1 consideration.
Emacs is good operating system, but it has one flaw: Its text editor could be better.
I reckon economics is, like theology, euclidean geometry and the propositional calculus, a formal system, which doesn't really apply to the real world
Are you kidding ? Except for the really large (think much larger than earth) and really small (lookup planck scale) euclidean geometry is king.
I dare you measure a deviation from euclidean geometry in, say, a month.
Working for necessity's mother.
I remember the days when my father's knowledge of cars pretty much had to be what my knowledge of computers has to be today. You had to be a rather proficient technician to get full use out of the technology unless you had the very large amount of money needed to have someone else do things for you.
Today, cars have become vastly more reliable and almost impossible for any but the very elite to do more than minor things to. And the minor things are best delegated to service providers like Jiffy Lube or the Costco Tire Center if your time is worth anything to you. Even the professional mechanics mostly use very expensive computerized test machines that simply tell them "replace part X". They don't really understand how the cars or the diagnostic machines do what they do.
The analogy can be stretched too far, of course, but I've noticed for years how the importance of programming ability for computer users is going the way of mechanical expertise for drivers. Even professional IT work is increasingly a matter of app selection and "configuration" these days, not what we used to think of as "programming".
"Those who have never entered upon scientific pursuits know not a tithe of the poetry by which they are surrounded."
In the case of India, a large percentage of Indian families undergo targetted abortions that kill female fetuses. The sex ratio at birth (SRB) of males to females is now 1.20. The normal SRB is 1.05.
Western nations like the USA and Japan have a normal ratio.
By the way, the parent article is deceptive. The bigotted writer claims that India has all the good features of the West. If this claim were true, then the SRB would be normal. Wife burning would be nonexistent. Further, the quality of life in India would be as high as the quality in the USA, Australia, and Japan.
1955 Japan had a higher quality of life than 2004 India. You be the judge.
As the grandparent article noted, we should terminate all trade with India, Mexico, and China until they Westernize. Westernization includes the adoption of Western values.
Remember East Timor? Remember the butchery by Indonesian thugs in 1999? Indian troops and Chinese troops did not lift a finger to help the East Timorese women and children. Only the Australians (a premier Western people) sent their military to East Timor to stop the bloodshed -- without approval from the United Nations (which is dominated by 3rd world bigots like the Indians).
By the way, I am an active participant in several international seminars, and I always take the time to condemn India, China, and Mexico.
I believe that object oriented programming, in a very large program, makes things much, much eisier, when it is properly done, which is quite easy to do. I believe one reason it tends to make programs more maintainable, more understandable, and easier to get familiarised with, when we instead of creating one big monolithic software program and instead create modules each which has a particular purpose, is it keeps the code in the module smaller, shorter, simpler, and eisier to understand. Good documentation is also important to describe what the module does and its interface. I have done OO programming and done right it makes things simpler, makes less code, keeps the code reuseable, ans breaks up a big program into smaller parts which are eisier to understand.
I'd prefer something like: "Don't bother implementing things you don't know you'll ever need." That still rules out the time-wasting feature creep, while acknowledging that it's often far more efficient to plan for likely future developments from the start rather than constantly evolving a system without any sort of "grand plan".
All evolutionary development causes overhead, most of which is unnecessary if you can anticipate major future developments with reasonable accuracy. Despite all the nay-sayers, I have never yet worked on a real project where this was not the case.
If you disagree, post your argument. (-1, Overrated) isn't your personal censorship tool for views you don't like.
That's an easy one: who is qualified to decide who makes a good engineer, and what is truly best practice?
In civil engineering, we have many years of experience to draw upon in deciding this. If a bridge-building technique fails, the bridge falls down. After building a few thousand bridges, you can see that some techniques are more reliable than others, and you adapt your practices accordingly. It's evolutionary, but with a huge base of experience to draw on by this point.
Software development just doesn't work that way, at least not today. For a start, almost no-one has shown they are capable of reliably developing large-scale systems on spec, on time, on budget, and with a low bug count. In the absence of many examples of things that worked, who's to say which of today's buzz-phrases, if any, represents the best approach to use on the next attempt?
Similar arguments apply on numerous other levels, everywhere from programming language design to project management and planning/estimation. IT simply has many difficult, unsolved problems, and until we start solving them with at least some significant degree of success, we can't evolve a true engineering culture that builds on that foundation.
Unfortunately, with the eternal divide between academic research and the pragmatism that works in the commercial industry -- currently around 20 years, it seems -- it's going to be hard to spot those successes we do have, and bring them to the front of the line on future projects. This is ultimately the difference between software "engineering" and real engineering.
If you disagree, post your argument. (-1, Overrated) isn't your personal censorship tool for views you don't like.
On the lighter side of things, I find it funny that IT departments often "re-invent" themselves by changing their name and acronyms--a complexity in themselves. You find acronyms such as MIS, CLC, ITC, CIT, CITS, etc. But in essence they all stand for the same thing.
My university's acronym is CITS (Computer and Information Technology Systems), and before that they were just the CLC (Computer Learning Center). Imagine if they kept the name "Learning" in the acronym somewhere, it could've been: Computer Learning and Information Technology Systems (CLITS). But somehow I don't see that happening.
Linux at home
This has to be a wind up!! East Timor! The Australian Government of Gough Whitlam stood by in 1974 whilst the Indonesian Government invaded East Timor then condoned Indonesia's control of the country. Until 1999 the Australian governments from both parties were happy to keep diplomatic ties with Jarkarta and sell weapons, and military training to the TNI (indonesian miliary). Weapons and training they had been made plainly aware was being used to suppress civilan dissent in East Timor, including torture and murder.
The brief intervention in East Timor was aberrant, rather than standard behaviour for Australia and was a tiny step back towards reasonable and respectable treatment of a loyal wartime (WWII) ally, East Timor.
Perhaps while you take the time to "condemn India, China and Mexico" you might read an international news paper... this is all in the public record.
Who's with me?! I SAID... WHO'S WITH ME!!??