Slashdot Mirror
Reuse Engineering for SOA
An anonymous reader writes "In most development organizations, software reuse occurs on a regular basis in at least an ad hoc manner. Code is shared across projects in an informal manner. SOA provides the mechanism for more formal reuse. So what are the issues? This article examines some of the challenges associated with the creation and usage of reusable services."
Would it kill submitters to expand acronyms? Or give a little background on the "frammazazz project" for those of us who have no idea what it is? I read some of these summaries and am even stupider than when I started. And that's saying something.
So you've rewritten your own versions of libgtk, libX, libxml, etc. that you understand? Cool. I'd do the same, but my time here on Earth is limited...
Too many managers are trying to jump on the SOA bandwagon. SOA is basically "runtime" reuse. But there are plenty of valid ways to do code reuse before runtime, hence functional decomposition, OO design, or componentized architectures. Don't fall for the marketing hype about SOA being able to fix every problem that ails you.
If you have a large company with a bunch of legacy or disjoint applications, SOA could be a great way to solve some of your business needs. If not, then keep an open mind and look for the right solution (and don't trust vendors).
I'm thinking you either work for yourself, or work for a company that doesn't monitor your programming habits. Otherwise, software reuse can be time-saving and is widely used. Even though knowledge of previously written code isn't always passed on to the next developer in such a way that they completely understand it, that usually does not warrant a complete rewrite. Code that is written to be reused, however ambiguous, usually can fufill its purpose.
For he today that sheds his blood with me shall be my brother.
Service Oriented Architecture (SOA), etc.
:)
In my limited experience, there are a lot of "software methodologies" out there, all claiming to make software better (i.e. more scalable, efficient, better re-use, etc.). Of course it all comes down to modular programming, good documentation, and agreement among the developers in an organization on a plan for how everyone is going to do things so that everyone is on the same page.
Also in my experience, more than half the developers at any reasonably sized organization are not really capable of dealing with abstractions like SOA, OOP, or whatever. No matter how well laid your intentions are, and how many rules you create, there will always be some new hack straight out of some college course who dives in and gets the job done, but manages to totally screw up the whole system you and the senior programmers had in place. Then it either goes unnoticed until it becomes a problem (when the next change has to be made), or you have to spend half a day undoing the damage they did, and doing it correctly. Either way, the new guy looks like a genius for getting it done in half the time it would have taken one of the older guys, and you look like an inflexible nimrod that's just getting in the way of productivity.
You want an acronym that works? Here it is: PR (peer review). Find some other smart guys in your company, and team up to review each others' work, share ideas, and build a common set of best practices. Don't let people outside that group touch your code.
"I have never let my schooling interfere with my education." - Mark Twain
I agree that reuse of objects is overrated - however reuse of components is widely used - the main killer is idiots that do not comment the code / provide documentation as to how a component / module / application works so other developers often find it easier to re-write a new component rather then reverse enginer a component to work out how to use it. This is a HUGE problem with our industry ( yes I am a coder and have been so for 10+ years)
Organizational impediments -- e.g., developing, deploying, and supporting systematically reusable software assets requires a deep understanding of application developer needs and business requirements. As the number of developers and projects employing reusable assets increases, it becomes hard to structure an organization to provide effective feedback loops between these constituencies.
Economic impediments -- e.g., supporting corporate-wide reusable assets requires an economic investment, particularly if reuse groups operate as cost-centers. Many organizations find it hard to institute appropriate taxation or charge-back schemes to fund their reuse groups.
Administrative impediments -- e.g., it's hard to catalog, archive, and retrieve reusable assests across multiple business units within large organizations. Although it's common to scavenge small classes or functions opportunistically from existing programs, developers often find it hard to locate suitable reusable assets outside of their immediate workgroups.
Political impediments -- e.g., groups that develop reusable middleware platforms are often viewed with suspicion by application developers, who resent the fact that they may no longer be empowered to make key architectural decisions. Likewise, internecine rivalries among business units may stifle reuse of assests developed by other internal product groups, which are perceived as a threat to job security or corporate influence.
Psychological impediments -- e.g., application developers may also perceive ``top down'' reuse efforts as an indication that management lacks confidence in their technical abilities. In addition, the ``not invented here'' syndrome is ubiquitous in many organizations, particularly among highly talented programmers.
On the other hand code re-use within organizations is rare, but I think that is mostly a process issue, not a technology one. In my experience product development companies have much better processes to foster such re-use, while non-software companies, where the IT division is more a necessary evil, rather than an asset, do not.
All you touch and all you see is all your life will ever be
Usually, I write a version of a popular library to understand it, and, having done that, realize how badly I have implemented the library in comparison to the original, and use it. It helps with both understanding and respect for your tools.
Neither the synopsis nor the article bothered to enlighten us plebes as to what exactly SOA stands for, so I googled it for the benefit of others:
A Service-Oriented Architecture is a collection of services that communicate with each other. The services are self-contained and do not depend on the context or state of the other service.
At least, I think it's that one. Then again, maybe it's this one"
Baldurs Gate II: Shadows of Amn.
Maybe they use multi-player mode to define the problem: "Okay, so the database is the castle, right? And the balrog is a stored procedure that we all need to be able to kill - uh, run, but we only have one magic sword, uh interface to do it with...."
Okay, maybe not.
Crumb's Corollary: Never bring a knife to a bun fight.
Warning, opinion ahead, intended for discussion, but some may see as flamebait. That's ok; flame me if you want.
For those who don't know the acronym, SOA means "Same Old Architecture."
It's a clever way for the folks who brought us distributed objects to resell "new" solutions and consulting. What you have is basically distributed objects, with the design patterns that anyone who had half a brain would have already implemented if they were using distributed objects. In the end, you'll probably end up marshalling all your data around via XML over protocols originally intended for other things, like serving up web pages, maybe getting to implement synchronous semantics over asynchronous protocols (or vice versa), all the while trying to keep things nice and reusable & decoupled, etc.
And you'll run into all the same problems you would have hit before, except your CIO will be cool with that because its SOA, you know, and that's hip.
I loved hearing a first rung manager at a bank insist on doing online trading transactions with an external partner over HTTP using XML back in 2000. What a visionary.
But hey, go ahead and explain to me how SOA solves all the old problems. People who couldn't implement robust services and reusable interfaces using CORBA aren't going to magically have all their problems solved with SOA.
This is exactly why software reuse doesn't happen often enough. I took a class in Software Reuse and Design and I paid attention long enough to gain the following two insights:
1. Software reuse is hard
2. It only happens if people want it to happen
You can build a completely usable system which enables effective software reuse and thus reduces development time, but it won't do a thing for productivity if no one wants to use it.
Companies can foster an environment of reuse, which helps with Number 2. Number 1? We didn't find a way around that one.
The big one is simple: "There's no such thing as reusable code, only code that has been reused.". It's very difficult to design code to be reused and get it right until after you've actually tried to reuse that code somewhere else and found all the problems. All code makes assumptions about how it's going to be used, and you usually don't realize which ones are true showstoppers until you go to use that code in a different way and get smacked in the face by them.
The rest of the problems with SOA are the same ones that've been around ever since someone throught up remote procedure calls. If you aren't familiar with RPC and XDR, what makes you think you won't make the same mistakes and face the same problems this time around?
1. Creating good interfaces is hard. Really hard. Most programmers create lousy interfaces that nobody but them wants to use. I know; I've written quite a few of them. Good APIs take thought, creativity, and a lot of effort, none of which are allowed in a typical business environment.
2. Reusing code will not necessarily save work. See point one for the first reason. The second reason is that it is often faster to reimplement the functionality and then refactor. This generates more code than reusing someone else's library, but may save development time. Saved development time is a good thing in the type of business that is always in crunch time.
Software designed for re-use will likely never ship, or ship far later than software designed to do something.
pooptruck
With that said, service oriented architecture strikes me as little more than the latest in a long string of TLAs so beloved by IT management and such (I.e. PHBs), but with very little in the way of real content behind it. The whole point of pretty nearly any software ever written is to provide some a service a user, so pretty clearly being "service oriented" is roughly as new as dirt.
Ignoring that, however, and taking web-service oriented software as somehow being revolutionary (even though it's really not) we're still left with a serious question about how in the world this would relate to software reuse. I'm reasonably certain the answer is that PHBs feel a need to sell their PHBs on the latest TLA, and IBM has thrown together a web page that tries to help them in that regard.
When you get down to it, however, the web page contains virtually nothing in the way of real information. It basically says that reuse is good. Whether you agree with that or not, the fact is they haven't really told you anything about how to facilitate reuse in general, or how SOA is supposed to contribute to that. They cite the usual reasons for reuse not working out well (e.g. lack of education and lack of software suitable for reuse). They go on to give the usual ideas that mentoring, careful analysis, etc., will help yield ideas for software to write that's worth reusing and more ability to reuse it.
Five years ago this article would have said "XP" instead of "SOA". Fifteen years ago, it would have been "OOP" instead. Twenty five years ago that would have been "structured programming". I wasn't around at the time to know for sure, but my guess is that if you looked carefully you could find something from the 1950's (or maybe even late '40s) talking about how the macro capability of the new assemblers wasn't resulting in as much code reuse as some people hoped, mostly due to 1) lack of education and 2) lack of macros worth reusing.
To make a long story short, "code reuse" has a long history of over-promising and under-delivering. Now, that may make it sound like I consider software reuse a lost cause, or something on that order, but that's just not true. The fact is that macros allowed some reuse of a fair number of (mostly) relatively small pieces of code, as long as there wasn't too much variation between the uses.
Structured programming helped a bit more, particularly by helping readability so you might be able to figure out what something did more easily than writing it all over again.
Likewise OOP allowed more reusability as well. Despite being the newest TLA on the block "SOA" is really little more than modular programming, with the modules in this case being relatively large. There's been a bit of work done on standardizing the interfaces between the modules, so it's a bit easier (at least in some cases) to plug them together, but in software that's pretty much what most architecture boils down to anyway -- designing interfaces.
Now, having that interface pre-designed (to at least some extent) undoubtedly makes it a bit easier to reuse a bit more software with less design specific to the problem at hand, and that's probably a good thing in general. OTOH, Brooks was right: there probably is no silver bullet, and even if there is, SOA isn't it. SOA will probably provide an incremental improvement over previous methods, at least in a few places under a few circumstances (given the amount of effort that's been put into designing the SOA interface "stuff", we'd better hope so, because it needs to help some people quite a bit to even break even).
Articles will be published crediting it with saving company X from total oblivion, triumphing over their opposition, etc. Other articles will be published blaming it
The universe is a figment of its own imagination.
No doubt the lack of reusable code is why the majority of code is buggy, bloated, and inflexible. Taking the time to build reusable components results in better code, smaller code, more flexible code, and in the end faster development time.
Novice programmers (even the experienced novices) think that quick development time is more important but all they end up doing is taking more time because they blindly recode everything instead of building upon what they've already built. Yes, it's faster the first time you code something to do it that way (if the project isn't very complex) but with each additional time your wasting time. Spend the effort up front and you'll save a lot more time in the long run.
I'm always horrified when I look at people's code and see how many of them don't even use functions let alone objects, components, and external services. I see programmers that should know better using cut and paste methods. Great.. that's quick.. until you have to change that code which means locating all instances of the code and applying changes.. which is of course made harder by the way most such developers don't comment code and often make several minor changes to various occurances of the code.
An example I attack a lot is how web-based services and applications are usually written. Rather than write a service that handles things like logins and credit card transactions you see this code just mixed in with application code. This is very standardized functionality that could easily be abstracted but instead it's rewritten over and over and ends up very buggy and insecure. You end up with everything mixed together as spaghetti code.
Try reusing that code. With a little practice and the gradual build-up of reusable parts it'll result in better code and faster development times.
It seems to me that people that actually like to program tend to reuse code whereas people who do it just as a job or to fill a need don't. Real geeks are lazy because we're overflowing with ideas. If we have to keep reinventing the wheel we don't find time to invent the cool stuff we really want to work on. That's why we invented shell scripts, pipes, pluggable components, functions, objects, services, high-level languages, development frameworks, etc. These things combine to let us do more in less time.
At what price learning? At what cost wisdom? The price is a man's peace of mind, and the cost is his life.
Wasn't OO supposed to be the panacea for reuse a few years ago? Never happenned... (it turned out that massively complex multiple inheritence trees were worse than the rewritten objects they were trying to replace).
Then 'Extreme Programming' did the rounds... use twice the number of programmers to produce the same code... yeah rock on dude... Not sure what they were aiming at there... My last boss took one look at it and said 'over my dead body'. End of that one.
Then UML... Everyone I ever met took one look at that and laughed.. PHBs tried to push it for a while though... was almost forced to do a project in it, until we pointed out that the design phase alone would take longer than the entire project deadline (which admittedly was only 2 weeks).
Now we have a new (unexplained... SOA to me means DNS records) fad that is supposed to make programs perfect, solve world peace, etc. As ever, it's 10% good ideas, 90% hype (I bet they sell a few books on the back of it though).
Good programming is about using every technique available, within the constraints of commercial reality - and that means tight deadlines, bosses who understand nothing but results, and users who are worse.
Code reuse is a very good idea, but in practice tends to happen within small teams (who often have a library of routines that they are familiar with and work with).. Commercial libraries often suck donkey (in fact very often... since they're subject to the commercial constraints listed above) - I once had to rewrite an entire development library because it sucked so hard.. took nearly 4 days (mind you, the original developer seemed quite proud he'd done it in less than 6 months... given the complexity of what he'd produced I could believe it - they still don't teach KISS in universities I see).
OTOH you can't become too wedded to code reuse.. sometimes the spec is just different, and constantly modifying the same routine to do multiple things it wasn't designed to do creates an unmaintainable mess real quickly.. that's where refactoring and rewriting starts (seems to happen in phases... you collect and modify the libraries over a period of years, then someone says 'look at this pile of steaming crap', and it gets rewritten... years later the cycle repeats.. if your stuff is modular enough you can do it without introducing bugs).
The title of this article does read a little strange for Dutch native speakers.
Google search on "Dutch SOA" http://www.google.com/search?q=dutch+soa shows what I mean.
Objects are easier than functions?
Not at all.. to write objects you have to think in an OO way.. that should come easily to an experienced programmer but I've seen a lot of things that are just linear programs with the word 'class' at the top.
Web services are a bitch... I've yet to find a really compelling use for them though - I could imagine an application that uses them but have never actually been called upon to write such a beast.
I personally like using XML-RPC to decouple components because it is easy to work with and easy to use from any programming language. Different languages can make different problems vastly easier or harder to approach so being able to pick the best tool for each job can be a real time saver. Something that is easy in PHP might be hard in C. Something that is easy in Prolog might be hard in PHP. Something easy in SQL might be hard in Prolog. And so on. Having the code bases sepperate makes debugging and maintainence a lot easier to. You can keep each service simple and direct and just use a little glue code to tie it all together. Code you can't reuse is the oddity in my experience and is usually the bits that change often - mostly UI stuff.
I have to agree that talent is important but I think talent combined with good methodologies can create magic. When you get a few talented people that are communicating and working well together wonders really do happen.
At what price learning? At what cost wisdom? The price is a man's peace of mind, and the cost is his life.
Objects can make more complex problems easier than functions alone can do.
:)
I've seen plenty of poorly designed objects too but that doesn't mean objects are a bad idea. I've seen horribly written software in general but I still think software is a good idea.
I'll assume that by web service you're refering to web-tech based components and not user apps like GMail. There are plenty of good reasons to use thse. If you're writing a complex program or set of programs that need to reuse functionality then they can be a good idea. You can reuse functionality without reinventing the wheel or sharing code (for security reasons, simplification, etc). Something as simple as handling users and permissions is a good thing to build into a service. There is no reason for every app to have it's own users and permission levels for those permissions. Why not abstract them into a service that all your apps use? Having this abstracted simplifies your logic both in the service and in the app, lets you more closely audit this important code for security flaws, and makes management of users easier.
At what price learning? At what cost wisdom? The price is a man's peace of mind, and the cost is his life.
Recently at PDC this topic was raised to these folks: Gregor Hohpe, David Ing, Tony Redmond, Steve Swartz and their response was exactly what you've read in these posts.
It's true that management loves this term. Have you seen the latest tech-management publications sitting on their desks? Headlines read: "Leverage SOA Now!" and such. Now, if you're someone that is held accountible for major architectural decisions and you're the one that the department manager poses these questions to, it's a fantastic opportunity to recalibrate expectations for things that you already do or have been told in the past that you cannot do because of time/cost limitations.
On the topic of code reuse, here's, humbly, how I've seen the most benefit:
My 2-cents.
http://www.inboxlist.com
Bad example.
I wouldn't trust user management to an external machine over the internet. ever. If that machine is ever compromised *every* app that uses it would be compromised. This is why MS Passport never took off.
Handling users and permissions is generally done by the OS anyway.. doesn't need an external module. Application-only permissions (things like 'can edit page', 'can ride quad-bike', etc.) are not generic and change for every application, so can't be stored externally anyway.
Really you're talking about something that probably takes about an hour of some junior programmers time, is about 20 lines long and never needs to be touched. Why introduce the complexity of a web service for something like that? Heck, just borrow it from the last application that needed it...
My opinion, the greatest drawbacks are:
1.) Versioning. Project A wants to add a feature to the service. Now we have to coordinate and test with projects B, C, D, and E. Two of which have no funding. The other is run by a moron.
2.) Reliability. I have 10 services on different machines run by different groups. All have to be up for my app to work. If they all have 99% uptime, mine is significantly less than that.
3.) Speed. Serializing data and calling across the network is slower than local. Period.
4.) Interacting with a 3rd party. If you are on a project and dependent on another group with different priorities and management, you are in for a few headaches and delays.
I could go on. I work at a large financial institution, so we have a lot of architects that prefer SOA. It has its place, but I hate to see people push it, when they don't back it up by detailing how they will mitigate the drawbacks that come with it.
This is probably the number one reason. I have worked on several projects where re-use was a goal, and the software produced was difficult to understand, error prone, badly documented, and inflexible. Re-use is possible though, the best examples are the Java and .Net libraries which are very good.
Another problem is that the technology changes rapidly enough that a re-usable library can become obsolete rather quickly. I don't know how many user interface libraries I have written (starting with an X-Window/Motif library) that are pretty much worthless in may current job.Over the Internet? You can ue web technology without running your code over unsecured portions of the Internet. Secure services should run deep inside your network on your most secure machines. There is some risk that reusing such code could create a single point of failure but that isn't much since someone that's penetrated your network to the most secure depth is likely to already have the run of the rest of the network.
Handling application users and permissions should not be done by the OS. There are many times when apps need to share users across multiple systems including systems of different types. Also there are many times when you don't want all application users to have user accounts of the systems running the apps. Would Slashdot want to give Unix accounts to every Slashdot user on the Slashdot servers? Hardly a good idea. Rules are rules are they are just as easy to abstract across applications as they are to apply to apps in general. To the apps it shouldn't matter.
You're approach is really why web apps are so buggy and insecure. It's not about how long it takes to write the code - it's about writing good code that is maintainable. I've yet to see code that 'never needs to be touched' in the real world either. Eventually everything needs new features or bug fixes. Perfect code is incredibly rare especially when written by inexperienced junior programmers in twenty minutes.
At what price learning? At what cost wisdom? The price is a man's peace of mind, and the cost is his life.
Service-oriented architectures is basically the UNIX philosophy: lots of little tools, often implemented as little servers. Yes, it helps with reuse, it helps with limiting the effects of errors, and a whole set of other problems. What else is new.
It's funny that this is now becoming popular among the UNIX haters (you know, like many object oriented developers, Windows developers, mainframe programmers, and all those guys). But, of course, they couldn't simply just use the approach, they needed a new acronym, massive amounts of new syntax and protocols, a constant stream of hot air, and preferably gigabytes of memory to implement it all.