Slashdot Mirror
When Do You Kiss Backwards Compatibility Goodbye?
Arandir asks: "Backwards compatibility is great for users. But it sucks for developers. After a while your normally sensible and readable code becomes a nightmare spaghetti tangle of conditions, macros and multiple reinventions of the wheel. Eventually you have to kiss off backwards compatibility as more trouble than it's worth. The question is, when? Should my code conform to POSIX.1 or Single UNIX 2? Should I expect the user to have a ISO Standard C++ compiler? What about those users with SunOS-4.1.4, Slackware-3.2, and FreeBSD-2.2?" This question is really kind of difficult to answer in the general sense. The best advice one can give, of course, is "when you can get away with it". Not much help, that, but the lost of backwards compatibility, like most complex decisions, depends on a lot of factors. The key factor in most developers eyes, of course, is the bottom line. Have many of you been faced with this decision? What logic did you use to come to your decision and what suggestions do you have for others who might find themself in this predicament?
Ironically I am doing it right now. Good part it is Saturday, and other developers do not know. Or they will lynch me..
<^>_<(ô ô)>_<^>
The two things I would say are, when you really reach the point where all the old crap is really clogging up the veins, fix it all at once. Make a clean break. Then people can at least keep in mind what is happening, what works with 2.x and what is still only for 1.x.
The other thing is, try to design to keep this from happening. Expose APIs that don't need to change much instead of the actual functions or objects that you use. One more level of indirection won't kill your performance in almost every case, but it will give you a whole lot more room to re-engineer when you decide you have to.
All that applies to the case where you control the interface and you need to change it. When you're publishing source code and want to decide what tools you can expect the user to have to make use of it, that's a marketing decision and not a technical one. You're talking about how many people will be excluded from your audience if you use GTK or assume a conformant C++ compiler. Technically the newer tools and libs are generally better, that's pretty clear. I think it's going to be a judgement call on the part of the developers as to how much they care about a lot of people being able to use their code. If they are willing to wait for the world to catch up before being able to use their program, then they can use the latest and greatest. If not, then they have to aim at a realistic profile.
I think there are several approach you may choose.
If you think your program/library has been widely adopted by many people, it would be very very hard to scrap the old one and start anew. You will provoke the wrath of other developers that use your program/library. If this is the case, then the possible compromise can be:
To do revamping, you'd probably want to look on how the Windows COM approach. I'd hate to say this, but this approach is generally good, but I don't know whether I can come up with a better solution.
Or alternatively do an OOP approach. OOP can help modularizing your code if it is done properly. (That's why KDE rocks)
That's my 2c, though.
--
Error 500: Internal sig error
God created the universe in 6 days because He didn't have to worry about an installed base.
Two rules of thumb:
1) Support whatever 90% of your users are using
2) Support the prior two versions
If you can't do the above, make a clean break and give it a new name or change the major version number and list the changes in the release notes.
If you have to make a clean break, if possible:
1) Provide a migration path
2) Provide an interop interface
And above all, listen to your users.
It takes a lot more than OO design to solve this problem. The fact is that initial designs are almost always naive, and lack the specific flexibilities (and inflexibilities) the Ideal Solution would require. So, very often, are second iterations. But the third time's usually the charm.
The need to depart from backward compatibility is the result of correcting design flaws. And design flaws happen, regardless of your programming methodology. Design flaws are less a product of programming methodology, and more a product of not completely knowing the problem.
Java accommodates design flaws a little bit better not by being object-oriented, but by relying so heavily on dynamic binding. While this can make recovering from Bad Design simpler, it only lets you create bandaids that do nothing to fix the original design problems. And all that dynamic binding costs Java in performance across the board.
Release a set of updates, only change the minor version number, break one critical function in each update, fix it and break a different critical function in the next. Repeat until users no longer depend on the functionality that you want to change, then introduce the new functionality.
;)
But first, go read "How to write bad code," and start following those suggestions too.
Often, Backwards compatability problems can be avoided by careful design. Leave room for improvements. Designate certain structures as ignorable. Presume that the current incarnation of the code is not the final version.
Design for elegence. If the current code is relatively clean, then chances are that it will be easier to tack on an addition later on. Include stubs for improvements that you can forsee adding later on -- even if you can't percive the exact form of the improvement at the time. When you tack on the addition, try and do that elegantly too.
With languages, you can sometimes avoid backwards compatability problems by not using the latest and greatest features just because they're there. (it also allows you to avoid creeping featurism growing pains).
If using a new feature makes a big difference in the implementation of a solution, then use it, but at least document it. It keeps you more conscious of the break, and makes life easier on the people who have to rip out your code and re-implement it on the older system that you thought nobody was using.
Anecdote: A friend of mine recently found out that that the security system where he had a storage locker was run on by apple IIc. The box was doing a fine job of what it was designed to do 20 years ago. Just because it's old, doesn't mean it won't work.
Sometimes boldness is in fashion. Sometimes only the brave will be bold.
I like Microsoft's solution, which I'm sure was done somewhere else first. At any rate, here's the situation...
:)
:)
:)
Microsoft face the issue that they want to maintain backwards compatibility with everything, so they can leverage off the existing popularity of their platform. This means they can't let new libraries break old applications.
They had a problem once with MFC, where they updated its memory allocation scheme to make it more optimal. Problem was, a whole lot of old apps happened to work because they accessed memory incorrectly, but the old memory allocation scheme didn't reveal their bugs (I think they were doing buffer overruns, but the old scheme allocated a bit of extra room). Anyway, Microsoft released an updated MFC DLL, and suddenly old applications started breaking. It wasn't really MS' fault, but it was a big event, and I think it was the last time they touched old code like that!
Anyway, this is where they have their "DLL hell" problem too. Different apps are written against different versions of a DLL, many times with workarounds for known bugs in those DLLs. A new DLL comes along (installed by another application) and suddenly working applications start breaking.
So here comes COM. I've encountered it with DirectX, and it works like this. When you request something like a DirectDraw object or a DirectDrawSurface object, you also tell the class factory what *version* of the interface you want. Then you're provided with that actual implementation of the library. If you write your game against DirectX 2, but the user has DirectX 5, well your request to the DirectX DLL will actually give you the version 2 implementation! Which is cool, because if you worked around old bugs, those bugs are still there; they're not fixed for you!
As far as I know, you're not getting an "emulation" of the old implementation either; I'm pretty sure they just include binaries for each version of the interface. They could easily dynamically load whichever one you request.
Of course it means bloat, but there's no other solution. If you want backwards compatibility, you can't fake it, because it's all the really subtle things (like bugs that people worked around) that will bite you in the butt. It's been Microsoft's nemesis but its strength through the years, just as it has been Intel's too. Both companies need to remain backwards compatible with a *LOT* of stuff out there, and so they're forced to have all this legacy architecture. It would be nice to start from scratch, but then they'd be levelling the playing field, and that's no fun
Of course, this backwards compatibility drama affects everyone, not just Windows people. Just the other day someone installed a new PAM RPM on my Mandrake Linux. It installed fine, being a newer version, but some subtle change in behaviour meant I could no longer log into my own box. That's no fun either
- Brendan
Commit yourself to a strict policy: nothing in a minor version will break anything since the last major version. If your code is at 1.9.99, it should be backwards-compat with 1.0.0. If your code is at 1.1.0 and backwards compatability breaks, move it to a 2.0 release.
Typically, users expect breakage--or, at the very least, problems related to upgrades--with major versions. With minor versions, they don't expect breakage.
Follow the Law of Least Surprise. If you break backwards compatability, up the major by one.
Insofar as when to break backwards compatability, that's a much harder question. The obvious answer is a little philosophic: not all engineering problems can be solved by saying ``screw backwards compatability'', and some engineering problems cannot be solved without saying it.
The trick is learning which is which.
A lot of the discussion seems to be related to issues of things like programming languages and operating systems (which are important). But what about keeping up with old formats and protocols? I think the issue is more one of what your project works with, than it is what language you choose (including the OS as part of the former).
I'm not so much looking for specific answers to the above questions, but rather, a general idea of how you think one should go about deciding those issues to come up with the best answers in some given situation.
now we need to go OSS in diesel cars
Code was ugly and hideous
Someone forked the tree a while back, and now we had to support 2 seperate source trees (this one was really annoying, because if fix a bug and change some behaviour in one side, since both sides had to be able to talk to each other, you'd have to introduce a corrosponding "fix" in the other side.
The core architecture was woefully outdated and ineficient
Speed was an issue, and the current architecture was limiting, and the code was optimized about as well as it could be (this was also part of the uglyness problem)
we were spending about 70% of our time fixing bugs in the old code, it took this much time because they were little and stupid and with the code in the state that it was in it took forever to trace things down.
Well, we had been wanting (desperately) to redesign from the ground up for a while, but the powers-that-be wouldn't give us the time, until one customer asked for a feature, marketing promised that they'd get it, and we said "Know what? We can't do that. Not with the current infrastructure." So the powers-that-be said "do what needs to be done!" and we said "yipee!".
Moral?
How much extra time is spent during debugging code that is due to the current state of the code?
How does the core architecture compare to what you will need in the future? Will it support everything?
How efficient is the old codebase, and how efficient does it need to be?
Can you get the time required to do so? This is a one-step-backwards before two-steps-ahead thing.
Do you trust the people that you are working with enough to be able to competantly and efficiently create the new architecture? This is a serious question, because I have worked with some people that are good if you tell them exactly what to do, but I wouldn't trust them to recreate everything.
Will you be required to keep the old codebase going while you are in the process of converting? If old customers need bug fixes, you might be forced to keep the old sourcebase around for a while.
Can you make the new design backwards compatible? If not, can you provide a wrapper of sorts, or some easy way to convert your customers from your old version to the new one?
If you are going to be redesigning the user interface or the API at all, then you must also think about the impact on your customers.
Just some food for thought.
If God gave us curiosity
"Programming is like sex. Make one mistake and support it for the rest of your life."
This is a confused discussion. A lot of people are mixing up "backward compatability for users" with "not making significant changes to the code base". The two are largely unrelated, except that screwing up the latter will also mess up the former.
What your user sees is, and always must be, decided by your requirements spec, not programmer whim. The only people who can get away with doing otherwise are those developing for their own interest (hobbyists, people involved in open source development, etc).
To put it bluntly, blanket statements like "meet 90% of your users' BC needs" are garbage. In many markets (notably the bespoke application development market) if you drop 10% of your users in the brown stuff, your contract is over, and your reputation may be damaged beyond repair. Look at MS; years later and in the face of much better libraries, MFC still survives, because people are still using it (including MS) and they daren't break it.
This is far removed from rewriting things significantly as far as the code goes, which is where things like the standards mentioned come into things. I'm sorry, Cliff, but you don't do this "when you can get away with it" if you're any good.
Every time you rewrite any major piece of code, "just to tidy things up", you run the risk of introducing bugs. You need to be pretty sure that your rewrite is
- necessary to meet your requirements, or
- fixing more than it breaks and not breaking anything unacceptable
or, preferably, both.If the rewrite is justified using these objective criteria, then you do it. When you do, you try to minimise the number of changes you make, and to keep the overall design clean. You retest everything that might conceivably have been broken, and you look very carefully at anything that didn't work -- it's quite possible that the people who originally wrote this code months or years ago made assumptions they forgot to document and you've broken them. Finally, if and only if your rewrite is performing acceptably and all the tests are done, you decide to keep it. If not, you throw it away and start rewriting again.
And for the record, yes, I spent most of last week rewriting a major section of our application, as a result of a code review with another team member. We kept the overall design, tweaked a few things within it, and rewrote most of the implementation. Now we need to retest it all, update all the docs, etc. This little exercise has cost our company thousands of pounds, but in this particular case it's justified by a needed performance increase and the significant reduction in bug count. But you can bet we thought very hard about it before we touched the keyboard.
If you disagree, post your argument. (-1, Overrated) isn't your personal censorship tool for views you don't like.
A key to providing backward compatibility is "design intent"; i.e., closely examine the backwards compatibility issue when you are first thinking about creating a piece of software. Internal data structures, external file formats, APIs, etc. are all influenced by the design constraints placed upon a project. If one of those constraints is backward compatibility then these structures will all be built differently than in the case where no backward compatibility is ever required.
.MIF files influences coding decisions.
FrameMaker is a great example of an application that appears to have been architected from day one to provide backward compatibility: every version of FrameMaker imports and exports Maker Interchane Files (.MIF files) and so it is trivial to move files between releases of the application. While I'm sure this causes the developers some headaches from time to time, I know from personal experience that a constant anchor point like
Having done work on an ASCII interchange mechanism for a multiplatform application, I can be fairly certain that the FrameMaker decision isn't very difficult to implement: each release of the application has a pair of small functions, one to walk the internal data structure and emit the ASCII interchange format, and another that parses the ASCII interchange file and produces an internal data structure.
When we designed our application, the ASCII interchange functionality was deemed important; this influenced the internal data structures, which in turn influenced the binary data files. If we had tried to bolt backward compatibility on at a later date (i.e., in version 2.0) it could have been a lot of work; whereas, building it in from day one didn't cause any extra work.
Conscious design intent is the key to making backward compatibility a non-issue.
The API has barely changed in the last 25 years. A friend of mine has an application that's been running unchanged since the 1970s. It has contined to work across generations of hardware. And it's in assembler.
They had the advantage that their OS was decades ahead of its time. UNIVAC had symmetrical multiprocessing with threads in a protected mode environment thirty years ago.. And threads were designed in, not bolted on like UNIX.