When Rewriting an App Actually Makes Sense

vlangber writes "Joel Spolsky wrote a famous blog post back in 2000 called 'Things You Should Never Do, Part I,' where he wrote the following: '[T]he single worst strategic mistake that any software company can make: They decided to rewrite the code from scratch.' Here is a story about a software company that decided to rewrite their application from scratch, and their experiences from that process."

Comments

Here's my short list

[Anonymous Coward]

5. When it is written in Visual Basic. Always.
4. When I'm getting paid by the hour and it is written in Visual Basic. Always
3. When it was written in a mid-90s WYSIWIG bastard child of a mid-80s interpreted language.
2. When it uses a thousand "IF-THEN-ELSE" when it means to use regular expressions
1. When it is written in Visual basic.

Re:Here's my short list

[beakerMeep]

See if someone put in a bunch of cryptic RegEx, I would re-write it as If-then-else statements.

For me readability > length of code.

I always find it funny when developers are struggling to fit their code on as few lines as possible. Like the forgot how to scroll or something.

Still, if it was written in Visual Basic, I'd let you take care of it.

Re:Here's my short list

[Anonymous Coward]

> See if someone put in a bunch of cryptic RegEx, I would re-write it as If-then-else statements.

Rewrite it as regexml! http://code.google.com/p/regexml/ ;)

Re:Here's my short list

[PRMan]

5. When it is written in Visual Basic. Always.
4. When I'm getting paid by the hour and it is written in Visual Basic. Always
3. When it was written in a mid-90s WYSIWIG bastard child of a mid-80s interpreted language.
2. When it uses multiple five-to-ten thousand line case statements
1. When it is written in Access basic.

This is why I am currently rewriting everything from scratch in .NET at my company.

Re:Here's my short list

[rockNme2349]

Do you really think that a thousand if-else statements is more readable than a regex statement. If it takes you one hundred lines of code to test for one condition, then the code becomes less readable, even if each piece makes more sense. RegEx is optimized to test for formatting of a string. This is what it was meant for.

Pseudo-code:
'If Email Address
If (String.RegExMatch("^([a-zA-Z0-9_\-\.]+)@((\[[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.)|(([a-zA-Z0-9\-]+\.)+))([a-zA-Z]{2,4}|[0-9]{1,3})(\]?)$")) Then
'Do Stuff
Else
'Do Other Stuff
End If

As proof of the point I'm not going to try and program the same functionality with If-Else statements. If you are using regular expressions correctly you leave a human-readable comment explaining what the regular expression matches.

But I think the important point to take away here, is that we can all agree that VB sucks. =)

RegEx copied from regexlib.com

Re:Here's my short list

[Anonymous Coward]

Still, if it was written in Visual Basic, I'd let you take care of it.

I'd definitely do a rewrite. What's the other option? Maintaining an old VB program.

Microsoft had a crack-team of developers working on maintaining an internal management app written in Visual Basic 3 around 1998. The code got jumbled into something vulgar... Something evil. After a couple gallons of whiskey, well... we know the rest.

Re:Here's my short list

[Tanktalus]

Regarding regexes, I've gone both ways: I've had my developers remove regexes where they were trying to use them, and I've had them add regexes where they were trying too hard to avoid them. Regexes aren't the answer, they're a single tool. You need to use them right. And the /x modifier in Perl helps a lot, allowing you to put useful whitespace and comments in the code (and a regex is code as much as any other language).

As for "as few lines as possible" - you need to do it right. When you span your code over 3 pages, that's not readable anymore. I harp on this with my developers all the time: SCOPE. Scope applies to variables, comments, and functions. The less I need to look at to understand what you're doing, the more likely it is I'm going to understand it. A huge 3-page if-elseif-elseif-else is going to be something I'm not going to understand, as I'll have forgotten all the conditionals by the time I get to the end to know really what scenarios are left - sequential access. A concise regex, on the other hand, is something that I can skim over just by moving my eyes - random access. These concerns aren't just valid for storage media (tape vs DVD/HD). Of course, 40 characters of regex special characters with no whitespace (either horizontal or vertical, preferably both) is generally going to overwhelm most readers, and is going stupid in the other direction.

Yes, readability trumps length of code. But sometimes that means to use a regex (or two or three - why make one big cryptic one when multiple simpler regexes can do the job?). And sometimes, that means avoiding them when what you really want to do is better done by another piece of code.

My favourite new-to-regex example recently has been someone trying to pull apart colon-delimited text with a regex. Woops - there are better language constructs for tokenisation, whether that's strtok in C or it's split in Perl (or better, Text::CSV_XS). Got rid of that regex in a hurry.

Re:Here's my short list

[Anonymous Coward]

Still with the MS love eh?

Re:Here's my short list

[hjf]

dude, THIS is the regex to validate an email address: http://www.ex-parrot.com/pdw/Mail-RFC822-Address.html

Re:Here's my short list

[Surt]

You should think about investing in learning tools. Regex is a well documented, well understood, capable feature of all modern languages. How many weeks will you spend debugging / refining your thousands of if/then/else when you could trust years of testing that has been done on regex engines? Your statement reminds me of novices who avoid ?: in favor of if/else because it's 'cryptic'.

Re:Here's my short list

[Surt]

That has to be the +5 funniest thing I've read in a month. Thank you, I just about fell out of my chair.

Re:Here's my short list

[Anonymous Coward]

It's a pity that compiler therory is so underused. Not applying Computer Science to write systems leads to lower quality software.

Re:Here's my short list

[WDot]

You could just put in a comment that says "This regex tests for an email address."

I've massaged if-else code into regexes before. Having several if-elses for a piece of data is rickety and (in my opinion) would take more work to rework than a regular expression if the data changed.

Re:Here's my short list

[Sarten-X]

My supervisor is (rightfully) wary of regexes, due to their reputation for unreadability. My favorite solution (in Java): Make a bunch of string constants to match various elements, then assemble those strings together into more strings, which are finally assembled into a single real regex. Sure, it takes about 15 lines for a moderately-complicated regex, but readability is superior to using 15 lines of indexof and substr.

And now, to just bring up a topic that I have a personal interest in:

Nobody knows how to write parsers anymore. I've never seen a recent university CS curriculum that covers parsing with respect to different parser constructs for different languages. Sure, the students learn to load up the chosen XML parsing library and pull in XML, and they learn to take text from stdin, but there's seldom any emphasis on what to do with more screwy formats. Maybe, just maybe, they might get exposed to different languages through a compilers class, but generally not how to process different languages outside of lex/yacc. This bothers me greatly.

Re:Here's my short list

[TheLink]

Looks like replacing that with a combination of if-else and regexps would be more readable :)

Re:Here's my short list

[fishexe]

3. When it was written in a mid-90s WYSIWIG bastard child of a mid-80s interpreted language.

In what universe is a language developed in 1964 and first marketed by Microsoft in 1975 a "mid-80s interpreted language"?

Re:Here's my short list

[turgid]

Sweet baby Jesus and the orphans! Whatever next?

Re:Here's my short list

[istartedi]

Bingo! You stole my thunder. Mod parent up Insightful. The solution to the infamous regex problem is to comment them. Your style guide should even have something like, "any regex longer than N characters should have a comment next to it". Developers who live, breathe and eat regex should be mindful that it might look like line noise to the rest of us, or that we might be able to parse it in 15 minutes whereas we can read your comment in 2 seconds.

Re:Here's my short list

[Provocateur]

WYSIWIG

For the noobs, thats What-You-See-Is-What-I-Got, normally after some software that was promised to deliver ABC gets XYZ instead, after paying M thousand dollars, and is in the process of being reported to upper Management.

Re:Here's my short list

[Anonymous Coward]

Can you use a regex to break up something into its constituents (for a C++ class perhaps)? Say, for example: //MYSERVER/C:/Folder1/SubFolder1/myfile.txt.

Re:Here's my short list

[TheThiefMaster]

What if there is a misplaced character in a 50 character (or more) regex? If it only broke in some obscure cases, even with a comment saying "this matches an email address", how the hell would you debug it?

Re:Here's my short list

[koreaman]

6. When it was a product of outsourcing to a foreign country. God, maintaining that shit is the bane of my existence. At least they've upgraded to VB 2005.

Re:Here's my short list

[willy_me]

If there is a problem with the regex you simply have to debug it - this is programming after all. The reason why the comment is important is that the debugger must know exactly what the regex is supposed to do. Is is any valid email addresses? Are there any exceptions? Once a programmer knows exactly what the code is supposed to do they can go about fixing it. Attempting to derive what the coded is supposed to do from the currently broken code is a recipe for disaster.

Re:Here's my short list

[Anonymous Coward]

That's odd that you mention that CS doesn't cover parsers anymore; it seems that's all we did in our second and third year classes (the first covering introductory topics and the last covering more advanced and specific topics like networking or advanced algorithms).

Re:Here's my short list

[jsdcnet]

Yes. The feature you are looking for is called capturing. Generally you use parens to indicate capture groups. The specifics of how you then access the captured components vary wildly from language to language.

Re:Here's my short list

[Calinous]

Using a RegEx could simultaneously decrease development time and reduce the number of bugs - everything the code do is located on a couple of lines of codes.
      Conversely, this might make it very hard/impossible to understand and change by someone that isn't a wizard in regular expressions (or might contain hard to identify bugs). Also, moving from ASCII to Unicode might be difficult.
      Regular expressions and pattern matching (see Erlang) are very powerful technologies, allowing for very interesting mistakes

Re:Here's my short list

[Calinous]

That RegEx matches an email address like alpha.123@0.333.999.0, so it isn't quite the perfect example (and I know almost nothing about regular expressions).
      This regular expression stuff is for regular expression wizards :)

Re:Here's my short list

[Foofoobar]

Sounds like you are robbing Paul to pay Peter.

Re:Here's my short list

[Anonymous Coward]

How much time do you need to debug an if...then...else block? it's not a complicated construct. How many times have you had a small typo in a long regex explode into something completely unexpected?

Re:Here's my short list

[Anonymous Coward]

C# email address syntax checker: try {new System.Net.Mail.MailAddress("joe@example.com");} catch (Exception e) {/*Invalid*/}

Easier to understand than a RegEx, and any tests it fails on will end up with other issues anyhow.

Re:Here's my short list

[Anonymous Coward]

FYI, I graduated with a CS degree in '05, and the first half of the undergraduate compilers course (required for all CS degrees) was devoted to lexical and parsing theory, gCFG, LR (botton-up) vs. LL (top-down) parsing, etc. before moving into the second half which was dominated by re-implementing core functionality of lex and yacc (because that is what is generally used now). So, if one payed attention, one would learn the skills you so covet (processing outside of lex/flex and yacc/bison) -- one would also learn that there is little reason to bother, since there are these handy tools that do a lot of the grunt work for you. Why become a butterfly farmer when you have C-x M-c M-butterfly?

Re:Here's my short list

[Surt]

If the if/else is nested a hundred deep to emulate the capabilities of the regex ... potentially quite a while.

Re:Here's my short list

[jeremyp]

Sorry, you're going to have repost that, it looks like your modem had some problems and there is quite a lot of line noise in it.

Honestly, if you think that is readable you need to get out more.

Re:Here's my short list

[jeremyp]

You can step through a large block of code bit by bit with the debugger. You can't do that with a regex.

Re:Here's my short list

[AlgorithMan]

dude, the rules for E-Mail adresses were written in the 1950's, when people had no f*cking clue what they were doing... just look at the SMTP RFC and you'll see that pretty much every design decision about E-Mail is totally idiotic!
the protocol isn't even unambiguously decodable when you think about that "boundary" bullshit or the separation of different mails by a single . which can also be part of the message body - and don't get me started on the CRLF bullshit...

[0-9a-zA-Z\._]+@([0-9a-zA-Z_]+\.)+[a-zA-Z]+

would be a perfectly fine regexp for email adresses, if they hadn't screwed up everything so badly, back then... the 255 byte limit in favor of stupid static buffers - this is what really blows the regexp up so much...

Re:Here's my short list

[beakerMeep]

You missed my point. Why do you think somehow the engine that parses the RegEx is where the long hours of debugging are going to happen? We're not talking about compilers, SDKs or runtime environments here, we're talking about writing code. What kind of weird apples to oranges strawman is that? Should we compare the engine that parses if-else with the guy that is reading RegEx?

Anyone writing a RegEx is equally capable of making a mistake as someone writing an if-else. The difference is that one is easier to read later on, when you need to figure out the mistake.

You're also painting a false dichotomy. Lots of the work people use RegEx for can be moved to re-usable utility functions like stripWhitespace( str ); etc. It's not just RegEx or if-else with no other options.

I like how you threw in a nod of insulting bravado in there too. Now there's a convincing way to get people to use RegEx.

Incidentally, I love = you ? true : false;

Re:Here's my short list

[beakerMeep]

//This code is not self commenting

To each their own, but RegExs, to me, are far more rickety. You're essentially adding a language on top of a language, and the one your are adding has no cues for your brain to pick up on like it would reading English based code.

Re:Here's my short list

[BiggerIsBetter]

Your statement reminds me of novices who avoid ?: in favor of if/else because it's 'cryptic'.

It's not about being cryptic. I use if/else and I've been cutting code for decades. if/else and ?/: do the same thing, while one is easier to read while you're scanning through lines of code (it's just English after all). That the novices can understand what's been written is a bonus.

Re:Here's my short list

[beakerMeep]

Replying to both you, Sarten-X, and the parent post: I think those are excellent solutions and ones I wish more people would take. Ultimately, if the code is so complex as to require a long regex or many if-else statements, you are trying to do too many things at once and should separate the tasks out to functions/objects/classes for better cohesion/encapsulation.

Re:Here's my short list

[dbIII]

The universe where BASIC morphs into something that is really Pascal then turns into something that is really Java - in other words the things Microsoft decided should have that label slapped on them.
Thus it should really be a "mid-90s interpreted language" because the thing squirmed about as fast as a cat trying to avoid a bath.

Re:Here's my short list

[Anonymous Coward]

This farcical example highlights the shortcomings of regular expressions; you're actually better off writing a real parser in this case.

Re:Here's my short list

[Anonymous Coward]

Nope. That regex is auto-generated from a real parser's grammar, and is used expressly because it is faster than loading the real parser.

Re:Here's my short list

[Acuram]

I'll never understand all the hate for VB.

Re:Here's my short list

[Zerth]

You can step through a large block of code bit by bit with the debugger. You can't do that with a regex.

There are many tools to go through a regex step-by-step(e.g. regex-coach).

And even if there weren't, you could just break it up into the constituent pieces and evaluate them individually and sequentially. If you have trouble with that, other regexp testers like regexpal have syntax coloring. I imagine several IDEs do, as well.

Re:Here's my short list

[Sarten-X]

You become a butterfly farmer to raise any kind of butterfly you want, rather than just what you already have. Chaos theory is great and all, but embedding infinite complexity into your programs is a good way to fail that ever-upcoming performance review.

This is exactly what I'm referring to. If the grammar isn't readily obvious, can you still write a parser for a language, and do so in a manner that's readable to others?

Last I checked, flex made some pretty large files just to parse out a simple language. How will your coworkers feel about having to maintain a few thousand lines of unnecessary parser code? (Note that this is based on looking at this example.)

What about a network protocol, where the whole structure may not be available at once, and may in fact be larger than your computer's memory can handle?

Sure, you may have learned how lex works, but can you actually apply that to writing a good parser yourself?

Lex, and all compiler theory, is great for compilers. If you have your entire structure in a nice well-defined form, and you're only dealing with a tiny amount of data at a time, compiling practices work fine. Once circumstances get weirder, you need to be able to adapt. Lex doesn't like adapting, and its processes are so abstract as to be ridiculously excessive for most jobs.

Re:Here's my short list

[Sarten-X]

Here's my personal checklist. If, off the top of your head, you can come up with a good plan to approach these formats and situations (using no libraries), I'd love to know where you went to school:

• CSV, including escaping
• SGML
• HTML (including common errors)
• WAV
• PNG
• JSON
• Brainfuck, with an extra instruction to interpret the output as brainfuck
• A serialized tree structure where each line is a node, and depth is represented by indentation
• ID3v1 (possibly including the enhanced tag)
• ID3v2
• All of the above involving a 1 terabyte file
• All of the above streamed over a network, and interrupted at any point. Your parsers must salvage whatever data they can.

Given that it's 1:45 AM, I reserve the right to amend this list when I'm thinking clearly.

Re:Here's my short list

[Anonymous Coward]

I perfectly agree that readablility is amongst the most important tasks when writing code. But by replacing regexes by if then else statements for the things you usually try to achieve with regexes you generally won't gain so much readability. And if you want to clearly express the intent of your code, why don't you make use of unit tests which I regard the best way of code documentation (well not really the best... for that is readability actually).

Re:Here's my short list

[BikeHelmet]

Nobody knows how to write parsers anymore. I've never seen a recent university CS curriculum that covers parsing with respect to different parser constructs for different languages. Sure, the students learn to load up the chosen XML parsing library and pull in XML, and they learn to take text from stdin, but there's seldom any emphasis on what to do with more screwy formats. Maybe, just maybe, they might get exposed to different languages through a compilers class, but generally not how to process different languages outside of lex/yacc. This bothers me greatly.

Warning: Long rambling post follows. To summarize, I haven't found any courses that covered anything close to parsing, except maybe invoking XML parsing libraries and stuff.

Before I took any programming courses, I figured out how to create an XML parser in javascript. I had a dream of making a game, which needed server-side storage, but I never got around to finishing it. Quite an experiment, though.

After that I went on to help decode Outpost 2's compiled map format. (maps were DLL files - oh the design choices of old games!) I wrote an editor for it in Jamascript (a mirror of Javascript - see Clickteam), and wrote my own tile renderer with smooth scrolling. It worked quite well, except in XP SP2. Lack of further interest from the community caused that project to die, but got me interested in compiled languages - with a compiled language, I'd be in control of whether it worked on an OS or not.

So I decided I needed to learn a compiled language, but since I didn't like C, I started by porting my XML parser to Java, and added writing capabilities. I used it for some projects where I didn't need to deal with DOM nodes and stuff; I just needed to spit out data as XML <key>value</key> pairs, which it did just fine. Although the parser was simple(200 lines, max?), and couldn't read complex XML, it could generate valid simple XML, which any other parser could read. Oh, and it failed gracefully if the file was too advanced. (that was my biggest accomplishment, and required rethinking how I had designed it)

Then I changed it to pre-pend itself to other files as a sort of metadata. That way I could attach info like the number of frames, size of frames, etc. to a tileset PNG. You can see where I was going with this... I wanted to make a game. But somehow I got bored and stopped working on it again.

All this working with Java led me to a few inevitable conclusions. I just plain don't like some of Java's syntax verbosity. Some of the choices make the code stretch on and on and on, when readability could be enhanced by doing the opposite. I wrote a code pre-processor, to enable (among other things) keywords like public: and private: - also var, so that I could easily deal with statements like this:

public ArrayList<SomeObject> myList = new ArrayList<SomeObject>();
->
var myList = new ArrayList<SomeObject>();

It just felt cleaner to me - perhaps because of my javascript roots. I had a dream of creating an entire IDE, with background colours (changing based on scope, public/private, etc.), but I never actually got around to writing that... in the end I went back to other IDEs which lacked my features, but had stuff like code completion. (quite handy)

Now I'm working on many other projects - but I must say, I haven't encountered a course yet that covered any of this stuff. I'm probably looking at the wrong college courses, or maybe there's just not enough jobs that demand it? I had no trouble finding, Ruby, Python, Java, C#, and C++ courses, but none cover this stuff. I just had to tinker to learn it.

Re:Here's my short list

[the_arrow]

Last I checked, flex made some pretty large files just to parse out a simple language. How will your coworkers feel about having to maintain a few thousand lines of unnecessary parser code?

Why would you maintain auto-generated code? You just maintain the flex input file, and whenever it changes a new big c-file is generated.

Re:Here's my short list

[GigaplexNZ]

That's not a fair test. To be able to come up with a parsing solution to all of those formats off the top of ones head, they need to know in advance all about all of those formats. Not memorising formats that one has never encountered doesn't indicate the inability to design parsers.

Re:Here's my short list

[styrotech]

In what universe is a language developed in 1964 and first marketed by Microsoft in 1975 a "mid-80s interpreted language"?

Maybe that referred to AmigaBASIC that MS wrote for the Amiga in the mid 80s. AmigaBasic was a version of BASIC with event handling and built in support for the WIMP interface. AmigaBASIC seemed to me like an 80s stepping stone between the traditional BASIC of the 60s and 70s and VB in the 90s.

Re:Here's my short list

[Sarten-X]

So the alternative is to say "This magic file is magic. Don't touch it." and expect that to be better? Let's hope there's nobody has to debug it.

Re:Here's my short list

[Anonymous Coward]

Your statement reminds me of novices who avoid ?: in favor of if/else because it's 'cryptic'.

Code should always be written for the novice, because the novices exist. Want to educate the world? Become a teacher.

The greatest challenge of programming is to write simple code.

Re:Here's my short list

[Sarten-X]

All those formats are readily available, and I'm not asking for detailed implementations. For example, CSV is just a state machine with a buffer. SGML's simplest parser is a primarily a stack. ID3v1 is a fixed-length read from a fixed location.

It's not about being able to write a given parser from memory. It's about recognizing language types and the parser patterns that can handle them efficiently.

Re:Here's my short list

[flux]

In that case I believe you must often whip out the disassembler to weed out bugs. After all, object files are where the real code lies in and they are merely generated from some (for example) .c-files.

Re:Here's my short list

[haxney]

Pseudo-code:
'If Email Address
If (String.RegExMatch("^([a-zA-Z0-9_\-\.]+)@((\[[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.)|(([a-zA-Z0-9\-]+\.)+))([a-zA-Z]{2,4}|[0-9]{1,3})(\]?)$"))
[...]

Well, it WOULD be more human-readable if you didn't IMPROPERLY escape "-" and "." within your character classes! </nerd-rage> ;)

This may actually be a VB (or whatever library you are using), but generally (at least PCRE and Emacs), "." inside of a character class statement (square brackets) doesn't need to be escaped, and likewise for "-" if it the last character in the character range.

So your first capture group could be:

([a-zA-Z0-9_.-]+)

since "." doesn't have a special meaning inside the character range, and same for the "-", but only because the engine can know that you didn't mean something like "A-Z", since it is the last character.

And that is your random regex lesson of the day ;)

Re:Here's my short list

[jesset77]

Sorry, you're going to have repost that, it looks like your modem had some problems and there is quite a lot of line noise in it.

Honestly, if you think that is readable you need to get out more.

Look, it's quite simple. Show us how you would do that as if-then-elses. Once you do, then I'll have a witty retort about how your mess isn't readable (or isn't practicable, I'll have to wait and see what you post to know what problems it will have. ;3)

Re:Here's my short list

[jesset77]

You can step through a large block of code bit by bit with the debugger. You can't do that with a regex.

I think this post really illustrates the gulf between those who fear regex and those who do not.

A lot of coders today simply do not know how to code (or know how to and avoid doing so), they know how to set breakpoints in a debugger looking for large duplo-blocks of code to comment out or alter in predictable ways until the code outputs what they expect from it.

Now if we are lucky, if we are very lucky the coders who do this are forced to at least write regression tests for their changes. But at the foundational level, they don't know what they changed or why it worked, they just banged the code with a hammer until the rattling stopped. Not having to do so again next month is above their payscale.

Once you abandon this dangerous approach and assume that debugging involves actually grokking the code, then regex suddenly becomes the right tool for a wide variety of parsing jobs. While obviously very terse, it is also rich in it's ability to comment, build larger pattern matches out of smaller, atomic, easily tested pattern matches, etc.

Regex requires someone familiar with Regex to debug. Pages of if-then-else require someone with an automated debugger and no hope of seeing the bigger picture to debug. Regex simply wins this battle, so long as the author is careful to comment and build up well labeled match strings atomically. I am not trying to say that a majority of regex slingers do The Right Thing, I am saying that with if-then-else alone it is not possible to do The Right Thing and no amount of debugger-friendliness changes that.

Re:Here's my short list

[jesset77]

To each their own, but RegExs, to me, are far more rickety. You're essentially adding a language on top of a language, and the one your are adding has no cues for your brain to pick up on like it would reading English based code.

The thing is, the correct way to maintainably invoke regex is to build long pattern matches out of constants and keep the constants both simple and easy to separately test.

That way, the constants you build the regex out of are both self-documenting and "cues for your brain to pick up on based on English words".

The inherent advantage of humans knowing what "IF", "THEN" and "ELSE" means is completely nullified after many pages of it. I understand that code folding IDE's can make such efforts more manageable, but IMO relying on the next debugger using a compatible code folding IDE is just as dangerous as relying on the next debugger being as much of a Regex wizard as you are. (and, apologies if mentioning fancy IDE is a strawman, I just can't pre-emptively guess how else you might try to defend pages of if-then-else. Just being pro-active. ;3 )

Re:Here's my short list

[xiong.chiamiov]

Your statement reminds me of novices who avoid ?: in favor of if/else because it's 'cryptic'.

It's not about being cryptic. I use if/else and I've been cutting code for decades. if/else and ?/: do the same thing, while one is easier to read while you're scanning through lines of code (it's just English after all). That the novices can understand what's been written is a bonus.

Personally, I find that the conditional operator is easier to get meaning from, which is more important. For instance, in
foo = isBar() : bar
? isBaz() : baz
: qux
the fact that you are determining the value of foo (and only determining the value of foo) through this logic is obvious (to someone who understands the conditional).

Aside: forgive my poor formatting; it appears that /. doesn't like my pre tag :/ .

Re:Here's my short list

[xiong.chiamiov]

Also, I appear to not be able to write ternary ifs correctly >. .

Re:Here's my short list

[mr_mischief]

You can do that with most regex systems. With Perl's you can do even so much more. If you have perl installed, try 'perldoc perlre', or you can readthe perldoc for perl regular expressions on perldoc.perl.org instead.

If you're intrigued by that, stop by Perl.org and PerlMonks. It's a much-maligned language, and for really superficial reasons. The modern best practices for the language answer most of the problems people ever had with it. The rest of the problems are personal preferences stated as fact, PR failures, and hyperbole.

Re:Here's my short list

[mr_mischief]

Perl's regex engine, at least, supports Unicode. I'm sure other languages's regex systems will soon if they don't already.

Re:Here's my short list

[mr_mischief]

Adding a language is a problem? It used to be that people said to use the best tool for the job at hand. That often included writing parts of a system with different needs in different languages.

C and assembly is one example. Regular expressions, EBNF, and C is another. Perl and regexes is a third (although in Perl regexes are basically a sub-language, like printf in C but on a bigger scale). PHP, SQL, and JavaScript seem pretty popular together. What exactly do you use for a database, and is it accessible with SQL? Are your applications written in SQL?

Re:Here's my short list

[Anonymous Coward]

I've only ever seen novices use ?: instead of an if/else. Experienced coders prefers the readable code that does the exact same thing without errors.

Regular expressions, on the other hand, do work that could not practically be done without making a very elaborate attempt at if/else string parsing. And I can't recall ever having troubles understanding one either.

Re:Here's my short list

[Surt]

I think experienced coders tend to use language tools without errors that egregious. :-)

Or if it's user error you're worried about, how stupid would you have to be to get that wrong? Probably stupid enough to get it wrong with the if/else as well.

Re:Here's my short list

[Anonymous Coward]

Yeah, I never use if, only the ? operator. Shows the world how leet I am!

Seriously: it isn't the regexp engines that are the problems, it is that even with a lot of experience, you will still write regular expressions that don't mean what you think they mean. And you won't know, because it's read-only code.

Although... some years ago, some crazy haskell hacker wrote wrappers for a bunch of regexp libraries, and then supplied the formal regexp language definitons and ran quickcheck. He found bugs in a lot of widely used implementations, including the libc one in FreeBSD and OSX. So maybe regexps are hard to implement, as well as write?

Re:Here's my short list

[Sarten-X]

If the project were written in assembly, then yes. In my opinion, projects should try to avoid adding extra unnecessary languages. Sure, if you need 500 different parsers for an otherwise-C project, lex is a good way to go. To just read those few data files you need, you should be able to write your own parse, and not significantly increase the maintenance requirements of your project.

Re:And why?

[neumayr]

Just a reading comprehension test.

You fail by the way. Epically.

Re:And why?

[XnR'rn]

I am about half way through the article in the second link, and it is really interesting, and informative. :>
Maybe not news, but it is worth your time (or at least mine).

Missing the point

[Anonymous Coward]

Joel was pointing out a case where an application was rewritten IN THE SAME LANGUAGE. These guys rewrote their application to move it to a new language / platform. It's not a REWRITE, it's a PORT.

Re:Missing the point

[mikael_j]

Actually, from what I got from the article it seems they also felt that the basic design of the original version of application just wasn't good enough, that it was in fact seriously lacking and that a gradual rewrite would take longer and not accomplish what they wanted (to clean up and future-proof their application).

Re:Missing the point

[mwvdlee]

Besides, it's not like a clean-room implementation when you do a rewrite.
Sure, the architecture and language may be different, but a lot of the old code would likely still be used as "inspiration".

Re:Missing the point

[commodore64_love]

Well here's a story from the stone knives and bear claws-era (early 80s):

Two programmers were tasked to convert the Atart VCS/2600 game Pitfall 2 to a Commodore=64 and Atari 800 computer. One said, "The Atari console is so primitive that it's easier to recreate the whole game from scratch," and the other said, "No just copy the 6502 code and then modify it for the varying graphics/sound chips." They then went their separate ways.

- The Commodore=64 programmer recreated the whole game from scratch, and produced a slightly-flawed but decent port.

- The Atari 800 programmer simply dumped the code directly, and then modified it. He produced a port that played identical to the original PLUS he had enough time left-over that he added a whole other game (basically Pitfall 3). So Atari 800 purchasers got two games for the price of one.

Reworking is faster than starting over. Even if the design is a complete mess, there's typically SOME modules that can be reused, and that's time saved

Re:Missing the point

[mikael_j]

You're assuming that the original design isn't an organically grown mess of code that's grown and mutated over the last 10+ years (this is pretty common on the business world), preferably written in some proprietary and deprecated language (ASP + VBScript is a current classic, most likely pushed to the company as a good "business language" by some MS sales drone). After ten years of "organic" growth of such apps cleaning them up generally takes longer than just rebuilding from scratch in a sane language with proper separation of presentation, business logic and data storage, exporting the database to the new database and calling it a day. Really, I've done this more than once and sometimes these code bases end up so rotten that it's painful to fix minor bugs, stuff that should take thirty minutes ends up taking the better part of a day because the entire codebase is a total mess with include-o-mania run wild (anyone remember that from the PHP3 days as well? lots of if(condition) { include("filename.php"); } crap that almost impossible to follow).

Re:Missing the point

[St.Creed]

Whenever I hit my thumb, I blame the stupid hammer as well.

Or, in other words: a fool with a tool is still a fool.

You can use assembly and have decent code, with clear separation of concerns. Or you can have a 4GL programming tool and still make a mess. Which is exactly why some programmers are 10 times more productive than others.

So where I worked we had ASP+VBScript (supplemented with VB6 COM+ modules running with transactional integrity on an Oracle database) and clean modules, separation of concerns and code that we could easily understand and maintain (even the junior programmers had no trouble getting used to it in a few weeks). We built most of the business apps in the last place I worked on such a design. It still works, is very easy to maintain and transfers cleanly to IIS 8 and Windows 2008. All our database code is in a single (small) module, same as the business layers. Presentation layers is a bit more complex but when transferred to .NET you can just get rid of it altogether because .NET takes over that part. Which is exactly what is happening now, ofcourse.

Don't blame the tools for the lack of ability of most programmers.

Re:Missing the point

[mikael_j]

I didn't blame the tools and since asking you to re-read my post will probably do nothing I'll try to explain it.

Certain tools are more dangerous than others. Few people can cause large-scale damage with a screwdriver but most people can do serious damage with a few sticks of dynamite even though both can be equally useful for different purposes. A bit extreme perhaps but the point is that VBScript is not a very good language for apps that you wish to keep maintaining and adding features to, add into this the fact that it is often mandated from above ("You will use this dynamite when erecting the wall, I don't care if you think a hammer is a better tool, I'm the CxO DAMNIT!"). These business applications also tend to be built according to specifications which are missing crucial features and when upper management realize that features are missing they feel that "it's just a few lines of code" so they allocate one or two programmers to add the feature with a three day deadline even though properly implementing the feature would require more programmers several weeks in order to properly restructure the code, instead you end up with ugly hacks. After a few years this codebase begins to look like a building held together by twine, duct tape and the occasional 2-by-4 hammered in place just to keep a wall from falling down. Now when you're working with a codebase like this it tends to get to the point where you end up accepting that things will break when you fix other things because all the different parts seem to be connected in the strangest way ("Why does the foo module crash when we fix the divide by zero bug in the bar module?" "Oh yeah, I think a couple of years ago someone decided to use part of the bar module for some of the foo processing to check if the input data was erronous").

Basically, the tool may not be useless but when coupled with an environment in which the tool is only being used "because I say so" and there is never enough time to do stuff properly certain tools can definitely be more dangerous than others (IMO one of the advantages of object oriented languages and designs is that with a proper base design it is a lot easier to keep parts of an application separate from each other which makes rewriting only part of the app easier).

An example of a typical "VBScript horror" is the "include-o-mania" I mentioned, this was also common with a lot of non-OO PHP code. Basically you'd have programmers put dozens of include statements in their code in the fashion I detailed above which made the code close to unreadable. While the language may not have outright encouraged this it sure made it tempting.

Re:Missing the point

[pommiekiwifruit]

They got source code? That makes it a lot easier. :-) source code with English comments would be even better...

(Japanese beat-em-ups for example are notorious for vast wodges of copy-and-pasted assembly code that is hard to understand if you can't follow the Japanese comments).

Anyway, the kernel of a 2600 game (which is half the code) is completely unlike a C64 game engine. The game logic would have been easy to port so it would make sense to keep that. And I would hope the graphics and sound were much improved anyway (I quite like pitfall 2 on c64 as it happens).

But "dumping" the code (i.e. reverse engineering a 2600 game from just the ROM) is time consuming especially back then when the tools would have been non-existent. Now we would use a debugging edition of MESS but on a 4.7MHz PC it would be another matter altogether, and a lot of time things were converted "by eye" instead.

Re:Missing the point

[commodore64_love]

>>>But "dumping" the code (i.e. reverse engineering a 2600 game from just the ROM)

Why on earth would the two Activision programmers need to do that? They could just walk down the hall to David Crane's office and say, "Hey the boss wants us to port Pitfall 2 to the Atari and Commodore computers. Can we have your 2600 source and notes? Thanks." Simple.

As for the C64:

It doesn't play "right". I first played the original game on the Atari console, and later the C64 game. The character doesn't act the same. But on the Atari computer version, it feels identical to the original console version.

Re:Missing the point

[Hal_Porter]

Actually back in the 70's and 80's licensing worked like this

http://www.dadhacker.com/blog/?p=987

I should explain how Atari’s Arcade conversions group worked. Basically, Atari’s marketing folks would negotiate a license to ship GameCorp’s “Foobar Blaster” on a cartridge for the Atari Home Computer System. That was it. That was the entirety of the deal. We got ZERO help from the original developers of the games. No listings, no talking to the engineers, no design documents, nothing. In fact, we had to buy our own copy of the arcade machine and simply get good at the game (which was why I was playing it at the hotel — our copy of the game hadn’t even been delivered yet)

I.e. the license was to make a clone of the game. There was no source code or documentation provided.

Re:Missing the point

[Calinous]

Also, I've had some issues with a C/C++ program using Fortran libraries and modules written in VBA (which the customer could modify). Trying to improve that was very very hard, and a port to more recent technologies was impossible (due to the customer needing to use the existing models/modules). So, in some cases, backward compatibility is quite difficult or impossible to do.

Re:Missing the point

[Vellmont]

Pfft. An interesting story, but irrelevant. You assume the outcome is based on the approach, and not the individual programmers or the original code. Given two different programmers with different skill sets and a different code base, how sure are you that the outcome wouldn't have been reversed?

Reworking is faster than starting over. Even if the design is a complete mess, there's typically SOME modules that can be reused, and that's time saved

Only if:
There's a module to begin with that's not dependent on some other crazy parts of the code.
The module was written in a sane manner that doesn't require a crazy architecture to support it,
Supporting the modules in question isn't more difficult than just starting over.
The modules themselves are either well written and easy to modify, or would never have to be modified to support a new feature.

Your example is also poorly chosen, since it's a game. Typically games are developed once, sold for a couple years, and then the vast majority of the code is simply discarded. There's no real maintenance of the thing, and nobody really reuses any of the code after the game is finished. This obviates much of the advantages of re-writing the software. Most software isn't a game, is maintained over decades, and typically need new features tacked on.

Re:Missing the point

[pommiekiwifruit]

You were the one who used the word "dumping" which implied object code rather than source code :-)

Re:And why?

[Ixitar]

If you bothered to RTFA, then you would see that this was a posting about a blog entry that is only two days old. It is a discussion about a current rewrite project and their comparison to the blog posting of ten years ago.

Re:And why?

[vlangber]

Hi!

I'm the author of the blog post. The Joel article is 10 years old, but not the one I wrote(the the second link).

Vidar Langberget

Sometimes to move forward

[rxan]

you must stop looking backward.

Re:Sometimes to move forward

[dotancohen]

you must stop looking backward.

And loose all your users. They are in your 'Backwards" direction.

I have filed or triaged well over 1000 bugs for KDE since KDE 4 came out. Forget about the missing features from KDE 3 to KDE 4, they don't care about breaking compatibility from KDE 4.n to KDE 4.n+1. Even Kaddressbook, one of the core KDE apps, had severe functionality loss from KDE 4.3 to KDE 4.4. Amarok, Koffice, and other lose severe core functionality when their "new versions" were actually "technology previews" with dot-oh version numbers.

Re:Sometimes to move forward

[Anonymous Coward]

The new Amarok just blows period. They could have actually done mockups and put it to a vote using a poll on their site, produced two or three demo UIs with only basic functionality implemented, put it to another vote and gone with that. In the end, the devs really just wanted to reinvent the UI because table views are so 2001.

The result is slow as a pig, ugly and more confusing then the old one.

Hell, I hate it so much that I now just use a Konsole instance running MPlayer instead. mplayer album/*/*.flac FTW.

Re:Sometimes to move forward

[Anonymous Coward]

Something like this perhaps?

http://pyvm.hobby-site.org/

Re:Sometimes to move forward

[SomeKDEUser]

This comment is disturbing in many ways.

1) There was and is a KDE player with the table paradigm: juk. It works very well. In the 1.x days, I would use it for tagging, and amarok for the listening, now, I do everything in Amarok, because the tagging works now so much better than t used to, whether in the tagging dialogue or in the playlist.

2) Amarok was always about meta-data around the music: similar tracks/recommended artists, WP articles, lyrics, moodbars and whatnot. In the 1.x days it was so, in the 2.x, it is still so. There never was any other reason to use it, except for the sorry excuse that due to KDE3's broken media system, in many case, amarok was at times the only player actually working. This was and is a wrong reason to use a particular player.

3) Current amarok UI is completely customisable, in fact you can have it identical to the 1.x series. This alone makes your comment inane.

4) clearly if command-line playing does the trick for you, then you never actually wanted to use amarok, whether 1 or 2. maybe MPD is more suited to your usage pattern.

5) The usage pattern of many people is precisely yours: put everything in one big list, and then play it at random (sometimes) . Now for some reason, they need to have the whole list, and will not accept only playing at random. If you are not mentally defective and realise that the outcome is the same, amarok dynamic playlists are much, much better, allowing also for biases in favour of artists/tags etc.

6) Amarok is in fact much faster than it ever was in the 1.x days: if you think otherwise, you have a tiny collection of files. Because search is now instant and was not before. If you care about launch time, then you are relaunching your player a significant amount of times: either there is a bug, or you have OCD. This is not a flaw in the player...

7) Integration with last.fm/podcasts/etc. is cool. You may not need/want it, but who are you to deprive me of a feature I like?

Re:Sometimes to move forward

[Anonymous Coward]

But be fair: we never claimed KOffce 2 was more than a technology preview. Except for Krita. Krita 2.2, to be released next week, will be better than 1.6.

Boudewijn, KOffice developer and Krita maintainer.

Re:Sometimes to move forward

[Draek]

And loose all your users. They are in your 'Backwards" direction.

Having your users loose in your backwards direction does not sound pleasant. I mean, I can understand one or two at a time, but all of them?

I have a newfound respect for all of you, F/OSS developers. Way to sacrifice yourself for the masses, in more ways than one.

Re:Sometimes to move forward

[Anonymous Coward]

remember when gnome lost functionality in one clean break at 2.0, justifying it by the difficulty in debugging a program with absurdly many code paths, and combining it with a search for default behavior that makes sense? and now is back to normal?

still, the main difference between gnome and kde users isn't philosophy, but whether they believe in cxx as a language.

Re:Sometimes to move forward

[dotancohen]

Thank you Boudewijn for commenting.

The "N.0" moniker is an accepted and long-standing convention for a software release. Hijacking that convention and appending a "developer preview" subtitle does not change the implied stability of an N.0 release.

Re:Sometimes to move forward

[dotancohen]

Why is this modded Troll? Slashdot used to be for discussion, not bashing.

Re:Sometimes to move forward

[lanner]

I would mod this up more, if I could. Specifically, about KDE developers and their nonsense "rewrite it for fun with less features" bullcrap. KDE4 is pretty much a disaster, and I would recommend anyone thinking about using a GNU/Linux desktop to avoid anything based on KDE. It is really really sad.

Re:Sometimes to move forward

[SomeKDEUser]

Basically, I am questioning the common wisdom the KDE4 and Amarok 2 are failures. I question them mostly because whatever reproach there might be usually is stale. Stale as in: "this was fixed a year ago, now".

But hey, questioning common wisdom on slashdot is good for burning karma. Remember kids: thrashing free projects based on false impressions you got a year ago is fine. Defending them with actual evidence goes against truthiness, and we can't let a fact-based discussion take place.

But this is something I noticed: I have been hanging on slashdot since 1999 (lost my password/login at some point). It used to be that the general knowledgeability of the audience was much higher. As in, you could actually expect the average slashdotter to know some basics of computer science and of having made some contribution to a free project. Or at least be a scientist/engineer of sorts.

Now, it is mostly ranting wanabees and crazy libertarian/ultraconservative mouthpieces that do most of the talking. Not that they are a majority, be who wants to argue with these...

Re:Sometimes to move forward

[dotancohen]

I am giving a presentation tomorrow on just that subject. The problem with KDE 4.0, Amarok 2.0 and Koffice 2.0 were not that they were failures. The problem is that they were branded as X.0, i.e., stable releases. They were then packaged as such and that's how the users got them.

Re:Sometimes to move forward

[dotancohen]

Lanner, go test KDE 4.4 or 4.5 when it comes out, and let me know what is missing for you. KDE 4.0 and 4.1 were bad, but the newest releases have achieved 99% feature parity with KDE 3.5.10, and there are lots of new features to be had as well. You can reach me personally, my Gmail username is the same as my Slashdot user name.

Thanks.

Re:Sometimes to move forward

[losinggeneration]

I don't really disagree with that, but I did watch the development blogs as KDE 4 was being developed and they painted a different picture of what the KDE 4.0 release meant (at least to them.) I believe the logic behind it was for a couple of reasons. (These are in relation to KDE 4, and not Amarok or Koffice)

Release early release often. This was a cited reason. They didn't want the development life cycle to drag on for another few years just till a perceived stability of a dot release could be achieved. The libraries were at a fairly stable point. The main Plasma was still basically at a tech preview state when 4.0 was released.

Now, I'm not sure how well KDE would have faired if they would have waited till it was at a stable point before releasing. Because it would have been at least a year more. Some devs said KDE 4.2 was the stable release of KDE. I personally feel it was closer to 4.3. In the time from 4.0 to 4.2-4.3, they received a huge number of bug reports and feature requests. I think at least some of these were fixed otherwise the current versions would still be as bad as early versions.

So, it may have been almost a necessary evil for them to release KDE 4.0 when they did and as they did in order to get the large scale feedback they (IMO) desperately needed. There is also the fact that most distributions didn't even begin to pick up KDE 4 as the default until 4.1 or 4.2.

Re:Sometimes to move forward

[Bigjeff5]

It's funny, Microsoft tried this.

It took years and years to develop, because they were practically re-writing it from the ground up.

Know what it was?

It was called "Windows Vista", and it was an unqualified failure for the only group of users MS really cares about - business users.

Turns out, sometimes you can't stop looking backward even if you want to.

Re:Sometimes to move forward

[dotancohen]

Yes, 4.0 was the release of stable the stable API and libraries. Everything you've written is true. Even the "download" page on kde.org listed 4.0 as a "developer preview" and 4.1 as "for early adopters". KDE 4.2 was the first in the 4.x series that was deemed ready for end users.

Re:Sometimes to move forward

[xiong.chiamiov]

You can either install Amarok 1.4 directly (I use Arch, and there's a pkgbuild for it), or use one of the forks that are around. I am aware of Pana and Clementine.

Re:Sometimes to move forward

[mr_mischief]

This is exactly why only my play machines went to 4.x before 4.3 and my work machines stayed with the 3 series. I'm not sure why some many people think that newer is always better. I love new things, but in the end better is better.

some other scenarios

[ascari]

First, I suspect everyone agrees that rewriting an app rarely makes sense for a small company with limited R&D budget like Spolsky's. (Better spend whatever money there is on comfortable chairs and parties for the summer interns, right?)

But if you have the budget and market share you have some good reasons to rewrite from scratch:

1) You can generate fresh revenue by forcing your users to upgrade due to icompatibility issues.

2) You can ease the burden of backward compatibility by creating a new baseline.

3) You can strengthen your old brand by coming out with a "new coke" that is so crappy that everybody will clamor for the old stuff.

4) There might be (unforeseen) technical or architectural issues that make a rewrite a sensible option.

Re:some other scenarios

[Quarters]

Maybe you should read Spolsky's 2000 article. He doesn't use his small company as an example. He uses the rewrite trials and failures of such other 'small' companies as Microsoft, Borland, and Netscape.

Re:some other scenarios

[mr_mischief]

There is no disconnect between something being a strategically poor choice and having to do it anyway due to unforeseen technical or architectural issues. Sometimes when backed into a corner deeply enough, strategy devolves into tactics and hope.

Pick opensource

[Anonymous Coward]

You take the plunge and do a full rewrite from ASP ... to .Net 2.0?

That was a golden opportunity to move to an opensource stack. If it had been PHP for example, the complete rewrite probably won't have been necessary. Just replace the depreciated functions and constructs as required.

(PHP is just an example. No language flaming idiots please.)

Re:Pick opensource

[vlangber]

The rewrite was mostly based on problems with the underlying design, not the platform switch. When we had decided to rewrite it from scratch, .Net was chosen as the new platform.

If you think the effort needed to make a new web content management system that can compete successfully with established players like SiteCore, EPiServer, Alterian, Ektron and Kentico (basically the mid-range platforms and products as defined by CMS Watch ) is a matter of changing to PHP and replacing functions, you're insane.

Re:Pick opensource

[Anonymous Coward]

(PHP is just an example. No language flaming idiots please.)

Your entire post is equivalent to a language flaming post, you idiot.

Why should they go with an open source stack? If they're MS-oriented folks, they have more in house experience with those products and they'll be easier, faster, and simpler for them to use.

I'd personally have ported to something multi-platform (no, sorry, mono doesn't count), but that's mostly my technical background and the fact that I prefer to be vendor independent. Other people or organizations without my particular history and concerns can, and do, come up with equally valid solutions to things that differ from mine.

Ugh

[drinkypoo]

Drupal does indeed brutalize your database (see second link). So looking forward to D7 to clean this up. That alone was sufficient justification to rewrite the application :p

Unfortunately the author goes on to display his ignorance before this is all over: There are also other examples of total rewrites that have been successful. Apple's transition to OS X is a very good example. The classic Mac OS was an operating system with some good ideas, but the core architecture had a lot of problems. Did Apple make a bad decision to start from scratch with OS X? I don't think so. They brought over and improved the best bits from OS 9, and merged it with the solid Darwin core (much like we did). Uh, no, you are totally and completely fucked here. They started with NeXTStep, last updated in 1995; it was itself based on an older version of BSD (4.3?) and an older version of Mach. OSX is not a complete rewrite of anything. Its legacy stretches back into the 1980s, and so does its code.

Re:Ugh

[morgan_greywolf]

But Apple did rebuild their GUI shell, application APIs, etc. from scratch. Sure the underlying OS is based on legacy code stretching back into, really, the 1960s (BSD itself was, after all, based on AT&T's Unix), but the GUI and application APIs were totally new.

Re:Ugh

[Suiggy]

A better example of complete rewrites would be iterations on game engines where vast swathes of legacy code get thrown out and are completely rewritten from scratch.

Re:Ugh

[drinkypoo]

But Apple did rebuild their GUI shell, application APIs, etc. from scratch. Sure the underlying OS is based on legacy code stretching back into, really, the 1960s (BSD itself was, after all, based on AT&T's Unix), but the GUI and application APIs were totally new.

That is totally and completely false. "Cocoa is one of Apple Inc.'s native object-oriented application program environments for the Mac OS X operating system. It is one of five major APIs available for Mac OS X; the others are Carbon, POSIX (for the BSD environment), X11 and Java. [...] Cocoa is the continuation of several frameworks (primarily the App Kit and Foundation Kit) from the NeXTSTEP and OPENSTEP programming environments developed by NeXT in the 1980s and 1990s." "Carbon descends from the Toolbox, and as such, is composed of "Managers". Each Manager is a functionally-related API, defining sets of data structures and functions to manipulate them. Managers are often interdependent or layered.
Newer parts of Carbon tend to be much more object-oriented in their conception, most of them based on Core Foundation. Some Managers, such as the HIView Manager (a superset of the Control Manager), are implemented in C++, but Carbon remains a C API." And I should HARDLY need to touch on POSIX, X11, or Java, the former two of which were already functions of NeXTStep, and the latter of which was not invented at Apple.

There is NOTHING in OSX which can reasonably be believed to have been invented (in any sense of the word) for OSX! Period, the end. ALL portions of OSX contain legacy code, from the microkernel to the presentation layer. They didn't go from Display Postscript to Display PDF because it was easy, they did it because they couldn't just throw over Display Postscript and be able to utilize existing GUI code. They didn't throw out the GUI and replace it, they replaced the engine under it, then made it work on that engine. The appearance of the GUI is still customizable, as it was under NeXTStep, by mangling the images it's made of.

Re:Ugh

[Anonymous Coward]

Unfortunately the author goes on to display his ignorance before this is all over:

The discussion is about code reuse and whether or not to rewrite applications. It is not about how much code Apple reused in one of the examples... or how much you know about Apple's kernel. Try commenting on the points TFA was making. Like most here you are focusing and complaining on details that are not germane to the discussion. So what if he is off a little in one point (as stated elsewhere, Apple did rewrite most if not all of the GUI and APIs, if not the OS kernel)?

Re:Ugh

[jeti]

The original rewrite of MacOS was Copland, which used up Apples development resources for five years before it was cancelled.
http://en.wikipedia.org/wiki/Copland_(operating_system)

Re:Ugh

[drinkypoo]

The original rewrite of MacOS was Copland, which used up Apples development resources for five years before it was cancelled.

Yeah, I remember that, it's what convinced me to stop being a Mac user. I had the INIT or CDEV or whatever that made your MacOS look like Copland. Then they cancelled it and brought out another shitty MacOS, and I ran like hell and never looked back. All the years from System 7 through System 9 were sad, abusive times to be an Apple customer. They kept bringing out inadequate hardware at astounding prices and almost spent all their cachet. Another year or two and they wouldn't have had enough fans left to float OSX.

Re:Ugh

[noidentity]

And Mac OS X isn't an application, either, so he's doubly wrong. (BTW, your subject line is uninformative)

Re:Ugh

[Blakey Rat]

Not to mention:

They brought over and improved the best bits from OS 9,

Apple *threw out* the best bits of OS 9. Right into a dumpster. Oh, they half-heartedly pretended to care for a short while (re-adding some features like Labels in Finder), but it was obvious they did not. They certainly didn't "improve" Labels when they *finally* got around to porting it. And many OS 9 Finder features they never bothered to port in the first place.

Not to mention abandoning the entire philosophy OS 9 was based around-- a spatial GUI. Not just the code itself, but the entire underlying development philosophy was chucked in a bin.

Pretty much every point in that paragraph is false. Wrongness like that pisses me off.

Re:Ugh

[morgan_greywolf]

Interesting bit of history; I had always thought that Carbon and Cocoa were new APIs implemented in the same fashion as NextSTEP/OPENSTEP, but not derived from its APIs like AppKit, Foundation Kit or Toolbox, but apparently this isn't even remotely true. Thanks for that; now I know. :)

Re:Ugh

[Vellmont]

We're getting into the realm of semantics and definitions here, which usually isn't terribly interesting.

So the question is really "What does it mean to re-write something from scratch"?

What I think it doesn't mean is starting from 0. I think it means starting over and abandoning the vast majority of the original code base for a new one. Nobody really goes and tries to re-write every single line of code of something without using some pre-built components. Just like in cooking you wouldn't try to create every ingredient yourself. (Who makes their own Worcestershire sauce for instance).

So while I think your distinction is important, it doesn't really degrade the original point of the article that starting over has certainly worked in the past.

Re:Ugh

[willy_me]

There is NOTHING in OSX which can reasonably be believed to have been invented (in any sense of the word) for OSX!

In any large update there is always going to be some reuse of code. But I believe what the grandparent post was referring to was the fact that the design of OSX was completely new thereby requiring a complete rewrite.

Just look at the Core Foundation classes - they were created for OSX. NeXTStep implemented the required base classes in their Foundation Kit. But OSX required the ability to run Carbon apps as native applications - difficult when your base classes are in ObjC. So they rewrote the base classes in C or C++ thereby allowing them to be called by traditional Carbon apps (and providing a speed boost). An additional layer (Cocoa) was then added to allow ObjC apps to also make use of these base classes via the traditional NeXTStep API.

So the point is that OSX was a complete rewrite. Sure, components were reused - as they should be. But OSX is not simply an evolved version of NeXTStep. Using the required car analogy - Apple didn't put a new motor and bodywork into that old mustang frame, they cut the frame up and built a new one. Parts from the old mustang were only used when they fit the new frame. Many new parts had to be built from scratch.

Note - the term "invented" isn't really accurate. When replacing a legacy system with a complete rewrite, one does not have to "invent" anything. All was already "invented" by the legacy system. A rewrite is simply a new implementation of a system - one not based on the previous system, but designed to replicate the functionality of said system.

Re:Ugh

[drinkypoo]

But OSX is not simply an evolved version of NeXTStep.

OSX is to NeXTStep as Windows 7 is to Windows XP. There's probably closer comparisons to be made but this one will do. Basically the same shit, with some things implemented with the new shit, and some things implemented with the old shit. An obvious improvement, but staggeringly slower in some areas which don't matter to most users because they have so much more hardware for so much less money. Riddle me this: Does OSX really do so much more than NeXTStep that it justifies the dramatically higher overhead and yet annoyingly higher UI latency?

Re:Ugh

[drinkypoo]

So the question is really "What does it mean to re-write something from scratch"?

What I think it doesn't mean is starting from 0

When I cook biscuits from scratch, I don't start with baking mix, or leftovers. I start with flour, and eggs, and water, and milk, and baking powder, and I end up with biscuits. I am lazy enough to use the food processor, but it replaces sifting and mixing. I certainly don't use pieces of old biscuits.
The simple truth is that if you don't start over from New Project, or a blank editor, you're not starting from scratch. There's benefits and drawbacks to each approach. Starting over allows you to make major architectural changes that would otherwise require changes to large swaths of code anyway, without having to worry about whether you changed it (and properly) in every occurrence. Not starting over means, in theory, writing less code. In practice, almost all significantly-sized applications utilize external libraries beyond those considered part of the core operating system, so even most software projects which start over "from scratch" really have to reinvent everything they use. But if they're accepting external libraries as black boxes or even just as something they don't want to maintain themselves, then it's reasonable to exclude that code from the assessment.

Re:Ugh

[Vellmont]

When I cook biscuits from scratch, I don't start with baking mix, or leftovers. I start with flour, and eggs, and water, and milk, and baking powder,

All of which are components you didn't create yourself. You didn't, for instance start out with flour seeds, chickens, cows, hydrogen, oxygen, an acre of farmland and whatever you'd need to create baking powder.

The point of my analogy is of course that much like in cooking, everyone in software development starts with some ready made component. Is taking NextSTEP off the shelf and starting with that really any different than buying eggs at the grocery store?

Re:Ugh

[kobaz]

You mean like duke nukem forever ?

Re:Ugh

[EpsCylonB]

yeah they both become unusable really quickly

Re:Ugh

[halcyon1234]

All the years from System 7 through System 9 were sad, abusive times to be an Apple customer. They kept bringing out inadequate hardware at astounding prices

I take it that means Apple is currently on, what, System 8?

Re:Ugh

[Bigjeff5]

And Mac OS X isn't an application

I've got to disagree with you there, it's just an application that runs at a lower level to make it easier to run higher level applications. It's known as an Operating System, but it isn't fundamentally different than any other application, and it is identical to other applications with regard to re-writes and such.

There are actually a few good reasons

[Opportunist]

The problem is that companies usually rewrite for all the wrong reasons.

A good reason would be the emerge of a new technology that supports your problem much better, to the point where redoing your code from scratch means easier maintainance later. Usually this goes hand in hand with an old technology (the one you used so far) getting abandoned by its maker. A good example would be how I had to maintain a client/server app written in VB6 using DCOM. Not some quick hack, a full blown client/server solution it was never meant to be, that also has to communicate with a SAP interface and a few more nifty things that cried out "please rewrite me". The overhead to maintain it soon turned from insane to nuts and even the tinyest change required a team of four people to work for weeks.

Unfortunately, the reasons why something gets rewritten are usually different. Like at my next gig. A pretty well designed and fairly solid piece of code was thrown out when the original creator was fired and someone with an ego that required its own office took over and insisted we use "his" collection of libraries instead of that "old" libraries. We trashed about 2 manyears of labour down the drain to end up with what we had before.

Re:There are actually a few good reasons

[jonwil]

Often the problem is that there are things that should be rewritten (either whole apps or specific sections of code) but no-one is willing to green-light such a rewrite even though the rewrite will take less time than bolting hack after hack on top of the old system.

Re:There are actually a few good reasons

[coryking]

even though the rewrite will take less time than bolting hack after hack on top of the old system

Often the problem is we developers are wildly optimistic in our estimates of how long such a venture might actually take. Especially if we are talking about code we dont know much about. Until we really get to know the problem domain, it all sounds so easy :-)

In my opinion, what Joel really is against is a wholesale "drop everything you are working on and rewrite the entire app". From hard-won experience, he is right... When you drop everything you are working on, you stop pushing out exciting new features to your customers. Said customers get pissy that nothing seems to be changing and start to look elseware for their software fix.

Doesn't matter if the product is internal or something for the public. If it is internal, the teams who use your product start getting pissy with your team and look to out you. If it is external, your customers look to buying or using other competing products (like jumping from Netscape to IE5 and IE6).

If the old code really does suck (and if it is a combo of classic ASP and vbscript, it really does suck), the best route is to look at how to migrate your infrastructure a bit at a time to a better one.

For example, when you improve the admin site that uses ASP/vbscript, all new code has to use javascript and .net 4. Then hack the load balancer so that the requests get routed either to the legacy platform or the new platform.

Basically when you add a new feature or improve an existing one you port it to the new platform. That way you still maintain excitement by pushing out cool things while in the background you are improving your codebase. Yeah it might take a bit more time to crank out each new feature or fix, but at least you are cranking out bits and pieces instead of nothing for 5 years.

Course, this is my opinion. Yours may very.

Re:There are actually a few good reasons

[Kijori]

I quite like the way that the article claims that it shows an exception to Spolsky's rule, but actually isn't at all: they claim to have started off as a successful CMS company with "big name" clients, embarked on a rewrite that took them off the market for two years and ended up as a tiny player with "more than two hundred web projects [...] in Norway".

As far as I can tell this is a project that went exactly the way Spolsky predicted: they had a decent product, they embarked on a rewrite that took longer than they expected and they lost the market by doing it.

Re:There are actually a few good reasons

[vlangber]

You're correct, we were reasonably successful before the rewrite started, but we started to see limitations in the original architecture that would become much bigger problems in the future. So yes, we traded growth for a while, in order to get a solid foundation. We still think we made the right decision, as we feel our future prospects are so much better now, than if we had continued with the old system. Time will tell.

Re:There are actually a few good reasons

[Kijori]

Well I wish you all the success in the world, but the article - and particularly, surprise-surprise, the /. summary - are extremely disingenuous. Let me quote from Spolsky's article:

Netscape 6.0 is finally going into its first public beta. There never was a version 5.0. The last major release, version 4.0, was released almost three years ago. Three years is an awfully long time in the Internet world. During this time, Netscape sat by, helplessly, as their market share plummeted.

It's a bit smarmy of me to criticize them for waiting so long between releases. They didn't do it on purpose, now, did they?

Well, yes. They did. They did it by making the single worst strategic mistake that any software company can make:

They decided to rewrite the code from scratch.
[...]
First, there are architectural problems. The code is not factored correctly. The networking code is popping up its own dialog boxes from the middle of nowhere; this should have been handled in the UI code. These problems can be solved, one at a time, by carefully moving code, refactoring, changing interfaces. They can be done by one programmer working carefully and checking in his changes all at once, so that nobody else is disrupted. Even fairly major architectural changes can be done without throwing away the code. On the Juno project we spent several months rearchitecting at one point: just moving things around, cleaning them up, creating base classes that made sense, and creating sharp interfaces between the modules. But we did it carefully, with our existing code base, and we didn't introduce new bugs or throw away working code.

With a few name changes that could be the story in the article and your explanation of why it was necessary. There doesn't seem to be any example here of "when rewriting makes sense" - you did exactly what Spolsky said not to do and now, well, no-one on Slashdot even seems to have heard of you.

Re:There are actually a few good reasons

[Anonymous Coward]

Ss someone who still needs to work in VB6 I feel for you. Since VB6 was released in 98 it's missing so many crucial Windows technologies that you can spend more time reimplementing them than writing your actual program. Even working with the registry wasn't implemented natively and you have to do it all with external API calls. I've seen some VB6 apps that were almost all external calls...which makes you wonder why they didn't just do it in C++ and avoid all the trouble of converting the data types and pointers to work in VB. Even simple GUI enhancements like icons in menus is a long and hard coding adventure which involves subclassing (crashes the IDE when the app is run too btw) and can be done so simply in MFC it makes you cry.

Re:There are actually a few good reasons

[Kjella]

I guess it depends on what you would see in the crystal ball, if they'd be the hot big name in CMS or they would be slowly bled to death as their big names left them one by one draining their margins until they're without the financial muscle to reinvent themselves before finally making their exit as an outdated and insignificant solution. Yes, there's a few that have managed to ride the wave from crap to dominance but plenty that have sunk as well.

I've worked with many large companies and they will use software that are way below state of the art simply because of all the costs involved in installation, administration, user training and not least of which the productivity lost getting back up to speed, cost of errors they will inevitably do with new software and so on. Every piece of software has its own idiosyncrasies, things that aren't logical but that you just *do* because you've learned that's how it is done. Or because there are customizations or configurations or integrations and all of these are sunk costs that essentially have to be redone with new software. What does that mean? Well it means that you'll have a long tail but you won't be getting any more new business unless you can improve. When you finally get the boot, you aren't just slightly behind the competition you are way behind. So it might be comfortable for a while, but it's not like it'll just last...

Re:There are actually a few good reasons

[Anonymous Coward]

I think you are willfully misinterpreting the "two hundred web projects" figure - that's new projects that they have completed with the new system.

Nothing says they "lost the market" - they didn't write "all out old clients left us". It is possible to support two versions of a product at once you know.

Re:There are actually a few good reasons

[Bigjeff5]

The overhead to maintain it soon turned from insane to nuts...

Huh, I always thought "insane" was crazier than plain old "nuts".

Re:There are actually a few good reasons

[mr_mischief]

Don't forget that although they started consulting with it (meaning their people implemented it for the customers), they weren't ready to sell copies of it until 2010. That's three additional years for documentation and testing, as admitted in this post earlier this thread. So it's really a five-year rewrite of something they'd been using only 5 years, if you consider documentation/testing/making it ready to actually sell to your customers part of the process of replacing the old product.

6 KB wasted on fucking VIEWSTATE data.

[Anonymous Coward]

I just looked at their article, and 6 KB of the page was near-useless VIEWSTATE data. If they can, they really should disable the generation of that. It's a useless artifact of the broken ASP.NET WebForms approach, which isn't really even necessary for a blog like theirs.

Seriously, with a typical Slashdot posting resulting in 80,000 unique hits for the target site, they're going to waste over 480 MB of bandwidth serving up just that useless VIEWSTATE data.

Re:6 KB wasted on fucking VIEWSTATE data.

[vlangber]

Thanks for noticing..

The blog module uses templated controls. When you bind data to a templated control, all that data get's stuffed into the viewstate as default. To fetch the comment content, the template for a comment has this snippet:

<%# Container.CommentContent %>

As default in .Net, a literal control is created for that data. Strangely Microsoft decided to make EnableViewState enabled by default for automatically created controls. The reason the viewstate is so big on that page is because I forgot to put the above snippet in a Literal control with viewstate enabled:
<asp:Literal runat="server" ID="litBlogCommentContent" EnableViewState="false" Text=""></asp:Literal>

I usually check the size of the viewstate, but none of the blogposts I checked had any comments, so I didn't catch it when we created the site. I'll update it as soon as traffic slows down a bit.

Re:6 KB wasted on fucking VIEWSTATE data.

[Rockoon]

If they can, they really should disable the generation of that. It's a useless artifact of the broken ASP.NET WebForms approach, which isn't really even necessary for a blog like theirs.

Are you suggesting a rewrite?

Re:6 KB wasted on fucking VIEWSTATE data.

[K. S. Kyosuke]

Why does ASP.NET pass by value something that can be passed by reference?

Re:6 KB wasted on fucking VIEWSTATE data.

[goonerw]

In ASP.NET 4.0 it's possible to disable the viewstate on the page level and have all the controls inherit that state. Then you can selectively enable the ones that you want to use viewstate. Ref: http://msdn.microsoft.com/en-us/library/system.web.ui.control.viewstatemode(VS.100).aspx

Re:6 KB wasted on fucking VIEWSTATE data.

[Anonymous Coward]

Web server farms and/or session timeouts. It's possible to have an original request served by server a, and a subsequent request served up tomorrow server b in an entirely different country, and through the magic of passing by value stuff just works by default with no effort and not even requiring a database much less a replicated database.

Trademark

[Anonymous Coward]

"App" is Trademark(TM) to App le computer inc. If you try to use it for your own stuff not app rove by app le expect to receive in iNjunction.

The best break all the rules

[angryphase]

It's a bad idea to roll your own. In fact, Joel Spolsky (Who's comment this blog post is the basis of) even went on to explain that in some cases, that's a complete lie (http://www.codinghorror.com/blog/2008/10/programming-is-hard-lets-go-shopping.html)

So really "It's a bad idea to roll your own, except when it's a good idea".

Re:The best break all the rules

[mr_mischief]

The best know which rules to break when, and why. Not all of us are the best. I'm not. I know which few rules I can break some of the time by trial and error. I'm not about to break all of them. Reinventing wheels is one you can get away with fairly often if you actually need a different wheel from the start. Rewriting from scratch is one you can get away with sometimes, but not nearly as often.

There are times for a rewrite and port.

[AnonymousClown]

The article was interesting in how they made their decision and I don't think it was necessarily a bad decision - assuming their assessment of the old version was correct. The process looked like it broke down in the design and planning phases. The article didn't go into detail on the design and planning; which kind of leads me to believe that may have tried to do the "superstar" programmer thing and rewrite it over a weekend. Anyone whose been in development long enough has heard the "he coded it over the weekend"stories and the programmer "street cred" it gave - at least back then.

I've seen rewrites/ports go quite well. Systems that were originally on mainframes and needed or wanted to be moved to cheaper hardware for cost - if it was the proper thing to do (Sometimes you really need the metal).

Another rewrite that went well was a bunch of code that over the decades became so convoluted to be a maintenance nightmare - modify one thing or add on functionality and then break a shit load of other things.

Just do these basica things and it'll work out.

Go back to the specs and start there.

Talk to the stake holders - yep, there will be creep but also feature reduction because there are things that they never used or because it doesn't make sense anymore.

Plan, plan, plan. No cowboy programming and hacking out shit. And document everything.

It can work.

Re:There are times for a rewrite and port.

[fishexe]

Just do these basica things and it'll work out.

Fixed that for you.

Re:There are times for a rewrite and port.

[mr_mischief]

If your old code is so convoluted you can't possibly refactor it, then you've already made major strategic mistakes, and a lot of tactical ones, too. Being backed into a corner by one strategic mistake and throwing your hands up to let the superior tactics of your troops bail you out is not a new strategy. It's a prayer.

The exceptions Joel should have included

[michaelmalak]

Joel's article implicitly had some tunnel vision because it did not state any exceptions. The general rule of thumb as to when to rewrite an application are:

• Change in platform paradigm. This has happened five times thus far (post-ENIAC):
  1. Mainframe to MS-DOS
  2. MS-DOS to Client/Server
  3. Client/Server to Windows
  4. Windows to Web
  5. Web to mobile
• Vast improvement in software technology (and you have a software project/product that requires significant expansion or improvement), such as:
  1. Machine code to Assembly
  2. Assembly to spaghetti
  3. Spaghetti to structured
  4. Structured to OO
  5. Proprietary unsupported language for obsolete sluggish database to SQL

Re:The exceptions Joel should have included

[16K Ram Pack]

The other problem is with scale.

When a company starts a new project or an entrepreneur starts a small business it's sensible to build something quite small and straightforward. It might take off, but it might not, so don't spend more on the tech than you really need.

So, something gets produced that's VERY hacky. And for a short term solution and from a business perspective, that's quite bright.

The problem is that scale it up a little and it starts falling apart. It becomes hard to maintain because the fundamentals are so poor. It's generally much better to spend the time rewriting at this point than living with what you've got.

Re:The exceptions Joel should have included

[afabbro]

This has happened five times thus far (post-ENIAC):

1. Mainframe to MS-DOS
2. MS-DOS to Client/Server
3. Client/Server to Windows

What are you talking about? That's not even chronologically correct.

Client/Server predates MS-DOS. And no one moved any code from the mainframe to MS-DOS. I doubt seriously any significant application was ported to it. Seriously...lots of people moved from mainframe to iSeries or other minicomputer systems, or perhaps to Unix.

What makes you think Windows isn't client/server? Or the Web? Or Mobile? They're all client/server. For that matter, there were plenty of pre-Windows MS-DOS-based/-era client/server apps, e.g. Novell.

Re:The exceptions Joel should have included

[michaelmalak]

What are you talking about? That's not even chronologically correct.

Client/Server predates MS-DOS.

If you know of some examples, I'd be interested to hear them.

And no one moved any code from the mainframe to MS-DOS.

"Moving code" probably doesn't qualify for "rewriting from scratch", the topic at hand. I was talking about paradigm shifts so vast that software written for platform B that does the same thing as software written for platform A would require a rewrite. An example in this case would be word processsing.

What makes you think Windows isn't client/server? Or the Web? Or Mobile? They're all client/server.

I didn't say Windows threw client/server concepts out the window, only that when switching from MS-DOS + Client/Server to Windows + Client/Server, the code should be rewritten from scratch.

For that matter, there were plenty of pre-Windows MS-DOS-based/-era client/server apps, e.g. Novell.

Indeed, I had Novell in mind in my timeline. Novell gained prominence in the latter half of the 1980's -- between MS-DOS and Windows.

Re:The exceptions Joel should have included

[DragonWriter]

Neither of those (changes in platform paradigm or changes in programming language/style) typically requires a big-bang rewrite, and if it is avoidable, you are usually better off doing incremental rewrites than big-bang, particularly in critical production systems. For instance, on any multitier networked application, a paradigm shift for the backend implementation can usually be handled piecemeal by putting the old-style implementation behind a facade using the new paradigm, which proxies to the old implementation for the functions that have not been reimplemented.

Piecemeal replacement for programming language changes is, of course, even more obvious how to do.

Re:The exceptions Joel should have included

[mr_mischief]

• FTP
• Telnet
• FTP mail (which predated SMTP)
• UUCP
• Kermit
• IBM IMS
• INGRES
• the original MUD
• MRDS

• Maybe you should just read A Brief History of the Internet which lists FTP, email, telnet, NCP, TCP, UUCP, and BITNET gateways as being developed and in production before MS-DOS was released in 1982.

  CBBS was a dial-in BBS (think web forum without the WWW, HTTP, browser, or pretense) in 1978 on CP/M.

  IBM had terminals connecting to remote servers -- a serious precursor to proper client/server -- in 1964. The first ATM -- clearly a client-server technology -- was installed in 1970. Usenet was invented in 1979. Essex MUD was in 1979. Ask the Computer History Museum about networking.

  Creeper traversed networks in 1971. Wikipedia calls it a virus, but the description they give is more of a worm. It ran on TENEX and infected systems over ARPANET.

  In 1971, RFC 189 described a method and system for submitting jobs to a mainframe from remote systems over ARAPANET and to receive the results. It includes the connections, protocols, the mapping of ASCII to EBCDIC for systems that used ASCII to not need to translate on their end, a name for the whole thing (NETRJS), and several details of implementation.

  Need any more examples?

  Why yes, yes, computer history is a hobby of mine.

Re:The exceptions Joel should have included

[michaelmalak]

Those are communication protocols, not client/server. FTP, Telnet, etc. were the protocols for one peer to communicate with another. Prior to DHCP and NAT, everyone had a static IP and everyone's computer was a server.

The term "client/server" refers to a fat client like Visual Basic accessing a central database such as via ODBC. The earliest occurrence of the term "client/server" on Usenet is 1993.

Re:The exceptions Joel should have included

[mr_mischief]

I don't give a fuck about the term, especially about when someone happened to mention it on Usenet. There were dedicated client programs and dedicated server programs before the buzzword.

I also don't care if a computer could be both a client and a server, as we're talking about the programming model and not dedicated machines in racks. "MS-DOS", "Windows", "Web", and "mobile" don't separate machines into desktop dedicated clients and rack-mounted dedicated servers, either, but DOS could be a client or a server, Windows is made for both, and the Web involves both.

Telnet involves a client and a server. You can't use two client programs together. FTP is the same way. IMS is a database server. INGRES is a database server. MRDS is a database server. CBBS is a BBS program. These all use a client/server model. They all predate MS-DOS.

RFC 189 was a job serving system called NETRJS. In the RFC, which you had a link to follow if you'd actually taken the time to read any of it, there is mention of needing a "user process" on the remote host to make use of the "server process" on the "server". That's client/server programming. A "user process" is also known as a "client".

Most of these were based on Multics or its descendant, Unix. Unix is what real computers ran when DOS came out. It's built to be a multi-user, multi-tasking, client/server system from the very beginning. Hence the Internet, which also predates DOS.

Re:The exceptions Joel should have included

[michaelmalak]

Calling Telnet client/server is like calling XModem client/server.

Re:The exceptions Joel should have included

[mr_mischief]

Hardly. Xmodem is a non-interactive data transfer protocol. Telnet enables a user to request multiple different and possibly unrelated services from a server and receive multiple different responses. Protocols like XModem and ZModem can even run within any telnet that is 8-bit clean. Calling telnet client/server is much more like calling the Web client/server than calling XModem client/server.

Re:The exceptions Joel should have included

[michaelmalak]

The Web is client/server only insofar as it can run Javascript (or ActiveX, or Flash, or Java, etc.) Telent has no similar capability. If we call Web-without-Javascript-et-al "client/server" then we'd have to do the same with VT100.

But I see where you're going with this. There are probably some UNIX-based client-server applications out there -- perhaps a Star Trek or a MUD that is server-centric and not peer-to-peer. But just as with all technology, it takes 30 years to progress from lab to retail. The pieces, such as Visual Basic and Powerbuilder weren't there for mainstream business client/server until circa 1990.

Re:The exceptions Joel should have included

[mr_mischief]

VT100 is a smart enough terminal, but it was never a client. It was a peripheral. Telnet was a software system with both a client module and a server module to enable two separate full computers to interact, not a terminal with a computer. As a client goes, a telnet client is a very thin one. However, thin vs. thick is a very different distinction from client/not client.

To say telnet isn't client/server draws the line at systems using X11, NX, VNC, RDP, HTTP, SMTP, POP3, and many more protocols not being client/server either simply because the transport protocol is simple.

Be clear here that I'm not saying the Telnet protocol is a client/server system. It's just a protocol. The programs that used it, starting from the telnetd daemon with a login and an OS shell on the server side and the basic telnet client on the client side, were simple client/server applications. What's more is that other systems used the same protocols and had thicker servers and thicker clients.

I'm not sure why you're so tied to Microsoft technologies, but the programming language has nothing to do with whether the finished implementation of a project is client/server. Sure, a language and its libraries can make developing a client/server model easier than doing the same project in another language. It doesn't make anything possible that some suitably flexible other language like C can't do, though.

Treat a rewrite as a new product

[davidwr]

How long did it take you to bring "version 1.0" from concept to new product to the "2.0" version that the market considered useful and bug-free?

Use that as a starting point for how long it will take to do a rewrite, then adjust up or down for things like:

*how many features will be in the rewrite that weren't in the 2.0 version (e.g. a rewrite of MS Windows today would take much more effort than writing MS Windows 3.1).
*time already invested in internal projects that will be used in the rewrite version (e.g. experience writing the NeXT OS greatly reduced the time it took to write MacOS X 10.0).
*other factors too numerous to list here.

Sometimes it makes sense to treat your pre-release product as a throwaway prototype rewrite BEFORE you go to market. Yes, this means starting the testing cycle all over again, but it gives you an opportunity to correct design flaws or maintainability-killing bugs while you still have a chance. Unfortunately, sometimes the competitive marketplace forces you to go to market with what you have, a bug-ridden, nearly impossible-to-fix product that will make a little money so you can have the funds to redo it from scratch for version 2.0, er, version "II, 1.0."

Re:Treat a rewrite as a new product

[drinkypoo]

*time already invested in internal projects that will be used in the rewrite version (e.g. experience writing the NeXT OS greatly reduced the time it took to write MacOS X 10.0).

As I covered elsewhere, OSX is just the latest revision of NeXTStep. I did make ONE error in that comment: there is one thing in OSX that didn't come with NeXTStep or come from legacy MacOS (like the toolbox code) in some form: Grand Central Dispatch. That's not a central API, it's totally optional, but it is an exception to what I said, so it's fair to mention it. It's not the experience that made NeXTStep 3.4, er I mean OSX 10.1 so quick/easy to release, it's the fact that it's made of more legacy code than new stuff. It is simply not a useful example when talking about complete rewrites since nothing that came with OSX originally was completely rewritten, and by nothing I mean absolutely nothing. Not Cocoa, not Carbon, nothing. This statement is not intended to take anything away from it as an Operating System, only to correct this revisionist view of history that you and others seem to have about OSX.

Re:Treat a rewrite as a new product

[larry bagina]

Quartz, Core Graphics, Core Animation, Core Foundation, etc.

Re:Treat a rewrite as a new product

[Bungie]

I agree with you on that point, most of OS X was completed while OS 8 was still current. People often forget about OS X Server 1.0 which was released 3 years before the client OS X version everyone knows today. Before that there was also Rhapsody Blue Box which was a complete transitional build of the OS and technology for developers. Carbon provided a fairly compatible API to the classic MacOS toolbox so they could transition code pretty easily. A lot of stuff had to be re-implemented in Cocoa after the release of 10.0, probably because it was using Carbon.

I'm sure Apple would've open sourced the classic MacOS a long time ago if it didn't contain a lot of their proprietary technologies and code which is still in use.

Re:Treat a rewrite as a new product

[drinkypoo]

Quartz Extreme is an extension to Quartz which is a reworking of existing libraries. Core Animation is a wrapper between Quartz and OpenGL. Core foundation is largely new functions which do the same thing as old functions, some of which were written to replace existing code and slid into place where the old stuff was. Core Animation is the closest to being something actually new, but it really isn't.

Why I want to rewrite

[Aladrin]

The article says that any developer will tell you that they want to rewrite the code they are working on. It claims it's because it is harder to 'read' code than 'write' it.

I disagree.

I want to rewrite my old code at work... But only for one reason: I am a lot better programmer now than 5 years ago. And 5 years ago, I was a lot better than 10 years ago. And in 5 more years, I have no doubt I'll feel the same way.

Code I write today is cleaner, easier to read, more efficient, and easier to work with for new projects.

Re:Why I want to rewrite

[Abcd1234]

I want to rewrite my old code at work... But only for one reason: I am a lot better programmer now than 5 years ago. And 5 years ago, I was a lot better than 10 years ago. And in 5 more years, I have no doubt I'll feel the same way.

There's actually one other reason most programmers would like to rewrite what they're working on: They've solved the problem once, and now they understand it.

IMHO, you can't solve a problem properly without solving it twice. Unfortunately, that's just not, in general, tenable in the industry, and so instead we have things like XP, which encourage prototyping and refactoring, which accept that maxim and attempt to allow for it in the process. Unfortunately, *that* requires preeminent design skills, and that's something lacking in your average developer.

Re:Why I want to rewrite

[Vellmont]

But only for one reason: I am a lot better programmer now than 5 years ago. And 5 years ago, I was a lot better than 10 years ago. And in 5 more years, I have no doubt I'll feel the same way.

These strike me as not terrible reasons to re-write your code, but also not particularly good ones either. It seems more ego driven than an economic one.

In the end, the major expense of software development is time. Time to fix bugs in the code, time to add new features to the code, etc. It doesn't matter if its OSS code for your own pet project, or something you're making a million bucks off of. If by re-writing the code you think there's a justifiable payoff in terms of time you'll save in bug fixing and maintenance, and is not time better spent elsewhere, then do it. The key word is of course "justifiable", and entails an honest attempt at calculating how much less time you'll spend maintaining the code compared to the cost of re-writing it as well as the risks of being off significantly in both calculations. But no, I don't think being a better software developer is a particularly good reason to re-write code in and of itself.

Re:Why I want to rewrite

[Blakey Rat]

I disagree.

I want to rewrite my old code at work... But only for one reason: I am a lot better programmer now than 5 years ago. And 5 years ago, I was a lot better than 10 years ago. And in 5 more years, I have no doubt I'll feel the same way.

Code I write today is cleaner, easier to read, more efficient, and easier to work with for new projects.

And what if your company started a re-write, and you left 2 weeks in? What would happen then?

(Or are you a one-man shop, in which case pretty much nothing in this thread applies to you.)

Re:Why I want to rewrite

[Aladrin]

I wasn't arguing the reasons a company should rewrite. I was only talking about the reasons why a programmer wants to, with myself as an example.

Re:Why I want to rewrite

[Aladrin]

I disagree. I've written plenty of things that solved the problem properly the first time and rewriting them would not solve the problem any better. The only improvement would be that it was more maintainable or efficient due to cleaner code, using global libraries, or or reasons. The actual solution to those projects wouldn't actually change.

IMHO, If you have to solve a problem twice, you didn't solve it the first time at all. You just hit keys until it worked, and then tried to figure out why afterwards. It wasn't designed, it was grown.

Re:Why I want to rewrite

[Anonymous Coward]

Second system syndrome? Has no one here studied software engineering? Aside from that, even Brooks doesn't believe in "Build one to throw away anymore".

Re:Why I want to rewrite

[Abcd1234]

IMHO, If you have to solve a problem twice, you didn't solve it the first time at all.

If you believe that, you've never built a complex piece of software before.

We're not talking about a single module, here. We're talking about systems. When building systems, you need to understand the technical and business requirements, and particularly in the case of the latter, often *no one* knows enough about the problem to fully specify those requirements until you've already embarked upon building a solution. This is exceedingly common when one is building software to slot in to an existing workflow, as there is often hidden requirements that aren't uncovered at the outset, for any number of reasons, ranging from domain-specific knowledge hoarding, to people simply taking for granted what they do every day. In the end, this manifests as vague or shifting requirements throughout the course of the project, as new information is discovered during implementation.

Re:Why I want to rewrite

[Abcd1234]

Second system syndrome? Has no one here studied software engineering?

Software engineering doesn't magically solve the problem of poorly specified requirements or a lack of experience in the problem domain. It simply gives you tools to mitigate their effect (such as XP's approach of build-and-refactor).

Only 80000 lines...

[AdamInParadise]

... and most of them run in a controlled environment (server-side). So lots of Joel's advice is not going to apply.
Then they have the balls of comparing their move to the transition from Mac OS 9 to Mac OS X!
In the end, while their story is interesting, it adds really little value to the "Rewrite/No Rewrite" debate.

Joel's article needs an update...

[MoHaG]

Netscape's rewrite might have cost them market-share at the time, but Internet Explorer is back at trying to catch up with Firefox, which might be easier to extend because of that newer codebase...

(And from when I forgot to log in: And WTF is up with the captcha? It doesn't change, even when it is obviously unreadable (A 8 / a / 0 with a line through is hard to make out with 2 guesses))

Re:Joel's article needs an update...

[Rockoon]

Actually it seems to me that Firefox is right back where Netscape was...

...too bloated to fix. Where its not bloated, it seems to suffer from Fancy Design Syndrome.

Re:Joel's article needs an update...

[Bigjeff5]

That's exactly why I don't use it much.

I much prefer Chrome - so clean, so shiny, so fast. FF is a major hog on my system, generally the most memory intensive program running (currently 3x the size of McAffee, #2, and 5x the size of Outlook, #3, with only 8 tabs up), yet all it's doing is displaying web pages - most of them static.

Simple rule...

[McNihil]

rewrite ONLY if it means it yields less lines of code to maintain with the same functionality. Use whatever language necessary to keep the length of code to a minimum.

If the code doesn;t look as if it has been refactored even once... junk it.

Re:Simple rule...

[Vellmont]

rewrite ONLY if it means it yields less lines of code to maintain with the same functionality.

I have to register a huge protest to this idea that "less lines of code" == "more maintainable". Perhaps on a grand scale across several orders of magnitude this is true, but on the small scale of say a factor of 2 or 3 it's most definitely false.

I don't know that there's any one single simple rule that correlates well with maintainability. Maintainability has something to do with complexity, and quality which also can't be defined in simple terms. You could have a LOT of really simple, well written code that's easily understandable and simple to maintain, or you could have a small amount of complex, hard to understand code that blows up every time you touch the thing.

Re:Simple rule...

[Chibi Merrow]

rewrite ONLY if it means it yields less lines of code to maintain with the same functionality. Use whatever language necessary to keep the length of code to a minimum.

Rewriting my well-documented if-then-else tree as a series of nested ternary operators would reduce the number of lines of code, but I don't think that makes it more understandable/maintainable.

Or I could write it in brainfuck, the entire program would fit on one line then.

Re:And why?

[WrongSizeGlass]

I'm going to forward your blog post to several clients and colleagues. The clients need to see why I make sure they know what they want before we start development (they never understand how hard it is to change infrastructure on the fly or just "do it again"), and the colleagues because they need to stop selling one product as something else that can "do what you need with just a little bit of tweaking".

We do some small custom web apps for clients, and even a few that have the potential to grow into bigger (but not big) products. While rolling out one of them this week, to the first of three clients who have ordered it, I'm already designing a complete re-write of the core of this product. What started out as a small helper app for one client has turned into a PITA to scale up and out for these other clients. Clearly it's better to fix it now than to keep patching & splicing in order to make it work for them ... until we sell it to someone else who wants "just one little change ...".

So very much better than the Sausage Machine

[lawrence.a.sim]

We had one of our apps Sausage Machined, very sad... Translate one chunck of code into worse code in another language and different platform. A complete from scratch re-write would have been SO much cleaner (and maintainable).

One typical problem

[jeti]

Rewriting an application can work if the developers know what they're doing. However, these rewrites are often side projects. If considerably more resources are spent to extend the original application, the rewrite will be behind until it gets cancelled. On the other hand, a rewrite is always a risk and betting your company on it is insane. In most cases it's safer to refactor the original implementation, even if it's more work than starting from scratch.

What, no plus sign?

[tepples]

I noticed that your regular expression doesn't allow the plus sign as a valid character in e-mail addresses. For example, a lot of e-mail providers allow receiving mail at, say, pino+regexlib@example.com, which gets put in the same mailbox as pino@example.com but tagged with "regexlib". See here for instance.

Re:What, no plus sign?

[rockNme2349]

Sorry about that. I just used that as an example and did a quick search for a code. I agree that I am frustrated when a program doesn't validate, but the entry is to specification. The poster after you posted a link to a proper email regex string.

Re:What, no plus sign?

[drachenstern]

You are aware that parsing an email address is not the same as saying pino@example.com is equivalent to pino+whatever@example.com ... the + is something done by the server, nothing to do with the standard.

Granted, most email validators don't follow the RFC. I still go with "the only way to validate an email address is to send an email to the address and monitor for a valid response" (meaning user interaction, bounceback, etc).

Re:What, no plus sign?

[Anonymous Coward]

You are aware that parsing an email address is not the same as saying pino@example.com is equivalent to pino+whatever@example.com ... the + is something done by the server, nothing to do with the standard.

Yes, but you still need to allow the user to specify the + in the first place. The stuff about what the + might be used for isn't necessary for the point, it's just a bit of extra information.

Re:What, no plus sign?

[KlaymenDK]

I find that kind of email-variance very useful. I also find that the majority of sites (esp. those I MIGHT suspect of leaking my info) do not support that format. Hrmpf.

Re:What, no plus sign?

[drachenstern]

Oh, quite well aware of that. The standard is actually very forgiving, yeah? something about "garbage in, valid data out"?

Re:What, no plus sign?

[mgblst]

That is the problem isn't it. People like you, copying and pasting regular expressions they don't understand, pasting them in their code. Now I can't add a plus in my email addresses half the time, so I can't track the pricks sending me spam.

Re:What, no plus sign?

[Antity-H]

Among these e-mail providers is Gmail, and I hate websites that prevent me from using the + sign.

Re:What, no plus sign?

[Anonymous Coward]

Send a message to the websites. I do it for every one I notice, and sure I can count on one hand the responses I've gotten, but someone has to educate these people.

Here's a good example

[Low Ranked Craig]

When your "enterprise" ERP application is written in VB6 with a roll your own alternative to ADO, half your business logic is in stored procedures that can be 5,000 lines long and 30% to 40% of your code was written to spec B.

5 points if you can name that product

Re:Here's a good example

[spells]

Siebel?

Re:Here's a good example

[Anonymous Coward]

I'd take VB6 and long stored procs over my daily mess.

I'm dealing with Delphi, BDE (unsupported for years!), Embedded Sql and Trigger based business rules and relational integrity any day. About 20% of the database tables have no discernible purpose. And thats an EMS

Don't pay so much attention to Joel Spolsky.

[Animats]

Look. Spolsky runs a dinky little software development firm that sells a little project management program. And it's still a dinky little software development firm after a decade. It's not like this guy runs a company that does anything critical, like avionics software, or anything really big and tightly integrated like Facebook, or financially significant like Chase's banking system, or leading-edge robotics like Boston Dynamics, or cutting-edge manufacturing like HyperMill. No, they just do Windows and Mac desktop apps. That's trailing edge technology at this point.

Some of the better shops don't hesitate to rewrite. Some have systems which intercommunicate by message passing and are designed to be rewritten in sections. (Facebook, works that way internally.) The bigger shops may have one team on the new version and a maintenance team on the old one; a small firm may not be staffed for that.

Re:Don't pay so much attention to Joel Spolsky.

[kmike]

Yes, I never understood why so many pay attention to Joel's inflammatory rants.

I don't even want to start on his company's product (Fogbugz). Seriously, ASP/VBScript translated to PHP? And then inventing a new programming language just for a web app with ability to output in several other languages? Ugh.

Re:Don't pay so much attention to Joel Spolsky.

[Vellmont]

Yes, I never understood why so many pay attention to Joel's inflammatory rants.

Partly because he's a fairly good writer, and partly because he seems to offer simple solutions to the larger problems of software, and partly because he's right at least some of the time.

I DO think he's just dead wrong on this issue though. In my experience starting from scratch is something that should be very carefully considered, but rejecting it outright on general principles is wrong. I really don't care much about his bona-fides or how successful he's been. People with big impressive resumes working for some fashional part of the industry writing in the cool language of the day can say some really stupid things.

The problem starts when people start believing someone PURELY on the reputation alone, and turn their brain off. There's a few figures like that that gain a lot of hero status. Joel is one of them, Bernstein is another. A lot of people actually don't like thinking very much, and prefer ANSWERS to hard questions (even if they're the wrong answers). The cult figures in software offer these answers, defend them to the death, and attract people with similar attitudes. It winds up looking like these guys have large followings, when in reality they have minuscule, but extremely dedicated and vocal ones.

Re:Don't pay so much attention to Joel Spolsky.

[Anonymous Coward]

There's a few figures like that that gain a lot of hero status. Joel is one of them, Bernstein is another. A lot of people actually don't like thinking very much, and prefer ANSWERS to hard questions (even if they're the wrong answers). The cult figures in software offer these answers, defend them to the death, and attract people with similar attitudes. It winds up looking like these guys have large followings, when in reality they have minuscule, but extremely dedicated and vocal ones

Not too familiar with Bernstein, but Spolsky would have to work 50 hours a day to be as annoying as those guys from 37signals.

Re:Don't pay so much attention to Joel Spolsky.

[Vellmont]

Not too familiar with Bernstein

Bernstein is a bit of a blow-hard that thinks the major component of software has to be is SECURE. He wrote some small securely designed applications like DNS, and then goes on to challenge everyone else about how THEIR product isn't secure while HIS is. He likes to ignore the fact that real life often intervenes, requiring software to be more complex than you'd really like it to be.

I've never heard of the 37 signals people. I've used basecamp though. It's OK, but it seems over hyped for what essentially amounts to a discussion board linked to email with some todo items thrown in.

Re:Don't pay so much attention to Joel Spolsky.

[inf4mia]

It's not like this guy runs a company that does anything critical, like avionics software, or anything really big and tightly integrated like Facebook, or financially significant like Chase's banking system, or leading-edge robotics like Boston Dynamics, or cutting-edge manufacturing like HyperMill. No, they just do Windows and Mac desktop apps. That's trailing edge technology at this point.

Yeah, it's not like he's ever done anything significant...

Spolsky started working at Microsoft in 1991[4] as a Program Manager on the Microsoft Excel team, where he designed Excel Basic and drove Microsoft's Visual Basic for Applications strategy.[5] He moved to New York City in 1995 where he worked for Viacom and Juno.

http://en.wikipedia.org/wiki/Joel_Spolsky

Re:Don't pay so much attention to Joel Spolsky.

[sootman]

Dinky company, perhaps, but quite successful at a personal level. In less than ten years, Joel took his company from zero to seven million dollars per year by my accounting.

http://joelonsoftware.com/articles/BionicOffice.html
$700 per employee in the original office

http://joelonsoftware.com/items/2008/12/29.html
"built for 18 employees" = about $12,600/month

http://www.inc.com/magazine/20080601/how-hard-could-it-be-adventures-in-office-space.html

"When we moved into our current offices, our rent had been equal to 15 percent of revenue, which was high. But the company grew, and today our rent is only about 2 percent of revenue."

So revenue was $84,000/mo ($1,008,000/yr) and is now about $7,500,000/year.

So he's not a complete waste of space. And he may not be God but that doesn't mean he's never right and/or never worth listening to. READ THE F ARTICLE about rewrites--plenty of Slashdotters (you included) have been here long enough to know that at least, his example about Netscape/Mozilla is 100% accurate. They lost YEARS because they chose to rewrite everything.

And judging by the comments here, I think a lot of people are reading the title and thinking he's saying "never make any changes." That is 10000% NOT what he is saying. He's saying "never throw away 100% of your code and start over from scratch." If you actually read his original article (I know, I'm new here) you'll see a lot of really good points.

Joel isn't God, but he isn't just some stumbling moron either. There IS a continuum between those two extremes, you know.

Re:Don't pay so much attention to Joel Spolsky.

[kriston]

When your boss quotes Joel Spolsky, it's probably time to move on.

Re:Don't pay so much attention to Joel Spolsky.

[CarbonShell]

To be honest, I don't know about Joes or his company and don't really care.

But the statement you are trying to make is fundamentally flawed, because success does not mean YOU are doing something correct. It basically only means you are able to turn a profit.
I've been through my share of companies and can tell you that success != doing stuff right. Some of those I have been to basically just had the right product at the right time, allowing them to throw anything on the market and it would be bought up, regardless of quality or how good the company worked internally.
And the sad fact was, internally the companies were plain chaos.

As soon as the cash-cows died, the companies started to have BIG problems, because they did not realize it was not them that were not the reason for the success.
They were so obnoxious that they basically stared blaming the market for not seeing how great their stuff was.
Then most of them started doing some really crazy stuff, basically proving they had no idea what they were doing. Especially adding more management overhead.

At least that was my exp. in 80% of the comps.
YMMV

Re:Don't pay so much attention to Joel Spolsky.

[mr_mischief]

And, you know, five years or so as project lead for Excel, designer of Excel Basic, champion of VBA, five-time published author on the subject of software development, graduate summa cum laude from Yale, StackExchange, running a profitable business in one of the most expensive cities in the world, and surviving that whole IT sector bust in 2000 despite being founded that year.

I'm not saying he's perfect, but he's pretty qualified to speak on the subject of managing teams of developers. It's what he does, and his company is doing better than many. Maybe he's not always right, and maybe he's opinionated. He's qualified to be brashly and authoritatively wrong once in a while, though, because he's doing something right most of the time.

Size of company is only a measure of success if you choose to make it so, by the way. Many people prefer to be the alpha male in a pack than the lead sheep in a flock, beholden to Wall Street mutual fund managers or grow-quick-and-sell VCs. Fog Creek was bootstrapped and is growing on its own funds in a custom-designed office in Manhattan with what many would call extravagant furnishings, equipment, and benefits. I'd say that's pretty successful after just 10 years of cash-flow growth.

Porting games written in assembly

[tepples]

Two programmers were tasked to convert the Atart VCS/2600 game Pitfall 2 to a Commodore=64 and Atari 800 computer. One said, "The Atari console is so primitive that it's easier to recreate the whole game from scratch," and the other said, "No just copy the 6502 code and then modify it for the varying graphics/sound chips."

But then how would one have ported a 2600 game to the ZX Spectrum, ColecoVision, MSX, or Sega Master System, all of which use a Zilog clone of Intel's 8080 CPU?

Summary ignores the single most important quote.

[Anonymous Coward]

From the article: "we experienced a lot of the hardship and trouble that he talks about, so we can easily see how many companies could get into trouble during such a process. If the board of directors had known in 2005 that we wouldn't be ready to re-launch in the international market until 2010, I doubt we would have gotten the green light for the project. "

And that says it all. In this case, it worked out; but it was quite possible that during that 5 year window with no releases that a new competitor could have emerged with a few new features, and these guys wouldn't have been able to respond since their code rewrite was still years away from being ready. Everytime a customer said "We'd love to buy your product but your competitors have feature X and you don't", these guys lost a sale since they couldn't counter "well, we have a new release of our coming out in a couple months with new features". Spending five years rewriting code from scratch instead of incrementally improving the existing code and doing a new release every year could quite possible have caused a company to run out of money and go out of business as customers moved on to the competition.

The reason to iteratively refactor your existing code rather than rewriting it from scratch isn't cause it produces a better end result -- it's cause iterative improvements allow you to periodically stop and ship a release every six months or a year (with one or two new features and a slightly cleaner code base) and then continue refactoring again for the next release. But rewriting from scratch means you can't ship a release for (in this case) ***5 years***. And the world may well have forgotten you by then.

Similar Experience

[eulernet]

We faced the same challenge at our company recently.

My company publishes a CMS for specialized niches.
The product started around 2002, and we developed it without real direction, since we added functionalities as we needed them.
The first years were great, since we were the only ones to release such a tool, but some concurrents emerged after a few years, proposing a much cheaper alternative.

As we built the project without long-term view, adding new functionalities took more and more time, since the codebase was hard to maintain (for information, we mainly use VB.NET with a lot of XSLT).

Two years ago, we realized that the interface could be improved for the customers, but it was impossible to do it with a reasonable amount of effort.
Thus, the CEO decided to start a new product, and we used an interesting approach: the customers approach (or technically, BDD, behaviour driven development).

The idea is to describe scenarios and implement them one after another, using agile methodologies.
For this kind of approach, you really need to hear about your customers, and write the scenarios that they need, and it's obviously impossible when you start your company.

The project is now radically different, but not yet released, since it's really a subset of the previous project.

What we discovered is that we needed to maintain the old codebase, because our consultants still used it, and it is difficult to sell the new project, since it's really different.

So I would suggest that if you want to totally rewrite your codebase, you need to place a few resources on maintaining the original project, since it won't probably die before a long time...

Re:Similar Experience

[Frankie70]

For this kind of approach, you really need to hear about your customers, and write the scenarios that they need, and it's obviously impossible when you start your company.

The project is now radically different, but not yet released, since it's really a subset of the previous project.

When you release it you will find the following things guaranteed
- Customers have forgotten to mention 20% of the scenarios they need which is present in your current release.
- 20% more have been misunderstood by you.
- Your new software may be a little more flexible for adding features as compared to the old one, but as much as you had wanted it to be when you decided on a rewrite.
- There are a lot of obscure, difficult to reproduce & difficult to fix bugfixes hidden in your old software, which will not be carried over to your rewrite. You will end up spending a lot of time on these once your rewrite is released.

Re:Similar Experience

[mr_mischief]

Not only will there be fixed bugs in the old software some of which will likely be bugs again in the new rewrite. There will probably also be bugs the customer got used to and maybe even thought of as quirks instead of bugs. By fixing some of those bugs in the new version, you'll be screwing your customers' workarounds.

Re:And why?

[tomhudson]

Here' let me help you :-)

• Mistake # 1: "CMS was developed using Active Server Pages, and consisted of around 80,000 lines of VBScript code"
• Mistake # 2: "new CMS based on Microsoft .Net 2.0"

  Einstein said something about making the same mistake and expecting different results ...

Re:One some /. er commented on Spolksky...

[Anonymous Coward]

I agree. I don't know why slashdorks (or anyone else) sucks his e-cock. He's a former MS mid-level manager. If anything he writes strikes you as intelligent, insightful, or advice worth following, you're probably totally lacking in common sense or real world experience.

Re:And why?

[fishexe]

Einstein said something about making the same mistake and expecting different results ...

No he didn't. That was Rita Mae Brown in her book Sudden Death , but that never sounds as dramatic or important as claiming it was Einstein.

Re:One some /. er commented on Spolksky...

[Sarten-X]

Mostly agreed, and I'll sign my name to it.

Spolsky's got some decent ideas, and knows a lot about what he talks about, but his idea of "good" and "bad" makes me cringe. He's one of those folks who's built up a following on what little they had to talk about, then ceased to think further. Now (and for several years), he writes about half-baked lists of do's-and-don'ts that never even come close to covering exceptions.

Re:And why?

[The End Of Days]

Don't worry about refuting a zealot, the two technologies have pretty much nothing in common anyway.

Not that I'm a fan of either approach, exactly, but this was just a cheap jab at Microsoft based on prejudice. You should be expecting that around here by now, your user id is low enough.

Re:And why?

[Chris Newton]

I'm the author of the blog post.

In that case, would you mind explaining something that is confusing me, please? It sounds like you started the rewrite in early 2005, planned to release in mid 2006, and actually had a finished product in early 2007, which isn't on schedule but is hardly unheard of for a software project. However, you then seem to jump to saying the new version wasn't available across your entire international market until 2010. What happened in between?

Re:And why?

[Z00L00K]

I have been through a similar project - rewriting a solution that did run under OpenVMS using Basic, Java, C++, C and a bunch of DCL scripts (that confusingly enough for DOS persons have the file extension .COM)

Target environment was Linux and language used was Java 1.6.

My experience when rewriting a legacy system that have a crapload of varying solutions that has evolved during 25 years or so you will find that there is always yet another functionality that nobody told you about - effectively doubling the development time. (This "Multiply estimated time by PI factor" statement isn't that far off.)

And there were some traps involved too - migration of the system had to be seamless for the users as much as possible and with minimal downtime. Since there were over 400 different customers with everything from 1 to 1000 users each involved this was to say the least "tricky". Especially since this was a 24x7 system. The solution was to write a replication protocol that replicated data between the old system and the new. The old system used OpenVMS indexed files while the new system runs a MySQL database and the data structures were different too, which made it necessary to write a replication solution. So when a customer was migrated it was effectively done by setting a flag that redirected them from the old system to the new system and they could continue working.

Of course there were bugs in the beginning, and user errors since the new system did have different functionality and behavior compared to the old. Bot none of them were causing any irrecoverable problems. Invoice printing was delayed, but no major amount of money was lost. The majority of the problems appearing didn't affect the end users at the customers, only the helpdesk service personnel and they were prepared for limitations ahead of time.

The amount of downtime for the system during the two years it has been operational has been very low. And this has given a different concern - too few "problems" with a system is also a problem because tech support will almost forget that it exists.

Specific problems with the application - especially in the beginning has been running out of PermGen space in Java. This at least partly due to design mistakes. But memory leaks that grows over time are very low. And the use of FindBugs has been very useful to trap a lot of errors (potential and real).

What the application does? - It's a management application for short-term lease of telephony at hospitals and similar (almost 400) and other services (a few) which enables and disables phone extensions, assigns numbers, allows instant move of an extension and provides invoicing for the rental and phone usage through processing of CDR:s.

Re:Summary ignores the single most important quote

[fishexe]

To paraphrase your post in the words of Steve Jobs:

Real artists ship.

For those who don't want to follow the link:

[fishexe]

I tried to C&P the link contents in but /. said "Filter error: Please use fewer 'junk' characters."

Old crappy code

[Frankie70]

That old crappy code has undecipherable bug fixes for 100's of obscure difficult to reproduce bugs. Many of them which could be hit only under really strange customer usage scenarios & which tooks days to reproduce, days to fix & many more days to fix whatever the fixes broke.

So unless you original software fully automated testsuites covering every functionality & you have added a bug fix verification test for each bug which has been fixed since then, do not rewrite something from scratch.

Re:Old crappy code

[Vellmont]

That old crappy code has undecipherable bug fixes for 100's of obscure difficult to reproduce bugs. Many of them which could be hit only under really strange customer usage scenarios & which tooks days to reproduce, days to fix & many more days to fix whatever the fixes broke.

Sounds like code that was just poorly written in the first place, and then endlessly patched and re-patched to try to "get it right this time".

Listen, software development is about software developers and the development process not about software. If a boob wrote it in the first place, that same boom is unlikely to produce anything much better the second time around. Boobs produce crap. If your process produce an enormous amount of scope creep, and the code suffered because of it producing all your crazy errors, then its the process that's at fault. Bad process produces crap. Re-writing it without the scope-creep involved in the process is likely to produce better quality code.

So unless you original software fully automated testsuites covering every functionality & you have added a bug fix verification test for each bug which has been fixed since then, do not rewrite something from scratch.

An utterly ridiculous statement. What makes you think you'll make the same mistakes when re-writing the software, and not completely different ones? If you do it right you should get LESS bugs to begin with, which is the justification for re-writing the thing. Test suites are nice and all, and they'll help you. But the idea that they're the only road to quality is silly at best. You can't test all scenarios, period. You've also made the grand assumption that the test suite is testing the right things in the first place.

Re:Old crappy code

[Frankie70]

Do you have experience rewriting huge enterprise software which started small, got lot of features added over the years & now has 5-10 teams of developers working fulltime on it? This kind of software is a disaster to rewrite. Yeah, if you want to rewrite something mid-size like a browser, it would probably not be as bad.
If you want rewrite huge software, do it in small chunks across different releases after adding a lot of automated test suites & bug fix verification tests.

Re:Old crappy code

[Vellmont]

Do you have experience rewriting huge enterprise software which started small, got lot of features added over the years & now has 5-10 teams of developers working fulltime on it?

Yes.

If you want rewrite huge software, do it in small chunks across different releases after adding a lot of automated test suites & bug fix verification tests.

It's an approach. It may be a good approach given the specific case. Too many people like to generalize their own specific experience across all cases. Sometimes a complete re-write really is the best approach.

The general idea I'm trying to present is you need to look past the actual code, and look at how you arrived at what you have. The process that produced the mess is as important as the mess itself. Maybe your specific case it makes sense to go the incremental route. But I guess I'm just extremely wary of this approach, given what I've seen what scope creep produces. I've also seen a lot of bad code written by people that didn't know what the hell they were doing in the first place. In that case, there's nothing really that's salvageable as often times the code wasn't written in any sort of modular way, so you can't re-write in small chunks. In any case, I'm mostly opposed to the idea that there's one approach that works for everyone and every situation. YMMV.

when rewriting, go wider, not narrower

[catmistake]

TFA says they released their software 2006, and expected it to be in use for at least 10 years. I realize the benefits of DotNET, quick development, compatible with the most popular platform. I don't think there was any mistake in starting from scratch, in and of itself. And I don't think the downside of developing in DotNET has even materialized yet, and so long as the product pays for itself before it becomes obsolete, I think they're OK. But I have a feeling that, and I admit my prejudice, that the devs were blind to any other possible solution because of Microsoft. DotNET was they're ONLY option... never even considered that it was, never considered anything else... never considered. If a mistake was made, that was it... and again, it may not make any difference because the product may still be successful even with lock in. But there is no doubt they're software will not be a major player in 2016 having been developed in DotNET.

Re:when rewriting, go wider, not narrower

[catmistake]

DotNET was they're ONLY option... never even considered that it was, never considered anything else... never considered. If a mistake was made, that was it... and again, it may not make any difference because the product may still be successful even with lock in. But there is no doubt they're

*their *their man I am sloppy today

Re:And why?

[fishexe]

Don't worry about refuting a zealot, the two technologies have pretty much nothing in common anyway.

I don't think the difference in technologies is actually relevant to his point. If company X is known to make unreliable products for reasons relating to corporate culture, then "use tech A from company X" and "use tech B from company X" are essentially the same mistake even if tech A and tech B are automobiles and recombinant gene therapies. I think that was more of his point. Having used Visual Studio fairly extensively (in the pre-.Net days) I'm also inclined not to use ASP or .Net based on my experiences even if they bear nothing in common with old VS other than company of origin.

I was just taking issue with the Einstein citation, which bugs me every time I see it because it's so blatantly false and it contributes to spreading ignorance.

Rewrite if existing s/w congeals

[presidenteloco]

Often a rushed, under-resourced or under-skilled s/w project will congeal into a large, brittle, solid clot,
which is not extendable without breaking things in mysterious prohibitive to fix ways.

Congealment comes from insufficient or ill-conceived architecture, and/or rushed or ignorant ill-fitting extensions or mods or "fixes",
combined with insufficient continuous re-factoring.

This code may be worth keeping on expensive life-support if there are many existing customers depending on it,
but make no mistake. Your codebase is already dead, even if its heart still beats.

So then, if you still need software with similar but slightly updated or extended functionality, you should rewrite,
and in doing so, make sure you get good architecture, take sufficient time to build each part or layer, evaluate the quality of
all third party libraries or frameworks used (on the "volleyball" principle that the weakest member of the team drops the ball
and determines the team's i.e. the system's quality), use continuous refactoring, with technical-risk based work prioritization
(biggest risks dealt with first, always), document the classes and methods
sufficiently, and include unit tests and/or invariants and pre-postconditions, so that there is a lower probability that
further extensions will start congealing into brittle, excess complexity.

If you can succeed at maintaining that discipline without going bankrupt, then it will have been worth it, because the value
of your new software capital asset will be much greater than previously.

Of course you should have done it right the first time, (and should have had management enlightened enough to let you,)
because it would have been much cheaper to do it carefully once, than the punishing expense of the original crappy
development and maintenance plus the rewrite. There IS a valid argument that by the time you let your s/w congeal into
a complex, brittle clot, you are already too late, and you should pull the plug, shed a tear, and walk away.

Code doesn't rust, but skill does improve

[Mike216]

I've always thought his article on this topic skipped over dealing with the fact that rarely, if ever, is code so perfect that it wasn't at least a little 'rusty' in the first place. There's only so far you can go with refactoring before it would just be more efficient to start from scratch.

This is a very simple question

[Chris Newton]

Whether to do The Big Rewrite always boils down to one very simple question: do the expected gains outweigh the expected losses?

Usually, the argument against doing a rewrite boils down to two key points:

1. it takes time and resources just to get back to what you already had, which confers no immediate business benefit; and
2. you risk losing the bug fixes and special cases that have accumulated during the real world use of the original implementation.

Those are certainly valid concerns, and IME it is often true that their impact is underestimated. However, what the doomsayers tend to ignore is all the potential benefits from writing a second version of something from scratch but with the experience gained from doing it once already:

1. you can design based on the knowledge accumulated during the real world use of the original implementation, giving code that might be easier to maintain in future and/or allowing you to add new functionality that was not realistic before;
2. you can refine your requirements based on that same experience, cutting out things that haven’t helped in practice and cleanly integrating requirements that weren’t anticipated the first time around, leaving you with a code base that is fitter for its purpose;
3. while you lose all the old bug fixes and special case handlers, you also get to clear out all the old hacks and bolt-on workarounds that are maintenance hazards and a high risk of causing future bugs; and
4. the best tool for the job the first time around might not be the best tool for the job any more, and a rewrite lets you revisit that decision and take advantage of any relevant advances in development tools, programming techniques, industry knowledge, etc.

I’m sure some rewrites really are just because a developer wants to write something new instead of working with what is already there, and those are almost always a bad idea IMHO. On the other hand, it can be annoying if someone comes in assuming that this is the only possible motivation for a rewrite, without considering whether there is another justification for the decision.

Re:And why?

[Anonymous Coward]

He spent 3 years dead for tax reasons.

My $.02

[gyrogeerloose]

I'm not a real programmer. I used to play with BASIC back 25 years ago or so and I've fooled around with writing C code for microcontrollers a bit lately but my skills are very limited to say the least. None the less, I was able to parse that regular expression without too much trouble. That leads me to believe that it can't be all that hard for someone who codes for a living to understand it. Still, it would be helpful to spend a minute or two with some helpful commenting.

Re:And why?

[vlangber]

The consulting part of the company used the new version since 2007, but it's a totally different ballgame to sell licenses to partners around the world, than implementing projects within a relatively small group of developers. If you're selling to 3rd party developers, you need a stable and well documented system, with all features found in competing solutions.

Re:And why?

[mweather]

but this was just a cheap jab at Microsoft based on prejudice.

Prejudice is judgment without facts. I'd say this was more of a cheap jab based on decades of experience.

Rewriting works sometimes, without a doubt

[crispytwo]

I worked for a company that rewrote the same application three times in different technology. 2 time using MS .net tech - aspx, .net desktop, and 3rd with PHP.

The first incarnation was a disaster from a performance/scalability point of view, but we did learn the (surprisingly complex) business requirements very well.
The second incarnation was good, developed quickly, but missed the target market -- nobody wanted a desktop app
The third performed much better than either predecessor, was simpler to maintain, and actually made money... still is.

There was some desire to rewrite it a fourth time, but that has been abandoned in favor of extending the existing system has been the current methodology.

The lesson here is "if it sucks, rewrite it". And more often than not, it is too costly to not rewrite it. Specifically, if the system was not rewritten a third time, the company could not (and would not) exist. As it sits, the product is quite good at what it does. The balance is looking 5 or 10 years forward and looking 1 year forward. If the system you are using won't last 1 year forward, let alone 10, a rewrite may be needed, duh!

The second lesson is "if it is not perfect, fix it". (In the third case it would last a year, but fixing it would maybe make it last 3 or 5 -- it's going on 5 now, and is looking good for another 5)

What this person seems to be faced with is the same thing as that of COBOL ... asp is old tech, nobody really wants to work with it, and it definitely is not perfect. A rewrite is the only sensible thing. (And yes, anything in COBOL should be rewritten too.)

Joel contradicts the IEEE

[Todd Knarr]

Joel's position contradicts a paper I read years ago in an IEEE software journal that basically said you needed to plan on rewriting your application about every 7 years or have it collapse on you. The logic in the paper was based on two things I've found to be true in the real world. First, the world changes. Individually it's small changes, but looking at it on the half-decade-to-decade scale it can add up to huge differences in what's needed in the software. Second, software isn't infinitely extensible/adaptable. Any software has a basic architecture and world-view, and a limit beyond which it can't be pushed without an exponential increase in the time and effort needed to successfully make the changes. The two combine to mean that at some point it simply becomes technically infeasible to extend and adapt an existing system. The requirements have changed too much and you're having to fight the system trying to make it do, not just what it wasn't designed to do, but what it was actively designed not to do.

Now, business doesn't like this. It doesn't make sense from a business perspective, and it'd be much better to simply keep adapting and extending what's already there. But that ignores the fact that something must be technically feasible before you can even ask whether it makes business sense. If you've got the best idea in the world that'll make the business tons of money while giving you a virtual monopoly in the field and reducing costs by 99%, that basically is from a business standpoint the absolutely ideal thing to do, but it requires the manufacture of say room-temperature superconducting wire by the mile, then it just ain't gonna happen. How desirable it is from a business perspective doesn't matter because it just isn't technically possible at this point in time.

I also liken it to building a 20-story office tower. It's tempting to start with a simple one-story building and slowing add to it until you've got what you want, but the foundation of a one-story building just isn't going to be able to support a 20-story tower. You might be able to get 2 or 3 stories out of it, but at some point you're going to have to tear the whole building down and re-do the very foundations themselves to support the greater weight.

Re:Joel contradicts the IEEE

[paradigm82]

I think the IEEE view is totally backwards. My advice would be to never start out by thinking of secondary factors like like how old the code base is, how messy it is, programming language it is written in etc. Instead, look at the bottom line: What does it cost for the business right now that the code is bad. Is there a new feature we can't make, or can we see it is taking us too much time to make changes, do we have quality problems with those changes. Further, it s important to at least try to quantify each of these points and how much they cost. Then quantify the cost of a rewrite (including risk of cost overruns, setback in the market, lower quality in the beginning etc.). Then discount the cost of a rewrite with the rate of interest and compare the long term costs of either approach. Only then can you make a qualified decision. Often this isn't a hard exercise, because very often one will quickly see that the cost of the rewrite isn't worth it - the potential productivity gains from the rewrite can not catch up with the cost of the rewrite even over longer periods of time. Probably the rewrite should only be undertaken if it can pay for itself over a relatively short timespan since there's a major risk the product won't even exist or be relevant at that time. In most cases (when we are thinking about an app that has some maturity) only a major refactoring could make sense. Only very rarely does it make sense to throw out everytime. One of the situations that are hard are: What do I do with this app written in strange unsupported language XYZ, functioning fairly well, maintainable with some effort, but looking totally outdated (say GUI that looks like a Win 3.1 program etc.) and that can't inter-op with other languages etc. If the provider of XYZ doesn't provide a path to move the code to another platform, and one isn't available in the market you could try to do it yourself, which has risks as well and might not get all benefits (see my other post of such a project). Otherwise you might be either stuck with an app that appears outdated (and may be accumulating problems due to a buggy and unmaintained runtime etc.) OR have to pay the full cost of a reimplementation. Given that the conversion project might not be that bad after all and may not be as hard as you think.

Re:Joel contradicts the IEEE

[luis_a_espinal]

Joel's position contradicts a paper I read years ago in an IEEE software journal that basically said you needed to plan on rewriting your application about every 7 years or have it collapse on you. The logic in the paper was based on two things I've found to be true in the real world. .

I don't think Joel's position contradicts those findings. Joel is referring to a complete rewrite that happens "just because" and without a plan originating from the original system. The way I understand the IEEE finding is that the rewrite plan is part of (or stems from) the decommissioning at the end of a system life cycle.

The plan has to take into account the existing architecture: does the architecture allows for the rewrite to be organic, does it allow for its decommissioning? Also, take into account that the suggested rewrite that should occur every 7 years does not have to be a full-blown, all-or-nothing, from scratch rewrite (which is what Joel advice against.) It can be incremental and be done as a part of a plan in the system's life cycle.

I would actually interpret Joel's position of being one that describes how not to follow the IEEE advice on rewriting.

Re:Joel contradicts the IEEE

[luis_a_espinal]

To better expound my previous post - barring a cataclysmic change (like a merger or a vital piece of technology unexpectedly going out of the window), if we find ourselves having to do a full rewrite from scratch, it probably means the end of 7-year-rewrite cycle caught us with our pants down. That is, either we didn't plan how to decommission the system, or the system (sometimes out of necessity) was written in a manner that was not amenable for incremental replacement and decommissioning.

Re:Joel contradicts the IEEE

[DragonWriter]

Joel's position contradicts a paper I read years ago in an IEEE software journal that basically said you needed to plan on rewriting your application about every 7 years or have it collapse on you.

Does it, though? You can completely replace a system every seven years without ever replacing the whole system from scratch if you are doing fairly frequent replacement of components. "You shouldn't ever replace a system all at once" and "you should plan to completely replace a system every 7 years" aren't contradictory positions.

Re:Joel contradicts the IEEE

[SlideRuleGuy]

It's all about the quality of your people and the financial state of the organization maintaining it. I have seen small applications (~30KSLOC) that a team of people with 10+ years of experience each could not keep running beyond 7-8 years. It collapsed under its own weight. Nobody had the skills/wisdom to refactor the code periodically, nor did the organization have the money to be able to afford it. (In fact, their poor financial state meant that they never hired the best developers. They had to take what they could afford.)

I have also worked at a different company that kept a system (3 million SLOC) going for over a decade with only periodic refactorings of portions of the system. They had good the financial status to be able to afford it, plus the skill levels to know when/how to do it. (And their good financial state meant they could --and did, hire better developers.)

The quality of one's engineers and the prosperity (or lack thereof) of the organization determines how long you can hold off "software entropy" and keep software evolving forwards.

Re:Joel contradicts the IEEE

[Anonymous Coward]

Joel's position contradicts a paper I read years ago in an IEEE software journal that basically said you needed to plan on rewriting your application about every 7 years or have it collapse on you. The logic in the paper was based on two things I've found to be true in the real world. First, the world changes. Individually it's small changes, but looking at it on the half-decade-to-decade scale it can add up to huge differences in what's needed in the software. Second, software isn't infinitely extensible/adaptable. Any software has a basic architecture and world-view, and a limit beyond which it can't be pushed without an exponential increase in the time and effort needed to successfully make the changes. The two combine to mean that at some point it simply becomes technically infeasible to extend and adapt an existing system. The requirements have changed too much and you're having to fight the system trying to make it do, not just what it wasn't designed to do, but what it was actively designed not to do.

Now, business doesn't like this. It doesn't make sense from a business perspective, and it'd be much better to simply keep adapting and extending what's already there. But that ignores the fact that something must be technically feasible before you can even ask whether it makes business sense. If you've got the best idea in the world that'll make the business tons of money while giving you a virtual monopoly in the field and reducing costs by 99%, that basically is from a business standpoint the absolutely ideal thing to do, but it requires the manufacture of say room-temperature superconducting wire by the mile, then it just ain't gonna happen. How desirable it is from a business perspective doesn't matter because it just isn't technically possible at this point in time.

I also liken it to building a 20-story office tower. It's tempting to start with a simple one-story building and slowing add to it until you've got what you want, but the foundation of a one-story building just isn't going to be able to support a 20-story tower. You might be able to get 2 or 3 stories out of it, but at some point you're going to have to tear the whole building down and re-do the very foundations themselves to support the greater weight.

Do you rember the author of the IEEE paper you mentioned or something about the title? I'd like to read it.
Thank you,
Sam

Re:One some /. er commented on Spolksky...

[grimJester]

The first article of his I read was on exceptions vs return values. It was on or just after April 1st, so I thought is was an April Fool's article, since every single thing he wrote seemed wrong. After reading it I scrolled up to verify the date and found out the text was several months old.

Re:And why?

[NuShrike]

Usually, that's called wisdom and experience. Once burned, twice shy.

I went through the following conversion effort

[paradigm82]

I went through porting a ~400K application to Java. It was written in an unsupported, obsolete, early 90'ies "drag and drop" RAD system with a relatively simple language core. The program was highly mature and very functional even for competing programs. To port it, I wrote a conversion program that converted it function-by-function to Java. Additionally, I reimplemented the runtime library myself. The system had a couple of interesting specialities that made it fun to convert, because they were challenging to emulate in Java. Some of it called out to native C libraries (via a JNI mechanism, that I also had to implement) but also a strange type of global variables where a variable was shared among multiple processes. Additionally, it was possible to link fields in the GUI directly to storage locations (i.e. row in a table could be linked to the text fields of a dialog) so that the two would update it each other. There were various types of reflection possible and some simple message system for GUI. It took around 6 months of work. The project had to be done very fast for strategic reason. A multi-man effort to rewrite the application had spent 2 years to port just parts of the application. Of course not all the benefits was derived from my automated conversion. The original code was not that readable by today's standards, and it became slightly worse through conversion. But not all the "hand-written" converted code was that readable either, and some of it has already gone through heavy refactoring. No one included myself got much understanding out of the code but at least modern tools like Eclipse can be used to inspect, debug and extend it. Still, now 4 years after the completion of the project the code is still in production. New functionality has been written from scratch but the old code is still a significant (decreasing but probably still around 50%) part of the full application and has been maintained via bug fixes. There's no plan to reimplement just for 'reimplementation's sakef' but they might get reimplemented when they have to be significantly extended. It was also a fun project for me to. However, the code for the runtime library I wrote, that bridges the gap between the conventions of the original runtime system and Java's ssystem, is a bit horrifying to say the least. Fortunately, once it was done it worked quite well and has required no fixing for years. However, actually I now have to look into it again to make it work on another platform. It feels like reopening the Chernobyl concrete sarcophagus ;-) Fun and challenging, but I only change something on the day's where I feel at the top of my game :P

Re:I went through the following conversion effort

[luis_a_espinal]

For the love of everything that is holy, paragraphs!

Re:I went through the following conversion effort

[paradigm82]

Right, I noticed that after I posted - there paragraphs were there in what I pasted in :-)

Rebuilds usually fail

[salesgeek]

The original 10 year old article is a classic and should be read by every manager who takes on managing software. It's been my experience that most rebuilds are driven by the idea that:

If we get on platform X, we will have a world beater because platform X will give us so many more modern features.

The problem is that most developers don't understand that rebuilding in platform X may take as long or longer to get to market as the original software will. Most often it is because the original development process was very organic and inexpensive and as the firm grew, more rigorous testing and project management skills were rolled in to deal with change management, preventing regressions on upgrades and such. All the benefits of platform X are eaten alive by increased costs and lost agility.

Re:Rebuilds usually fail

[paradigm82]

You are absolutely right. Also, if you rewrite you constantly have to make the choice "Do I keep this thing working the same way as in the old app". This could be on multiple levels, i.e. how you present something in the GUI, how something is internally in the code, how it is represented in the database etc.

More often than not, there will be a lot of pressure to keep at least some of the things the same as before. And often keeping those things the same will force you into a lot of choices of the old app. You might find youself just cloning/hand-converting the old app.

On the other hand, every time you change something as compared to the old app, there will be complaints from customers since likely there will be some scenario that wasn't as easy as before. Furthermore, even if every change is for the better, just the fact that it works differently may upset customers as they have to retrain their employees. Even if everything works the same but just looks slightly different due to different GUI, that can be a huge issue as well since all training material with screen shots have to be updated!

As described in my previous post, I went through this whole process and this was for an automated conversion process which by its nature preserved the functional aspects. But just the slight variations in how the GUI looked were a huge issue but those we managed to get the customer swallow. However, other aspects I would also consider minor and generally equivalent (i.e. the exact tab order of elements, exact keyboard behaviour in list boxes) turned out to be huge issues whenever there was a difference - so I went to great lengths to preserve as much as I could. Consider how hard and not the least uninteresting it would be if you are not just auto-converting the code, but are implementing from scratch!

Lesson: Only convert or rewrite when you really have to. Make sure your existing customers' cost with the change is factored in as well as they will surely factor it in when considering your product.

Re:Rebuilds usually fail

[mr_mischief]

Not only is that cost born by your customer considered for when to upgrade, either. It's also considered vs. the cost of jumping ship to another vendor's product. The former costs you money now. The second costs you money for a long time to come.

Re:And why?

[Vellmont]

Your problem is not one of software development, but a business problem. It sounds like you're trying to please every customer with every feature they ask for. Try to learn there's some customers you don't want, and there's really certain features they think they want, but really don't need. Some people get into this mode of trying to find every special case they might ever possibly need. Those kind of customers will bleed you dry unless you can either convince them otherwise, or fire them.

It also sounds like you're trying to generalize your products and re-use them. That's good, but if you're going to do that you really have to be disciplined about what you will do and won't do, and be willing to let a customer go if it doesn't suit your larger business strategy.

That Regex is Generated

[Anonymous Coward]

And if you had the book "Mastering Regular Expressions" you would have seen that that regex was created by other code which refers to the grammar in RFC822, rather than being a magic blob of code stuck in the middle of the program.

Also, just how would anyone write *that* with a bunch of if-then-elses? You'd spend 8000 lines and it'd be slow. That regex, for all of its complexity, is actually very fast.

Re:That Regex is Generated

[Anonymous Coward]

Also, just how would anyone write *that* with a bunch of if-then-elses? You'd spend 8000 lines and it'd be slow. That regex, for all of its complexity, is actually very fast.

You'd write if/else statements for a finite state machine and reproduce what is effectively the compiled version of that regex. It might be 8000 lines, but it'd be anything but slow. You could almost certainly tweak the crap out of it and beat the regex code. I don't know what it would do for readability, and it certainly wouldn't be as nice as a one line regex and a one line comment to explain its purpose.

benefiting from previous work

[luis_a_espinal]

If this is true, then a rewrite of an existing application will always be a huge mistake. Joel is basically saying that developers don't benefit from previous experience, [b]a notion that I think is plainly wrong[/b]... All the [b]good developers[/b] I've ever worked with have been more than capable of seeing what works and what doesn't work in a solution, and would have been able to do a better job on the second try.

vlangber, see, benefiting from previous work only occurs with good developers, not with mediocre ones. So we have to look at what Joel said on that from that context. Many of Joel's writings deal with crappy/mediocre developers.

So it is true as you said, that it works... for good developers. Sub-par developers can also benefit from prior work if it was performed with competent developers, and only if the new work occurs in the same organization (and with the same competent developers from whom to learn.)

On average, and sadly, this is not the general case. There is more mediocrity than competency in software now (thanks to the Java/.Net universities), individuals who work year after year doing crappy work on crappy jobs with lowered expectations and dysfunctional modus operandi. They keep repeating the same crappy year of experience over and over. They wouldn't benefit at all from previous experience if they were tasked to rewrite something they did. Not in a million years.

Re:benefiting from previous work

[vlangber]

You could say that it's a problem with the universities, but I also believe that the companies that employ those people are to blame. Very interesting topic IMO. Maybe I'll write a blogpost on the subject..

balancing principles

[Xtifr]

The infamous "second system effect" can be countered by following the old principle, "write one to throw away" as long as you make sure that it's the second system that you throw away! Of course, this is often what actually happens in practice, but it is rarely what's planned. :)

rewriting is an exercise in collective...

[Organic Brain Damage]

...stupidity. Write a new application, don't re-write the old one. Don't promise to completely match the function of the legacy system with no downtime. If that's all that matters to your users/customers then they should stick with the legacy system and spend their money on something other than re-writing a working system in a fancy new programming language. Most of the time, a system grows over a decade or more and becomes difficult for new programmers to understand. Customers want a few simple changes and lazy, immature programmers say: "we can't change it because the old technology is no good, we have to rewrite it with the new technogloy" when they really just are too lazy or stupid to learn to use the old technology to make small changes to the existing system. I've seen this happen more than once. Never with a good result.

Re:And why?

[sohp]

Einstein DID say, "We cannot solve our problems with the same thinking we used when we created them", which may be what confuses people.

Don't dispect Joel!

[Rogerborg]

I heard, this one time, a developer pulled a 40 hour shift and the image of Joel appeared in their coffee and wrote their code for them. Just accept Joel as your personal guru and saviour and you too will ship!

Quoting Joel Spolsky is the new Godwin's Law

[Anonymous Coward]

Quoting Joel Spolsky is the new Godwin's Law.

I'm just saying.

launchd

[Anonymous Coward]

There is NOTHING in OSX which can reasonably be believed to have been invented (in any sense of the word) for OSX! Period, the end.

What about launchd? According to Wikipedia:

launchd is a unified, open source service management framework for starting, stopping and managing daemons, programs and scripts. Written and designed by Dave Zarzycki at Apple, it was introduced with Mac OS X v10.4/Darwin v8.0, and is licensed under the Apache License.

Doesn't sound like a very big project...

[CPE1704TKS]

A project that starts October 2006 with a deadline of Dec 2006 doesn't sound like a big project. In fact, even including the slippage of 3 months to March 2007, that's not a lot of code complexity we're talking about. I've had a single feature that took more time than 3 months!

From the sounds of it, it doesn't seem like this project really counts. Completely rewriting an app that take 3-6 months is not what I think could be considered a high risk project. The rewrites that I've seen were for much much larger projects and they took years. One ended in success, and the rest were failures.

Re:Doesn't sound like a very big project...

[vlangber]

I think you misread the article. The pilot project using the first basic version of the rewritten CMS started in October 2006, and was completed in march 2007. The rewrite of the CMS started in february/march 2005.Since first implementation project finished in 2007, we've been busy improving and adding features to the system. It's only recently that we've felt that the system has all the features needed to compete successfully in the international market, and that's why we have delayed the international launch of the new system to this year.

Re:Doesn't sound like a very big project...

[mr_mischief]

How about a five-year rewrite (Feb 2005 to early 2010) for a project to replace a five-year-old project?

I'd rewrite, too, if the government paid for it

[afabbro]

luckily we qualified for R&D grants from Innovation Norway, several years in a row, which helped quite a bit. It gave us the required time to make the base framework very solid and well-thought out.

Well, yes, if the government gives you money, why not spend it? Free money really skews the ROI.

Re:I'd rewrite, too, if the government paid for it

[vlangber]

We have much higher corporate taxes in Norway than in the US, and the grants were tax grants, so we got back a certain percentage of the tax we paid for the time we spent on research and development the previous year. So it's more like we are able to compete on equal terms with US companies.

Word

[Anonymous Coward]

Agreed, it would me no mistake to rewrite Word from scratch and dropping 99% of that useless "functionality" that never really worked on the way.

Rewrite when it is worth the effort

[n2rjt]

I use several rules to guide when to rewrite.
They aren't hard rules: some of them contradict.

* If it works, don't fix it. Only rewrite if the cost of rewriting is exceeded by the benefit.
* Learn from history. Never rewrite without understanding the old version.
* If at first you don't succeed, try try again. If I wrote it, I can rewrite it because now I know how to do it better.
* Don't throw the baby out with the bath water. When I rewrite it, I try to reuse the good parts.

Re:Rewrite when it is worth the effort

[n2rjt]

More:
* Always prototype new projects
* Always throw away the prototype

Re:And why?

[SnowZero]

Invoice printing was delayed, but no major amount of money was lost.

Looks like it turned out better than rewrites in 1999/2000:
Nobody would accept the program. Entire crops were lost.

SPOLSKY IS A FUCKING RETARD

[Anonymous Coward]

and so is anyone who listens to anything he says.

Re:And why?

[tomhudson]

Are you saying that it's impossible that Einstein also said the same thing? That there's some Pauli Exclusion Principle of Aphorisms?

I'm sure that lots of people have independently come up with the same quote - because it's so OBVIOUS and so TRUE.

Re:And why?

[fishexe]

Are you saying that it's impossible that Einstein also said the same thing?

Yes. Not because of some exclusion principal. Because it's so stupid. People quote it as though it's a technical definition out of the DSM or something. Even colloquially, it's a piss-poor definition of insanity; it's really much closer to a definition of stupidity. Insanity and stupidity are not the same thing, any more than lightning is the same thing as thunder.

Re:And why?

[tomhudson]

... but it's funnier with Einstein - (though I prefer to say that it doesn't take an Einstein to figure out that doing the same thing and expecting different results is stupid.

People quote it as though it's a technical definition out of the DSM or something

That's not such a great example. The DSM is pretty superficial, and sometimes dead wrong. Use it for insurance purposes, or as a shorthand to communicate (the latter is its' real use - but it's not the equivalent even of a man page). This is the same privately-published manual that had earlier versions saying lesbians and gays were mentally ill. Turns out that even today some of the psychiatrists on the various committees have "issues" of self-loathing with their own sexuality. They need to see a shrink ...

Insanity and stupidity are not the same thing

I'm not so sure that they're not related (playing devil's advocate). An insane genius is a pretty stupid thing. Insanity would then be a subset of stupidity. Speaking of which, since stupidity is relative, the supply of stupidity is limited (Einstein got that one wrong, just as he got the "unlimited universe" wrong :-)

The waters diverge

[Anonymous Coward]

Water likes A to B with the shortest route possible, and some take this approach to program design.

Others know the water stops flowing sometimes and know that whoever is around to experience it can read the river bottom to find out why.

Design for Supportability.

Not everything requires elegant logical precision or lean pictographic representations to get the job done.

Re:And why?

[fishexe]

People quote it as though it's a technical definition out of the DSM or something

That's not such a great example. The DSM is pretty superficial, and sometimes dead wrong.

My point had nothing to due with the accuracy of the DSM, but with the difference between technical and colloquial usage. The DSM gives technical definitions of each of the conditions, such as "A person has ADD if they regularly display six of the following ten behaviors:..." whereas it's more colloquial to say "You have ADD if you're often distracted by shiny things!" or "You lost your keys again? God, that's so ADD of you!" People often give the Rita Mae Brown quote as though it were a precise definition rather than a silly aphorism.

Insanity and stupidity are not the same thing

I'm not so sure that they're not related (playing devil's advocate).

And I'm not sure thunder and lightning aren't related (my previous example). In fact I am sure they are related. But they're not identical, so it would be rather silly to define lightning as "that loud booming noise that comes from electrical sparks between clouds and the ground." That would just confuse the issue.

Re:And why?

[mr_mischief]

So the project wasn't really complete in 2007? I mean, I've always considered testing and documentation part of a software project. Don't you? So your rewrite of the code was done in a couple of years, but the complete project, I'd say, took closer to five.

Re:One some /. er commented on Spolksky...

[mr_mischief]

Apple didn't rewrite OS X from scratch. They bought NeXT and made their OS look vaguely OS 9ish.

dude, THIS is the regex....

[Anonymous Coward]

Dude,

Your regex is a good reason to rewrite it. Here is alternative:

http://www.regular-expressions.info/regexbuddy/email.html