Arjen writes "Artima has had a conversation with Martin Fowler, one of the gurus on software development today. It consists of six parts. Parts one, two, three, and four are available now; the rest will appear the next weeks."
Developers love him; Managers hate him
by
dagg
·
· Score: 5, Interesting
Martin Fowler's books that outline such things
as extreme programming and refactoring are tops
in my opinion. But in my experience, many
middle-tier and upper-level managers think that
such concepts are useless and timewasting.
Re:Developers love him; Managers hate him
by
Ars-Fartsica
·
· Score: 1, Interesting
As a developer you can explain how cleaning up code is preventative maintenance to make development easier in the future, but if you're not a developer that's a very hard thing to measure.
Even then it is often a wash. How many people "refactored" to support OLE? CORBA? All of those would be a waste of time now. How many people refactored to get OO into their code? Dubious gains there too.
Refactoring is often about injecting the last good idea you had into working code.
I won't dispute that there are legitimate cleanups, but if a developer couldn't do it right the first time, I have my doubts that round two is going to be much better.
Re:Developers love him; Managers hate him
by
Textbook+Error
·
· Score: 2, Interesting
Refactoring is often about injecting the last good idea you had into working code. I won't dispute that there are legitimate cleanups, but if a developer couldn't do it right the first time, I have my doubts that round two is going to be much better.
I think you're probably right on both counts...:-) My experience has been that refactoring is most successful when applied to code that was in pretty poor shape to start with.
Often this is because it was the first attempt by someone who didn't really know what they were doing, they kept hacking on it until it limped into life, and then it got shipped. "Refactoring" in these cases is really just a face-saving way of cleaning up someone else's mess...
If you've been working professionally for more than 5 years, and you're reasonably good at it, then writing clean maintainable code should be second nature ("refactoring" as a new concept is really for people who didn't pick up "structured programming" the first time round...).
--
Nae bother
Re:Developers love him; Managers hate him
by
Leeji
·
· Score: 5, Interesting
Refactoring is often about injecting the last good idea you had into working code.
I've got to disagree with you on this one. Refactoring isn't about injecting good ideas into working code, it's about making sure that those new requirements you got last week didn't create spaghetti code. In a development world were you design from a static set of requirements, Refactoring really doesn't have much of a place. In that world, you have a valid point: if a developer can't do it right the firt time, round two won't be much better.
However, that's not the world I live in, and it's not the world that anybody I know lives in. Then again, I don't have friends at NASA. We live in a more realistic development world that changes reqirements after we've designed our framework. We live in a development world where the evolving system makes the customer reconsider their original decisions. Since you didn't design the system with these new requirements in mind, you're usually patching functionality on top of what you got.
Refactoring is about reviewing this patch-work to make sure that the code is written as though it were designed that way from the beginning.
Certain agile methodologies (especially XP) treat refactoring as a tenet. Customers never give you a complete spec, and you never want one. They give you "stories" that you implement quickly and minimally. You don't over-engineer to support infinite extensibility via Web Services and a plugin architecture (etc,) unless the story asks for it.
When your latest minimal implementation starts looking ugly or hacky (ie: 4 page switch statement) then you refactor until the code properly "expresses its intention."
You had some good points, but I think you are a little mistaken about when refactoring comes into play.
--
It all goes downhill from first post...
Re:Developers love him; Managers hate him
by
jorleif
·
· Score: 2, Interesting
You certainly have a point about sucky coders creating sucky code. However many times programs that are initially well designed get bloated with all sorts of cruft over time when features are added and requirements change. Requirement change tends to turn good designs into bad designs, since the original good design was for a different problem than the current program solves. This mess can often be cleaned up with refactoring. Often one is forced to work with code created by someone else, and if that code is messy, but not completely hopeless wonders can often be achieved by refactoring it.
Refactoring may very well be a waste of time
by
Ars-Fartsica
·
· Score: 2, Interesting
I don't think its just the PHB crowd that is suspicious of refactoring. My own opinion is that you should simply try your very best in the first iteration and leave it at that. Constantly reworking code that does the very same thing that it did before is often a pointless exercise - by time you have the internals "clean", new requirements demand a drastic addition or total rewrite anyway.
Requirements tend to move rapidly. That has been my experience. Dwelling upon one particular snapshot of requirements presumes that your code will have a long active lifespan, which it often doesn't. In my own case, my code has lived on in productive use for about four years on average after it has been written (after which it is often junked entirely), with a creation time often near a year. Those type of stats don't lead me to believe that refactoring is worthwhile.
Funny to see old terminology being the 'new' thing
by
Junks+Jerzey
·
· Score: 5, Interesting
Pick up any Forth book from the early 1980s and a major theme is that of properly factoring your code. I was always surprised that no one ever picked up on the term--until the last several years, that is.
Refactoring vs. Maintenance
by
Shackleford
·
· Score: 4, Interesting
It's quite interesting to see an interview from Martin Fowler just shortly after attending a lecture in a software engineering course in which maintenance was discussed. The lecturer, in his discussion of software maintenance, compared software maintenance to other forms of maintenance. Ususally, when people speak of maintenance, it is simply the act of ensuring that something is working as intended. In the context of software, however, when maintenance is done, so much about the software is changed that it maintenance would be an inaccurate term. When I hear about refactoring, however, I think that it is a more accurate term for it could be "maintenance" simply because it does not change the system, but improves the way in which it is built.
Just as in any other case in which maintenance must be done, it is quite important that this maintenance be done. It may not change the functionality of the code, but it can help make the software more easily adaptable. It can also help developers understand their own code, view it differently, and find different ways of implementing their systems. It may be more popular with Dilberts than PHBs, but perhaps those in the latter category should understand that even small amounts of refactoring can help save much time later on.
This is one of my favourite books on programming/software engineering and one of the many topics it covers is refactoring. I'd say that it does a good job arguing the importance of refactoring and how to convince those PHB types to accept it. But if you're just interested in refactoring itself, I suppose that this one is the best reference on the topic. I must say that for quite a few reasons, refactoring is something that should not simply be considered just another trend/buzzword, but an important part of maintenance, which in turn is an important part of the software development life cycle.
Re:Refactoring is a waste of time
by
Anonymous Coward
·
· Score: 2, Interesting
Your requirements will not change drastically
You are wrong with this assumption. Refactoring (like other XP practices) are for software whose requirements changes constanly.
Your code will be needed for a long time
You don't need the code for a long time because if you never refactor your design (not only your code), the system, with a lot of patches, becomes a Big Ball Of Mud, and then you need to throw away your code.
The point isn't only about "Refactoring", is about the evolutionary code (refactoring is only a small technique for to allow future evolution in parts of the code, cleaning the code to allow more drastic design changes).
The best example about software evolution is Smalltalk (note that the book "Refactoring" is based on ideas taken from Kent Beck's book "Smalltalk Best Practices Pattern").
The base code of Smalltalk comes from the 70's. And the recent version of comercial Smalltalks (with support for all the buzz tech like XML, WebServices..) still uses a lot of objects who evolve from the original implementation.
(offtopic - Talking about Smalltalk, I don't know why any company doesn't made an object based OS, based in the concepts of Smalltalk, for example Squeak is a good example of an Smalltalk Enviroment -or Object Enviroment- that can be used as a complete OS desktop).
Refactoring is the PROCESS!
by
alder
·
· Score: 2, Interesting
From the book:
Refactoring is the process of changing a software system in such a way that is does not alter the external behavior of the code yet improves its internal structure
This process is not a rigit set of procedures one must perform in order to make a refactoring happen. The refactoring could be as simple as introduce explaining variable or extract method or even rename method, which take less then a minute to complete. While they are simple and easy to complete, especially if your IDE/editor is equipped with refactoring support/tools, they provide tremendous value - the code becomes cleaner and more maintainable.
I had to perform those tiny refactorings zillion times on my own and other peoples code to better express its purpose in code itself. Sometimes without preliminary refactoring it was simply impossible to understand someones code at all (one prominent example - class had a method that consumed 12 Letter pages in 8pt Courier). It was my experience that most of the time refactoring never takes more then 10 min. to complete. Then you switch your activity to coding, and only later may want to do some more refactoring. You may not have notices this, but if you are a good software engineer you do refactor you code even if you do it subconsciously and think that you don't!
Refactoring, like XP, is not a religion and does not attempt to be "all or nothing" for you. Use as little (or as much) from them as you find convenient. Spend as little (or as much) time on a particular activity as you find useful. Most importantly - use what you find useful and keep open minds for other parts as you may find them useful in the future. After all refactoring stems from the same roots as design patterns - "a core of the solution to the problem which occurs over and over again in our environment". They just provide value to different domains - design and implementation.
Their problem is, it's very difficult for them to tell if refactoring was actually worth it - their developers start with a program that does A, they stop working on new features for the next release for 3 months, and they come back with a program that still does A ("but better!").
S'funny, most of my refactorings take about three minutes, and come back with a program that still does A exactly the same (but better organised).
Occasionally, as I've done over the past couple of days, I do spend several hours rewriting a piece of spaghetti into manageable form, just so I can sensibly work with it. I am, at this moment, the living embodiment of the poor soul who is fated to work with hack-on-hack code forever, such that every little change he makes that logically is fine actually causes a different result in the fifteenth decimal place.
This is the sort of time when refactoring is essential, and of course, if the code has been refactored in small increments along the way, it would never have reached SpagCon 5 and required me to do a big refactor now...
The only catch, alas, is that now my nice systematic code has to have special cases in to reproduce exactly the behaviour the old spaghetti had in relevant cases, right down to that error in the fifteenth place.:-(
-- If you disagree, post your argument. (-1, Overrated) isn't your personal censorship tool for views you don't like.
From the trenches...
by
cheezehead
·
· Score: 4, Interesting
Ok, a true story.
Years ago, I worked on a big software project for the aerospace industry. This was in the early 90s, and I am proud to say we were a bit ahead of our time. Iterative design, unit tests automatically running overnight, peer reviews, refactoring (although we called it "rewriting"), collective code ownership, we did it all.
After extensive integration testing (and finding and fixing many bugs), we installed the system at the customer site. After running for two weeks continuously, the system froze. Fortunately, the operating system was kind enough to report that it had run out of memory.
The cause of the problem was obvious: we had a memory leak somewhere. We had never run our code through a memory leak detection tool. The reason for this was that management did not want to spend money on something like this (...). Fortunately, we happened to have an evaluation copy of such a tool when the problem was detected. Installing the tool took 20 minutes, finding the leak took 2 minutes. It also found a memory leak in on of the OS libraries, but that was a one time leak. Our problem leak was only 8 bytes in size, but since the leaking routine was called several times per second, the system ran out of memory eventually.
Anyway, the leak was all my fault, and fixing it took about 20 seconds. Rebuild, and everybody was happy again.
So, what did we do wrong?
1. We should have had better checks for memory leaks in the first place. So, blame management...
2. We should have tested for a longer period than we did. Our integration tests included stress tests, but we never ran tests for more than 24 hours or so without rebooting (rebooting obviously hid the memory problem). Running for two weeks would have revealed the problem, but that doesn't always cure everything (read on for part two...)
Two years later. I had moved to another country, and was doing consulting work at a customer site. I got a phone call. The system had frozen again, but this time not after 2 weeks, but after 6 months running continuously. I investigated the problem, and after lots of debugging I isolated the problem to a (vendor supplied) device driver. Code inspection revealed that this driver incremented an internal counter every time it was called. This counter was a signed 32-bit integer. So, after 6 months of calling it several times per second, the counter rolled over, and the driver stopped working. Of course, a reboot fixed it, and the system was good to go for another 6 months. I'm not sure if this driver was ever fixed. You could very well argue that it's not worth the effort: just reboot once every 6 months, and you're fine.
What is the point of all this? Well, a lengthy testing period would have revealed our first bug. However, to find the second one, we would have had to do a stress test for more than 6 months. No matter how enlightened management or your customer are, they'll never agree to something like that in real life. Besides, there is a known defect in this system that will manifest itself in 2036 or so... Also, where should you stop testing? We trusted the 3rd party driver to be coreect, and in this case we were wrong to do so. I see no practical solution for this, though.
Lesson: you can do (almost) everything right, use proper methodologies, test early and often, have a team of brilliant people, etc., and still you can have problems.
Ironically, the fact that we had stable software running on a robust operating system (UNIX), caused the bugs to manifest themselves. Had we been writing unstable software or running on Windows, reboots would have happened more than once every two weeks, and neither bug would have shown up...
--
MSN 8: Now Microsoft even has bugs in their ad campaigns.
Design Patterns
by
jtdubs
·
· Score: 5, Interesting
Everyone all hopped up on Design Patterns that has any background in high-level programming needs to read:
Revenge of the Nerds, by Paul Graham http://www.paulgraham.com/icad.html
Paul Graham is a Lisp guru, among other things. His perspective on "patterns" is quite in contrast to that of the XP fab four.
Specifically, I refer to this fabulous quote:
"If you try to solve a hard problem, the question is not whether you will use a powerful enough language, but whether you will (a) use a powerful language, (b) write a de facto interpreter for one, or (c) yourself become a human compiler for one. We see this already begining to happen in the Python example, where we are in effect simulating the code that a compiler would generate to implement a lexical variable.
"This practice is not only common, but institutionalized. For example, in the OO world you hear a good deal about "patterns". I wonder if these patterns are not sometimes evidence of case (c), the human compiler, at work. When I see patterns in my programs, I consider it a sign of trouble. The shape of a program should reflect only the problem it needs to solve. Any other regularity in the code is a sign, to me at least, that I'm using abstractions that aren't powerful enough -- often that I'm generating by hand the expansions of some macro that I need to write."
He's my hero.:-)
Justin Dubs
Re:Missing some crucial bits there...
by
Mark+Wilkinson
·
· Score: 2, Interesting
Every artifact that you create, and then decide to keep, will need to be maintained over time. [...] Every time you decide to keep a model you trade-off agility for the convenience of having that information available to your team in an abstract manner (hence potentially enhancing communication within your team as well as with project stakeholders). Never underestimate the seriousness of this trade-off. [...] a development team that decides to develop and maintain a detailed requirements document, a detailed collection of analysis models, a detailed collection of architectural models, and a detailed collection of design models will quickly discover they are spending the majority of their time updating documents instead of writing source code.
Also check some of the practices, particularly "Discard Temporary Models" and "Update Only When It Hurts". In brief they suggest that you learn to throw things away when you've finished with them, minimise the number of models that you keep, and don't fret about making sure all your models are consistent just for the sake of it.
Slashdot Mirror
Sex - Find It
Requirements tend to move rapidly. That has been my experience. Dwelling upon one particular snapshot of requirements presumes that your code will have a long active lifespan, which it often doesn't. In my own case, my code has lived on in productive use for about four years on average after it has been written (after which it is often junked entirely), with a creation time often near a year. Those type of stats don't lead me to believe that refactoring is worthwhile.
Pick up any Forth book from the early 1980s and a major theme is that of properly factoring your code. I was always surprised that no one ever picked up on the term--until the last several years, that is.
Just as in any other case in which maintenance must be done, it is quite important that this maintenance be done. It may not change the functionality of the code, but it can help make the software more easily adaptable. It can also help developers understand their own code, view it differently, and find different ways of implementing their systems. It may be more popular with Dilberts than PHBs, but perhaps those in the latter category should understand that even small amounts of refactoring can help save much time later on.
This is one of my favourite books on programming/software engineering and one of the many topics it covers is refactoring. I'd say that it does a good job arguing the importance of refactoring and how to convince those PHB types to accept it. But if you're just interested in refactoring itself, I suppose that this one is the best reference on the topic. I must say that for quite a few reasons, refactoring is something that should not simply be considered just another trend/buzzword, but an important part of maintenance, which in turn is an important part of the software development life cycle.
Your requirements will not change drastically
You are wrong with this assumption. Refactoring (like other XP practices) are for software whose requirements changes constanly.
Your code will be needed for a long time
You don't need the code for a long time because if you never refactor your design (not only your code), the system, with a lot of patches, becomes a Big Ball Of Mud, and then you need to throw away your code.
The point isn't only about "Refactoring", is about the evolutionary code (refactoring is only a small technique for to allow future evolution in parts of the code, cleaning the code to allow more drastic design changes).
The best example about software evolution is Smalltalk (note that the book "Refactoring" is based on ideas taken from Kent Beck's book "Smalltalk Best Practices Pattern").
The base code of Smalltalk comes from the 70's. And the recent version of comercial Smalltalks (with support for all the buzz tech like XML, WebServices..) still uses a lot of objects who evolve from the original implementation.
(offtopic - Talking about Smalltalk, I don't know why any company doesn't made an object based OS, based in the concepts of Smalltalk, for example Squeak is a good example of an Smalltalk Enviroment -or Object Enviroment- that can be used as a complete OS desktop).
I had to perform those tiny refactorings zillion times on my own and other peoples code to better express its purpose in code itself. Sometimes without preliminary refactoring it was simply impossible to understand someones code at all (one prominent example - class had a method that consumed 12 Letter pages in 8pt Courier). It was my experience that most of the time refactoring never takes more then 10 min. to complete. Then you switch your activity to coding, and only later may want to do some more refactoring. You may not have notices this, but if you are a good software engineer you do refactor you code even if you do it subconsciously and think that you don't!
Refactoring, like XP, is not a religion and does not attempt to be "all or nothing" for you. Use as little (or as much) from them as you find convenient. Spend as little (or as much) time on a particular activity as you find useful. Most importantly - use what you find useful and keep open minds for other parts as you may find them useful in the future. After all refactoring stems from the same roots as design patterns - "a core of the solution to the problem which occurs over and over again in our environment". They just provide value to different domains - design and implementation.
S'funny, most of my refactorings take about three minutes, and come back with a program that still does A exactly the same (but better organised).
Occasionally, as I've done over the past couple of days, I do spend several hours rewriting a piece of spaghetti into manageable form, just so I can sensibly work with it. I am, at this moment, the living embodiment of the poor soul who is fated to work with hack-on-hack code forever, such that every little change he makes that logically is fine actually causes a different result in the fifteenth decimal place.
This is the sort of time when refactoring is essential, and of course, if the code has been refactored in small increments along the way, it would never have reached SpagCon 5 and required me to do a big refactor now...
The only catch, alas, is that now my nice systematic code has to have special cases in to reproduce exactly the behaviour the old spaghetti had in relevant cases, right down to that error in the fifteenth place. :-(
If you disagree, post your argument. (-1, Overrated) isn't your personal censorship tool for views you don't like.
Ok, a true story.
Years ago, I worked on a big software project for the aerospace industry. This was in the early 90s, and I am proud to say we were a bit ahead of our time. Iterative design, unit tests automatically running overnight, peer reviews, refactoring (although we called it "rewriting"), collective code ownership, we did it all.
After extensive integration testing (and finding and fixing many bugs), we installed the system at the customer site. After running for two weeks continuously, the system froze. Fortunately, the operating system was kind enough to report that it had run out of memory.
The cause of the problem was obvious: we had a memory leak somewhere. We had never run our code through a memory leak detection tool. The reason for this was that management did not want to spend money on something like this (...). Fortunately, we happened to have an evaluation copy of such a tool when the problem was detected. Installing the tool took 20 minutes, finding the leak took 2 minutes. It also found a memory leak in on of the OS libraries, but that was a one time leak. Our problem leak was only 8 bytes in size, but since the leaking routine was called several times per second, the system ran out of memory eventually.
Anyway, the leak was all my fault, and fixing it took about 20 seconds. Rebuild, and everybody was happy again.
So, what did we do wrong?
1. We should have had better checks for memory leaks in the first place. So, blame management...
2. We should have tested for a longer period than we did. Our integration tests included stress tests, but we never ran tests for more than 24 hours or so without rebooting (rebooting obviously hid the memory problem). Running for two weeks would have revealed the problem, but that doesn't always cure everything (read on for part two...)
Two years later. I had moved to another country, and was doing consulting work at a customer site. I got a phone call. The system had frozen again, but this time not after 2 weeks, but after 6 months running continuously. I investigated the problem, and after lots of debugging I isolated the problem to a (vendor supplied) device driver. Code inspection revealed that this driver incremented an internal counter every time it was called. This counter was a signed 32-bit integer. So, after 6 months of calling it several times per second, the counter rolled over, and the driver stopped working. Of course, a reboot fixed it, and the system was good to go for another 6 months. I'm not sure if this driver was ever fixed. You could very well argue that it's not worth the effort: just reboot once every 6 months, and you're fine.
What is the point of all this? Well, a lengthy testing period would have revealed our first bug. However, to find the second one, we would have had to do a stress test for more than 6 months. No matter how enlightened management or your customer are, they'll never agree to something like that in real life. Besides, there is a known defect in this system that will manifest itself in 2036 or so...
Also, where should you stop testing? We trusted the 3rd party driver to be coreect, and in this case we were wrong to do so. I see no practical solution for this, though.
Lesson: you can do (almost) everything right, use proper methodologies, test early and often, have a team of brilliant people, etc., and still you can have problems.
Ironically, the fact that we had stable software running on a robust operating system (UNIX), caused the bugs to manifest themselves. Had we been writing unstable software or running on Windows, reboots would have happened more than once every two weeks, and neither bug would have shown up...
MSN 8: Now Microsoft even has bugs in their ad campaigns.
Everyone all hopped up on Design Patterns that has any background in high-level programming needs to read:
:-)
Revenge of the Nerds, by Paul Graham
http://www.paulgraham.com/icad.html
Paul Graham is a Lisp guru, among other things. His perspective on "patterns" is quite in contrast to that of the XP fab four.
Specifically, I refer to this fabulous quote:
"If you try to solve a hard problem, the question is not whether you will use a powerful enough language, but whether you will (a) use a powerful language, (b) write a de facto interpreter for one, or (c) yourself become a human compiler for one. We see this already begining to happen in the Python example, where we are in effect simulating the code that a compiler would generate to implement a lexical variable.
"This practice is not only common, but institutionalized. For example, in the OO world you hear a good deal about "patterns". I wonder if these patterns are not sometimes evidence of case (c), the human compiler, at work. When I see patterns in my programs, I consider it a sign of trouble. The shape of a program should reflect only the problem it needs to solve. Any other regularity in the code is a sign, to me at least, that I'm using abstractions that aren't powerful enough -- often that I'm generating by hand the expansions of some macro that I need to write."
He's my hero.
Justin Dubs
Also check some of the practices, particularly "Discard Temporary Models" and "Update Only When It Hurts". In brief they suggest that you learn to throw things away when you've finished with them, minimise the number of models that you keep, and don't fret about making sure all your models are consistent just for the sake of it.