Slashdot Mirror
Joel Test Updated
An anonymous reader writes "In 2000, Joel Spolsky wrote the Joel Test, an excellent and simple way to evaluate a software company. While the test is still used, it's getting outdated, as many companies are moving to web technologies, and new development tools exist. In his blog, Marc Garcia wrote about what could be an update to Joel Test."
I think the original Joel questions still work fine.
Who needs a distributed source control system if everyone on my team works in the same office.
Also, I don't want end customers submitting directly into my bug tracker. I'm OK with them having a web based way to submit problems, but then QA should verify the defect and translate customer speak into something that makes sense. Then the defect can be entered into bug tracker with a good set of steps to reproduce and given a proper severity. To a customer, everything is critical.
Joel Spolski is a guy who runs a software company, and he used to be a program manager on Excel. However that's not why people give a shit about what he thinks. People read (past tense) his articles because they were pretty good, and explained stuff lots of people in the software business need to know but often don't. For instance if somebody doesn't understand Unicode, I often point them to his article explaining it .... because he did a pretty good job.
The real question is, who is Marc Garcia and why does the article submitter think we should care? In fairness, he says the "updated list" is just his personal opinion rather than something generally applicable, which is good because pretty much every software company I know would fail at least one or two of the points on there, including Google.
*Shrug* - just comes off as a wish list of how this developer thinks software companies should work. IMHO part of the attraction of the original Joel list was that it was more or less applicable regardless of product audience / build tools etc. The core principles *really were important*.
But getting back to this, Garcia's list appears to be fairly sound. I have some comments on two of his modified questions:
Do you use a distributed source control system? Why should I care about distributed source code control in a monolithic commercial development environment? I can see its value in a distributed open-source project, but I really don't understand the necessity otherwise.
Do you fix bugs before implementing new features? All bugs? Some bugs? This tells me nothing about prioritization. Sometimes you need to do both at once. Sometimes it's not worth it to fix a bug if the circumstance is rare enough.
Dog is my co-pilot.
This is of course not "Joel" updating his list of requirements for good development, but some joker trying to take advantage of Joel's reputation.
Example: Allow users direct access to a bug database? It's hard enough to train testers to give you good bug reports. You won't get anything usable from an end user without some severe filtering.
Do you sell your car every year to buy a new one? There's a cost to converting, so you have to make an engineering decision about making the conversion. The automated tools don't always work with old and complex repositories.
There's a difference between getting customer feedback, which is good, and allowing that feedback to go directly into the bug database, an exercise in insanity. How many of those bug reports are going to be accurate and descriptive enough so that whomever gets stuck reading it will actually be able to identify what needs to be fixed, especially if there's not a crash report, or other error messaged included?
Unless your users are professional developers/programmers, the signal to noise ratio is going to be horrid.
The guy's apparently from Belgium, so English is quite possibly his fourth language, so I won't bother ripping on his grammar. His content is another matter...
Original: Do you use source control?
New: Do you use a distributed source control system?
My current Big Work Project has a whopping four coders, so I can't speak to when distributed source control is a big deal and when it's overkill.
Original: Can you make a build in one step? Do you make daily builds?
New: Do you have automated build or deployment procedures?
Clearly inferior. An error-prone 20-step process that you run once a month is still automated, just not automated enough and not used enough.
Original: Do you have a bug database?
New: Do you use a bug database where users can report bugs directly?
The BWP is still small enough to get by on Excel lists, with changes manually merged back into the master copy by the project manager, or just e-mail for quick-turnaround items. Excel is noticeably clunkier than an automated system, but you may want to start there to get a feel for what the automated system should do (e.g. separate status fields for "the coder did some testing and thinks it's fixed" vs. "the tester did some more thorough testing, confirmed that there were no misunderstandings, and couldn't find any more edge/corner cases").
Original: Do you have testers? Do you do hallway usability testing?
New: Do you have a testing protocol, and specific resources for testing?
I hate calling people "resources". Also, your protocol should stick to the right things (e.g. "when you find a problem, report X and Y and Z"); an example of a wrong thing is "test this specific way of using the system", because real users will go off the rails.
Original: Do you fix bugs before writing new code?
New: Do you fix bugs before implementing new features?
More or less equivalent.
Original: Do you have an up-to-date schedule? Do you have a spec?
New: Do you have a roadmap, and you don't make important changes to the short term priorities?
These have become fuzzy for no good reason that I can discern.
Original: Do programmers have quiet working conditions? Do you use the best tools money can buy?
New: Do your team work in good conditions (quiet environment, flexible schedule, freedom to choose development software, fair paycheck...)
More or less equivalent. "Fair paycheck" is so blindingly obvious that it shouldn't need to be pointed out. "Flexible schedule" is a genuinely good addition; I've personally gained some peace of mind by saving some tasks for evenings/weekends when I knew I wouldn't be interrupted by other work stuff (family stuff is another matter, but easier to control), and consequently taking it easier during normal business hours.
Original: Do new candidates write code during their interview?
This has been omitted completely for no good reason that I can discern. Maybe he's lucky and hasn't had to clean up after a bad coder yet.
This test probably isn't applicable to some open source projects, but there's probably a modified version of the test that could indicate whether an open source project is likely to succeed. Many of the questions on the test should be common sense by now, but would you want to work for a company that didn't use source control? Even if you were working alone you should be using some kind of CMS. Some times the best tools money can buy don't actually cost a thing because they're FOSS. Initial and support costs aside, how is using the best tools available bad advice?
If you think Microsoft products are bad, imagine how much worse they would be if Microsoft wasn't answering yes to a lot of those questions. Also understand that the article was written in 2000, when the computer world landscape was vastly different than it is today. Google and Apple where nowhere near as relevant, Linux wasn't a viable option for grandma, and it appeared as though Microsoft would continue to dominate the industry just as it had in the 90's. If you wanted to name drop a company that everyone would know and recognize as a leader in the software industry, you would have used Microsoft as well.
Put a $10 bill, or the local equivalent, in an envelope on the company bulletin board. On the outside, write, "I need change for $10 please" without any indication of who you are. Do this every six months or so. If you ever come back and find that the envelope is empty, your company is too big. You have hired a thief who does not care about his or her fellow employees.
I piss off bigots.
If it means redoing all of your established routines and teach people the new routines, rewriting all your automation, and obsoleting existing ticket or work log systems (or otherwise run two repositories in parallel, with the problems of authorativity that entails), "nicer" doesn't necessarily cut it. There has to be a gain that measurably outweighs the inconveniences.
as many companies are moving to web technologies, and new development tools exist.
Web technologies change nothing in his test. And his test does not mention any specific tools, just general classes of tools. "Do you use source control?" and the catch-all "Do you use the best tools money can buy?" are asking if you use the general types of tools that distinguishes good shops from bad shops. You could add "Do you use a mock-objects framework?" but now it isn't universal, because that doesn't always apply and could be subject to debate.. Then it just becomes someone's checklist of "Have you used every tool that I use and endorse?" The Joel Test is universally applicable, covering the kinds of things all shops should do.
Most of the updates in his blog are a pedantic rewording of the existing ones.
The idea that borland didn't know about the performance of loading a segment register is ridiculous. It's in the intel manual. Everybody I knew who cared about the performance of software had that manual handy. Then eventually the compiler just took care of it for you and we all stopped caring.
"Who is the Journal of Quantum Physics going to believe?" --Stephen Hawking
I imagine Joel thought you would be smart enough to apply the guidelines to your own situation. You've done that to a degree, but you're making the list needlessly inflexible.
- Do you use source control?
If your source control does not actually control the source effectively, it isn't source control. It's just a thing you put your source code into to make your life a living hell.
-Can you make a build in one step?/Do you make daily builds?
This one you have a small point on, but the obvious purpose here is to automate the process of publishing the latest version of the software to the team in order to avoid mistakes in the build/deployment process. Server side scripting, for example, isn't compiled or "built", but all the pieces do need to be in place and everyone needs access to the most current version. "Building" in this case would mean deploying the new code to the internal test server so all the developers know what the most current version of the web site is and can actually use it to verify that everything is working. This prevents you from making changes that are so large they are difficult to trace (if you have source control and nightly builds, you know exactly who screwed up and when). There should be an automated process to ensure all of the needed components are, in fact, there. It's just as critical for things that don't build as it is for things that do, it just looks different so you may not realize it.
"make this a 'deploy package' in one step" I hope you mean deploy to the test server in one step, and not publishing it out to the world. If not you totally missed the point of nightly builds (it's to catch major bugs before the code needing debugging becomes too large).
I could accept Garcia's "Do you have automated build or deployment procedures?" as a replacement for "Can you make a build in one step". The point is automation to avoid procedural mistakes, not compiling/deployment itself.
-Do you fix bugs before writing new code?
"Use unit testing here" - There is no need to be so specific. Unit testing is an effective modern tool, but it will not catch all bugs, particularly systemic bugs. The point is that you need to fix the major, money sucking bugs before you add new features.
-Do you have a spec?
"overrated" ??? The specification is the thing that describes what you are trying to do! How the hell can you write anything but hodgepodge software, especially with more than one developer, if you don't have overall design goals written down somewhere where they can be referenced? You can change the specification if your goals or needs change, but you should always have one! I suspect this is an especially serious flaw for open source projects that don't have strong leadership, given the distributed nature of open source.
-Do programmers have a quiet working conditions?
"ditch the phones" ?? What if your programmer works from home, how are you supposed to effectively communicate? Email isn't good enough for all situations, IM is better but still doesn't quite cut it, and frankly it encourages people to interrupt you more often. Quiet working conditions are what you need, not a simple lack of phones. I think if you were to expand this you should add "free from distractions" to the end of that. These days, it can be very quiet in your office yet still be extremely distracting with emails and IM notifications popping up all over the place.
Security is mostly a superstition... Avoiding danger is no safer in the long run than outright exposure. - Helen Keller