Slashdot Mirror
Joel Test Updated
An anonymous reader writes "In 2000, Joel Spolsky wrote the Joel Test, an excellent and simple way to evaluate a software company. While the test is still used, it's getting outdated, as many companies are moving to web technologies, and new development tools exist. In his blog, Marc Garcia wrote about what could be an update to Joel Test."
I think the original Joel questions still work fine.
Who needs a distributed source control system if everyone on my team works in the same office.
Also, I don't want end customers submitting directly into my bug tracker. I'm OK with them having a web based way to submit problems, but then QA should verify the defect and translate customer speak into something that makes sense. Then the defect can be entered into bug tracker with a good set of steps to reproduce and given a proper severity. To a customer, everything is critical.
Joel Spolski is a guy who runs a software company, and he used to be a program manager on Excel. However that's not why people give a shit about what he thinks. People read (past tense) his articles because they were pretty good, and explained stuff lots of people in the software business need to know but often don't. For instance if somebody doesn't understand Unicode, I often point them to his article explaining it .... because he did a pretty good job.
The real question is, who is Marc Garcia and why does the article submitter think we should care? In fairness, he says the "updated list" is just his personal opinion rather than something generally applicable, which is good because pretty much every software company I know would fail at least one or two of the points on there, including Google.
Joel Test for my scientific endeavors.
Hi! I make Firefox Plug-ins. Check 'em out @ https://addons.mozilla.org/en-US/firefox/addon/youtube-mp3-podcaster/
*Shrug* - just comes off as a wish list of how this developer thinks software companies should work. IMHO part of the attraction of the original Joel list was that it was more or less applicable regardless of product audience / build tools etc. The core principles *really were important*.
But getting back to this, Garcia's list appears to be fairly sound. I have some comments on two of his modified questions:
Do you use a distributed source control system? Why should I care about distributed source code control in a monolithic commercial development environment? I can see its value in a distributed open-source project, but I really don't understand the necessity otherwise.
Do you fix bugs before implementing new features? All bugs? Some bugs? This tells me nothing about prioritization. Sometimes you need to do both at once. Sometimes it's not worth it to fix a bug if the circumstance is rare enough.
Dog is my co-pilot.
This is of course not "Joel" updating his list of requirements for good development, but some joker trying to take advantage of Joel's reputation.
Example: Allow users direct access to a bug database? It's hard enough to train testers to give you good bug reports. You won't get anything usable from an end user without some severe filtering.
really most principals can be applied to any kind of work and the employee doesn't really matter (a developer should pair with a sales person, if that's the kind of experience you want to swap for instance)
Code repository = Auditable whatever, and filing.
Bug / Feature tracker = CRM or whatever.
Usability testing = Product evaluation and feedback.
etc....
Best tools money can buy, should really be most appropriate tools etc...
thank God the internet isn't a human right.
The "sentence" is total fail.
I find the irony of that statement absolutely hilarious.
Security is mostly a superstition... Avoiding danger is no safer in the long run than outright exposure. - Helen Keller
Do you sell your car every year to buy a new one? There's a cost to converting, so you have to make an engineering decision about making the conversion. The automated tools don't always work with old and complex repositories.
So, why exactly does everyone need a distributed source control system? Just because anything distributed is automatically cooler?
Also, yeah, having the users report bugs directly in the bug database is just stupid.
sic transit gloria mundi
Thanks for proving Santa does not exist. However by doing so you have also proved Trolls exist.
What one fool can do, another can. (Ancient Simian Proverb)
There's a difference between getting customer feedback, which is good, and allowing that feedback to go directly into the bug database, an exercise in insanity. How many of those bug reports are going to be accurate and descriptive enough so that whomever gets stuck reading it will actually be able to identify what needs to be fixed, especially if there's not a crash report, or other error messaged included?
Unless your users are professional developers/programmers, the signal to noise ratio is going to be horrid.
The guy's apparently from Belgium, so English is quite possibly his fourth language, so I won't bother ripping on his grammar. His content is another matter...
Original: Do you use source control?
New: Do you use a distributed source control system?
My current Big Work Project has a whopping four coders, so I can't speak to when distributed source control is a big deal and when it's overkill.
Original: Can you make a build in one step? Do you make daily builds?
New: Do you have automated build or deployment procedures?
Clearly inferior. An error-prone 20-step process that you run once a month is still automated, just not automated enough and not used enough.
Original: Do you have a bug database?
New: Do you use a bug database where users can report bugs directly?
The BWP is still small enough to get by on Excel lists, with changes manually merged back into the master copy by the project manager, or just e-mail for quick-turnaround items. Excel is noticeably clunkier than an automated system, but you may want to start there to get a feel for what the automated system should do (e.g. separate status fields for "the coder did some testing and thinks it's fixed" vs. "the tester did some more thorough testing, confirmed that there were no misunderstandings, and couldn't find any more edge/corner cases").
Original: Do you have testers? Do you do hallway usability testing?
New: Do you have a testing protocol, and specific resources for testing?
I hate calling people "resources". Also, your protocol should stick to the right things (e.g. "when you find a problem, report X and Y and Z"); an example of a wrong thing is "test this specific way of using the system", because real users will go off the rails.
Original: Do you fix bugs before writing new code?
New: Do you fix bugs before implementing new features?
More or less equivalent.
Original: Do you have an up-to-date schedule? Do you have a spec?
New: Do you have a roadmap, and you don't make important changes to the short term priorities?
These have become fuzzy for no good reason that I can discern.
Original: Do programmers have quiet working conditions? Do you use the best tools money can buy?
New: Do your team work in good conditions (quiet environment, flexible schedule, freedom to choose development software, fair paycheck...)
More or less equivalent. "Fair paycheck" is so blindingly obvious that it shouldn't need to be pointed out. "Flexible schedule" is a genuinely good addition; I've personally gained some peace of mind by saving some tasks for evenings/weekends when I knew I wouldn't be interrupted by other work stuff (family stuff is another matter, but easier to control), and consequently taking it easier during normal business hours.
Original: Do new candidates write code during their interview?
This has been omitted completely for no good reason that I can discern. Maybe he's lucky and hasn't had to clean up after a bad coder yet.
If you were an alien and you came here in 1991 and you wanted to learn how to develop software, you would learn ten times as much at Microsoft as anywhere else, I think, because I watched these companies kind of flail making mistakes. There were things--really basic things, that companies did not know. Microsoft knew that loading a segment register on the 386 was a very time-consuming operation, and therefore on the 386 architecture you can't use far pointers unless you absolutely have to because it's extremely slow. Borland did not know that. Result: Microsoft Access loaded in 2 or 3 seconds; Borland Paradox for Windows took 90 seconds to get running. Because of something that Microsoft knew that Borland did not know. And that's one of a million examples.
Now Microsoft has forgotten all these things, and they've hired a lot of morons that don't know these things anymore. I think that now Microsoft is kind of a big tar pit where you can barely move forward because there's so much bureaucracy. But I learned a lot.
Source: http://www.foundersatwork.com/joel-spolksy.html
What one fool can do, another can. (Ancient Simian Proverb)
This test probably isn't applicable to some open source projects, but there's probably a modified version of the test that could indicate whether an open source project is likely to succeed. Many of the questions on the test should be common sense by now, but would you want to work for a company that didn't use source control? Even if you were working alone you should be using some kind of CMS. Some times the best tools money can buy don't actually cost a thing because they're FOSS. Initial and support costs aside, how is using the best tools available bad advice?
If you think Microsoft products are bad, imagine how much worse they would be if Microsoft wasn't answering yes to a lot of those questions. Also understand that the article was written in 2000, when the computer world landscape was vastly different than it is today. Google and Apple where nowhere near as relevant, Linux wasn't a viable option for grandma, and it appeared as though Microsoft would continue to dominate the industry just as it had in the 90's. If you wanted to name drop a company that everyone would know and recognize as a leader in the software industry, you would have used Microsoft as well.
That's it. No tools, methodology, procedures, or what have you will make up for the lack of good programmers. And good programmers will do well no matter what the tools, procedures, methodologies, etc are (barring Kafka-esque hindrances).
So here's my revised list:
1) Is the company full of good programmers?
Of course, acquiring and maintaining good programmers doesn't just happen. New hire interviews need to be technically based, the staff needs adequate compensation, and management should not get in the way of programmers trying to do their job. However, employees don't need to be treated like prima donnas. They just need management that commands respect and respects them likewise.
Put a $10 bill, or the local equivalent, in an envelope on the company bulletin board. On the outside, write, "I need change for $10 please" without any indication of who you are. Do this every six months or so. If you ever come back and find that the envelope is empty, your company is too big. You have hired a thief who does not care about his or her fellow employees.
I piss off bigots.
If it means redoing all of your established routines and teach people the new routines, rewriting all your automation, and obsoleting existing ticket or work log systems (or otherwise run two repositories in parallel, with the problems of authorativity that entails), "nicer" doesn't necessarily cut it. There has to be a gain that measurably outweighs the inconveniences.
as many companies are moving to web technologies, and new development tools exist.
Web technologies change nothing in his test. And his test does not mention any specific tools, just general classes of tools. "Do you use source control?" and the catch-all "Do you use the best tools money can buy?" are asking if you use the general types of tools that distinguishes good shops from bad shops. You could add "Do you use a mock-objects framework?" but now it isn't universal, because that doesn't always apply and could be subject to debate.. Then it just becomes someone's checklist of "Have you used every tool that I use and endorse?" The Joel Test is universally applicable, covering the kinds of things all shops should do.
Most of the updates in his blog are a pedantic rewording of the existing ones.
So I call myself a software developer, but I've never worked on any group project that required builds at all. I've done java and c++ projects of my own, but any time there was more than just me, it was a web development environment where things were broken up to the script level and it was very rare that one person's work would break another's. Launching individually tested scripts was fast and asynchronous. It seems to me that is a superior model for web development. I know that the place I used to work switched over to java for a lot of stuff, and now they have build headaches and compatibility issues between the communication layers. I'm not sure what the advantage is there for web development. Isn't the whole idea of builds a pointless carryover from desktop software?
The idea that borland didn't know about the performance of loading a segment register is ridiculous. It's in the intel manual. Everybody I knew who cared about the performance of software had that manual handy. Then eventually the compiler just took care of it for you and we all stopped caring.
"Who is the Journal of Quantum Physics going to believe?" --Stephen Hawking
I imagine Joel thought you would be smart enough to apply the guidelines to your own situation. You've done that to a degree, but you're making the list needlessly inflexible.
- Do you use source control?
If your source control does not actually control the source effectively, it isn't source control. It's just a thing you put your source code into to make your life a living hell.
-Can you make a build in one step?/Do you make daily builds?
This one you have a small point on, but the obvious purpose here is to automate the process of publishing the latest version of the software to the team in order to avoid mistakes in the build/deployment process. Server side scripting, for example, isn't compiled or "built", but all the pieces do need to be in place and everyone needs access to the most current version. "Building" in this case would mean deploying the new code to the internal test server so all the developers know what the most current version of the web site is and can actually use it to verify that everything is working. This prevents you from making changes that are so large they are difficult to trace (if you have source control and nightly builds, you know exactly who screwed up and when). There should be an automated process to ensure all of the needed components are, in fact, there. It's just as critical for things that don't build as it is for things that do, it just looks different so you may not realize it.
"make this a 'deploy package' in one step" I hope you mean deploy to the test server in one step, and not publishing it out to the world. If not you totally missed the point of nightly builds (it's to catch major bugs before the code needing debugging becomes too large).
I could accept Garcia's "Do you have automated build or deployment procedures?" as a replacement for "Can you make a build in one step". The point is automation to avoid procedural mistakes, not compiling/deployment itself.
-Do you fix bugs before writing new code?
"Use unit testing here" - There is no need to be so specific. Unit testing is an effective modern tool, but it will not catch all bugs, particularly systemic bugs. The point is that you need to fix the major, money sucking bugs before you add new features.
-Do you have a spec?
"overrated" ??? The specification is the thing that describes what you are trying to do! How the hell can you write anything but hodgepodge software, especially with more than one developer, if you don't have overall design goals written down somewhere where they can be referenced? You can change the specification if your goals or needs change, but you should always have one! I suspect this is an especially serious flaw for open source projects that don't have strong leadership, given the distributed nature of open source.
-Do programmers have a quiet working conditions?
"ditch the phones" ?? What if your programmer works from home, how are you supposed to effectively communicate? Email isn't good enough for all situations, IM is better but still doesn't quite cut it, and frankly it encourages people to interrupt you more often. Quiet working conditions are what you need, not a simple lack of phones. I think if you were to expand this you should add "free from distractions" to the end of that. These days, it can be very quiet in your office yet still be extremely distracting with emails and IM notifications popping up all over the place.
Security is mostly a superstition... Avoiding danger is no safer in the long run than outright exposure. - Helen Keller
The "nobody takes this seriously part" requires relating the 1991 date of the quote with MS buying Fox Technologies in 1992, along with their product Visual FoxPro, which I am told was renamed to "Access" and released in 1992.
"Science flies us to the moon. Religion flies us into buildings." - Victor Stenger
Do your team work in good conditions...
The sentence fragment is "correct" in countries like England where English escapes them, and they don't understand that a team is a singular entity
In over 30 years of living in England, I never saw or heard this solecism committed by an English person.
It is a mistake — not a local variation — and is never correct.
There's something to be said for 'bad' use of DVCS in a private company. But here are the good usage patterns IHMO
1) checkin after every logically complete operation (for What The Fun Just Happened moments)
2) checkin every night (so if I'm sick tomorrow people can get to my work)
3) my-code-doesn't work, collaborate with someone down the hall or geographically remotely
4) I want to experiment with an alternate code path (but don't want to deal with the politics - remeber coders have egos)
4a) I want to experiment with an alternate code path, but don't want any risk to the trunk
4b) This code is too specialized, we need a much simplified version for this use-case (but need to maintain the original code path)
5) Let's say we suck at graphics, so we outsource to a 3rd party company. How the hell does this happen with central version control. F-no do we give them direct access. And if they email us a zip file of the final product, how do we keep in sync from there-on-out? Their or our changes will get over-written or go into non-versioned-hell. With DVCS, we can provide read-only access (possibly via emailed repository-clones). DVCS allows trivial re-integration. Security is maintained, reconciliation is trivial. History and auditing is somewhat maintained (you can obviously fake it). And most importantly we can switch a new NEW 3rd party contractor at any time, possibly even AT THE SAME TIME.
6) rebase (not DVCS specific). If I've got 10 branches (in svn or any place else), do I know for sure that after a while, the history has gotten too complex? In git at least, we can say, ok, these feature-branches should all be thrown away - lets 'rebase' to produce a pristine trunk and quite literally throw away all the branches by flattening them.
7) central 'owner'/'maintainer' of a given project. Make sure someone knows everything that's going on with the project by having them integrate or 'bless' an integration. With central repositories, this requires they do the merging. with DVCS, you do the merging as a 'candidate' and they either accept it as their own or not (e.g. fast-forward merging of your repository with theirs).
7a) as with linux, for larger projects development-teams, you can have lieutenants that perform step 7 for sub-sections of the larger project. For which each lieutenant will 'trust' each other's official repository and auto-fast-forward-merge. The singular project-manager can then choose for political reasons (because we are political in nature) to disagree with lieutenants decisions - as they are they primary responsible party (at least in closed-sourced commercial solutions). This works because lieutenants can continue with their private fork until they can form a mutiny - so ego's are maintained.
-Michael
Daily builds have never made much sense to me. If someone breaks a build, the fix is easy - revert their commit and tell them they screwed up.
If you aren't doing daily builds, how do you know when the build is broken? The idea is to catch it soon after the problem occurs, so it's easier to fix.
Having seen visual foxpro in action I can assure you that they did not just "rename it and release it as access," unless you count castration as a usual part of the renaming process.
I want my Cowboyneal