Joel Test Updated

An anonymous reader writes "In 2000, Joel Spolsky wrote the Joel Test, an excellent and simple way to evaluate a software company. While the test is still used, it's getting outdated, as many companies are moving to web technologies, and new development tools exist. In his blog, Marc Garcia wrote about what could be an update to Joel Test."

Comments

Old system is fine.

[Anonymous Coward]

I think the original Joel questions still work fine.
Who needs a distributed source control system if everyone on my team works in the same office.
Also, I don't want end customers submitting directly into my bug tracker. I'm OK with them having a web based way to submit problems, but then QA should verify the defect and translate customer speak into something that makes sense. Then the defect can be entered into bug tracker with a good set of steps to reproduce and given a proper severity. To a customer, everything is critical.

Re:Old system is fine.

[Bigjeff5]

His "updates" just sound like re-statements of the original questions for a particular situation (i.e. less applicable to all modern software companies than the original).

Joel Spolsky assumed you would be intelligent enough to adapt the list to your specific situation.

For example, what good is "source control" if it doesn't effectively control the source code? There is no need to specifically mention distributed source control; if your source control is doing its job then you have good source control. If it isn't doing its job because you've got developers all over the country, then you need a distributed source control. It's built in to the question.

Customers directly reporting to a bug database, as others have mentioned, can be disasterous. However, Joel's flagship software is bug tracking software, and from what I've heard it's very, very good. His bug tracking uses a combination of silent reports from the software, direct customer input, and support service input. Specifically stating bug tracking must be entered directly by the customer is stupid and inflexible, and does not apply to all situations. The point of the software test is to apply to all situations.

It goes on, most of them are similar, but this one is egregious:

Do you have automated build or deployment procedures?

What the hell does he think "Can you make a build in one step?" means?! That's automated build and/or deployment!

Also:

Do you fix bugs before implementing new features?

Uh... frankly, that sounds worse than "Do you fix bugs before writing new code?"

Do you have a roadmap, and you don't make important changes to the short term priorities?

A) That's not the programmers job nor responsibility, and B) "Do you have an up-to-date schedule?" Hello?

Seriously, what does this guy think all these words mean? Just because they were written 10 years ago doesn't mean the meanings of the words changed. Apply them to your situation, they fit just fine.

Last but not least:

Do your team work in good conditions (quiet environment, flexible schedule, freedom to choose development software, fair paycheck...)

That's a dream of every office worker in America, and if you refuse to work at companies that don't have an office culture like that, well, you won't be working much unless you are seriously hot shit.

A serious question

[Scareduck]

Has Joel Spolsky done anything that's worth a damn? I am a long-time user of Fogbugz, and can attest to that product's general lack of attention to detail in its design. It's almost as if it were written by people who hated each other and didn't want to communicate. Several of my co-workers attended a release conference with him present, and the uniform reaction I got back from them was that he had moved on from Fogbugz, wasn't interested in the problems we had found in its implementation, and was fascinated by some other product.

But getting back to this, Garcia's list appears to be fairly sound. I have some comments on two of his modified questions:

Do you use a distributed source control system? Why should I care about distributed source code control in a monolithic commercial development environment? I can see its value in a distributed open-source project, but I really don't understand the necessity otherwise.

Do you fix bugs before implementing new features? All bugs? Some bugs? This tells me nothing about prioritization. Sometimes you need to do both at once. Sometimes it's not worth it to fix a bug if the circumstance is rare enough.

Re:A serious question

[rudy_wayne]

Has Joel Spolsky done anything that's worth a damn?

Not really, but he seems to be an expert bullshitter who throws around the fact that he once worked at Microsoft every chance he gets. As for what he's done, let's see:

City Desk - some sort of program for creating and managing websites. Little or no mention of it on his website anymore and the City Desk forum is long gone from the website.

Fogbugz - The Forgbugz forum is also gone from his website. Here's a blurb from Joel this past September: "Thanks to the hard work of the Fog Creek team, including ten great summer interns, we have just released amazing new upgrades to FogBugz " Thank god for free labor.

Co-Pilot - a remote access program that was written entirely by summer interns. Really. Thank god for free labor.

Stack Overflow - Not an actual application but simply a website where people can ask questions.. This doesn't stop Joel from proclaiming "I’m the CEO of Stack Overflow".

From what I can see on his website, his main business now is ads for programmer jobs.

Re:Total failure

[Frosty Piss]

You won't get anything usable from an end user without some severe filtering.

Indeed. This attitude is one of the biggest failures in Open Source software development, and why companies like Microsoft flourish. Microsoft software has many issues, but listening to the End User is not one of MS's problems. On the other hand, pipe up to just about any Open Source project about End User issues, and "STFU and submit a patch" is about the nicest thing you'll hear. Even as responsive as Apache and Mozilla are, End Users still feel this wrath. It a fact, most companies that want their products to flourish and make money are responsive to the people that actually use them. The GIMP Team? Not so much.

Re:Who is this guy?

[rokkaku]

Do you sell your car every year to buy a new one? There's a cost to converting, so you have to make an engineering decision about making the conversion. The automated tools don't always work with old and complex repositories.

Simple test for when a company is too big.

[EWAdams]

Put a $10 bill, or the local equivalent, in an envelope on the company bulletin board. On the outside, write, "I need change for $10 please" without any indication of who you are. Do this every six months or so. If you ever come back and find that the envelope is empty, your company is too big. You have hired a thief who does not care about his or her fellow employees.

Re:Total failure

[Kjella]

Example: Allow users direct access to a bug database? It's hard enough to train testers to give you good bug reports. You won't get anything usable from an end user without some severe filtering.

The question is whether you are better off leaving your users to work out their bug corresponence via mailing lists or email and only let a blessed few enter bug reports, or is it better to have the full case history going all the way back to what the customer actually reported along with any logs or screenshots. Or if you just drop it only the floor saying "LALALALA our software is perfect, all problems are PEBCAK problems."

Personally I'm a big fan of wine appdb's "*** This bug has been confirmed by popular vote. ***". If enough people are experiencing a problem, you have a problem whether you get anything useful from the logs or not. Don't forget that crappy bug reports and crappy logging often go hand in hand, when the application just goes boom without giving any useful information about why the developer is just as much at fault for making it impossible to debug.