Slashdot Mirror
How Can You Measure a Wiki's Worth?
moldar writes "I have been involved in a major project to migrate documentation from multiple sources to a wiki (Media Wiki, if you must know). Now, the PHBs are all asking questions about how organized the data is. I've already googled for various wiki metrics ideas, but mostly they focus on page counts, average page sizes, rates of edits, etc. Can anybody suggest better ways of measuring the quality of a wiki? Things like uncategorized pages, articles that are too small, etc? Any help or fresh ideas would be appreciated."
See how many phone calls/emails you get complaining.
PHBs will always insist on metrics for things that can't properly be quantified.
What I would do is suggest that he task a team from another department to run a survey on satisfaction amongst wiki users. Has the wiki helped them? How often do they use it? What would they like to see changed.
This should distract the PHB for a while, while diverting your responsibility to come up with a metric.
http://blindscribblings.com - Tasty pop-culture in conceptual fashion.
It depends a lot on what your wiki is for, but usage pattern statistics might be a good metric. For example, repeat visitors are people who are getting some value out of the wiki. Trampoline pages (pages people hit and then bounce elsewhere from a link in the page) might be another good thing to track. As well as abort-and-retry pages, where instead of following a link the user goes to the search box and tries something else.
You get the idea.
If the wiki is valuable to the people who are using it, you should be able to tell this from the logs--even if no one ever mentioned wikipedia in public, they could tell how much we care by noting that we keep coming back.
--MarkusQ
Many Moons (I apologize for the hosting company in advance, the site is not mine) is a childrens' story everyone in IT, Sales, and Marketing should familiarize themselves with.
The truth of the matter is: You're asking the wrong people. You should be asking the suits what they would imagine in a report and what kinds of numbers they're looking for. If they're general and obtuse, tell them straight up they'll get a general report, or just make up numbers to keep them happy. Create a widget they can access from their desk that shows them numbers generated from the database that hardly matter. Does it matter what you deliver if they don't know what they want?
They're trying to imagine some great power-point presentation you'll be showing with pie charts and red/green/yellow/blue graphs popping out and wowing them? Make it so. They're trying to imagine a spreadsheet? Easier and more concise. They just want a status report at the end of the day on what percentage of the documentation has been migrated? You can probably get away with a few page count written on a napkin.
Remember the Alice's conversation with the Cheshire cat:
"Would you tell me, please, which way I ought to go from here?"
"That depends a good deal on where you want to get to," said the Cat.
"I don't much care where -" said Alice.
"Then it doesn't matter which way you go," said the Cat.
I am the richest astronaut ever to win the superbowl.
The problem is that he's trying to measure the wrong thing. He's trying to get metrics from the wiki itself. He should be getting them from the users.
So the simplest solution is to do a user survey about what's good and what needs improvement.
Look, the pointy haired boss doesn't really want or expect a numerical metric of organization. Immagine asking him if he has a function in Word that will tell him how organized an outline or document is. He doesn't, and if he never needed it for Word, he doesn't need it for a webpage.
Possibility 1: he wants to kill the project, and came up with this as a way to find an excuse. The best response to this is to get your resume out there on monster right now, and walk into his office tomorrow morning and say "do you want to kill the wiki, and is this metric a way to get an excuse ?" . Let that conversation go where it will.
Possibility 2: he read in USA Today that anyone can add stuff to wiki's and they are chaotic, so he's worried. He's never actually looked at any wiki including this one in his life. This is the most likely possibility. If you still have a job after asking the question in Possibility 2, go for this, or maybe start with this.
Here, you want to talk to him for about 5 minutes using the following words as much as possible, without actually lying: "review", "control", "process" and "catagorize". If you want to can reveiw everything he has ever said and come up with your own set of buzzwords, based on the words he himself likes to use, but these will do. Then you say things like "We have a PROCESS to CONTROL the QUALITY of employee submissions. I occasionally REVIEW the most visited pages in the wiki, and make sure they are ACCURATE and properly CATAGORIZED. I also REVIEW the least visited pages, and if they are not visited because they are improperly CATAGORIZED and linked, I fix that." Actually talk like that, and kind of shout the buzzwords at him. He's either stupid enough that he needs it, or smart enough to figure out what's going on and move on to something important.
All the geeks in this discussion who started actually talking about measuring connectedness of graphs and crap are totally off in the weeds.