On Measuring the Effectiveness of a Search Engine?

younix asks: "I am working at a major telecom provider in the Netherlands and we are implementing a new system for knowledge management. Our system works quite good at the moment, but there is no way to actually measure the effectiveness of the search engine we are using. With effectiveness I mean how much effort do I have to make to find a particular solution that is present in the knowledgebase. Is it possible to express this in a number? Has anyone in the slashdot community done anythin similar before? I would really appreciate any pointers to work done by others on this subject." I think that the single most important aspect of a search engine is the mean time it takes to get useful results out of it. Now, about regarding measuring such a thing (or any other usable characteristic) in an objective manner?...Therein lies the problem.