Slashdot Mirror
IDC Analyst Dan Kusnetzky Explains the Numbers
First, Dan Kusnetzky says,
I'm responsible for research on access-point device management software, operating environment software (embedded, client and server), clustering and high availability software, web-centric computing software, and some portions of storage software. You'll note that PCs, workstations, and servers were not listed anywhere. My team follows software, not hardware.Now, on to the questions...Other groups within IDC conduct extensive research into hardware (PCs, handheld devices, appliance servers, servers, storage systems, etc.). My team and I are part of IDC's software research group.
1) I just have to ask...
by Ded Bob
Might he have the stats on the BSD's. People from the BSD community are curious. :)
Kusnetzky:
BSD revenue or paid copies are tracked as part of IDC's research on Unix. In 1999, just over 8,000 paid copies of BSD server software were shipped. This would give BSD about 0.9% share of worldwide Unix server operating environment shipments.
The 2000 figures have not yet made it through IDC's publication process and so I will not pre-release them here. IDC has a policy that subscribers see research first.
2) Funding
by ritlane
In studies of market share (or studies in general), we often hear quotes about who funded them. This seems to somehow imply that those who funded the study had some influence in how the data was gathered/interpreted.
My question is: Do those who fund a study influence how the study turns out (i.e. Microsoft studies show higher MS market share). Or is it that these corporations only decide to fund groups who they know will most likely return results in their favor.
Kusnetzky:
Over 95% of the research which IDC produces has not been sponsored. Therefore, our research studies are completely independent. In the few cases where IDC performs specific research for a client, IDC has strict guidelines and review processes in place to ensure the objectivity of our research. IDC never does market share studies on a sponsored basis.
3) More breakdown needed
by BillyGoatThree
When People magazine does an issue devoted to "what's hot" in fashion, do they interview Jane Doe from Des Moines, Iowa? No.
So why are OS numbers reported with equal rating? Not all users are equally suited to *choose* an OS, therefore not all users *choices* are equally interesting. I'd really like to see a breakdown of OS by user-type (levels of education, field of degree if applicable, occupation, etc). Keep in mind this applies just as much to business. A technology company presumably put more informed thought into their choice of server than an art supply house or whatever.
Kusnetzky:
IDC conducts research which will help its subscribers make better business decisions. This often means surveying decision-makers who make the purchasing decision rather than the people who actually use the products. This also tends to mean that academically interesting, but not commercially viable, research may not be done at all.
The complete findings of IDC's research are supplied only to subscribers. So, the snippets of data that Slashdot community members may have seen in magazines only includes information IDC chose to make public. Quite often, IDC's intent is to interest companies in purchasing the entire study. 8^) Another point is that it is quite possible that the findings which are mentioned in the press were taken out of context or mentioned incorrectly.
4)Self-fulfilling analysis?
by dvk
Do you think that there exists a possibility (or can even provide examples of) self-fulfilling analysis, such as "analysis says X is losing market share=people get skeptical about X=X loses market share although it may not have done so otherwise"?
If it is possible or already happened, do analysts in general (and you in particular) find it a worrisom possibility, and if so, are there any attempts/ideas to deal with the issue?
Kusnetzky:
Having studied a great deal of physics in college, I understand the Heisenberg uncertainty principle. That theory indicates that it is impossible to observe a system without interacting with it in some way.
While I suspect that studies have been done to detail how analysts (and journalists for that matter) influence the creation of products and their success in the market, I have not read them and, thus, can not comment on what they may or may not say.
5)Data origins
by m2
Do you base your data mostly on marketing analysis or do you actually go a pay a consultor to scan machines on the net? If there are scans involved, how do you pick the IP blocks to be scanned and what's the uncertainty associated with such a method (and how is this uncertainty guessed)? If there are no scans involved, why not? If this is "maket analysis", can you defined that for me? Which factors are involved? And a different question: who's the target market for this kind of study? How much does such a thing cost?
Kusnetzky:
IDC conducts two types of research -- demand-side and supply-side. Demand-side (end-user) research is designed to shine light on who is buying and using information technology products, how these products are being used, and what plans decision-makers have for future purchases. A scan of the network is not involved in this process. Contacting people and working with them is the basis of IDC's demand-side research. Network scans, while interesting, are not part of IDC's current research methodology.
Another important point is that a network scan would only show systems which were up and available at the time of the scan. Any systems protected by a firewall wouldn't appear. This means that the study would not be able to shine light on what organizations were using inside of their firewalls.
The cost of demand-side studies is driven by how many respondents are needed, how many questions must be asked, and how many countries are included. A small study conducted in North America which asks a few questions can be conducted for tens of thousands of dollars. The data gathering phase of such studies might be completed in a few weeks. A large, multi-country study which must be segmented by company size, market type, etc. might cost many hundreds of thousands of dollars. The data gathering phase of this type of project might six or more months to complete.
Supply-side research shines light on revenues, shipments, technology trends, etc. IDC gathers information from the financial reports of public companies and from public statements made by executives of privately held companies. This data is then segmented into 102 different software markets, 9 different operating environment platforms, and 6 different geographical areas. The software markets, by the way, are defined by a software taxonomy which attempts to define a list of markets which is both exhaustive and mutually exclusive.
The company data segmented using this model is offered to representatives of each of the companies. After the company representatives have had a chance to review the models, an IDC analyst then contacts the representative to verify the segmentation.
Once IDC has completed this process, the company models are examined by internal and external reviewers. Any questions which arise during the review process are brought back to the company representatives and resolved to IDC's satisfaction.
IDC goes beyond determining the revenues to modeling revenue or paid shipments in some markets. In this case, the revenues produced by the first process are segmented using demand-side data which shows the average shipment value. The modeled shipment data is, as with revenue data, examined by both internal and external reviewers. As before, any questions are resolved before data is published.
IDC, by the way, is watching over 1,200 companies worldwide. Revenue data from these companies is collected on a quarterly basis, segmented into IDC's taxonomic software markets and then stored in the IDC software research group's software forecaster database for analysis.
6) *what* constitutes a Linux server?
by Anonymous Coward
At what point does a computer become a server? Many Linux desktops have ftp, telnet & http ports open, so do they count as severs too?
Kusnetzky:
IDC's software group is using a very simple definition. If it serves the needs of a single person, it is called a client. If it serves the needs of many people it is a server, regardless of the system configuration. IDC's enterprise server research team has detailed definitions of different server configurations, but I won't repeat them here.
Linux, as observed in a recent study (December 2000, N=1583 North American and Western European respondents), appears to have a different usage pattern now than seen in earlier studies. This recent survey showed that just over a third of Linux shipments were used as client operating environments, roughly a third of Linux shipments were used as server operating environments, and just under 30% of Linux shipments were used on a workstation which supported the workload of a single person and provided some service to a workgroup. We're calling that new category "Serverstations." For historical purposes, Serverstations are currently allocated across client and server operating environment shipment totals. IDC will be publishing data on each of these categories in the near future.
7) Polling questions
by cavemanf16
I have often wondered how biased polls are based on the questions asked, the demographics of the people polled, etc. When results about polls are made public, is it also possible to obtain information about how the poll was conducted in a simple, by request method? Now if the answer to that question is, no, how much can we rely on polls, since we have no way of verifying if the questions asked and the people interviewed were heavily biased to favor one outcome over another? (Such as in the recent large discrepancies of the 8% vs. 24% use of Linux as a server results that we've seen on Slashdot recently).
Kusnetzky:
I believe that I've already covered this area in previous answers. It is clear that IDC's server operating environments data is being compared to another research firm's server adoption data. This comparison really isn't valid for several reasons including the fact that supply-side research is being compared to demand-side research and the fact that software research is being compared to hardware research.
IDC's extensive hardware research shows that Linux only holds a small share of the market when one examines shipments of servers and then segments it by the operating system which was installed at the factory. IDC's software research is showing that Linux server software is being installed on both older and new systems and that the configurations being used as servers include PCs, workstations, appliance servers, and more traditional server configurations.
8) What about the so-called "third world"?
by Kareem Abdul-Lamarr
Do these analyses factor in the so-called third world? Most of these analyses are US-centric or some times do include the continent across the pond but what about Africa and Asia? Do these analyses *really* take inputs from these continents?
Kusnetzky:
At this point, IDC has offices in 43 countries. IDC's software research team is represented in North America, South America, Europe, Africa, and Asia. IDC's operating environments research included revenue and shipment data from over 140 different Linux distributions, and we have collaborative input from analysts in geographic areas around the world. I think that this means that the answer is yes, IDC is examining operating environment revenue and shipment data worldwide.
9) Who Keeps track of the Predictions
by dmccarty
I have a question on the area of predictions in general. For example, fellow IDC analyst Jill House has been severaly negative on Palm over the years, with regard to the Win CE operating system and devices. A sample quote from her in Feb. 2000 read, "If I was Palm, I would be beside myself with panic."
The issue is, that over the last 3 - 4 years she's been predicting the demise of Palm and the rise of Win CE, a claim that has never materialized. Who verifies the reliability of these predictions and keeps the analysts accountable. With the frequent sound bytes and one-liners that they give to the press, these analysts have significant influence over public perception of the issues. But how is policing done when the analysts don't analyze very well?
Kusnetzky:
I really can't speak to this issue directly. Jill is not part of the software research group.
I know that a prominent part of the software research group's annual forecast and analysis reports for each software market is reviewing last year's forecast, how close it came to this year's findings. If the findings differ from last year's forecast, the analyst(s) attempt to determine what changed and why. I'm told by subscribers that this analysis is often is very helpful.
10) Re:Question(ADDNUM)
by HeUnique
I just remembered an Idea that a friend of mine suggested:
What if IDC could work with the Linux distributions (RedHat, Mandrake, SuSE, TurboLinux, Debian) to add a small program which will run after the first internet connection has been succsessfull..
When this program runs - it will ask the user to "register" his copy of the distribution. If it has been purchased from one of the distributors then the user can add his serial number. Some other questions like will this distribution be used as a server, a workstation, or combination of the 2, or a development workstation. The survey SHOULD be anonymous (unless the user wants to give some details about himself)
By that way - the distributors can give the numbers back to IDC - and IDC can publish a report which will tell that the number of Linux installations - and that number is X. X is combined of Y free download version and Z purchased copies of Linux.
What do you think, Dan? (And what do Slashdot readers think about it?)
Kusnetzky:
I agree this would be interesting information. I don't believe that this would be a good business for IDC to be in. Our analysis is that many users of Linux don't want their organizations or their competitors knowing what they're running. They'd rather get kudos for a job well done than criticism for how they got the job done. Not only that, but some vendors that have added features to track activations have also faced a great deal of criticism. I doubt that Linux users would appreciate such tracking operations.
What about Antarctica? After all, we know what OS all those penguins are using.
Personally I didn't think that any of your answers were "fluffy," but I am glad that someone did :). Your answers to random posts in the forum have been nearly as enlightening as the original interview.
Thanks!
Yet at my office out of 100 people perhaps 15 of them have iPaqs, and another 30 have Palm.
The iPaq is extremely popular, and I'm starting to see them everywhere. Not as often as the Palm, but still...
Which is great for your supply-side surveys. But surely demand-side clients are far more interested in actual deployments rather than paid for shipments. When making a decision about an OS, I don't care how much revenue it's generated for a given company, but I am interested in how widely it's being used, and in what areas. For example, of our 25 or so servers running Linux, Solaris and OpenBSD, only 5 or 6 have paid-for copies of the OS, and I know that plenty of others are in the same situation. Going purely by paid for shipments, your figures are going to be wildly inaccurate.
"The invisible and the non-existent look very much alike." -- Delos B. McKown
Could you imagine a scientific journal that only contained abstracts? It would be laughed at. The methodology is absolutely essential to understanding the trustworthiness of the data. There needs to be full disclosure of these procedures before they release their snippets into the media.
Spreading unfounded information is as good as lying.
David E. Weekly
David E. Weekly
Code / Think / Teach / Learn
h4x0r for
Bzzzt! convincing math but it's wrong.
If 50% of the linux market is desktop then it has 10% of the total, not 0.3%. Your mistake came from assuming that 150:1 win32 client:server ratio even applied to linux(you said 1:1). In fact, this shows that windows servers have 0.6% of the server market, whereas Linux has 10% of the server market or 1500 times more servers.
Client side it's 10%(linux) vs 39.4%(windows). So windows has 4 times the market share. I do consider 10% cracking the market.
I believe your largest mistake was lumping both segments together and comparing them.
NB. True number are closer to
Win32 - 90%
Mac - 5%
Linux - 5%
for client
You don't exist. Go away. --SysVinit Halt
IDC IS doing an analysis based on actual usage surveys, not just market sales, so they should be (assuming their methodology is sound) producing numbers that reflect how Linux is actually used.
Er, I should have said "doing both an analysis based on actual usage surveys, and one on market sales, so they should be (assuming their methodology is sound) producing numbers that reflect how Linux is actually used, as well as how it's being sold, assuming you look at the right sets of their numbers."
-
The important bits that I get from this are:
IDC IS doing an analysis based on actual usage surveys, not just market sales, so they should be (assuming their methodology is sound) producing numbers that reflect how Linux is actually used.
Dan knows his mouse from a hole in the ground.
Dan knows the difference between Linux, BSD, and Unix.
IDC seperates Linux from Unix and BSD because they think that division matters to their customers. They don't seperate BSD from Unix because they think that division doesn't matter to their customers.
IDC gives a shit whether it's predictions turn out correct, and tries to improve their accuracy by examining their results.
In all, it sounds like Dan's a geek, knows what Linux is, and is concerned first and foremost with producing accurate, truthful, scientifically-gathered information.
Now watch me get modded down for not following the party line.
-
It sounds like there's a lot of stuff Mr. Kuznetsky can't bring up because that information is reserved to subscribers of their research. Maybe /. should ask a subscriber to share what they get out of the data and how they use it? I imagine they're not allowed to just release the whole report, but some info on the general conclusions that they reached from it and how those differ from the results of the public portions of the research might be interesting.
Caution: contents may be quarrelsome and meticulous!
Your right to not believe: Americans United for Separation of Church and
It would be probably a bad idea to send registering e-mail, even if anonymous. However,
it would not be so bad to point them to the
Linux Counter
You have the technical point of view. They have the business point of view. From a technical point of view, their metrics are wrong. But from a business point of view, they are not that bad. If they want to estimate the market share in number of copies sold, not in number of copies used then they are right.
Again, you have a technical point of view. The best business decision may be very different from the best technical decision. Often, a manager will have to choose a solution that is not optimal from a technical point of view, but is good for the company because it minimizes the risks or because it is considered better by the shareholders or board of directors or whoever decides the future of the company and the future of this particular manager. Note that it does not mean that the solution is better, only that the shareholders perceive it as being better.
-Raphaël
Dude, I hate to tell you this, but where I work if you don't do upgrades without testing them thouroughly on a control machine, you'll probably find yourself out of a job. It doesn't matter if the platform is windows, linux, mainframe, unix, or the doorlock on the bathroom. If it isn't tested before going into production, you are going to find trouble.
True true. I confess to a bit of hyperbole, but in our experience, the "tests" we do on control machines for linux/unix upgrades are usually successful, as opposed to what happens when you run Service Pack X on your perfectly stable Exchange machine.
The "tests" we do typically end up just being technicalities on the linux/unix boxen.. plus the fact that we can COMPLETELY restore the state of unix box from a tar file (or tape backup) with 100% reliability means the IT dept. sleeps much easier when they tweak a unix box.
I know 2K/NT backup solutions are SUPPOSED to do this, but getting back a PERFECTLY tweaked Exchange server from tape is not exactly painless.
Exchange is a very good product
There you go again, confusing "good" with "really unbelievably crappy".
Exchange is NOT a very good product. It happens to have a TON of *features* that PHBs really like, but nobody in their right mind would consider it stable, let alone good.
We have both sendmail AND exchange running. They both serve their purpose, and they are both stable *in their current state*.
The difference is that I *can* reconfigure the sendmail servers. I *can* perform upgrades. I *can* back it all off, and restore them, and expect them to come up just as they were. I *can* add different virus scanners. I *can* run other services on them.
The Exchange servers are black boxes. You don't run upgrades (of Exchange OR Win2k/NT) without testing them thouroughly on a control machine. You don't run other services on them. You don't expect a restore of the server to come up correctly.
Sure, they are stable, as long as you don't so much as look at them sideways.
It was announced this week that iPAQ will overtake the Palm very shortly
iPaq may eclipse palm in revenue (after all, it costs a lot more), not unit shipments, and is not even close in terms of installed base. Of course it may be revenue that counts to you - it depends on what you're looking for.
IMO Developer mindshare shouldn't necessarily swing to iPaq as fast as revenue has, since for a developer, Palm+Handspring looks just like one big market, cost of entry is lower, and the tools are good, and cheap (or free).
Some of his answers seem to say very little, using lots of words, and thus makes the interview a little less interesting.
Well, I'd better get used to this, as the election in Norway is coming in the autumn.
Lotus Notes and Novell Groupwise are the only two products with similar funcionality that I've been able to find. I've been stuck with Exchange for about a year and a half now, and I absolutely agree with nyet -- its stable, as long as you don't touch it. Try upgrading/changing/restoring anything, and you're screwed. And Exchange provides so little administrative feedback that its damn near impossible to fix the problems that come up after you touch it.
Notes seems pretty decent. The Domino mail server runs on Solaris (not sure about linux) and seems pretty straightforward. I've worked with an older version of it a while ago, and it was definitely a lot more predictable than Exchange. I'd switch to it in a heartbeat -- Just have to get management approval first.
If anyone can come up with any alternatives (free?) to Exchange/Notes/Groupwise, I'm sure I'm not the only one who'd be interested in hearing about them. Anything that does what your typical "groupware" does -- email, scheduling, and contact management I think are the biggies.
- Sean
The answer is deceptively simple:
Let's say Windows has about twice the market share of Linux in the server market (40% to 20%). Let's even assume that the client:server ratio on Linux is 1:1. Both of these are on the generous side to Linux (I believe).
Now, Windows clients outnumber Windows servers by about 150:1. This means Windows clients outnumber Linux clients by 300:1.
By anyone's books, 0.3% of the market is a LONG way from cracking it. Even if my numbers are a factor of 5 off (1.5%), Linux is still a long way away from cracking the market.
Fear: When you see B8 00 4C CD 21 and know what it means
No, you are wrong. I've assumed that the ratio of Linux servers to Linux desktops is 1:1, not 150:1.
If 50% of the Linux market is desktop and 50% is server then that means there are the same number of Linux servers as desktops.
If there are twice as many Windows servers as Linux servers, and 150 times as many Windows desktops as Windows servers, then it follows (from simple algebra that somehow eluded you) that there are 300 times as many Windows desktops as Linux desktops.
Or, mathematically:
Assume Ls = Ld.
Assume Ls * 2 = Ws.
Assume Ws * 150 = Wd.
Then Ld * 2 * 150 = Wd.
or Ld * 300 = Wd.
I have no idea where you got your 10% from because I was not lumping both market segments together in any way. Perhaps you need to reread my post??
(I know what the true numbers are - I'm just going to show how stats can tell all sorts of wonderful lies).
Fear: When you see B8 00 4C CD 21 and know what it means
Damn I wish I could mod this up. Computer science is black and white - things work, or they don't. Social science is grey - it depends on how you approach it. To assess the validity of the results, all you can do is evaluate the methodology chosen while considering the constraints it was done under. It's easy to say "it would have been better to portscan every computer on the Internet three times a day over a period three months", but even ignoring the cost factors in such an approach, servers running Linux may not even be connected to the Internet. His methodology sounds ok to me. There may have been other better ways to do it, but it doesn't look like there were any significant flaws in his approach.
People on /. have asked sampling questions - our normal explanations on how and why we structured our sampling go for 5+ pages of highly technical documentation (that relies on theory most people here probably wouldn't understand or be interested in, such as the benefits of 5 point likert scales vs 7 point, scaling and weighting difficulties, and techniques for accounting for measurement error). You can see why a complete answer to these questions won't be presented here. Our methodology section in our reports will often go for 15+ pages.
As an economist / statistician and part-time programmer / sys-admin, I like the comments about "all things being held equal". The reason we do that is to define the problem in such a way that we can answer it. Interacting with the real-world is a lot messier than coding a program. Stats and research is not easy. It's hard work coming up with something that's defensible and will give you quality outputs.
This is your life, and it's ending one minute at a time.
Linux has traditional been a techies operating system. That being the case the IT guys (who use gnu/linux at home) would just install linux without going through the standard excutive process (which would have probably not allowed linux on the network). Well linux gained it's reputation by siletly doing the work, I THINK that is what he was talking about.
Huh?
I think you misspelt "But we've corrected the error, and you can buy the new survey for $1,999.95!"
Dude, I hate to tell you this, but where I work if you don't do upgrades without testing them thouroughly on a control machine, you'll probably find yourself out of a job. It doesn't matter if the platform is windows, linux, mainframe, unix, or the doorlock on the bathroom. If it isn't tested before going into production, you are going to find trouble.
I am as big a linux zealot as they come. All of my home machines are linux-only machines, and I quickly repartition any work hard disk so that I can have a copy of linux on it. I publically hate microsoft. BUT I don't think that it's good advocacy to start telling people that our software can be used without extensive testing. Sure it's a worse plan for M$ software, but it's a bad plan for any software.
I do understand your point that Exchange is crappy software, especially when compared to some of the opensource/free alternatives. But that doesn't justify ignoring good change control and change management policy.
$.02
--
Key to financial independence: Spend less than you earn. Save and invest the difference. Do it for a long time.
Exchange is *NOT* a good product.
At the consulting firm I used to work for, I had quite a bit of exchange exposure. IMNSHO, any mail server that crashes the OS when a poorly formatted message arrives is crap.
This was not a one time incident. It happened at multiple clients of ours and was repeatable (by sending the suspect message again).
Microsoft's response was "That's weird. We have no idea why that's happening. We'll take a look at it"
We never heard back from them or noticed the problem in their KB.
The answer is pretty simple. If someone paid for the software as a packaged product or paid to download the software, it is paid software. If the person didn't pay for the software, it isn't paid software.
IDC screens the lists of potential survey respondents very carefully. Every reasonable effort is made to make sure that the surveys or focus groups include the appropriate people in the study. Since the studies usually include hundreds of respondents and in some cases thousands of respondents, the impact of a single person lying is not likely to be very large.
You left one out that is equally important.
"Not everything that counts can be counted and not everything that can be counted counts." Albert Einstein
While conducting interviews for surveys or meeting with clients at their offices, I often ask "How much Linux do you have installed?" or some similar question. Executives often tell me that they have absolutely no Linux running. Later when talking with folks in various business offices, I ask the same question. At times, I'm told by these people that there are quite a few systems running Linux (or BSD) which are supporting Web-based applications, file/print services for a workgroup, or messaging. Software development is another popular use.
When I ask how it is possible for so much Linux (or BSD) to be installed without executives being aware of it, I'm often told something along the lines that within the organization it is much better to get congratulated for doing a great job and solving a problem than to listen to criticism about what tools were chosen to solve the problem.
I believe that a portion of Linux (and BSD) usage is hidden. This means that traditional surveys of IT decision-makers won't find it. Interviews of computer users in business units might turn up clues to its presence.
No. That would not be an accurate rendition of my comment.
IDC conducts many types of research. A great deal of this research is published use of subscribers of one or more of IDC's 300+ "continous information service" subscribers. It is correct to point out that the subscription fees pay for the research. No subscriber, however, controls the topics researched, the methodology, or the findings beyond the choice to subscribe or not to subscribe.
A small portion of the findings may (or may not) be released in a press release, in a media interview, or in a presentation at an event.
The press releases, interviews, presentations, etc. all serve to publicize IDC's publications and services. In some cases, this encourages individuals or companies to purchase the document or become a subscriber.
Yes, you are right. The resulting revenue allows the company to stay in business and continue conducting research for people.
I'm sorry that I appeared to be providing "fluffy" answers to your questions. Let me try a more direct approach.
IDC's software research group has an editorial review board. All forecasts are reviewed before they are published. Since IDC's name and reputation are on the line, every attempt is made to make sure that the assumptions are reasonable and that they have been applied in a supportable way. They still, by the way, may prove to be wrong in interesting and highly public ways.
IDC analysts review previous year's forecasts each year when the report is published and discuss whether or not they were accurate. If they were not, the analyst tries to point out how market trends differed from those projected in the previous report. This is an attempt to help the subscriber gain some value from what the analyst learned from thet time the first forecast was published.
The bottom line, however, is the review provided by subscribers. If they renew their subscription, they found value in the program. If they don't, then they didn't.
The questions we (/.) asked were overwhelmingly methodology questions. We got methodology answers. So don't complain that his responses didn't tell you anything.
/. Readers are (like myself) accustomed to black and white answers to our questions. You _know_ the problem domain and you have complete control of the solution set. This is not the case in other professions, especially statistics. There are commonly repeated phrase in Economics & Econometrics: "all things held equal" and "it depends". They are commonly repeated for a reason. Inevitably, when you are performing any type of statistical analysis, you have to make judgement calls on how you gather data, knowing full well that what your results will be affected by them.
As a former statistician, let me tell you that performing this type of analysis is not cookie-cutter stuff.
The methodology for gathering information, and the feedback mechanism he describes, seem prudent to me. I see little fault with his analysis, given the constraints on his access to the target data set. I think he did a good job explaining it to a group of non-statisticians. Nice interview.
~Religion is O.K., as long as it gets you laid.
IDC is paid to provide research to institutional investors (for the most part). They don't care what the best technology is, only if it will make them a bunch of money if they buy the stock.
I would like to see someone perform research that was of more use to a techie like myself, but I think we get that job ourselves. Since we aren't willing to pay millions for that research, they don't provide it. 8-)
Well, I posted the above question you replied to, so I guess I'll take a crack at your reply. First let me say that I think you're taking the story you're quoting (available here) out of context. The report states that Compaq may soon start shipping more iPaqs than Palm does Palm devices, not taking into account that Palm already has a lot of devices on the shelves, the closer-to-reality average device cost numbers of Palm, return rates of CE devices, etc. And that doesn't mean that Palm's install base of 10,000,000+ devices is going to evaporate if more iPaqs are sold, a la IE vs. Netscape.
Secondly, I don't think that she was right all along, although if Palm doesn't solve some short-term problems they will have made her right through no insight of her own. I do think that her predictions had some influence on the people that had buying power and very little technical knowledge.
When she first started posting pro-CE reports, anyone worth their salt would have laughed out loud. In 98 or 99, CE was absolutely horrible and Palm was nothing short of golden. Palm devices actually synced better with Windows than Pocket PC devices (back then they were called Palm-size PC's) did at the time. So we--some of the old school Palm developers--read her forecasts and had a good chuckle.
So to bring this post back around on-topic, now the situation is one where the IDC analyst may end up becoming right, even though for the wrong reasons. She'll probably proclaim, "I was right all along!" but there is no way she could have known that Palm would make several key missteps, more CE device makers would not jump ship (as they were when she was predicting) and that the corporate world would blindly buy inferior devices on the whim of an uninformed purchaser. Which brings this question around full-cirle: there doesn't seem to be a mechanism for analysts to be shown wrong, or say "oops, I screwed up--sorry" or be held accountable for poor predictions. And until there are, it's not prudent to make important decisions based on their reports.
Have fun: Join D.N.A. (National Dyslexics Association)
So I was pretty surprised to read his interview. After thinking we were going to get some "real" answers, it was disheartening to read his corporate, fluffy answers to the questions. My question (question #9) was glossed over with a "that's not in my group" answer, so the question of who keep the analysts honest is still a very open one. Someobody other than yourself has to keep you honest. Accountants have audits, programmers have code reviews and people in construction have inspectors. Reviewing your own data from last year isn't good enough without either a) an oversight organization to make sure that your mistakes won't happen again or b) a watchdog agency to make sure that your mistakes won't happen again. Apparently, IDC has neither.
Have fun: Join D.N.A. (National Dyslexics Association)
Help save the critically endangered Blue Iguana
Announced by who? And overtake how? For raw sales I might believe it, I'm guessing the Palm market is pretty saturated by now, so that Handspring/Palm sales are not exactly brisk.
But I have yet to notice one person using an iPaq where I work (or anywhere), whereas out of 100 people in my office, I'd guess there are at least 15 Palm users.
I do not have a signature
I agree that polling zealots will get you less useful results - regardless of which product they're zealous for.
sulli
RTFJ.
There will be a $4 billion market in bullshit analysis by 2006.
sulli
RTFJ.
IDC conducts research which will help its subscribers make better business decisions. This often means surveying decision-makers who make the purchasing decision rather than the people who actually use the products. This also tends to mean that academically interesting, but not commercially viable, research may not be done at all.
To summarize: they make recommendations to the purchasers based on comments received from the purchasers, without regard to what the users think!
Now I know why I have to use Exchange.
sulli
RTFJ.
Interesting to note that the operating system which is "a long way from cracking the desktop market" is used as a client workstation by two-thirds of the people who use it.
It was announced this week that iPAQ will overtake the Palm very shortly (assuming you could Palm, inc seperate from Handspring, etc). Maybe she was right all along? Or did her predictions have some influence? Personally, I think the problem is that palms were great in 1997 but they haven't really gotten significantly better since then. But it is interesting to think about.
Uninnovate - Only the finest in engineering.
-- .sig are belong to us!
All your
A feeling of having made the same mistake before: Deja Foobar
Yeah, that was the thing that jumped out from the IDC methodology and said hi for me.
The problem is this - let's look at Rogers Cable, up in Canada, who are putting out a set top box with a version of Linux. It could be BSD too, for all it matters.
The main problem is they may have only five paid copies and hundreds of thousands of boxes running Linux or BSD that they did not pay for a copy of the OS.
It's free software, open source. So long as the industry uses the wrong metrics, asks the wrong questions, they'll continue to underestimate the number of Open Source OS boxen out there.
Let's look at your average Fortune 500 company using Linux. They probably buy 10 licensed copies and a support agreement, but since they standardize on only three PC versions (laptop, desktop, workstation), that's all they need. And they have tens of thousands of PCs running Linux or BSD. They may have only one-third of their PCs running Windows (any flavor) and two-thirds of their PCs running Linux or BSD, but IDC and other industry groups will report that Linux and BSD have less than one percent of the OS share, because they measure only paid OS copies.
You must measure something correctly to be able to make correct decisions. In modern accounting, pollution has no cost, only trying to stop pollution has a cost. So a CEO will try to increase pollution, not decrease it.
--- Will in Seattle - What are you doing to fight the War?
Okay, our company is moving to Windows 2000 servers. Is it because...
1. The consulting company brought in by our PHSM uses IDC demand-side research (for PURCHASERS by PURCHASERS)?
2. We (the IT group) and our users are getting tired of our Novell file servers and *nix web servers never crashing?
3. Microsoft invested $600 million in our company a year ago when we really, really needed it?
And the bonus question: is it still called demand-side when it's being shoved down our throats?
(BTW, Deloitte Consulting believes in Microsoft software and IT outsourcing.)
"This also tends to mean that academically interesting, but not commercially viable, research may not be done at all."
Ummmm....what? Let's say I'm a purchaser who needs to know what the best file and print server is. If I didn't have IDC, who would I ask? A purchaser at another company or a tech who has previously setup and maintained a file and print server?
In other words, tell me the opinions of people who have valid opinions, not J. Random Loser. If that ain't "commercially viable" I don't know what is.
--
324006