Slashdot Mirror
A Rubric for IT Analysis
Aredridel writes "Zed A. Shaw has an insightful article on how analyses of software systems should be performed, and how they're often done wrong. It should be required reading for all IT journalists, and all readers of IT journals."
Even worse it works about as well as pricing soap at $1.95 instead of $2.00 to fool people into thinking it's cheaper.
I think $1.95 is cheaper, isn't it?
Better run it through the rubric...
8. Paper does not use the above terms correctly or calculates them incorrectly. Without the data you won't know the second part, but these 6 statistical concepts are very simple to calculate and get right.
I think it's broken.
When you have read that article, go and buy a copy of the 1954 classic How to Lie with Statistics by Darrell Huff, ISBN 0393310728.
The author of the rubric "carefully" lists examples of things that ought to be seen -- and then carefully extracts two graphs from a long analysis in order to "prove" his claim. Never mind that the things he argues one should look for would be embedded in the materials and metods or results section, not the conclusion or the paper summary. Never mind, either, that his objections are bogus (red versus black ink? Uh, wait -- if the winning system had been shown in red, it would have conveyed how burningly fast the system was.)
Oh, wait -- it's somewhich which shows that samba 3.0 is slower than w2k3. Never mind. This is slashdot, so the ditors have gotta troll for ad views.
Thats my exact question. When will someone make software that will analyze TFA and tell me if it is worth reading? Think about how much bandwidth could be saved with this app? Sort of like a stumble upon for news.
The usage of red and green determines the meaning, if the higher statistic was red, it wouldn't be the "bad" effect he is stating.
... and the graphs aren't necessarily misleading in the aspect of spacing, the graph seems to be trying to show the ratio of difference, not the difference amount. ... aside from what looks like a bad example of bad examples... there are some good points in the article...
The statement that green is good, red is bad, is not really true. Red is an attention getter, Green is an easy, inobtrusive color (relaxing, generally).
While it is easy enough to make the leap that 'red' is bad because red is often an 'alert' color, the reason red is an alert color is because it is an attention getter, not because it means bad.
Why else do you think so many people drive red sports cars? If red was bad, why wouldn't they drive green ones?
MoM++ - A Classic Expanded - [Master of Magic 1.5]
http://mompp.sourceforge.net/
I hate it when people lie with statistics. Even the BBC did it recently when they were trying to justify 1 million GBP on their new weather program. They said 7/10 people either liked the new system the same as the old one or preferred the new one. Perhaps they could also have said 9/10 liked the new system the same as the old one or preferred the old one? Who knows when you lump categories together like that without providing the raw data?
What he's stating seems rather obvious, but then again I might not be his target audience. One thing he seems to be missing is: who is paying for the test and is the one in whose favour the test turns out to be also the one who paid for it?
see a Text Widget
You have to be kidding me. The last three jobs I had, I got dinged if I did analysis of any sort. Most software developers skipped the analysis and design part, because Managers wanted them to start coding on the first day and not stop until it was ready for QA to look it over. I called it "Seat of your pants" programming. Often I had to fix problems in other developers' programs and they did not have proper documentation, source code comments, naming conventions, flow charts, or any sort of documentation to help me figure it out at all.
Requirements kept comming in, and they changed daily. Often what I started writing at 8am, was useless by 4:45PM when the requirements changed on-the-fly and adhoc and required me to program something else to replace it before I went home for the night. While I could have waited until the requirements were locked in, there was no such thing as that, any idea anyone had was instantly accepted by a manager and given to me to put into the program. Combo boxes became Listvues, then combo boxes again, then a text box, and then a Listvue again, and then a combo box. Database names for tables and columns were always changed, and of the thousands of SQL Queries in my programs that accessed them, they needed to be changed as well.
Management didn't think anything of it, and kept their "We cannot say no to anyone, no matter how insane the request" attutude.
Analysis, hooo haaaa! Yeah I wish! Corporate America apparently does not believe in it anymore.
Remember, Slashdot does not have a -1 disagree moderation, and no, troll, flamebait, and overrated are not substitutes.
Well, no idiot. When graphed properly, they look the same. Both tests show an absolutely compareable performance ratio. What does it matter that the faster machine runs both OSses faster? How does this skew anything? Is the concept of relative speed increases a new concept for the creator of the article?
A REAL loaded graph would surpress the y-axis or something to push the lower graph further down, or to skew the proportions.
Man, is today really shit article day on slashdot?
HI O WISE PRINCE. WHT TOOK U SO DAM LONG?
No, really. That's how it started, usually the title of a section, paragraph or similar.
Obviously the bit of red text contained something someone thought was important so eventually the word came to mean an important rule or important passage. These days it means an important set of rules.
http://www.dictionary.com/
htttp://www.m-w.com/
http://www.askoxford.com/?view=uk
Deleted
That pulling fragments of a text out of its context serves to confuse?
Nevertheless, Zed's enumeration can be extremely valuable in helping a discerning reader (who doesn't already know it all!) to critically interpret graphs in order to decide what s/he may conclude. For example, if system X appears to outperforms system Y, but the difference may be within the (unpresented) deviation, one should not accept the assertion that X is superior. Instead, one may conclude that X may be better than or comparable to Y.
Zed's article can help some of us tell the difference between lies and truth. That's a good thing. The unfortunate weakness of the article is that the example is not particularly compelling. It simply doesn't illustrate the most important pitfalls.
Actually, only having two values it is impossible to know how much difference there is. If you test 100 video cards made in the last 5 years, and their scores range from 500-11000, then these two cards are basicly the same speed. If your range is from 1800-1950 then they are radicly different. Numbers are meaningless without units, and units are meaningless if the user doesnt know what they are.