Slashdot Mirror
Truth in Advertising?
PerformanceEng wonders: "I work as an engineer for a large technology company in the U.S., and have been privy to what I find a interesting practice. It's well known that marketing data sheets often paint the best picture of a product while leaving the devil in the details. I've come to expect this, and when I am evaluating technology, I always have a skeptic's eye for claims made by the sales and marketing folks.
However, I've also witnessed our product go into test labs (usually for the purposes of running a series of tests for a 'bake off' in a trade publication). Not uncommon is the attempt to 'tune' the configuration of the device under test to perform in the best light (not unlike tuning your car to pass emissions tests). I have seen it go as far as exploiting weaknesses in the test that, if the test operator discovered, would be considered bad faith. To the other engineers: Are you aware of this kind of practice at your company? To the IT professionals: How much faith do you put in these sorts of publications and their 'bake offs'? To everyone: When does spin doctoring cross the line and become false advertising?"
It wasn't an uncommon practice for video card makers to tweak their drivers to perform better on benchmarks.
I've also witnessed our product go into test labs (usually for the purposes of running a series of tests for a 'bake off' in a trade publication). Not uncommon is the attempt to 'tune' the configuration of the device under test to perform in the best light (not unlike tuning your car to pass emissions tests). I have seen it go as far as exploiting weaknesses in the test that, if the test operator discovered, would be considered bad faith.
:-) Seriously though, this has been the whole problem with "benchmarks" like SPEC and others that ultimately results in pissing matches between manufacturers saying "my product is faster than yours" which for 99% of the users out there means nothing. In fact, even for that 1% of us where it does make a difference, specific optimizations to ones code or algorithms typically will get you more performance. So, what it really comes down to is how productive is the product + environment + task that you are assigning to the platform.
Oh, you work for Intel then.
To answer your question of false advertising, I would say keep to the standard that most of us scientists do: Specifically, peer review and ensure that your results can be duplicated by said peers. If results cannot be duplicated, then it is false advertising.
Visit Jonesblog and say hello.
What about all those people on Slashdot who claim First Post but fail it? Isn't that false advertising?
Is that a rhetorical question?
If you're naive enough to trust companies, rather than getting multiple reviews of products or services from unaffiliated sources... then you need to get your head checked.
I work in developing web applications. When choosing server technologies I have learned to conduct my own bake off using the application that will be deployed. Each application is unique. Comparing your custom app to a published bake off is usually an apples to oranges comparison.
The product brochure may lie or hide facts, but the product's technical details book (like the manual for Kyocera VMSE47 Phone) HAS to tell details and truth.
I always make it a practice to read the technical manual of any product i buy over the web. if the company can't provide the manual, then it isn't worth buying.
"Doing what i can, with what i have." ~ Burt Gummer
"To everyone: When does spin doctoring cross the line and become false advertising?"
Been reviewing the previous election, have you?
Consumer Reports is a such a respected publication because they have strict standards for the products they test. They don't accept items from the product makers, they go out into the marketplace and buy their test subjects using cash whenever possible. (Up until a while ago they even bought cars with cash, until they realized that car dealers began recognizing them as the only people who paid cash for cars, and the IRS requirement of reporting large cash transactions got in their way too.) As a result, their tests are immune to any tweaking...
It'd be nice if the tech publications could afford to do this, because at times they start to resemble the video game websites set up by kids who do it only to get prerelease copies of games for free under the guise of reviewing them. Such kids always have to write glowing reviews of everything they get because as soon as they post a negative review their stream of free stuff grinds to a halt.
Bottom line is that there's a foolproof way of preventing tampering in any review, but it costs money. Any review that involves accepting free stuff compromises the integrity from the start.
on his Hackers In The Free Market entry on his blog. I think it's pretty insightful, but he tends to be a little too libertarian for my tastes to. YMMV.
...what can one do if your company is dishonest? This has been happening a lot more lately in my company - billing is highly overflated (seven hours billed for two and a half done), things promised to clients that there is no intention of delivering, etc.
What can be done to show those in charge the error of their ways? As long as the money keeps flowing, it seems the problem will just get worse.
The marketting people are a pack of liars. In their work and in their life. They have been spouting bs for so long that it has permeated their very being. I don't trust word one out of any of their mouths.
As a person who worked in the advertising business side, I can say wholeheartedly that truth in advertising is a complete misnomer. The whole concept of advertising rejects the idea of truth. I don't sound bitter do I?
Stay tuned for new sig...
When does spin doctoring cross the line and become false advertising?
When you get sued or someone dies or both.
Flappinbooger isn't my real name
When does spin doctoring cross the line and become false advertising?
When you get caught!
Similar to the game journalism post earlier today, if you want honest impressions, you're going to get them from your buddy saying the stuff rocks/sucks than from any sort of review/preview/ad.
When we are in the market for hardware of a certain value, we require a loan of the actual device to test ourselves.
"Who are in control, they are not in control of anything - they don't even control themselves!" - Glen Beck
"By the way if anyone here is in advertising or marketing... kill yourself. No, no, no it's just a little thought. I'm just trying to plant seeds. Maybe one day, they'll take root - I don't know. You try, you do what you can. Kill yourself.
Seriously though, if you are, do. Aaah, no really, there's no rationalisation for what you do and you are Satan's little helpers, Okay - kill yourself - seriously. You are the ruiner of all things good, seriously. No this is not a joke, you're going, "there's going to be a joke coming," there's no fucking joke coming. You are Satan's spawn filling the world with bile and garbage. You are fucked and you are fucking us. Kill yourself. It's the only way to save your fucking soul, kill yourself. Planting seeds.
I know all the marketing people are going, "he's doing a joke... there's no joke here whatsoever. Suck a tail-pipe, fucking hang yourself, borrow a gun from a Yank friend - I don't care how you do it. Rid the world of your evil fucking machinations. I know what all the marketing people are thinking right now too, "Oh, you know what Bill's doing, he's going for that anti-marketing dollar. That's a good market, he's very smart." Oh man, I am not doing that. You fucking evil scumbags! "Ooh, you know what Bill's doing now, he's going for the righteous indignation dollar. That's a big dollar. A lot of people are feeling that indignation. We've done research - huge market. He's doing a good thing." Godammit, I'm not doing that, you scum-bags!
Quit putting a godamm dollar sign on every fucking thing on this planet!
"Ooh, the anger dollar. Huge. Huge in times of recession. Giant market, Bill's very bright to do that." God, I'm just caught in a fucking web! "Ooh the trapped dollar, big dollar, huge dollar. Good market - look at our research. We see that many people feel trapped. If we play to that and then separate them into the trapped dollar..." How do you live like that? And I bet you sleep like fucking babies at night, don't you?"
[We miss you, Bill.]
What were the skies like when you were young?
This isn't just a phenomenon in the IT arena. Have a look at medical journals some time... You have to be VERY careful when putting stock in the findings of studies -- the first thing to check is who *funded* the study.
I think it's just a fact of life: everybody wants their product to be seen in the best light, and to sell well (in the case of commodities or services).
That's why Amazon.com has reader reviews, sites like epinions.com exist, and Slashdot has moderator points. It's also why there are hardware review sites -- we can't just trust the manufacturer's PR now, can we?
So, people may be inherentely biased and often untruthful, but with proper monitoring (read: community involvement), the truth will out.
...sums up this article perfectly.
There's a Mercedes gap too. I want one and can't afford one, but it's not government's job to do anything about it.
companies are (nearly) ALWAYS unethical if it will make more money.
"When does spin doctoring cross the line and become false advertising?"
When your product fails to perform, as advertised, in the environment you specify.
Most of us call this lying to your customers.
OK, I really don't mean that but its the attitude you see in a lot of places. My last job was for a mega retailer, we had a strict policy about transmitting and storing any data with customer's credit cards and such, it has to be encrypted. But out auditors deliberatly ignored the fact that if you went online and bought a gift card the data between the client browser and web server were secure but a trickle feed between 2 internal systems sent the same data back and forth across the internal network as clear text. The attitude was that if you could nod your head and say "We use encryption" then you pass the test.
... but naive.. Come on, what were you smoking? Of course the benchmarks/testing/what have you will be done in such a way as to out the product to be sold in the best possible position. Your question is naive. Even us scientists do this when providing paper plans for our bosses. We paint the best possible picture, do serious window-dressing and interpret our results in the most optimistic manner compatible with science. If you think that an advertising campaign will feature objective (if such a thing exists in benchmarking) performance comparison, you really need to get a reality check. Or, if your conscience is giving you problems, find a profession that doesn't require a conscience. I hear that law is rather profitable these days.
----- One learns to itch where one can scratch.
Companies' own figures and test results are always generous and I believe that customers are aware of this. It just like with "60 minutes"..
To everyone: When does spin doctoring cross the line and become false advertising?"
In the consumer audio market, that's when.
From over-unity speakers (200W watts output from a 10W wall-wart), to "better-sounding" fiber optic cable, no claim seems too outrageous or fraudulent for a great many consumer audio manufacturers.
As an engineer who loves audio, it drives me nuts to see the bullshit that is constantly perpetrated in that market.
I'm sure there are tons of slashdotters who can post examples of incredibly unprofessional and possibly fraudulent specmanship in this arena.
Life is too short to proofread.
Some years ago when I was working for a certain SCSI RAID HBA company (that shall be referred to as company M) we were shock to find out a certain OTHER company (that shall be referred to company D) is advertising that their SCSI RAID HBA out performs ours by substantial margine in LARGE BOLD letters.
When we took a closer look at the disclosure (in fine print) it states: Company M HBA tested in single threaded mode (READ: Tag Queuing Disable.)
That's when I lost all respect for marketing people...
ELOI, ELOI, LAMA SABACHTHANI!?
Well its funny because I was having this discussion with my co-worker 10 minutes ago about how many people publish data in scientific journals that is at best massaged to an exten that it can't be reproduced. The problem, sadly isn't just in industries but has perpretaded academia too and probably has been there for many decades. I wouldn't know because I'm still young and naive:). But seriuosly, I think most of the corporate problems exist due to managers who get paid insanely high salaries and need to justify and maintain them by making engineers work like slaves and give them crap about TPS reports and fucking morale and what not. C'mon say it with me... get rid of managers, lawyers, insurance companies. What do you know, insane amounts of coffee while listening to Rage against the machine does make you hate the man. Aaaaaah
But the fancy numbers aren't for me, they're for PHBs that like to see lots of impressive numbers -- after all, the other product has them so if this one doesn't...
Looking at computer specs lately I'm beginning to think the principal point of them is to bulk out the specs -- make it look like it has lots of features, and the actual content of the specs is irrelevant.
for a major worldwide advertising company that happens to do ads for a major hardware manufacturer. As the "IT" person at this location, i am often asked to come in and help to "explain" what such and such a piece of hardware does.
It never ceases to amaze me that these people (paid quite well to "understand" us (IT folk) have absolutely, completely no clue about any of the products they are trying to sell, let alone about more complicated then "this is a laptop, that is a desktop".
Build it, and they will come^Hplain.
is to ask who isn't doing this. Is there anyone reading this that works for a company that doesn't use this kind of practise and instead relies on the quality of the product to stand on it's own?
Thats what I thought. If there was, that company can't expect to last long in competition. This world is plainly insane, but so are all of us willing to buy into it, so I suppose it doesn't matter.
More Caffeine. NOW
Unlike many popular forms of advertising, I don't trust testimonials. When a piece of equipment is reviewed, I judge the review by it's source, since perhaps as a tech I'm a bit happier with a "clumsy UI" than with sheer abilities of, say, hardware.
So I look to Toms, [H], Ars for reviews by people who seem to have similar knowledge as myself. Then, when tests are formed, I don't trust just one benchmark, nor just one test or review.
If a company is going to game the testing, I'm disappointed. This lowers the confidence I have in these tests. Since blatent ads and testimonials turn me off, where else do it look? I'll have to just rely on the repeatability of any review's scores. This usually uncovers companies that try to dupe the reviewer.
Sun Microsystems (SUNW) shrugged off accusations today of unfairly reporting test scores for the beta version of one of its Java compilers.
Pendragon Software yesterday said that Sun, using Pendragon's CaffeineMark benchmarking tool, inaccurately inflated the test results of the Solaris 2.6 just-in-time Java compiler by optimizing the compiler specifically for that test. Solaris is Sun's version of the Unix operating system.
Sun responded by calling such optimization standard practice.
"The idea is that you want people to optimize for the benchmark," said Brian Croll, director of marketing for Sun's Solaris products. "We'll do everything in our power to do really well on all the benchmarks we get our hands on."
A benchmark is a battery of tests that gauges the speed and performance of software running in various configurations. Several developers have created Java benchmarks; CaffeineMark, which Croll called "the best benchmark we've got," is available free off the Web.
But how much optimization is fair play? Pendragon president Ivan Phillips contended Sun inflated the test results of the Solaris 2.6 just-in-time compiler by lifting code from CaffeineMark and inserting it into the compiler.
"The logic test is contained in the 'logicatom.class' file, and almost 50 percent of that file appeared in the compiler," he said. "The probability that this code made its way there accidentally is infinitesimal."
Reusing such a large chunk of specific code risks diverting too much of the compiler's resources, resulting in lower performance once the compiler is deployed in the real world, Phillips added.
Croll denied that Sun used CaffeineMark code but said the company "optimized around it." It will be difficult to determine who is correct, given that the beta compiler in question is no longer available. Croll stressed that the compiler is designed to perform well on a benchmark because that's what determines good real-world performance.
"If certain things happen frequently in a benchmark, you want to make sure you handle them well," he said. "If it turns out the benchmark doesn't truly represent true application performance, you need to evolve the benchmark."
The charges come at a time when Sun and Microsoft are entangled in tit-for-tat lawsuits over Microsoft's use of Java in its Internet Explorer 4.0 browser.
In an October 20 press release, Sun bragged that Solaris had the "world's fastest Java performance" and ran Java applications 50 percent faster than rival operating system Windows NT. After taking issue with Sun's test results, Pendragon said it asked Sun to retract its claims and remove the compiler from its Web site.
Sun removed the entire JDK 1.1.4 for Solaris on October 29 because the beta evaluation period ended, according to Croll. The company didn't take down the press release or rescind its claims, however, and Phillips responded yesterday by publishing his accusations.
Pendragon doesn't usually double-check testers' CaffeineMark scores. But when it saw Sun's results--the Solaris compiler hit a score of 1.4 million on the "logic" test, while the previous high for that test was 22,000--the software firm decided to investigate, fearing that CaffeineMark contained a bug.
If Sun indeed took deliberate steps to skew its results, Phillips was surprised at the lack of subtlety.
"If a company really wanted to conceal what they were doing, they could do a better job," he said.
Take a look and feel free: http://www.PieMenu.com
To everyone: When does spin doctoring cross the line and become false advertising?
When you get caught.
"It's NOT a surprise at all. Product Reviews by companies about their products are like asking hens to protect their own coop."
These hens did pretty good.
Instantly. There is no gray area between honesty and dishonesty. You either tell the truth, or you tell a lie. Your company either attempts to subvert tests [i.e., lies], or it doesn't [i.e., does not attempt to lie]. No ambiguity exists in this case.
Your question reminds me of a question posed on the cover of a national "news" magazine in the wake of revelations that the New York Times had published falsified news reports. Their question was, to paraphrase: "Does this signal a new standard in journalism?". Of course it signaled no such thing; it only signaled that some publications, or at least reporters writing for them, were willing to be dishonest and to print lies. I wonder if the author of that question was perhaps a bit hopeful that, yes, this event did signal a relaxing of ethics?
...for about 5 years in the mid to late 90's. I started doing the testing on basic network equipment and graduated over time to oversee the testing methodology for every product comparison we ran.
I can tell you that, if the testers themselves are competent, it's a moot point. For instance, when testing server hardware by using a database application, I always insisted that the databases be identical and configured as identically as possible. Normal stumbling blocks were issues with stock disk sizes, but we always ensured that RAID configurations were as similar as possible within the realm of reason.
Testing is an art form. It requires a thorough and repeatable plan as well as a good bit of knowledge about real world usage of equipment and software (would it be realistic to enable a non-battery backed write cache on a raid controller in a database application?)
I can say that many, many vendors attempted to put one over on us. And it's entirely possible that I missed some of them, and they benefitted because of it. However, in general, professional test procedures should expose and nullify any sort of vendor tweakage of equipment or software.
Key principles for good testing:
- Set any basic configuration to manufacturer's public recommendations
- Don't let vendor representatives touch anything. If they need to send someone into the lab, allow them to recommend changes, and document all of those for later review / revocation
- If third party hardware/software is involved in a test, use the third party as a sounding board. If you're testing a layer 3 switch using streaming media, talk to the streaming media provider about realistic stream rates and usage patterns.
- If at all possible, wipe and reload vendor equipment and software. You should be looking at the setup process anyway, so that helps the test as well as helping to prevent shenanigans.
In short, good test procedures prevent, or at least mitigate, the kind of abuse in question. And, as consimers of reviews and tests, it's in all of our best interests to get educated and develop opinions about the competence, thoroughness, and honesty of any souce.
Cheers
-b
If I wanted a sig I would have filled in that stupid box.
I tell you when - when you reach down that hooker's pants and, "OH MY GOD YOU'RE A MAN!" - that's crossing the line.
I was an engineer doing testing of composite materials and structures for aircraft and spacecraft for ten years, and the clients were all very clear about wanting the testing done and the data reported in the most optimistic light.
One REALLY BIG COMPANY ate shit over this and lost a billion dollar contract for jet aircraft engines, resulting in massive layoffs that were, remarkably, not reported as the result of management ineptitude.
Surprise! Surprise!
Reminds me of the strips where Wally had to impersonate the demo.
When the image got fuzzy, they tried a razor.
... when I worked for a German owned plumbing fixture manufaturer's US subsidary, we had to have all faucets certified for lead contanimation (leaching from the solder and brass compounds). As it turned out, a lot of what we were already selling in the US market would not come close to passing. The Fatherland offered to send faucets that were garanteed to pass. All we had to do was tell them what levels that they needed to meet for a particular model (has a lot to do with the length of the flow chamber).
They seemed quite upset that the units had to be pulled at random from stock. Maybe they were just to use to cheating.
No incumbents, not no where, not no how.
Vote them out every term.
I can't remember a time where I haven't been asked to tweak things a bit for a better showing in a review, performance test, or bake off. As an engineer, I like things objective, and let the chips fall where they lie. But in business good results can be life or death for the company. Ethics are a very complex subject. Especially when it involves your livelihood.
Developer Quote Of The Week: "What we do is, given a benchmark, we try to do as well as we can on it, and make sure that our system is the fastest benchmark -- I mean, fastest system -- in the world." -- Brian Croll, Sun Microsystems' director of marketing for Solaris
Two weeks ago, Sun Microsystems got caught with its hand in the benchmarking cookie jar. Or did it? Depending on your point of view, Sun either grossly misrepresented the performance of its Solaris Java just-in-time compiler by fooling Pendragon Software's CaffeineMark performance test, o r Sun proved the CaffeineMark is not an acceptable measure of Java compiler performance.
For those who may have missed it, here's the background: In a Nov. 4 press release, Ivan Phillips, president of Pendragon Software, in Libertyville, Ill., a developer of software for personal digital assistants, accused Sun of engineering its new Java compiler to trick the CaffeineMark into reporting higher performance results.
When Sun's compiler detected a block of 600 bytecodes unique to the CaffeineMark (a technique known as pattern matching), the compiler bypassed data processing, and instead returned a value expected by the benchmark. This fooled the test into reporting performance results 300 times faster than the compiler would deliver in real-world use. Third-party developers subsequently validated Phillips' assertion. Interestingly, when Pendragon's engineers altered the test to appear different to Sun's compiler, the compiler's branching was short-circuited, and its performance plummeted. Java compilers under Windows 95, Windows NT, and the Mac OS delivered uniform results under both the original and altered tests.
Sun officials initially admitted no wrongdoing, and were quick to point out that optimizing software to improve benchmark scores is an accepted practice among computer technology vendors. "People are optimizing against the benchmark," says Brian Croll, Sun's director of marketing for Solaris.
Further, Croll maintained that the aberrant results indicate a fundamental flaw in Pendragon's benchmark suite, and do not represent any impropriety by Sun. "I don't know how valid the [CaffeineMark] is," Croll said. Then last week, during a day-long media briefing at Sun's Mountain View, Calif., headquarters, Sun officials updated their explanation of events. SunSoft president Janpieter Scheerder said the company was not trying "to do anything malicious;" rather, Sun engineers simply "optimized too much."
A Sun spokesperson at the event blamed the incident on human error, and said an engineering prototype somehow found its way through Sun's rigorous (you would think) development and quality assurance processes, and onto the Web, with documentation, and overblown press release in tow.
What if Pendragon officials had not discovered Sun's alleged trickery? What if Sun engineers tweaked their compiler to only improve its score 10-fold, instead of the eye-popping 300-fold increase that flagged Pendragon officials?
Sun's PR machine had already posted a press release, in which they touted their "new Web-enhanced Solaris operating environment" as delivering "the world's fastest Java technology performance." The release also claimed Solaris' compiler was 50% faster than the best Windows NT score, and cited the CaffeineMark as proof.
If Pendragon officials had not discovered the ruse, Sun's formidable sales and marketing machine would now be steam-rolling press and IT decision-makers alike, trumpeting Solaris' performance advantage over Microsoft's Window s NT, waving Sun's illicitly obtained CaffeineMark results as evidence in hand.
"Any benchmark, no matter what its original purpose, is subject to use as 'benchmarketing,'" says Larry Gray, board member of the Standard Performance Evaluation Corp. (SPEC), in Manassas, Va., a consortium that administers many well- known benchmarks. "I'd guess may
Take a look and feel free: http://www.PieMenu.com
Absolutely none, I rely solely on product packaging.
Seriously though, I hold the belief that all sales and marketing folk are born liars and will never change. I purchase solely on word of mouth (from people I trust) and my past experience with a particular brand/manufacturer. I am the person that advertisers hate because I sit in front of the TV and explain to my wife exactly which mind fucts the advertiser is utilizing. Sales and Marketing (S&M how ironic) folk are beneath lawyers, politicians and criminals in my book.
(B) + (D) + (B) + (D) = (K) + (&)
You are talking about cheating here, and representing things better than they are, but it's not uncommon to find flat out lies in advertisement.
I've seen advertisement claiming that the product was the cheapest or only, where I clearly knew different.
It happens outside advertizing as well. Steve Ballmer has made some very untrue statements, and so have certain people in the previous administration.
The worst thing is that people often believe the lies rather than the truth. It's like in Zen and the Art of Motorcycle Maintenance: "The truth knocks on the door and you say, 'Go away, I'm looking for the truth,' and so it goes away.
Please correct me if I got my facts wrong.
Here is how you can get good results from Bake Offs:
Outline what you are testing, how the tests were set up, and let the results point you to a conclusion.
Use the conclusion to make other predictions. Test the predictions with another experiment.
Verify.
If a user sets the same problem up on their own, they should see the same results. If not, then something was cooked funny.
Personally, I look to independent sources for reviews. I would not trust claims made by a manufacturer.
You must be new. I've been working for in high-tech for about 20 years now for various companies, and I would not want my products to be evaluated on a level playing field. I will put in any tweak necessary to win a comparison. This is not kindergarten...fair is nice, but I know my competitors are doing the same thing. And the old college try does not pay a very good Christmas bonus.
In the pay per click world of google adwords (those text ads you see when you search) I advertise a free service. But since this free service is bundled with other nonfree services I put the prices on the ad itself.
So although they may be looking for something free, I don't pay for the click unless they know they're going to pay *something*, the visitor is better informed, and I get a higher conversion rate from the qualified traffic.
So although this may not be on the exact topic of yours, I submit that honesty in advertising works, especially when you pay for performance.
You're comparing how products perform under a specific test that you have devised. (which ideally, is similar to your production environment).
Tuning can have a dramatic difference in performance, and unless you're familiar with all of the products involved, it's impossible to get the best performance out of each one.
The original poster is talking about where one of the systems has been modified so it is not a default install, and specifically customized before being sent to the testor, so that they will perform better. (like with ATI's Quake 'optimization').
As another example, there were some folks trying to get higher rankings in SETI@home, who would return bogus results -- as that was faster than actually performing the calculations. If someone knows that the results won't be checked for accuracy (or can't), and only for time, they can boost their rankings dramatically.
Build it, and they will come^Hplain.
Tweaking Java test?: Sun Microsystems has been accused of manipulating Java benchmark software and using the results to state that its Solaris "runs Java applications 50 percent faster than Windows NT." Pendragon Software, maker of the benchmark software CaffeineMark, has put out a press release that claims Sun found a way to cheat on the benchmark tests, and then advertised the bogus scores. Sun has since removed the Java compiler from its download page, Pendragon says, but the original press release remains on the Sun site.
Sun admits Java testing error
Sun Microsystems (SUNW) today conceded errors in the results of recent tests involving its Java programming language.
The company erred in not admitting that it matched code from a Java benchmark tool for one of its own Java compilers, Sun Software president Janpieter Scheerder said today. A benchmark is a battery of tests that measures the speed and performance of software running in various configurations.
Kicking off the "Inside Sun Software Day," Scheerder began his remarks with a mea culpa for Sun's actions, revealed last week in a report by CNET's NEWS.COM. At that time, Pendragon Software, makers of the CaffeineMark Java benchmark test, accused Sun of taking code from the CaffeineMark software and adding it to a beta version of the Solaris 2.6 Java just-in-time compiler. CaffeineMark is one of several developers that have created Java benchmarks.
Last week, Brian Croll--director of product marketing for Solaris, Sun's flavor of the Unix operating system--denied that Sun lifted the code. Today, however, Scheerder made it clear that Sun had made a big mistake.
"Nobody was trying to do anything malicious," Scheerder said. "We just optimized [the Solaris Java compiler] too much."
A Sun public relations manager called the episode a "big-time organizational breakdown" in which an engineering prototype that was never meant to go public was posted on the Web with all attendant documentation, along with a press release that touted the software's performance. Sun has also posted an explanation on its Web site.
"Sun committed an unintentional error when we published Java performance numbers for an engineering prototype that included code that specifically looked for a piece of code in the Caffeinemark 3.0 benchmark," according to a company statement.
In a release dated October 20, Sun bragged that, according to the CaffeineMark 3.0 test, Solaris 2.6 ran Java applications 50 percent faster than Windows NT. But it neglected to say that it had set the compiler to look specifically for a chunk of code from CaffeineMark. Reusing such a large chunk of specific code risks diverting too much of the compiler's resources, resulting in lower performance once the compiler is deployed in the real world, said Ivan Phillips, president of Pendragon.
After taking issue with Sun's test results, Phillips said he asked Sun to retract its claims and remove the compiler from its Web site. As of last week, Sun had not retracted its claims, so Phillips went public with his accusations.
Scheerder stressed today that the compiler, which was part of the Solaris 2.6 Java Development Kit 1.1.4 beta, was not shipping product. The company pulled it from its Web site soon after Phillips contacted them last month.
The news comes four days before the International Organization for Standardization (ISO) decides if Sun is qualified to be the official submitter of Java technology if and when Java becomes an international standard.
The official submitter has the responsibility to gather industry consensus and present it to the ISO's technical committee for consideration. There is some concern that Sun, which owns Java, might not be a neutral submitter. So far, 11 countries have voted yes on Sun's bid and one country--the United States--has voted no. A total of 27 countries are scheduled to vote by Friday.
Take a look and feel free: http://www.PieMenu.com
What are you, a Sharper Image salesman or something?
You probably shouldn't click this.
Marketing materials do not set out the faults of the product. This is not the role of marketing. Marketing aims to connect buyers to sellers. Providing information about faults does not help to make that connection. Also, many of the "tests" cited by marketers are labeled with titles such as, "Customer Success Story". This should be a clue that the material will not detail unsuccessful characteristics of the product.
Finally, marketers in most companies are not technical experts. They have to rely on the information provided by engineers and programmers. Many companies avoid ever telling the marketing department anything negative. As a result, in many cases, marketers aren't lying when they make claims -- they're explaining what they were told. Many of these marketers, especially the ones writing up collateral, are junior, new to the company, or even working on contract, so they don't have the depth of knowledge to tell that they've been given misleading information. Other people in the company sometimes lie to the marketers. It's not always black and white. (Not that all marketers tell the truth, of course.)
-- SYS 64738 --
And you know who else hates that type of benchmarking whoring? Linus Torvalds, that's who! Linus would never stoop to such a thing, because Linus is a great guy!
And you know who else would never do it? Apple Computer, the people who make the greatest computers in the world! They would never stoop to rigging benchmarks!
Or karma whoring.
Lawrence Person (lawrencepersonh@gmailh.com (remove all "h"s to mail)
http://www.lawrenceperson.com/
Truthful Advetising is an oxymoron.
we had a bake-off on one of our products...i was called in when the results didnt meet what management were expecting. after adding cpus(!) to the product under test (without the knowledge of the tester) we finally got a result which beat our competition. this was in 2001. i was later first on the chopping block in 2002 after i noted at a meeting that we should not try to publicize the results too much since it might backfire. the VP who canned me noted that if we got results we should publicize them as much as possible and i was an "impediment to future marketing capaigns"). i got an above average severance package tho so i guess they paid me off to leave quietly. ironically HP's results got beaten by IBM which simply threw money at the problem 4 months later and won.
In my experience, the lies don't stop with the advertising. My bosses are both salesmen. The only thing they do at the small company that I work at is sell our software, and they'll tell anyone whatever it is they want to hear so that they'll buy it. But I've noticed that this is definately not the end of their dishonesty. They treat their staff, me included, just like a buyer. They promise us stuff like compensation for working weekends, etc, but then just like our software, they fail to deliver.
I have a huge problem with salespeople and advertisements specifically because of my bosses. IMHO everyone who works in sales is nothing but a glorified 419 scam artist. Politics really aren't any different either.
That is why I like open source so much. Almost everything is free, so there's no reason to lie.
No one cares what your captcha was
Houston TX, USA
I'm not going to say who, but I know that some cars that we are getting ready for the Detroit Auto show right now have been given multiple coats of paint as well as a double clear coat. They paint the brake discs too...
Or, for a careless end-user, is that the devil sold his/her soul and make a killing?
... whatever.
:-P
Any which way you dice or slice it, it boils down to the "trial" run to overcome the buyer's skepticisms. Be that it may, a trial balloon, trial-by-fire, trial-by-jury,
In the case of Internet-based products, it takes a true network engineer to understand the fine subtleties between UDP throughput and TCP throughput (as well as any other application/presentation/session layered throughput combinations) and to procure an actual Internet traffic composition when placing the DUT (device under test) into operational mode (and under duress, no puns intended).
For an average I&T guy, the best way to evaluate an Internet product is to ask for a 30-day trial period and dedicate a portion of your corporate network. I'd say, sic it to the development group (hey, I'm one of them too!) as they should be focusing on their coding/design effort, not reading Slashdot
When such a DUT chokes under nominal traffic scenario despite publicized (and ominously rosy) one-sided benchmark, it usually a strong indicator that the DUT is a poor design and "SCREAMS" stay away.
30-day trial is your best friend. But the enemy of your enemy (the Devil's advocate) is also your friend.
I wish the world were so simple. Is it lying if you don't advertise that your product does poorly on a particular benchmark? Is it lying if you don't advertise that your product will fail under a condition that occurs, on average, once every 20 years?
Power supply makers always seem to derate their stuff. for instance, a 75 watt supply rated for use up to 60 Deg C may lose up to 40% of its output power from 50c to 60c. Thats 47 watts at 59c. Funny thing is, most manufactures will tell you this out right and provide the derating curve on their web site. others don't mention it.
The government which is strong enough to protect you from everything is strong enough to take everything from you.
It might be a problem if it were not so well known. However, it's very common to test a part under optimal conditions because numbers sell. A 1.2 GHz capable part sounds better than a 1 GHz capable part.
That being said, anyone who takes the numbers completely at face value gets what they deserve. We have an entire facility that acquires parts we are considering using, and "qualifies" them, and then publishes their findings in the company's part database. So somewhere between the vendor data sheet and the homespun one, the truth lies.
I work in the forklift truck industry and understandably the advertising there revolves around saving storage space, increasing efficiency per handling transaction which both in turn saves money.
The upshot of all of this is that when it comes to it, a prospective customer will usually say "prove it" and you well, have to. I for one took great pride in being part of the tech/development/demonstration team in that I had a say on what went into the sales literature as I'd often be the one proving it...
Needless to say, as it was MY arse on the line, I managed to complete demonstrations without any screw-ups.
I've always completely disregarded benchmarks, etc. other than those I've run myself.
It's kind of like Microsoft's BS-filled "Linux TCO vs. Windows TCO" ads here on slashdot. Sure, maybe Windows Server 2k3 is cheaper to operate than linux (What a bloody joke) in Microsoft's excessively convoluted idea of how servers/whatever might be run, but chances are extremely high that Microsoft has no damned clue about how my servers are run, what content they serve, etc. etc., not to mention the fact that there's rarely a way for individuals to verify the accuracy of the benchmarks in the first place.
Like the article already expresses, benchmarks and "tests" essentially always treat the competing products very differently, placing their own product(s) in far more favorable environments to skew results to their advantage. It's plainly obvious, and personally I would be amazed to have anyone disagree with what I'm saying.
As a fairly well educated and "aware consumer" (or something), I can assure you I don't really care what your company tries to tell me, I'll go by my friends' experiences with the company's products, and entirely ignore any sort of "factual studies" (which are 99% of the time done by some company that is paid to do them).
Of course I unfortunately speak for a rather small percentage of consumers, as far as I can tell. It's pretty depressing actually.
Does this sound familiar to Verizon's "It's $14.95 extra a month to add computers to your cable connection. Not so with DSL!" commercials? How about Microsoft's "Get the Facts" campaign?
Your ad here.
the existence of such practices is not all that suprising, in and of itself. but if this kind of thing is being done by professional engineers, then i am a little dissmayed. as an engineering student in college, ethics have been an important part of our education, and i would be quite disapointed to learn that this kind of unethical behavior is widespread among engineers in industry.
"Unlike many popular forms of advertising, I don't trust testimonials. "
Hi. I'm AC, and I give Slashdot two thumbs up. The staff is knowledgable and friendly. The moderation is always on the mark, and the community is insightful, and often funny. And with Slashdot's preview feature I can see upcoming stories before most of the audiance. A tech site that is technically above, viewable on all standard browsers. I highly recommend Slashdot, for all your information needs.
More often we become aware of it when the competitor does it.
About 20 years ago there were a series of "shootouts" between Novell, Microsoft, and 3COM, to see which network OS was faster. That was when I was literated to the fact that tweaking parameters can make a HUGE difference in test results. If you have even more control, you can even tweak the tests. We used to have to supply "debunking" documents that explained how the competing companies got the results they published. Sometimes it was hard to reproduce their numbers, even tweaking our own sofware in the worst ways.
These days a lot of journalists try to maintain a neutral position. They go to great lengths to be fair, and document even tiny things that might give one product a slight edge over another. It gives them more credibility to those of us that have been through these product wars.If companies can get away with spouting total bollocks (first 64 bit desktop anyone - my Mesh Alpha (from a consumer desktop computing company) is obviously now very valuable since it never existed?) and not get fined, what incentive do they have for telling the truth?
Lies sell, since most people are stupid and believe whatever they are told.
Beep beep.
Is that there aren't any truth in advertising. Long gone are the days of the devil in the detail, even though capitalism prepared us to what is happening now we still have to confront the harsh truth:
The single most unifying caracteristic of capitalist entities is to, at any cost, give you less than what they sold you in such way that you believe you are the bad guy, the one who overevaluated the product. Try hard, when was the last time you bought something and it worked as advertised, as implied, was as sturdy and functionnal as told?
We are so used to it we even consider people who are pissed at recieving a burger that looks like a quarter of what is found on the pictures morons or naive. they are not, we are, the more we let capitalism win the more we lose. Don't get me wrong, capitalism is potentialy a great thing but only if kept on leash, real tight leash and if it's teeth are removed. Right now any occidental citizen who believe he's free should take a serious look at his lifestyle, we are servant, nothing more. You work your entire life so that a few people live happily and rich, on our sweat, with our money.
Bake off in mags: pure bullshit, period. the best way to buy products, and this is what I personnaly do with for my company, is to order test versions of said product from the company for evaluation and test the shit out of those, only then you will KNOW what is right for you. Any salesman, any publicity, bake off, review, comparison chart is a load of bullshit, whatever the site whatever the source. Every tech review website (anandtech, hardocp, tom's hardware), are all bullshit, they are held by people with agendas. the review and advices on Slashdot seen in the forums, bullshit, again, written by someone with an agenda even if this agenda is to make him look knowledgeable, an expert in some way.
the only thing I trust is my experience and it works very well, you should try.
Isn't that like.....
Sexual freedom in Saudi Arabia
Fiscal accountability in corporate America
Bug-free programming in Microsoft products...
Intelligence and integrity in GWB?
Click here or a puppy gets stomped!
I worked on a product where the rumor was that a few thou slipped under the apartment door of an editor's girlfriend got it the "editor's choice" award in a well-known computer publication. The product failed, anyway.
If she looks like this, I just might hit it.
Or how about her?
Yes, I've seen this sort of thing at other places I work. It's inherently dishonest. It's justified via a) claiming that it'll help sales (dubious), and b) claiming that everybody knows that they're bullshit anyways. Note that the two justifications are mutually exclusive. Doesn't stop them from using them though.
No, I trust none of these "bakeoffs". Or any other IT advertising for that matter. There isn't a single mainstream IT rag which is even marginally trustworthy. Go ahead and, instead of reading just the bakeoff that you're looking for, read an article about something you already know about (through hands-on experience with all the primary alternatives, including a FOSS alternative if it's software and there is a FOSS alternative). Note how much stuff they get wrong, how shallow the article is, and how it almost reads like an advertisement. The same is true for cars too, largely, at least from what I've read. I can't comment on other industries since I'm not particularly familiar with their trade press. Note, however, that I still don't trust them at all - I expect they're just as bad. It's just that I don't make enough decisions relating to those industries' products to warrant reading the trade press - instead I go to the store and carefully examine the alternatives.
This sort of thing crossed the line into fake advertising at least a decade ago. Companies routinely make absurd claims and get away with it. There's just no political interest in enforcing it. At best they'll include fine print in their ad. If it's a print ad, maybe you'll be able to read it. It's been a while since I've seen an ad with fine print whose fine print didn't take up at least 10 lines of extremely small type. Television ads are a joke, it's impossible to read the fine print at broadcast resolution, regardless of the size of your TV, and it typically takes up a whole screen.
What can we do about it? Elect governments with some spine. These sorts of advertisements will continue to be successful so long as people are poorly-educated, and people will continue to be poorly-educated unless there is a strong collective agreement in place that says "yes, everybody needs some minimum level of education, otherwise they're prone to manipulation and our society is controlled by those who control the media or the other forms of information dissemination." It's funny, isn't it, how political campaigns in the US almost exclusively take the form of commercials? (Except for the "debates", which are a joke to everybody outside the country.)
Note that when the US was founded, everybody who advocated democracy made sure to point out that the requirements for democracy included an educated public, free speech, and free press. People have totally forgotten the education bit and the press bit. (A government-controlled press is no more effective at disseminating important information than a press controlled by an aristocracy - corporate or otherwise.)
Barclay family motto:
Aut agere aut mori.
(Either action or death.)
First Post!
When does spin doctoring cross the line and become false advertising? What's the fucking _difference_?
You have GOT to be kidding! Are you just out of school and lived a sheltered existence with your over-protective parents? Man, wake up!
....it is also rather unproductive.
A great deal of the consumers time is wasted in finding out what a product can really do.
Such extreamism, if not worst than that, is counter productive for the whole industry, as the computer industry has been doing pretty good showing how well it applies double speak or its ability to manipulate abstractions, be it in producing code or producing advertising text...
A good example is teh recent stories regarding spyware removal products, how the freeware is far better.... because the incentive to deceive the consumer is just not there.
Clearly, companies do manipulate their products to get the most favorable results. There was even a case of truck manufacturers manipulating the fuel injection parameters so they would switch to a different profile after the engine had been running for fifty minutes. They knew that the fuel economy test only lasted fifty minutes... so they had one profile for that, and another profile for when the semi was cruising along the highway. So yes, this happens.
But how important is performance anyway? Compare it to reliability, ease of use, intuitive interface (see Three Mile Island for an example of the problems of a bad interface) and many other factors. From what I can tell most computer hardware is very very fast these days. The weaknesses are in other areas. I remember I once had to work with Bay Networks routers. Sure, they had more CPUs than the competing Cisco routers but they were full of bugs and the control interface software was abysmally bad. The performance numbers might have looked slightly better in a bake-off and they may have had a cooler feature list (more CPUs, hot swapable stuff) but they sure had a lot more downtime due to bugs and ease of use issues.
For business majors at my university, they have a compulsory course called "Business Ethics."
I've always wondered, and this article confirms, that 'business ethics' is probably the widest used oxymoron.
follow the money. who pays for the work to be done to have a particular benchmark or test suite run against your product? who pays the people who paid for it? etc.
this is normal. get used to it. accept it. if you don't do it your product will suffer in the marketplace.
marketing is all about walking that thin line. if the marketing weenies happen to cross it and generate tons of negative publicity it might hurt a bit, but chances are they'll at least spell the company and product name right.
I don't blame companies for acting this way, as it is a sales force's job to sell. I just ignore all of these white papers. I do however pay a great deal of attention to what companies like Gartner say about various products. They are paid by us (the consumers) as opposed to the producer and are not quite as susceptible to false analysis.
The sneaky monkey wins.
Power to the Peaceful
As a writer of data sheets and user guides and such for 12 years, I think it is all spin. To the degree that relevant information is left out, or made to look better than it is (through generalization or whatever) it is a lie. Potential buyers need the truth and all the truth so they can make a decision without having to guess, or be surprised when the product goes on line.
the advertising industry has enough baggage of its own without needing to absorb the dishonesties of another industry. LOL.
Wow, I didn't know Dilbert read slashdot.
Coder's Stone: The programming language quick ref for iPad
This is difficult. I was canned from a job once under the "at will employment" clause in my contract a few days after pointing out that business practices were questionable (and in all likelyhood illegal).
I did in the hope that we could reform those practices (it was a small company working public contracts), but ended up on the street as a result.
Everything worked out extremely well in the end for me (I had a generous enough severance (which required an NDA about said practices) to keep me on my feet and found a much better job).
But this post does bring up a great question -- what can/should one do when faced with ethical question's about an employer's practices? Do we as the technology staff have an obligation to make sure what our company does is ethicall/legally sound? Or is our roll simply to take marching orders?
It's all defaults. I'm not in the business of optimizing other people's stuff. And I don't thwart products, either. Most optimizations are good for a single edition anyway.
I don't care if the results are therefore good or bad; I get paid to report on what the defaults do-- either way. Virtually anything can be optimized to give screaming defaults. I use only industry benchmarks, never a proprietary one so that others can extrapolate my results on their platforms. Sometimes the equipment used to benchmark costs money, and that's real life, too.
I don't believe that there are very many trade pubs in this business that optimize for results. If they do, they won't last long as people will see it for the BS it is.
My only wish is that there were more cross-platform, OSS benchmarks that couldn't be tweaked. It made good benchmarks, like Intel's IOMeter, into a meaningless tool.
---- Teach Peace. It's Cheaper Than War.
When does spin doctoring cross the line and become false advertising?"
...
When you get caught
About ten years ago, the local roads and traffic authority had put out a tender for development of traffic signal lamps that minimised the effects of phantoming - where light (usually the setting or rising sun) enters the module and is then reflected back out making it look like the signal has changed and turned on when it hasn't (mess often ensues). The conventional "fix" is to put a cross-piece of black material in the housing, at least in Australia, so that light entering from an angle is absorbed and doesn't make it to the reflector. Unfortunately, doesn't work very well, especially at shallow angles.
So, the CSIRO (Aussie Sci/Industry public org) analysed the (real) problem and came up with a solution that involved a Fresnel lens on the lamp and a somewhat elliptical reflector at the back of the housing. Light entering from pretty much any angle except straight-on was directed in such a way that it couldn't escape, very effective at reducing phantoming. The traffic authority's tender, however, spec'd a test that a light source placed at x metres and y angle would have reflection measured at b metres and c angle. The competitor's winning design was cheaper - IIRC, it had a dark spot at the focus point for that particular configuration. Passed the test perfectly. Pity it was useless in any real world situation. But it's ok-it was cheap! (To be fair, cheap and meeting stated requirement makes (commercial) sense. The acceptence method was the problem here - and the exploit of a clearly inadequate test when the real problem was obvious. And, incidentally, important to road users' health).
And so we still have people drive into each other because they both see a green signal. Yay.
When does spin doctoring cross the line and become false advertising?
It's quite a simple answer - misleading or misrepresenting anything whasoever is falsehood. There's not really any grey area, proposing the existence of such is a socially acceptable way of making the lie pallatable or discusable.
People generally have the common sense to know themselves if they're lying or not, but mainly prefer to not worry about it. The problem is that we live in a societies based on and that thrives on lies. Liars often win in a consumerist culture, because lies are usually selling people their own dumb desires right back to them.
The real issue is whether it is actually acceptable to lie. All politicians without exception lie and muddy the water, advertisers and PR people lie so much perhaps they don't even notice anymore. The alternative is too unpalatable to a mindless and uneducated society who want everyone to do their dirty work for them,
Most Americans would rather think that their army for instance is well equipped with modern and state of the art equipment. We like to think that our governments care about every soldier as we do our friends and family. Regardless of who's in power - the government is not a benevolent father who loves each and every one of us and watches down on us like a proud patriarch.
The reality is that dumb kids lives are cheaper than good equipment (regardless of who you vote for and who's in power). Another dead kid in Iraq isn't really top priority, unlike keeping the Whitehouse furniture and art restored. People don't like to admit that some dumb grunt isn't worth as much as a nice piece of Louis XIV furniture, so people pretend to care when in fact they don't terribly much.
The holy grail of technology is no different - the utopia of consumerist culture is just to tempting to refuse new technology for it's own sake. Nobody wants to know that the latest thing isn't all that good - hell most people don't really have an actual use for their computers as they're lives and work are usually fairly inconquential. We want to eat the dream of technology and time saving devices even though deep down we know that it's all make believe, and we don't really have anything to do with all our saved spare time anyway.
Si tacuisses philosophus mansisses. If you had kept quiet, you would have remained a philosopher.
I've seen this happen myself...
For five years, I worked in the QA lab of a company which is no more (one of its co-founders went on to become governor).
I remember one or two occasions when "tweaking" occured for a test, and I would not be terribly surprised if there were more instances.
I wasn't especially impressed that the Powers-That-Be felt the need to do that, since I felt that the product(s) should stand (or fall) on their own merits. I asked about the fairness of the practice, and I was told "The other guys are going to do it. If we don't we won't look good against them."
I still wasn't thrilled at the prospect, but from that point, I judged the product by its performance when I had it in my posession (preferably on a trial basis).
When politicians are involved, everyone loses.
Another one: just look at the old Dhrystone benchmark and all of the over-the-top "optimizations" that were used to get better compiler/processor results. The SPEC organization, created in a direct attempt to deal with this very kind of problem, still must update its bendhmarks regularly in order to deal with loopholes (and changing technology in general). A good example was when a particular benchmark (matrix300; ref is 2/3 the way down) was defeated because it took no input. That made it possible in principle to collapse the entire program to a constant, and at some point, somebody did. That last link also gives a good description of why initially good benchmarks go stale.
And while those two examples are old enough to show that this has been going on a long time, there are plenty of examples far older.
Benchmarking has always been an arms race.
I too have witness practices, habits, and customs
.....
in the work and lab environs, that, when taken as
a whole, posit the prior question as to what, if
any, ethical, moral, or value-derived basis there
exists, will exist, or could exist in this or
any other practice, with respect to the
aforementioned unmentionables, and the need
for fairness, due process, and respect in
ones dealings with others.
So I agree with the story submission, as
submitted, and it does a great service to
shed wanted light on this subject. But do,
if you, please allow me to add that
I could not agree with it more.
I found myself wondering: what the hell is this
guy talking about? Of course benchmarks are
cooked. What unethical crap is he talking
about? Is he talking about a compiler that
has benchmark-specific tweaks, or something
more over the line like holding a guy to
the product tester's heads? I mean, what
the fuck? Can this guy be more vague?
What does faith have to do with engineering or capitalism? People are competing. "Faith" and "lazy analyst" are synonyms.
--
make install -not war
This goes double for overclockers. I only bought one motherboard before I realized that reviews didn't hold up as well as the collective experiences of techies (this is especially important for overclocking results). Forum posts are direct links to specific experience and knowledge gleaned from those experiences, like how you shouldn't ever expect a Tiger Direct rebate. Reviews never stress the products to my liking - you need clumsy people and morons to do that for ya. Sometimes they can also tell you the step in the (dis)assembly that was missing from the manual. I guess that black heatsink gunk on the standard Intel HS can get really stuck to the processor. Stuck enough to rip the processor out of the socket with the pins still in the socket. Yeah, don't use intel thermal gunk. If you must, apply heat to remove. See? You learned something on a forum.
then then there's the truth! :-D
For example, at the company at which I work at, we always de-tune our applications so that every operation will take place at the worst-case scenario level. Then, when our customers buy our products, expecting a class C- performance, and our products then provide a class A+++++++ performance, they are thrilled, which causes them to:
- Buy more of our products.
- Tell their friends and neighbors, who then buy our products.
That is what to the pain means. It means I leave you in anguish, wallowing in freakish misery forever.I think you're bluffing.
It's possible, pig. I might be bluffing. It's conceivable you vomitous mass. I may only be lying here because I lack the strength to stand. Then again, perhaps I have the strength to stand after all.
What are you selling your customers?
Are you selling an emotion or some sort of strange detached feeling of satisfaction? Most companies nowadays do that.
Then just sell your IT product with a nice looking GUI and lots of nice little buttons and habe the marketing dept. take some pictures and add their phrases. They'll ask you about a noteworthy feature or two and present it in such a way you wont recognize your own product anymore. It will sell like hot cakes.
If, on the other hand, your selling really JUST the product and the truth that comes along with it - combine it with a consulting business that checks wether the product is the right thing for your customer or not.
I tell my customers flat out if my thing isn't the right one. I'm that honest. On the other hand they do get a remark from me when they buy third party crap and complain to me later on. Especially if I told them so before.
There are very few people who work this way and they even, naturally, have the habit of rejecting certain customers. Old Book Stores come to mind as an example. There you get exactly what your asking for. But be prepared to be told that you're not the kind to actually judge if the product you interested in even is the right one for you and you better go look somewhere else, cause they don't really want you as customer.
I'm actually looking forward to the time when my business has grown that far that I have a customer que were my (and my employees) biggest job is to pick the right ones and reject the others. That's the only way a business built on truth will be able to work in the long run imho. To be true you absolutely need a customer who knows what he wants.
All else is just the usual stuff.
We suffer more in our imagination than in reality. - Seneca
There's a package listed on Freshmeat for benchtesting supercomputers, and is used (I believe) by the top500 group. It has about 6 or 7 high-performance tests that, from what I understand, stress out just about every bit of hardware in the box.
Of course, you can beat all of these standardized tests, if you put your mind to it, and that is typically what manufacturers do. They want to sell their products. So long as the customer comes back, what the customer thinks is irrelevent.
Think about all the on-processor caches there are, these days. Do you think they really help users that much? Even a few megs of cache will quickly be flooded by handling oversized multi-threaded applications.
No, caches are infinitely more suitable for scoring well on benchmarking programs, which can typically fit entirely within said cache. This can "improve" memory access by an order of magnitude.
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
Uhhh...
Can a NDA that forbids the disclosure of illegal practices possibly be binding?
What are they going to do, sue you becaues you exposed their illegal business practices? In a sense, it'd probably be against the law NOT to report them, since you are witness to a crime and could be considered an accomplice if you don't...
-Z
I see it all of the time. You know that handset that won't provision over the air? Yup. We knew about it. Know that handset that won't work with Multimedia Gateway? Yep. We knew about that too. Sprint sux and they don't give a rat's ass about you, the customer.
It's like Reagan used to say: trust, but verify.
You can show me all kinds of wonderful data about how your software is the absolute best thing in the world. I may believe what you're saying. But I am going to verify that the software does what I need to do. I am also going to verify your claims.
I can't accept anything without verifying it. When I go to buy clothing, I do what I can to get the right size and style, but I also verify that it is the right size and style. When I buy meat, I still smell it before I cook it even if it is supposedly fresh that day. When I do anything, I trust but verify.
If these guys are really doing what you are talking about, then that is a problem. It is not because it is an ethics violation (it is, and that is very serious in and of itself.) It is because WHEN they get caught, it will cause irreparable damage, possibly far more so than the cost of doing things right. You may lose your job due to economic reasons due to management's questionable practices.
In fact, if you think about it, all of ethics is just behaving in your best interests. It is not in a company's best interest to sell a shoddy product or misrepresent their product, no matter what the marketing droids or peanut counters say.
The radical sect of Islam would either see you dead or "reverted" to Islam.
When you're quoting a case from 1997 it's helpful to say so up front. Otherwise people might get the idea that you had some meaningful statement to make about Java performance or current benchmark-optimization practices.
Breakfast served all day!
I remember when the PC Mag benchmarks were THE benchmark. I still remember the article on the card that had the test strings embedded in the BIOS, so the numbers were out of sync with it's performance in other tests. They disassembled the BIOS and found the test string.
To bad I don't remember the card manufacturer but then it's doubtful they are still in business. I think this was in the late 80's early 90's time frame.
One time I asked Oracle on a support call if I could disable the redo and undo logs, because my particular application did not need point-in-time recovery, and it constantly loaded massive amounts of data. As you might imagine, redo and undo were the biggest I/O bottleneck.
I got up to developer-level support and they told me about this undocumented, unsupported feature that you can put in your init.ora to turn off the redo and undo logs. They actually admitted to me over the phone that they use this feature for benchmarks against other RDBMSs. They continued to warn me that if I were to use it in production, I would get no recovery support, but they seem to think its ok for selling the POS.
I've since seen the light, switched to MySQL, and will never look back. MyISAM turns out to be exactly what I needed all along.
essentially deceit permeates our world. Once people figured out that words are really icons or labels for concepts and that most people are guided by their internal representation of things instead of reality. For those things outside of math, the world has very LOOSE definitions. Now you go to the store and buy a "pizza", but have a look at the ingredients, I wouldn't call that a pizza at all, but a collection of manufactured chemicals that are presented as a pizza icon. No one has officially defined that a pizza can only have certain ingredients or be made a certain way. Good enough to fool 99% of everyone. Politics and the media spin stories and distort reality to their choosing transformational vocabulary. For example the department of defense use to be known as the department of war. Bill Clinton said he did not have "sexual relations", depends on how you define what sex is. In communist countries they used to send people to "educational camp", is it an education or is it indoctrination/brainwashing?. Ideas like moral relativism, subjective experience etc. hold that there is no absolute truth, or things such as right and wrong and therefore its perfectly permissible to change the icon for things. People live inside their virtual worlds, their mind, and will distort the incoming information. When it comes to a companies bottom line of making a profit, of course they are going to do everything they can to sell it, right or wrong. They are going to push the limits of what they can legally get away with. Noticed I said "legal", which an independent concept from right and wrong. Just as a tyrant government would do to stay in power. If you noticed, scientists used to look for "truth" now they look for "models" or paradigms. It seems interesting that the great debates between the sophists and the philosophers have implications for today's world.
A few years back I interned at the Clorox R&D labs. In their industry, as in many others, most of their tests were industry standards that would be very difficult to tweak. They were very concerned that their results be repeatable and therefore verifiable, lest they have a few lawsuits on their hands.
Of course, the problem with the computer business now is that there are so many tests and not one agreeable "standard," which is what you get when a product is used for so many different tasks.
The difference is that a computer is so complex that it cannot be described by a simple quantitative rubric. Until a product becomes more like a commodity, it will always have this "truth" problem.
The story goes that one time it "optimized" a competitor's benchmark down to a single NOOP since (you guessed it) the benchmark didn't produce any output! They gleefully used the resulting performance comparison against thier opponent -- who compiled it with a "normal" compiler -- application against their opponent until someone called them on it.
True? Perhaps.
Funny? Definitely.
It crosses the line into false advertising when someone who has no financial interest in the company says "But that's misleading!"
That is to say, about 99.99% of all advertising out there today.
Maybe I'm being radical and pining for the good old days when only 90% of advertising was completely fabricated.
fifth sigma, inc.
I also work for a large tech company and experienced the same type of fluff in a recent product review.
It all started when my colleagues and I decided to do a simple technology comparison between our selves and a few competitors. It was originally a spare time project, devoting a couple hours here and there so we could learn a thing or two from the industry in general. Within a week or two, even though the evaluation was still quite fair, you could tell that each person had a small bias towards a particular product.
When management caught wind of our little evaluation 6 weeks later, the atmosphere completely changed. Almost instantly there was a strong pressure to show 'our' product in the best light. It eventually turned into a high profile, 6 month long, in depth evaluation. Testing methods were continually tweaked until people were happy with the resulting data. Guess which product came out on top? Hint: at one point I was told that my final conclusion was wrong and to re-look at the data. Things like this really make me lose faith in corporate culture, and i'm sure it's just the same in public culture as well. So much for honesty.
What I have learned is that when money is involved:
Companies don't want to hear that the technology their future is dependant upon isn't really the best thing out there.
Consumers don't want to hear that their choice of the latest gadget wasn't really worth the wad of money they just blew on it.
there's three kinds of prevarication: lies, damned lies, and benchmarks.
-Tom Duff
Speaking as someone who also benchtests as part of what I do for a living, I'd like to add a few more points.
:)
Test labs are useful for customers without the clout to engage vendors directly. Buying an $80 vid card ain't going to get you any attention from vendors, distributors or even sales outlets.
When you step up the ladder a few rungs though, things can get interesting.
For example, your grant has come thorough and you're about to assemble a cluster to run your nasty embarassingly parallel app. Now you can do some interesting things!
Chances are someone, somewhere is doing something vaguely similar already. Check the Net! Bound to be a mail list on a related topic. Ask a few loaded questions
For the chance of a big sale most vendors will lend you evaluation kit. When you know what sort of kit you'd like run - don't be shy... ask! Hardware, software, the whole shabang.
The best test is your application. Science, commercial whatever. SPECflopbongotatas with HT enabled switching fabrics don't mean squat! Cmme up with a good canned scenario for your app and benchmark with that. Wall clock time often sufficies as the measurable.
Things to consider with this approach. A lot of fuss is made about compiler flavours and "standardised" optimisation switches. Forget it. Go for the *most* optimised flavour you can muster. It's your app and the only thing that matters is the clock (for the sake of this example).
A nice little benefit from this approach is that it's often OK to let the vendors into the sandpit after you've played hard. Let them go crazy! Tweak drivers, compilers switches, CPU revisions etc etc. Doesn't matter (to a large extent) what they do. Your app is the reason you're doing this!!
Word to the wise though, don't share results from other tests until all vendors are finished. Sure you can let them know there are others but don't say who. This saves them from going into FUD-mode at this point. They'll do enough of that later when they're unsuccessful =)
Remember, you're in the cumfy chair. Use the power while you're got it (alas, it fades so quickly). The fun soon stops and you'll soon be grinding your new toy into the ground.
Enjoy.
Stevo
Forget the truth. Science is fact.
Yeah, I don't know how many headphones I've seen in packages blaring "DIGITAL READY." Yet they only have connections for ANALOG signals.
Even if someone came up with a pair of headphones that had an S/PDIF or AES/EBU interface, it would still have to have a DAC and an analog transducer, because my ears are not digital nor will they ever be. What's that you say about cochlear implants? Well, there's still has to be a DAC between whatever and your nerves/brain.
So don't give me this crap about "digital" headphones. There isn't any such thing now, and I don't see how there ever can be.
Give me my freedom, and I'll take care of my own security, thank you.
My worst experience was in doing a document management system for the Whitehouse. The project was run by a bunch of dress nice but clueless people - which is pretty common in the Whitehouse. The contract called for certain bonuses to be paid if delivery dates were met. When it became obvious that the design was a failure and that this thing would not pass a single prewritten test, the project manager called in the team to work over the weekend and re-write the tests so that the application would not fail. This all needed to be done before she went and taught Sunday school at her church.
I refused to work, eventually got fired (after I completely stopped showing up), and more importantly forever formed an opinion about most good Christians.
If I could remember the name of the company I would be happy to say it - I think these kinds of frauds needs to be exposed.
slashdot troll = you make a compelling argument I do not like the implications of.
I once worked for a marketing and management practice consulting company. While I was there, they structured their offerings so that unless the participants hit metric X, they would pay a reduced price.
Right around the same time the CEO approached me and had me make modifications to "update" our metric reporting tools - which coincidentally made it much more likely that the target point would be reflected by the participants.
I don't work there any more, having left as they imploded under that poor management and the economy began to pick up enough that I had some other way to provide for the family.
Any spoon would be too big.
Bullets don't lie.
We test our double-armored cable with rifles, saws, axes, and anything else we can find. And it passes.
Forget the engineers. Get some pissed off rednecks out at the rifle range and see how YOUR products stack up. Ours will pass any day.
The Underwriters' Laboratory certifies materials for fire resistance, in the US.
t ro forwebsiteFINAL.pdf
For decades, plastics manufacturers sent prepared -- unrepresentative -- samples in for testing.
Once they had certification, they used the more flammable, cheaper, more malleable plastic in products.
Glad you asked. Can't tell you how I learned that, but it was a long time back; I think the practice stopped a decade ago. I hope it did.
Still plenty of stuff out there too dangerous to use:
State Fire Marshals -- Products Too Unsafe for Use in the Home
http://www.firemarshals.org/issues/home/docs/In
I was raised by a small businessman who was raised by a slightly larger small businessman. I quit the family business for technology thirty years ago. My dad refused to hire a salesman saying "my customers are the only salesmen I need". The best salesman in our large high tech company, that I ever had the pleasure of working with, had a large set of loyal customers. The customers were loyal because he never sold them anything that wasn't fit for there application. He also got to know them very well (including how there kids were progressing, by name). His customers trusted him and I loved supporting them. It was quite strange how few other salesman really seemed to learn much from him.
I've seen this a lot. More in software and internet appliance companies, but some of the big boys do it too. Remember those stories about entire testing departments being sent out to see a movie so the testing could be pencil whipped by the sales people in their absence?
In short it comes down to most places not caring about quality, and most customers not caring either. If more places sued companies for not delivering on their promises, this would go away fairly quickly. But I'm not going to hold my breath, the MicroSoft 'good enough for the average Joe' model made them (deservedly) rich. So every Marketing and CEO 'genius' thinks they can do it too (but they don't know their market as well as MS does).
So they lie, cheat, and skimp on testing and quality, and too many customers let them get away with it. I can name a half dozen major software products that are complete crap and do this (but I won't cause I don't want to get sued myself!!), yet they still sell Millions, if not Billions of dollars worth of product every year.
So I don't blame the companies, I blame the consumers. The market rules after all...
My current employer does it. We have to. When we test our product to give marketing their numbers for publication, we have to make certain assumptions for the variables. Want to know how fast a Porsche goes? Straight line or curvy road? How curvy? Oh, straight then. Windy or calm? Sea-level or alpine desert?
There's no equivalence between marketing numbers and YOUR reality because the variables change. So in a bake off you have to try tuning for the customer's variables. And when you do, you find that the performance falls off because, duh, marketing published the optimized numbers. That's not deception, but a common denominator with the competition.
Think about it. When was the last time the "how fast" question was answered with worst case numbers? You get best case, always.
Then you bake off and it sucks.
Then if the vendor has a good Sales Engineer and Support team, they'll tune it for your variables and get the performance where you'll make a buy.
These opinions guaranteed or your money back.
a more than made up for it in the years since.
9 ,00.asp
http://www.extremetech.com/article2/0,3973,110525
nV's tuning was clocking over a thousand points more than when it was blocked. ATI was still cooking their drivers at the time, but only to the tune of a hundred or so points. ATI has since removed any such optimisations from the catalyst drivers.
The problem is, bake-offs and third-party testing are supposed somehow to differentiate product A from product B. Scores on some matrix developed by the testers are supposed to tell us which product to buy.
Here are just a few of the problems with this approach:
1) Everything's at the same "level" of importance. Reminds me of when I was applying to college. There was a computer program that was supposed to help me select a place to go. It had hundreds of questions, ranging from course of study preference to living arrangements to climate. The program ultimately decided that I should apply to the University of Miami, a cheerleader party school at the time (sorry if that's no longer true). What I really wanted was Cal Tech, CMU, MIT, Stanford. Fortunately I ignored the program and ended up at MIT. But obviously the program "weighted" my answers to those questions in some stupid and random way. How are buyers supposed to weight the apparently equal categories created by the tester? Same problem.
2) If anyone has a truly innovative program or approach, the testing can't show that. Instead, everyone gets rated on some set of lowest-common-denominator problems. Of course some asshole big company is going to score better on a feature matrix, they've been pissing in the same code for ten years, with nothing better to do than add YAF (yet another feature). The small company that really has something new and different? Screwed.
3) Intangibles. Your service is amazing. Your training is legendary. Your app is incredibly easy to use. You answer your goddamn telephone, and you don't charge $200 every time somebody calls. You fix bugs. You fix them right away, not six months later in "the next release." It's all washed out in the "testing." You're fucked; you look the same as everyone else.
No thanks. Anybody invites me to a bake-off, I'll tell them to stick it up their ass. In fact, if I get invited to a bake-off, I'll run for the hills. Nothing could be worse for my company, and nothing could be less appropriate to our value proposition.
OK, end of rant.
Oh, yeah. My former employer did this.
:-)
/. subscriber looking over my comment history could figure out the identity of my former employer.)
I used to work with a company that manufactured a kind of commercial kitchen fixture. We came to a time where the competitive situation forced us to refresh the product line. The marketing and sales folks got their desired specs together, and told the design folks to make it happen.
The first prototype looked okay from a construction point of view, and a couple more prototypes went into the lab for testing to see if it performed its various functions. It had to pass a set of UL tests, and also meet the efficiency targets that marketing had set.
It eventually passed the UL test, although the testing enviroment itself had to be tweaked to a comical (but allowed!) degree. Once its UL listing was pending, it went on sale.
However, the only testing we did on it was safety-oriented. It never underwent pre-sales testing for marketing's efficiency standards... they just claimed that it met the targets. (It may have been tested after I left, though I doubt it.) It never underwent realistic operational testing in a simulated normal kitchen. It was just thrown into a customer's actual operational kitchen and expected to work. We had exactly one "beta test" installation of one unit, and sold many more units before we ever evaluated the field performance of the beta test site.
Obviously, the new product ended up having some expensive problems in the field.
As far as I know, no licensed engineer ever signed off on any aspect of the project. I can say this with some confidence, because I was the lead manufacturing designer. I objected to several of these shoddy practices, but to no avail.
Management figured this process was okay, because they assumed all our competitors were equally shady in their "engineering". (It might even be true, for all I know.)
After this experience, I couldn't stomach that employer anymore, and embarked on a career change.
(I am posting anonymously, as a really determined
When you say "engineer," do you mean a degreed engineer from an ABET-accredited school? Or a "pretend" IT type? That would be like a tree surgeon passing himself off as a doctor.
Just curious.
I lost my faith in any type of review when my Startup won a NAS shootout, with a product that didn't exist. Apparently one of the owners of this Respected magazine, had a large stake in the company .
Sad but true.
It's all about the almighty dollar.
This is the reason I completely ignore benchmarks and spec-tests. I go straight for the heart of the matter: reviews by users. Go to Google and type in Any Product review, and you'll get your fair share of honest reviews and feedback.
When forty nine out of fifty reviews say something is a piece of crap, then you know two things:
1) It's a piece of crap
2) The fiftieth reviewer is the same guy who faked up the benchmark posting under a pseudonym.
UTF-8: There and Back Again
Your good.
You could at least give details about this product if you consider this practice to be a problem... unless PerformanceEng is your given name, and you fear the wrath of your Zardoz overlords at Yoyodyne.
And yes, I am aware of the irony of posting as an anonymous coward.</preempt>
so you must work for Nvidia
Ludwig Wittgenstein
Consumer Reports is an interesting case. I like their reviews in the sense that they *are* unbiased by ads and the rest as you have noted. Unfortunately, CR reviews so many product categories that they are not particularly well informed about many of the technologies involved. Their car reviews, for example, weigh reliability so high that someone with a preference for performance might not get much out of their reviews. Their computer reviews are similarly shallow, coming from a middle-of-the-road user standpoint. Interesting and informative, yes, but perhaps not very meaningful for a prospective geek user. It would be great, IMHO, if there were a magazine with the neutrality and consumer-focus of CR but with a more technical bent and a more limited scope of products covered. Say, for example, only cars, only electronics, etc.
HP has somehow managed to go from one of the leading producers of quality printers, for example, to one of the many cheapo vendors.
I've used a lot of HP test and analysis equipment over the years, and most of it has both performed well and been durable. I always thought of HP as an instrument manufacturer, even at their peak of printer quality. I was kind of suprised that the consumer products company got to keep the HP name, and they renamed the instrument company. I still use a lot of Agilent stuff, and still often call it HP out of habit. Their printers seem to be a mixed bag, and don't really stand out the way they used to.
"Advertising in legalized lying" - H.G. Wells
The truth is a commodity in pretty short supply these days in any arena. Corporate America lies to us, government lies to us, the military. And we just take it. It's so common it's considered routine. We know they're lying to us and as long as it supports a narrow political agenda, then it's fine.
That's our life, the big wheel of shit. - The Fat Man, Blue Tango Salvage
The Certance tape drives (manufactured by Seagate) crossed the line. The Travan 20 (STT20000A) and Travan 40 (STT3401A) claim to store up to 20 gig or 40 gig. Of course, this assumes 2:1 compression, which is a standard assumption in the tape backup industry.
The catch? These drives don't support hardware compression. Furthermore, they don't support a variable block size, so you also can't perform software compression. The *only* way you could compress your backups would be to compress them to a file on your hard drive, then back up that file. In other words, you can only back up a 10 (or 20) gig file of (possibly-compressed) data.
To top it off, they claim to support Linux, but their tech-support people have no clue about it. They can guess about using TAR (yes, they like to use caps), but that's about it.
I have a client that is a very large utility, and the engineers that work the plants are old school, by the book, calculators for woosies I can take a derivitive in my head types. They don't compromise safety and they go by the -book- on everything. Every time I talk to these guys I'm filled with the greatest of admiration.
This is my sig.
You said:
"When choosing server technologies I have learned to conduct
my own bake off using the application that will be deployed."
Care to share a few tips and tricks on what to do to conduct your own bake off ?
Thanks !!
Muchas Gracias, Señor Edward Snowden !
This sort of thing is why I prefer the world of open source. Its also why I trust open source software so much more than software produced at software mills.
When profit from the product is not the number one priority, things like preformance, and overall quality tend to get higher marks.
This is especially true when there is not pressure from a sales forces and marketing dept, to add X Y and Z features just so the product more favorably comparse with the competition.
I just wish hardware had the same awide variety of choice that software curently has.
-MS2k
There is nothing wrong with it. For example the SAT's are suppose to test what knowledge you know however they still offer classes on how to "take" the SAT's. Many people customize their resume to be specific to a job they are trying to get. How is that any different. They are basically applying for a job and are trying to present themselves in the best light. Nothing wrong with that.
NEVER read what marketing comes up with. That's like watching The Media report on a subject that you happen to know about. Ignorance is bliss.
"It wasn't an uncommon practice for video card makers to tweak their drivers to perform better on benchmarks."
Something that could never happen with an OSS driver.
It was chip supplier in question was IIT (Integrated Information Technologies), they later dropped out of the graphics (and math co-processor business) and retargeted themselves at video, becoming 8X8. Later they evolved from video oriented at videophones to move into VoIP.
The 2D ZD WinBench had a string "The quick brown fox..." it rendered in different colors and sizes using the Windows GDI, and the IIT BIOS embedded it. I believe the parts were still ISA based, so embedded the string in a ROM on the card would actually offer more bandwidth than passing it over the ISA bus did.
Celebrities have always been manufactured from scratch for the most part. They aren't usually born famous. (Some are, and always have been, by virtue of being born to famous and/or rich parents.) The difference is in who manufactures the fame, and how. Used to be you could get famous by being in the right places at the right time, and having the ability to perform. But you could also get famous just by knowing the right people and not sucking TOO hard (or by sucking genitalia).
Now, it seems that the "next big thing" is pre-determined by the trendmongers running MTV, reality shows, and the like, then they find someone who fits what they already want (or is willing to fake it), regardless of whether they actually have any ability or not. How far would Janis Joplin have gotten if she'd tried out for "American Idol"?
Mal-2
How is the Riemann zeta function like Trump rallies? Both have an endless number of trivial zeros.
When Microsoft paid Veritest to show that Windows 2003 Server was faster than Linux, they did some heavy tweaking to the Windows registry and other configuration switches. Give the lab credit for listing the tweaks in their reports. It would take more smarts than I have to tell whether the tweaks were legitimate and even-handed, but I'm pretty sure that they wouldn't be done by 99.99% of Windows 2003 Server administrators. And I don't see how it can be legitimate in a server comparison to tweak registry settings in the client computers.
This post got me thinking of a movie from the early 90's (?) called Crazy People. An advertising executive decides to write advertisements truthfully. Really quite funny...The movie was so far from reality though, it was sad.
One of my many jobs is participating in vendor selection for my company ([sarcasm]it's a beautiful committee process...[/sarcasm]).
Last year we had a certain computer company (IANAL, so the name is intentionally missing) come in and give a sales presentation on why we should dump our existing vendor and go with them.
For the most part, they had our existing vendor beat from a price point. But we had been burned by previous computer vendors...made all of the mistakes...and knew exactly what we wanted (and, frankly, had made our existing vendor comply with our requirements over the period of 4 years that we dealt with them)
We image all of our PCs, we have specialized software for ensuring that everything is up to a baseline and that our environment is as predictable as possible. We needed hardware that would be easily inventoried, and *consistent, long-term, globally available configurations.* There were several other requirements we laid out and prior to the "sales pitch" meeting, we supplied this vendor with these "absolute requirements."
Of course, we received a 45 minute long power point presentation that basically regurgitated back to us everything we told them were our requirements. (lesson learned: it's better not to give the marketing guys the game plan. They tend to be more honest when they don't have time to power-point the lies and instead have to provide answers off-the-cuff).
It's a running joke on our team because if we took the entire content of their presentation and crossed out every word in each bullet point that represented some sort of "promise", we'd be left with about four words repeated over and over for 20 slides..."The" "a" "and" and "but".
I don't trust *anything* from any marketing or sales rep. After testing this vendor's products and talking to friends of mine who's companies had used this vendor in the past, we knew they weren't going to live up to their promises.
From day one, the information they gave us about getting loaner PCs for testing was sold to us as "far more flexible" than it turned out to be, and this poor customer service was going on *while* we were evaluating this company to determine if we should sign the contract!
Unfortunately, as the story goes, our opinions were appreciated, but the decision to choose this company was made anyway.
Myself and another coworker were noted as objecting to the switch in our final meeting minutes. Of course, that meant nothing except for a future "I told you so." And there was nobody left to say "I told you so" to because in the end, we were the ones left having to compensate for these broken promises.
Never forget: Caveat Emptor.
"God is dead!" - Nietzsche
"Nietzsche is dead!" - God
...you're basically asking what we think of tuning a machine for favorable benchmarking results.. ..Did I read that right?..
Aaaah... am i the only that is painfully reminded about that article on "corporate e-mail making no sense" I'm not even gonna try and decrypt this one. Just turn my eyes away before i go blind and/or stupid.
When does spin doctoring cross the line and become false advertising?
As soon as it's posted, printed, released, etc., but then, I'm a bit of a cynic. If it's true then it's not spin doctoring.
Q: How can you tell they're lying?
A: Their lips are moving.
No, frankly, I don't. Good hardware, maybe, but terrible driver. I spent hours working around their bugs in my applications. What year are you thinking of here?
spin doctoring
you have to be a corporate whore not to see that.
Homer: "Hey, what are all these holes?" (points at bullet holes in car hood)
Car salesmen: "These are speed holes. They make the car go faster."
Homer (impressed): "Oh yeah, speed holes.
Shutting down free speech with violence isn't fighting fascism. It IS fascism!
I ran across a plugin for the web that claimed it could be inserted with a single line of HTML code!!! Being truthful, they should have said "..inserted with only a single line of HTML code that's roughly 150 characters long!"
"Derp de derp."
Yes, I *know* they help. I've developed plenty of software which paid careful attention to cache sizes and memory accesses, and could have retired in my 30s because of it...
Basic performance optimizations like moving constant code outside loops will exploit the on-chip caches. Maybe it's only compiler writers, OS hackers, and embedded programmers that know it, but the benefits are real and present in product after product after product.
Even when there's evidence that a product does not provide what it claims to, the offending person or company is rarely prosecuted. Advertising has all turned to shit.
These posts express my own personal views, not those of my employer
Thus, of course we want or sales reps to be able to match the competitors figures, but in many cases it wouldn't be comparing apples with apples. Because of that, we are very open with our customers what and how we are measuring, when we get our numbers. And, since we don't want the customer to be unsatisfied, we make sure we perform measurments in relevant ways, too. So when the customer performs his benchmarks, he will be able to receive figures in the same range.
If the customer wouldn't be able to reproduce our results, we would simply loose our reputation. And that simply wouldn't be worth it.
You can always trust people to be human, (Yogi? /')
any more than that, an I think 'To err is human' falls in place.
The Roots of Consciousness: Science:
http://www.williamjames.com/Science/ERR.htm
> When does spin doctoring cross the line and become false advertising?
Pretty much all of the time, but almost always just within the bounds of what is considered legal.
If simple un-deniable facts by themselves were enough to sell a product and allow a genuine and straightforward comparison between competing products, there'd be no need for spin doctoring. However, only the best product available at the time in each price bracket would win any volume of sales. If you had the second best product, you'd need to find a way of getting people to buy it from you instead - enter the marketing team and all their little helpers.
The simple truth is, that by tweaking test results it may result in additional sales, and from a business perspective, it's pretty much essential. At a very basic level, as long as you can replicate your results if required to do so, then they are valid from an advertising point of veiw.
As it is widely recognised that all manufacturers do this to at least some extent, the technically aware individual will place no real faith in benchmarks from the actual manufacturer and seek independent advice, probably from a trusted magazine or web-based resource, who will compare the products on a more level playing field and in more realistic situations. Not to say that these result are impartial, just that they are usually more realistic and so usually give a better comparison.
What you need to remember however, is that a large percentage of computer users out there don't know or care about the technology or companies involved, they simply want the most powerful XYZ for the money they have available. This is the area where clever advertising backed up by what may well be dubious benchmark results will invariably win business.
Q: What is the difference between a used car salesman and a computer salesman?
A: The used car salesman knows he is lying.
I don't know about the US, but in the UK if you find an ad that makes false claims, you can complain to the Advertising Standards Authority (via their convenient online form):n /
http://www.asab.org.uk/asa/how_to_complai
and if they find the ad to be misleading, they will make the advertiser withdraw it.
I've done this; it works.
I would have thought that any global manufacturer which had an ad censured by the ASA in the UK, would probably be forced to withdraw it in the US too.
From the NYtimes: Dan Aykroyd once played a toy manufacturer on "Saturday Night Live" who sold children perilous products like bags of glass. If he branched into fast food, Mr. Aykroyd's character would probably have come up with Hardee's new Monster Thickburger, an artery-clogging mountain of Angus beef slabs, bacon, American cheese and mayonnaise on a buttered sesame-seed bun. It weighs in at 1,420 calories and 107 grams of fat - quite possibly one of the most lethal pieces of food out there. The Center for Science in the Public Interest calls it "the height of corporate irresponsibility." Jay Leno joked that it was being served in little cardboard boxes shaped like coffins. But Hardee's is hardly alone. Burger King's Double Whopper with cheese has more than 1,000 calories and more than 65 grams of fat, and Wendy's Classic Triple with cheese has 940 calories, with 56 grams of fat. If restaurants want to serve food like this, they should print the calories and fat content on the overhead menus. But consumers have to be responsible, too, and start making the mental connection between gargantuan fast-food burgers and fries and heart attacks and strokes. What is driving Hardee's is a simple fast-food formula: poor nutrition sells. The company says its sales have been up steadily since it introduced its Thickburger line last year. In its rollout of the Monster Thickburger, Hardee's has gamely played up the new burger's sheer excess with the ad slogan, "Be afraid. Be very afraid." It is a setback for public health, but a triumph for truth in advertising.
The FTC would have to understand technology to bring a claim of false advertising. I tend to doubt that anyone there does. The only way tech companies will change is if individuals' bring law suits. "To establish a violation under the Lanham Act, consumers and competitors must prove the following: (1) the advertiser made false statements of fact about its product; (2) the false advertisements actually deceived or had the capacity to deceive a substantial segment of the target population; (3) the deception was material; (4) the falsely advertised product was sold in interstate commerce; and (5) the party bringing the lawsuit (known as the "plaintiff") was injured as a result of the deception."
http://www.poznaklaw.com/articles/falsead.htm
Like you, I've learned to look out for this (as do most of my engineering friends). If I look at an Analog Devices spec sheet, I know everything is dead-on. I trust the part. If I'm looking at a TI part, I'm a little bit more sceptical. I know there may be hidden catches. As a result, I'm more likely to design with AD parts. If I do use TI parts, my first iteration design will only use ones that leave a margin for error.
Having to go through an extra design cycle looks bad on me. Whenever possible, I just avoid companies that doctor spec sheets. I don't know if it is illegal, but I'm pretty sure it's bad business.
The other thing that gets me is when companies don't post prices. I usually understand I can haggle down if I'm buying a bazillion of them, and if I order from Digikey, I'll pay a couple times as much. Still, it's a lot easier to do rough project budget (this is especially valuable in the proposal stage) with rough prices, and as a result, your parts are much more likely to get used.
Is it where you cook the test data???
Tag lost or not installed.
When it violates a law.
Tuning something for a test isn't illegal. Neither is non-disclosure of security bugs or bugs that could be escalated into security bugs in Software before selling to customers or the government.
But it is certainly "caveat emptor" in this day -- even things that are promised in large print (if you do x and y, then you get z). But then you will find that the completion of X and Y is not as simple as doing the tasks but you will find there are fine details about completion criteria for X and Y. Like some of the bigger rebates...have to mail in old CD's or UPC's from previous boxes to qualify for them even though you registered your previous version with them and they have you on file to make the "special upgrade offer" to you. They have the unique serial number from the previous edition, you can't sell it to someone else who would register it or the serial # would get caught. You can still dup'ed the disk if you really wanted to make a copy of the previous program for a friend (or the new one). It's just they make you keep all the boxes and manuals (which are usually online with the paper manuals and boxes gathering dust), and sending a CD through the mail isn't
a 13^h^h37 cent stamp either...
It's capitalism, low-market control, whatever you can get away with that isn't explicity illegal is allowed (though most certainly, not what one might
call ethical). But ethical? When it comes to a 'game' where the sole objective is to make money?
Where's the profit in having ethics? Are we really the progenitors of the federation or the real future will have us being the Ferengi.
*psycho future!*
-l
Benchmark "tweaking" (euphamism alert) happens like this. You set up the benchmark on your machine and run it, but you find out your competitor's machine gets a better score. You know your machine is faster so you figure out what your competitor did to cheat. Then you either do the same thing or you don't eat. I can remember a transaction processing benchmark with a 256 drive RAID 0 array used for the main database. Never mind that a 256 drive RAID 0 array is a reliability nightmare and totally impractical: The real reason for it was that it made it unrealitically expensive for small vendors to get good numbers on the benchmark.
If you are a buyer, you run your own benchmarks or depend on benchmarks run by reputable third parties.
If you can't do that, then it's probably better to look at the raw power of the machines...
An engineer who ran for Congress. http://herbrobinson.us
Nope, it is simple: you can choose whether or not to lie. If you doctor your product so it will look better in tests, seeming better at some function than you know it really is ... that's lying.
... you are lying.
You don't have to advertise your flaws; that's a different question. If you make medicine, or potentially dangerous products, you should -- or we'll eventually gang up on you because you didn't [see: Merck, Ford, etc].
You don't have to disclose flaws in your product, but when it's discovered that you hid those flaws, others will regard you as a liar.
If you actively try to *hide* those flows, or deceive people into believing there are no such flaws, then
I draw the line at ads for photo phones that say in small print "Images Simulated".
Why not also say "It can call anyone in the world for free!" and in small print add "Charges apply"?
Why not also say "It gives you next week's winning lottery numbers!" and in small print add "Cannot fortell the future".
If your product's USP is visual, do not "simulate" the visuals.
Ceci n'est pas un sig.
In one of our "performance" meetings the question came out - why can't we just yank the ISDN connection out from under the app and let it shut itself down... I mean it had to survive a cable unplug anyway - so there wouldn't be any MORE bugs exposed.
Long story short - we did the hack, and left a "compatability" flag in for dealing with conformance.
Moral of the story is that sometimes there is a reason for a hack - many times there aren't.
I have mod points and I am not afraid to use them