Slashdot Mirror
Huawei Caught Cheating Performance Test For New Phones (techcrunch.com)
An anonymous reader quotes a report from TechCrunch: UL, the company behind the tablet and phone performance benchmark app 3DMark, has delisted new Huawei phones from its "Best Smartphone" leaderboard after AnandTech discovered the phone maker was boosting its performance to ace the app's test. The phones delisted were the P20, P20 Pro, Nova 3 and the Honor Play. "After testing the devices in our own lab and confirming that they breach our rules, we have decided to delist the affected models and remove them from our performance rankings," the company said in a statement.
For the Huawei case, the rules are actually a little fuzzy. Phones are permitted to adjust performance based on workload, which results in peaks or dips in performance for different apps, but they are not permitted to hard-code peaks in performance specifically for the benchmark app. Huawei reportedly claimed that the peak in performance seen during the run of the benchmark app was an intuitive jump determined by AI; however, when an unlabeled version of the benchmark test was run, the phones were unable to recognize it and, as a result, displayed lower performances. In other words, the phones aren't so smart after all.
For the Huawei case, the rules are actually a little fuzzy. Phones are permitted to adjust performance based on workload, which results in peaks or dips in performance for different apps, but they are not permitted to hard-code peaks in performance specifically for the benchmark app. Huawei reportedly claimed that the peak in performance seen during the run of the benchmark app was an intuitive jump determined by AI; however, when an unlabeled version of the benchmark test was run, the phones were unable to recognize it and, as a result, displayed lower performances. In other words, the phones aren't so smart after all.
You DO know Apple phones are made in China, right?
Cheating and spying are no way to go through life, son.
"Huawei reportedly claimed that the peak in performance seen during the run of the benchmark app was an intuitive jump determined by AI"
Whenever someone makes an AI claim you know they are lying.
If you know what chipset is in a phone, then you know as much as you usefully need to know about the real world performance of that phone relative to other phones with the same chipset, i.e. pretty much the same. If you see a benchmark saying one such phone is outperforming its peers, then you assume that, if it's not down to actual cheating, then it's a quirk of the benchmark, because you know the same cpu/gpu at the same speed gives the same performance. Does anybody actually look at benchmarks when buying a phone?
There is no such thing as an American-made phone. In fact, there is almost no American-made anything when it comes to electronics. Prior administrations have allowed the manufacturing sector to move almost everything to enemy states.
Make no mistake. If there were ever a war, the US would lose, and quickly, because we cannot maintain, repair, or build anything without China's help. We have nearly zero electronics manufacturing capability compared to what would be needed to prosecute a War.
The U.S. still does a lot of chip design and fabrication. It's just that the chips get shipped overseas for packaging and final product assembly.
Also, U.S. manufacturing output has been increasing year over year even though we were moving large chunks of it overseas. I think now would be a good time to start reinvesting in local manufacturing, but that would be done with machines. A lot of the jobs aren't coming back, but that's okay because it means that labor is free to do something that's more productive instead.
Stick with Apple. Buy American, support American companies and the Americans they employ, and put your money where your mouth is.
Don't you mean "Buy Chinese?" Apple phones are made in China, not America.
Alzheimer on devices - here we come.
If builders built buildings the way programmers wrote programs, then the first woodpecker would destroy civilization.
Could you remind me, when did the US stop being a foreign country?
Comment removed based on user account deletion
So now we know where ex-Volkswagen engineers end up.
Stick with Apple. Buy American, support American companies and the Americans they employ, and put your money where your mouth is.
Where do you think Apple products are made?
There are very few products that are "American made" and you can say the same thing about every other country too. Furthermore brands are useless in determining where something was made. I have a truck that is Honda branded but it was the most American made vehicle in 2017 (yes more than the Ford F150. But the content in it is from all over the globe. The concept of "American products" or "Chinese products" is largely a myth for any non trivial product. Manufacturing these days is a global enterprise and few products have all their content (including labor) from a single country.
Anyway if you want people to "buy American" then American companies need to make products people want to buy at competitive prices. I'm not going to buy something inferior and/or more expensive merely because it was made in the USA. American companies have to compete for my business just like everyone else. If my fellow Americans are as clever as we like to pretend then that shouldn't be a problem but the other 95% of the world's population has a lot of smart and hard working people too.
No, not all car brands. VW/Audi/Porsche all had variations on the diesel emissions cheat. They're in a class by themselves.
https://en.wikipedia.org/wiki/...
Organization? You must be joking..
Quake/Quack3 was 17 years ago and people still haven't learnt.
Sucking gold songs?
Also I'm Swede. I'm only allowed to buy political correctness and progressivism?
Guess that's in line with the parliament. They think people talking bad about the situation of the country are traitors. Preferably forbidden.
Just build a wall against them. Call it the Chinese wall to make a statement.
Standing in a soup line is much more productive.
Oh, no, wait,I forgot.... we're all supposed to re-tool for the new economy. Because businesses want to hire 40 year olds for entry level tech jobs....
"Prediction: within 10 years, Windows will be a Linux distribution." Me, 7-6-2016
Yep, those BRAND X phones are totally made in america.
"Prediction: within 10 years, Windows will be a Linux distribution." Me, 7-6-2016
Who wouldve thought the company that gives approval rights for their designs to the Chinese government would lie to us? But for sure there's no possibilty of any backdoors. Trust us!
You DO know Apple phones are made in China, right?
Name 1 that is manufactured in the USA.
Apple has an American design team, but outsources all production and is incredibly expensive....
It be nice to see more American production, and maybe some mid grade stuff at a lower price point....
Samsung's flagship phones are every bit as expensive as Apple's flagship phone.
Now where's your Hate against Samsung?
Me Chinese. Me play joke. Me go peepee in your Coke.
- Huawei CEO
The U.S. still does a lot of chip design and fabrication. It's just that the chips get shipped overseas for packaging and final product assembly.
Also, U.S. manufacturing output has been increasing year over year even though we were moving large chunks of it overseas. I think now would be a good time to start reinvesting in local manufacturing, but that would be done with machines. A lot of the jobs aren't coming back, but that's okay because it means that labor is free to do something that's more productive instead.
Ya know, for all the Apple-bashing around here, they have actually been doing manufacturing/final-assembly of at least one of their products here since 2013: The (often-maligned) Mac Pro.
Granted, it's not the highest-volume product Apple sells; but even if it represents just 1% of Apple's nearly 20 million Mac units sold in 2017, that still represents a quantity of nearly 2 million units of $2k-5k Mac Pros per year (for an average yearly income of $5 BEELION in gross sales), which is a production rate and income that many a company would die-for. Not so bad for a "failed" product!
And they are ALL at least Assembled in U.S.A
https://www.macrumors.com/2014... ...and Apple is at least TRYING to make even more products in the U.S.A.:
https://venturebeat.com/2018/0...
At least many Samsung phones are made in South Korea
But is this what GPU manufacturers have been doing for years?
Yes, it was wrong for them and it's wrong for Huawei now. Anandtech, HardOCP, Tom's, and others caught GPU manufacturers doing this and called them out just like now. Part of the reason for Anandtech's development of their own test suite was because they didn't completely trust that vendors weren't cheating the known industry benchmarks.
So far as I know, the above sites are still looking for cheating and GPU manufacturers have stopped. They've at least stopped obviously cheating by looking for when a known benchmark is running like Huawei got caught doing. They may still be tuning for benchmarks, and not real-world performance, but that's why real-world tests like Anandtech and HardOCP do are still useful for me.
Then they would have not caught this current bullshit...
-- You are in a maze of little, twisty passages, all different... --
Yeah, this is my thought too. Except they should run it in both modes, so they can easily detect a discrepancy and hence a cheating attempt.
They should be running the benchmark at least 2 or 3 times and averaging the results anyway, and so 1 or 2 extra runs with the unlabeled version, and see if the performance is out of line with the standard average, would not be increasing the overall test duration by much.
At least many Samsung phones are made in South Korea
Since the OP was talking about AMERICAN production, that is a Strawman argument.
Well he was talking about foreign production. The USA is as foreign to me as South Korea as I live in either.
But unlike China, South Korea actually has decent wages, which I thought was the point.
precisely - now we get the rash of oops - our benchmark was the result of cheating...we'll find the name of the engineer who was responsible...and force him to use our phone...
nothing to see here - move along