Slashdot Mirror
Huawei Caught Cheating Performance Test For New Phones (techcrunch.com)
An anonymous reader quotes a report from TechCrunch: UL, the company behind the tablet and phone performance benchmark app 3DMark, has delisted new Huawei phones from its "Best Smartphone" leaderboard after AnandTech discovered the phone maker was boosting its performance to ace the app's test. The phones delisted were the P20, P20 Pro, Nova 3 and the Honor Play. "After testing the devices in our own lab and confirming that they breach our rules, we have decided to delist the affected models and remove them from our performance rankings," the company said in a statement.
For the Huawei case, the rules are actually a little fuzzy. Phones are permitted to adjust performance based on workload, which results in peaks or dips in performance for different apps, but they are not permitted to hard-code peaks in performance specifically for the benchmark app. Huawei reportedly claimed that the peak in performance seen during the run of the benchmark app was an intuitive jump determined by AI; however, when an unlabeled version of the benchmark test was run, the phones were unable to recognize it and, as a result, displayed lower performances. In other words, the phones aren't so smart after all.
For the Huawei case, the rules are actually a little fuzzy. Phones are permitted to adjust performance based on workload, which results in peaks or dips in performance for different apps, but they are not permitted to hard-code peaks in performance specifically for the benchmark app. Huawei reportedly claimed that the peak in performance seen during the run of the benchmark app was an intuitive jump determined by AI; however, when an unlabeled version of the benchmark test was run, the phones were unable to recognize it and, as a result, displayed lower performances. In other words, the phones aren't so smart after all.
But how will you have electronics with no high performance chips?
Where will they get their memory?
You DO know Apple phones are made in China, right?
Apple has an American design team, but outsources all production and is incredibly expensive....
It be nice to see more American production, and maybe some mid grade stuff at a lower price point....
Cheating and spying are no way to go through life, son.
"Huawei reportedly claimed that the peak in performance seen during the run of the benchmark app was an intuitive jump determined by AI"
Whenever someone makes an AI claim you know they are lying.
but we don't see anyone "CAUGHT CHEATING" unless they're from China or South Korea, then it's to be buried quickly. Go figure.
No thanks; Fuck apple.
If you know what chipset is in a phone, then you know as much as you usefully need to know about the real world performance of that phone relative to other phones with the same chipset, i.e. pretty much the same. If you see a benchmark saying one such phone is outperforming its peers, then you assume that, if it's not down to actual cheating, then it's a quirk of the benchmark, because you know the same cpu/gpu at the same speed gives the same performance. Does anybody actually look at benchmarks when buying a phone?
There is no such thing as an American-made phone. In fact, there is almost no American-made anything when it comes to electronics. Prior administrations have allowed the manufacturing sector to move almost everything to enemy states.
Make no mistake. If there were ever a war, the US would lose, and quickly, because we cannot maintain, repair, or build anything without China's help. We have nearly zero electronics manufacturing capability compared to what would be needed to prosecute a War.
The U.S. still does a lot of chip design and fabrication. It's just that the chips get shipped overseas for packaging and final product assembly.
Also, U.S. manufacturing output has been increasing year over year even though we were moving large chunks of it overseas. I think now would be a good time to start reinvesting in local manufacturing, but that would be done with machines. A lot of the jobs aren't coming back, but that's okay because it means that labor is free to do something that's more productive instead.
Stick with Apple. Buy American, support American companies and the Americans they employ, and put your money where your mouth is.
Don't you mean "Buy Chinese?" Apple phones are made in China, not America.
Did you mark this as "Insightful" with your main account? You realize that the overwhelming majority of modern electronics are actually manufactured in China, because they are better at it than anyone else? Your iPhone? Yeah, entirely built in China.
Alzheimer on devices - here we come.
If builders built buildings the way programmers wrote programs, then the first woodpecker would destroy civilization.
Could you remind me, when did the US stop being a foreign country?
Comment removed based on user account deletion
So now we know where ex-Volkswagen engineers end up.
Stick with Apple. Buy American, support American companies and the Americans they employ, and put your money where your mouth is.
Where do you think Apple products are made?
There are very few products that are "American made" and you can say the same thing about every other country too. Furthermore brands are useless in determining where something was made. I have a truck that is Honda branded but it was the most American made vehicle in 2017 (yes more than the Ford F150. But the content in it is from all over the globe. The concept of "American products" or "Chinese products" is largely a myth for any non trivial product. Manufacturing these days is a global enterprise and few products have all their content (including labor) from a single country.
Anyway if you want people to "buy American" then American companies need to make products people want to buy at competitive prices. I'm not going to buy something inferior and/or more expensive merely because it was made in the USA. American companies have to compete for my business just like everyone else. If my fellow Americans are as clever as we like to pretend then that shouldn't be a problem but the other 95% of the world's population has a lot of smart and hard working people too.
Stick with Apple. Buy American, support American companies and the Americans they employ, and put your money where your mouth is.
Don't you mean "Buy Chinese?" Apple phones are made in China, not America.
Good thing then that all the Android phones are made in America.
It doesn't matter. Americans are too fat and lazy to fight a war, and most of them would rather be communists or socialists anyway because they're too stupid to know why those things are terrible.
Fat, lazy, and stupid. Americans deserve what they get.
They thought that they were supposed to build the phone based on the benchmark.
-1 false dichotomy No one needs a smart phone. There are non-apple non-android options for anyone's actual needs.
Dummy apple phones made in china and rip you off dollar for dollar when it comes to hardware... ðY'
Quake/Quack3 was 17 years ago and people still haven't learnt.
Sucking gold songs?
Also I'm Swede. I'm only allowed to buy political correctness and progressivism?
Guess that's in line with the parliament. They think people talking bad about the situation of the country are traitors. Preferably forbidden.
Dongs not songs. Damn autocorrect.
Just build a wall against them. Call it the Chinese wall to make a statement.
Fixed that for you.
Standing in a soup line is much more productive.
Oh, no, wait,I forgot.... we're all supposed to re-tool for the new economy. Because businesses want to hire 40 year olds for entry level tech jobs....
"Prediction: within 10 years, Windows will be a Linux distribution." Me, 7-6-2016
Yep, those BRAND X phones are totally made in america.
"Prediction: within 10 years, Windows will be a Linux distribution." Me, 7-6-2016
Who wouldve thought the company that gives approval rights for their designs to the Chinese government would lie to us? But for sure there's no possibilty of any backdoors. Trust us!
You DO know Apple phones are made in China, right?
Name 1 that is manufactured in the USA.
Apple has an American design team, but outsources all production and is incredibly expensive....
It be nice to see more American production, and maybe some mid grade stuff at a lower price point....
Samsung's flagship phones are every bit as expensive as Apple's flagship phone.
Now where's your Hate against Samsung?
For an American company that exercises oversight and QA.
This isn't the first time benchmarking has been gamed by executable recognition. And it is always discovered by changing the benchmarking executable's name and noticing a pronounced difference in performance.
Why don't they just randomize the benchmarking software's name BY DEFAULT so it can never be recognized?
Me Chinese. Me play joke. Me go peepee in your Coke.
- Huawei CEO
Great the fucking apple hate police is here.
FUCK OFF.
People can hate apple if they want and are not accountable to you or the other mindless apple shills.
The U.S. still does a lot of chip design and fabrication. It's just that the chips get shipped overseas for packaging and final product assembly.
Also, U.S. manufacturing output has been increasing year over year even though we were moving large chunks of it overseas. I think now would be a good time to start reinvesting in local manufacturing, but that would be done with machines. A lot of the jobs aren't coming back, but that's okay because it means that labor is free to do something that's more productive instead.
Ya know, for all the Apple-bashing around here, they have actually been doing manufacturing/final-assembly of at least one of their products here since 2013: The (often-maligned) Mac Pro.
Granted, it's not the highest-volume product Apple sells; but even if it represents just 1% of Apple's nearly 20 million Mac units sold in 2017, that still represents a quantity of nearly 2 million units of $2k-5k Mac Pros per year (for an average yearly income of $5 BEELION in gross sales), which is a production rate and income that many a company would die-for. Not so bad for a "failed" product!
And they are ALL at least Assembled in U.S.A
https://www.macrumors.com/2014... ...and Apple is at least TRYING to make even more products in the U.S.A.:
https://venturebeat.com/2018/0...
At least many Samsung phones are made in South Korea
But is this what GPU manufacturers have been doing for years?
Who gives a flying shit about apple?
Yes, it was wrong for them and it's wrong for Huawei now. Anandtech, HardOCP, Tom's, and others caught GPU manufacturers doing this and called them out just like now. Part of the reason for Anandtech's development of their own test suite was because they didn't completely trust that vendors weren't cheating the known industry benchmarks.
So far as I know, the above sites are still looking for cheating and GPU manufacturers have stopped. They've at least stopped obviously cheating by looking for when a known benchmark is running like Huawei got caught doing. They may still be tuning for benchmarks, and not real-world performance, but that's why real-world tests like Anandtech and HardOCP do are still useful for me.
Samsung's flagship phones are every bit as expensive as Apple's flagship phone.
Now where's your Hate against Samsung?
Well, maybe he is one of those buying cheaper Huaweis instead of Samsungs, making Samsung's market share drop like birds dying in flight. Better hope for him he din't also buy the Huawei for it's performance in 3DMark.
There is no such thing as an American-made phone. In fact, there is almost no American-made anything when it comes to electronics. Prior administrations have allowed the manufacturing sector to move almost everything to enemy states.
Sounds like you are a big fan of complete government control on private enterprise.
At least many Samsung phones are made in South Korea
Since the OP was talking about AMERICAN production, that is a Strawman argument.
Well he was talking about foreign production. The USA is as foreign to me as South Korea as I live in either.
But unlike China, South Korea actually has decent wages, which I thought was the point.
Fucking Idiot
People need to remember what a phone is for...MAKING AND RECEIVING PHONE CALLS!!! Any thing else that can be done with the phone is secondary, and of far far less importance! Far too many people these days have what I call phone-itis. They seem to just have to be playing games, watching videos, texting, or wasting time on "social media" every waking moment! God forbid that their phone should battery should go dead!! They would think that their life was over if they had to go back to a flip phone with no Internet access!
Being 58, I can remember a time before cell phones, when a phone was attached to a wall or had a cord that was attached to a wall. And we all got along just fine! I really never liked phones, but they were and are a necessity. Why? Because they allow people to interrupt you at the absolute most inappropriate and inconvenient times!
But unlike China, South Korea actually has decent wages, which I thought was the point.
Doesn't help much when Samsung poisons their workers. https://www.digitaltrends.com/...