Slashdot Mirror
Samsung Caught Boosting Galaxy S4 Benchmarks
A recent forum post at Beyond3D made an interesting claim: that the Samsung Galaxy S4's GPU ran at 532 MHz for certain whitelisted benchmark applications, and at 480 MHz for everything else. The folks at AnandTech decided to investigate and found out that the phone does indeed let its GPU run at a higher frequency when particular benchmark software is running. They found a similar oddity with the CPU — it wasn't restricted for other apps, but it was forced to run at max speed during benchmarks. Then they decided to look for direct evidence that this was intentional.
"Poking around I came across the application changing the DVFS behavior to allow these frequency changes – TwDVFSApp.apk. Opening the file in a hex editor and looking at strings inside (or just running strings on the .odex file) pointed at what appeared to be hard coded profiles/exceptions for certain applications. The string 'BenchmarkBooster' is a particularly telling one. ... Quadrant standard, advanced, and professional, linpack (free, not paid), Benchmark Pi, and AnTuTu are all called out specifically. Nothing for GLBenchmark 2.5.1 though, despite its similar behavior."
When every sixth topic on Slashdot is about the evils and perils of Government Regulation, why are we constantly seeing examples of companies misleading, blatantly lying, to their customers? We need more teeth on consumer regulation. I bought my Samsung Galaxy S4 on certain assumptions of power. Remember Hyundai blatantly lying about their fuel numbers for half a decade? They were doled out a punishment, but the boost in sales due to in part by their chain-wide efficiency offset any net losses.
Slashdot readers will remember this, and probably choose an S4 when faced with so few choices. Samsung sees no benefit to not skewing numbers in the future.
Source, proof, evidence or STFU.
No, they don't. There is a difference between optimizing a system and overclocking just for specific benchmark apps. Samsung could get fraud charges on this one if they advertised or published the benchmarked speed. It is less obvious if they did not do the publishing themselves.
No, they don't.
Though I agree the burden of proof is on the accuser, that is a very definitive statement, how do you know for sure? We didn't know this about Samsung either before recently, and digging into this on other phones could reveal more. Without such information I agree with the other response to the OP though:
"Source, proof, evidence or STFU".
Sony has betrayed consumers more than almost any Tech company can name. They're universally hated across all spectrum of Slashdot users.
Yet they're largely poised to win the next "Console War" and they're still one of the premier names in the home entertainment business.
Companies have NOTHING to fear from consumer retaliation. Consumers are by and large stupid, with an extremely short term memory.
I remember old articles where ATI and Nvidia were both caught out gaming benchmarks, in one case by embedding particular benchmark game strings in their driver, and short cutting a few algorithms to boost their score.
My rights don't need management.
I disagree. Any self respecting nerd SHOULD buy one of these. Then publish an app to overclock it for every application.
"Freedom in the USA is not the ability to do what you want. It is the ability to stop others from doing what THEY want"
No different than how Samsung made tons of commercials poking fun of iPhone users. If you make a better product just show the product. If you make an inferior product then take cheap shots at the competition.
Yep. Apple would never make adverts poking fun at the competition...
No sig today...
There seems to be an official answer from Samsung here: http://samsungtomorrow.com/4676
It's in Korean, but here is the translation, provided by sammobile.com:
"Under ordinary conditions, the Galaxy S4 has been designed to allow a maximum GPU frequency of 533MHz. However, the maximum GPU frequency is lowered to 480MHz for certain gaming apps that may cause an overload, when they are used for a prolonged period of time in full-screen mode. Meanwhile, a maximum GPU frequency of 533MHz is applicable for running apps that are usually used in full-screen mode, such as the S Browser, Gallery, Camera, Video Player, and certain benchmarking apps, which also demand substantial performance.
The maximum GPU frequencies for the Galaxy S4 have been varied to provide optimal user experience for our customers, and were not intended to improve certain benchmark results.
We remain committed to providing our customers with the best possible user experience."
Here you go
http://news.cnet.com/8301-1001_3-57593426-92/debate-sparked-about-benchmark-for-intel-arm-chips/
Meh. That link basically says that there are different results from different benchmarks. It says that it's a "not uncommon assertion" that companies "have attempted in the past to "manipulate" benchmarks", but that's not the same as finding code that overclocks the chip specifically when it's running benchmark programs
http://www.geoffreylandis.com
Nobody mentioned Apple. You mentioned Apple.
Shows how far behind Samsung is in terms of hardware engineering. They stack the deck and still can't touch a 9 month old phone. Both browser performance and gaming performance, the 2 most stressful use cases on a smartphone, are way behind Apple.
http://images.anandtech.com/graphs/graph6914/54294.png
http://images.anandtech.com/graphs/graph6914/54296.png
http://images.anandtech.com/graphs/graph6914/54300.png
http://images.anandtech.com/graphs/graph6914/54298.png
http://images.anandtech.com/graphs/graph6914/54305.png
That's nice. It's still unethical and should be treated as such.
SJWs are the new boogeyman. -Me
I disagree. Any self respecting nerd SHOULD buy one of these. Then publish an app to overclock it for every application.
Why would you do that? All you'd end up with is a device with a shorter battery life that is more prone to overheating.
I don't think we need to celebrate benchmarking phones, period. This was one of those flamebait trolling things that happened in the PC era where people boasted how superficially fast their beloved shoebox was by putting $10k worth of equipment into and liquid cooling it just to get some high number result in 3D Mark or some other meaningless program.
We don't need this for phones.
Yes phones play games, yes phones are getting faster, but realize that phones and tablets are a HUGE step back from the PC era in terms of performance so benchmarking them means you may as well drag out your dusty Pentium era PC and start boasting about good its benchmark numbers are.
Also when 80% of the apps on the Android platform are unstable POS then I don't care about how fast they crash. Even Chrome quits unexpectedly repeatedly and this is by the company that makes the Android platform on their own Nexus brand devices.
I haven't thought of anything clever to put here, but then again most of you haven't either.
Samsung copy everything Apple do, so it is Apple's fault :P
They lose TWO INTERNETS today!
Don't make us take away another one.
So those Apple Commercials where the Apple spokesman is a "hip guy" while the "PC" guy is some nerd are not making fun of the customers?
Except they're not overclocking anything because the GPU is rated for 533mhz.
They're just making sure that even if the benchmark apps don't tell it to work in it's most high performance profile that it does, because the whole point in benchmarks is to give a benchmark of the optimal performance of a device.
The danger is that if they don't do this then the benchmark programs will give a misleading view of the performance capabilities of the device because they'll only be running it in the more power saving oriented default mode.
What's the alternative, they don't do this and shitty benchmark apps that take no advantage of the optimisation options for the device suggest it's not as powerful as it really is and so they get slated for it being underpowered even if that's not true?
Newsflash. Apple do exactly the same thing, as does every other manufacturer.
by Anonymous Coward on 7:15 31 July 2013
So yeah, they're already here and they were mentioning Apple 20 minutes before TimHunter..
0 1 - just my two bits
Nope. Those ads are making fun of PC guys.
You know this how? There's no way to tell if Apple does this or not since their OS is closed source..
I not sure that is boosting, normal processing may be 480mhz, because that is what they need, does you computer clock always runs at top of its resources?
This is just like steroids in sports. Shameful.
Views expressed do not necessarily reflect those of the author.
/Rant
Build a Man a Fire, and He'll Be Warm for a Day. Set a Man on Fire, and He'll Be Warm for the Rest of His Life.
How is it not true that it's underpowered if it only runs in power-saving mode for the user?
There are two types of people in the world: Those who crave closure
Wow. Did you even read the article. Or even the summary? They aren't doing this to ensure the device isn't running in power saving mode. The enhanced frequency is _ONLY_ available to benchmark tests. The code even refers to it as BenchmarkBooster. What do you possibly think BenchmarkBooster does?
Seriously, you are the personification of "fandroid" right now.
But the device doesn't do 533MHz for the GPU in any other use case, the top clock for the GPU is 480MHz.
This isn't "forcing the system into optimal mode for benchmarks so that power saving, etc, doesn't futz the result".
This is forcing the GPU into a state that never can be attained by any other software on the system.
Of course I'm not ignoring the fact that another Samsung device runs at 533MHz and this was a bad cut and paste job onto the new device!
It doesn't. Other applications can run at 533mhz, it's just that that's not usually necessary and hence preferable to save power.
I suppose the old car analogy now applies to phones too.
Excuse me, but please get off my Pennisetum Clandestinum, eh!
And "PC guys" are customers of IBM PC compatible brand. so yes, they made fun of the competitions customers.
Well, then you get a phone that fails much earlier, use the warranty and get another phone!
Of course, that one will probably require overclocking too...
Plus, you can use them when camping in the mountains as a sleeping bag warmer.
There are two types of people in the world: Those who crave closure
Provide an example of a thirdparty app running at 533MHz on the S4.
No kidding!!! What do you say at this point?
Does it?
No kidding!!! What do you say at this point?
I always thought the point of a benchmark is to show what a device is capable. So if the "phone does indeed let its GPU run at a higher frequency when particular benchmark software is running", Its just showing what its capable of. Though I would definitely want to run benchmarks the whole day and see what happens.
Some of us learn from other people's mistakes and the rest of us have to be other people. -- Zig Ziglar
It doesn't. Other applications can run at 533mhz...
No. They can't. Please try reading the article.
http://www.anandtech.com/show/7187/looking-at-cpugpu-benchmark-optimizations-galaxy-s-4
...all other apps/games were limited to 480MHz.
Other applications can NOT run at 533. The only applications that have access to that speed boost are benchmark apps.
The ONLY apps.
Please read the article before you continue spreading your misinformation.
Other applications being other benchmarks?
There are two types of people in the world: Those who crave closure
"No. They can't. Please try reading the article."
A wrong article is still wrong. Samsung themselves have quoted a number of their apps that run at 533mhz. You'll have to let other vendors tell us whether they use the same optimisations or not.
Realistically the biggest problem is that most people don't bother to optimise specifically for the GS4 because it's extra work for not really any worthwhile benefit (an extra 53mhz isn't going to change that much, especially at the expense of higher power consumption) so not many apps will run at the full 533mhz and I agree that's an issue, but it doesn't change the fact that 533mhz is the correct clock speed to benchmark the phones optimum capability at.
You are a liar who's spamming the hell out of this thread with your lies. The original article clearly states that 533MHz is not available for any other apps nor games - it's only available for benchmark tests.
Stop spreading lies.
That's nice. It's still unethical and should be treated as such.
They are no different than any other modern corporation, just assume everything they say or publish is a lie and go from there. You'll never be disappointed and sometimes (though rarely) you'll be pleasantly surprised when the advertized hype is somewhat close to the truth.
You are feeding a troll/flamebait.
Easy to start a flamewar pointing to Apple, while the facts of the story have to do with GPU which is specced at 533 MHz but running it at that speed causes overheating. Apple does not use a GPU clocked at 533 Mhz - it uses a 350 Mhz GPU which is of a similar architecture, but without Direct X.
Your unstated major premise is "what Samsung has told me is accurate". This is a mistake. Samsung's explanation is a rival hypothesis to Anandtech's. At the moment you have to compare the two hypotheses with the presented data. That data tends to favour Anandtech's explanation.
No kidding!!! What do you say at this point?
If no apps other then the benchmarker run at 533Mhz, then overclocking is a fair word even if the CPU is rated for 533Mhz. If even 20% of the apps were allowed to run at 533Mhz then I would say otherwise, but NONE of the other apps are allowed to run at 533Mhz, just the benchmarker.
It is fraud.
The alternative is to do a benchmark of the performance that the user will really get. What's the point knowing the potential of the phone when at the end of the day it is configured in such a way that you will never reach them.
Huh... First post said it...
Which ones?
No kidding!!! What do you say at this point?
Is a time old tradition. Started by Oracle.
25 years ago, Byte was notorious for forcing Mac benchmarks they used to be compiled with SANE turned on, to make PC benchmarks look a lot better. SANE was Apple's "Standard Apple Numerics Environment", a library which took the results from the Mac float processor and massaged the last few bits to ensure they were the same as the Apple II.
Recompiling that stuff in Lightspeed C without it gave very favorable results.
(-1: Post disagrees with my already-settled worldview) is not a valid mod option.
(This is an HTC device rather than Samsung) When I origionally stumbled on this thread I assumed yea yea cyanogen must be doing something wrong using a shit driver or not doing something quite right using a conspiracy theory as an excuse to blurt out a lazy response.
http://forum.cyanogenmod.com/topic/75172-why-is-cyanogenmod-htc-one-so-slow-6250-on-quadrant-standard-instead-12500/
I suspect that A benchmark manipulation is not limited to Samsung. B there is still something screwy going on in cyanogen and C we're going to find out pretty quickly what the truth is.
It's not overclocking, it's just that Samsung underclocks their phones to save battery and to stay within the specified thermal envelope.
Only the benchmarking apps run at full speed, because they're the only apps that need the full power of the phone at all times.
Other apps can't handle the full power of the Samsung ecosystem, thus Samsung protects them from the overwhelmingly high power coolness that is the Samsung platform.
So really, everything we do is in the best interest of our customer. We protect our customers from experiencing the full power of our phones to preserve their mental cohesiveness. Anything less would open a wormhole in the fabric of reality, and we wouldn't want that.
Maybe they just need to rename the benchmarking binary (I vote quack.exe)?
I jest, but only partly. You see, there is truly nothing new under the sun.
Quo usque tandem abutere, Nimbus, patientia nostra?
They're just making sure that even if the benchmark apps don't tell it to work in it's most high performance profile that it does
If I run a normal app (say, a game), does the GPU get this "most high performance profile" as well?
because the whole point in benchmarks is to give a benchmark of the optimal performance of a device.
The point of a benchmark is to give a benchmark of REALISTIC performance on a device, as a user would get under normal daily usage.
Slashdot social media options: AIM, ICQ, Yahoo, Jabber and Mobile Text. Why no MySpace?
"If I run a normal app (say, a game), does the GPU get this "most high performance profile" as well?"
Depends on the app, for some yes, for most no.
"The point of a benchmark is to give a benchmark of REALISTIC performance on a device, as a user would get under normal daily usage."
I think this is where the real problem is, there's no real consistent definition of what benchmarks are meant to represent and I agree with other posters in this thread that the best option is to stop caring about benchmarks at all and just focus on how well the device works in practice. If you mostly just use Samsung's apps that do run at 533mhz then the benchmarks would be realistic for you, if you use a bunch of 3rd party apps that only run at 480mhz then this benchmark wont be representative and that's the problem with figuring out what a benchmark even means in practice.
It started getting awkward enough with cores/multiple CPUs where the type of processing can be effected drastically by concurrency, but when you have specific optimisation options too then it just becomes too messy to extract any meaningful comparisons from benchmarks. Even years ago this could be a problem where processors could be compared directly with benchmark applications but some benchmark applications didn't check for and make use of SIMD optimisations when they first came about meaning the result simply weren't fair because the processors with SIMD instructions weren't being used to their greatest extent. It's only got worse since then as we have ever more optional optimisations.
All the ones Samsung listed for a start:
S Browser, Gallery, Camera, Video Player
If the chips can benchmark at a speed but then de-clocked to be released to the public what is wrong with it? Every car maker been doing this for years when it come to Torque and Horse Power. The TQ and HP number are higher then if a person would take his/her to a dyno. But no one screams.
The other way to look at it... I can savelly overclock this phone to x speed.
...that's it?
No kidding!!! What do you say at this point?
So those Apple Commercials where the Apple spokesman is a "hip guy" while the "PC" guy is some nerd are not making fun of the customers?
One said "I'm a Mac" and the other said "I'm a PC". Not "I'm a Mac customer" and "I'm a PC customer".
Wow. My comment above asking for evidence has gone from +something to -1, off topic and back to +1, Insightful.
The comment rating system doesn't work anymore on /.
I've observed that quite often (almost always) in the last months. Say something against Google or Samsung or just be not a cheering fanboy and paid shills and fanboys vote it into oblivion. No matter how factual or objective. Doesn't happen to that extent with other companies.
I'm seriously depressed about the state of /.
It's not a nerd fest any longer. It's a battle ground between companies.
Android Army vs the Apple Cult.
Day after day. It's exhausting boring, annoying.
PLEASE : no more Samsung, Google or Apple news for a whole month. That would be refreshing.
And you believe them? Why?
Samsung isn't trustworthy.
Steroids in sports actually improve your performance. Speeding up benchmarks only affects the benchmark, not actual device performance.
I agree with other posters in this thread that the best option is to stop caring about benchmarks at all and just focus on how well the device works in practice.
Exactly. Now we just need to devise some way to uniformly measure how well the device works in practice, distilling that experience down to a single number, where larger numbers are better.
People who say "sheeple" have about as much sophistication as an AOL user, and in fact are probably actually AOL users.
If *anything* on /. 'seriously depresses' you, you need to get out more ...
Yes it does.
Sorry, teleporters just kill you and then make a copy. A perfect, soul-less copy.
PC guys are ... PC guys. What's not to make fun of?
Your unstated major premise is "what Samsung has told me is accurate". This is a mistake. Samsung's explanation is a rival hypothesis to Anandtech's. At the moment you have to compare the two hypotheses with the presented data. That data tends to favour Anandtech's explanation.
Hypothesis implies something that is not known. Samsung knows the clock speed on their phone, and seeing as you have read the article you surely know that this is verified when you run the benchmark tests. For the other side of the article: http://www.telegraph.co.uk/technology/samsung/10213672/Samsung-deny-performance-boosting-hardware-in-Galaxy-S4.html
Disagreeing with you does not make me a troll.
An article cannot be wrong?
S browser, gallery, camera, video player all run at 533mhz
Others can as well if they need to. It's just how many developers out there are going to optimize for one phone?
Just because you don't want to do any research your self because you lazy and would prefer to insult people does not mean that anyone else lied. It just means that you took someones word without verifying it.
Now apologize the the GP for calling them a liar.
Sorry, teleporters just kill you and then make a copy. A perfect, soul-less copy.
Because they are not going to hand you evidence of fraud.
Sorry, teleporters just kill you and then make a copy. A perfect, soul-less copy.
Why would others optimize their apps for one phone? I can see that Samsung would because those apps are designed for that phone.
So instead of being obtuse how about using a little common sense.
Sorry, teleporters just kill you and then make a copy. A perfect, soul-less copy.
Ahh, but you would have control! :)
"Freedom in the USA is not the ability to do what you want. It is the ability to stop others from doing what THEY want"
... buys a mobile phone based on benchmark scores anyway? It's a god damn phone, not a gaming rig, server, graphics workstation, etc.
Seems to me that it works well. After some up and down while moderators battle about what would be the best option, it has come out as somewhat insightful.
If you've been on slashdot as long as you claim, you must also have seen all those newbies whining as soon as their comment is down-modded, not realizing that there are actually more than two moderators reading, and that the score will vary during the course of a day or two. Don't be one of them.
c++;
You are a liar who's spamming the hell out of this thread with your lies. The original article clearly states that 533MHz is not available for any other apps nor games - it's only available for benchmark tests.
Stop spreading lies.
http://www.telegraph.co.uk/technology/samsung/10213672/Samsung-deny-performance-boosting-hardware-in-Galaxy-S4.html The original article is wrong.
Disagreeing with you does not make me a troll.
"Making fun" of PC's? Yes. Making fun of customers, no. "I'm a Mac" and "I'm a PC".
I'm really a low 5-digit Slashdotter, but this ID is where I am now.
In a word, yes. If the "optimisation options" for all other operations involve capping performance to save battery life/operating temperature/stability/whatever, then the benchmark should reflect the performance actually available to users.
The foaming-at-the-mouth morons would be all over this as a hate crime against humanity had it been Apple doing this. Samsung has a long and distinctly unvaried history of shady tactics and scams like this that tends to be quietly ignored for the most part. Both companies have their fanboys and their antifanboys that should be ignored, but this discussion is surprising for the turned tables in "benchmarking doesn't matter" versus every other mobile phone flamewar talking up benchmarking and performance numbers.
You know this how? There's no way to tell if Apple does this or not since their OS is closed source..
C'mon don't be so hard on poor Samsung, they copied Apple's devices, they copied the look of Apple's mobile OS and now they are seeking to copy Apple's weaselyness.
Only to idiots, are orders laws.
-- Henning von Tresckow
Nothing new here. Move along.
now we need to go OSS in diesel cars
prove it.
It's not an argument. "Regulation" is a code word for power. Either the government holds this power, or private interests hold this power. There's no middle ground, due to the convexity of the slippery slope. It's either a firewall configured with a default "block all" or a default "pass all". Those are your two choices, 100% mutually exclusive.
Besides, inhabiting the middle ground involves the tedious art of knowing the difference, which is not what people with power enjoy doing.
It is not cheating - benchmark shows you how fast it runs on a particular device.
They've caught out NVidia AND AMD (At that time ATI) doing some of the same sorts of hyjinx with PC benchmarks in years prior.
Having said this, they paid DEARLY for having done the deeds in question. Can we say the same thing from Samsung?
Maybe they just need to rename the benchmarking binary (I vote quack.exe)?
I jest, but only partly. You see, there is truly nothing new under the sun.
This.
Does anyone honestly think anyone is not optimising their product for the products favourite benchmarking programs? Does anyone honestly think they aren't using every dirty trick in the book to get a few points more?
I have no trouble believing that Samsung has optimised their phones for benchmarking, the same as I have no problem believing that Apple optimises their phones for benchmarking (Apple would be even worse, blocking benchmark applications that dont give them favourable results). The only difference is that someone is calling out Samsung.
ATI, Nvidia, Intel, IBM, Apple, HTC... Everyone is doing it because you (the general you, no the parent) have made these benchmarks important to decision making.
Also it's trivial to over clock an android phone. So an easy test for this would be to get an SGS4 and sert it to 480 MHz, test it then OC it to 532 MHz, test it again and compare both results to a non OC'd SGS4.
Calling someone a "hater" only means you can not rationally rebut their argument.
The point of a benchmark is to give a benchmark of REALISTIC performance on a device, as a user would get under normal daily usage.
Is it? In my mind, the point of a benchmark is to test the maximum performance of the device. If I wanted to measure normal daily usage, I'd test it under, y'know, normal daily usage conditions. Which for most people is checking email and playing Angry Birds on Power Saver mode, in which case I doubt the CPU and GPU would hit half of either of their (claimed or otherwise) max clock speeds. So a benchmark should only test the CPU clock speed for low-demand applications? Sure, whatever you say.
It depends on whether S4 allows third party apps to run at 533MHz at all.
If yes - this benchmark makes sense, if no - I'd prefer to have numbers for 480MHz that would let me know what to expect from games running on my phone.
Apps that they mention in their reply as running at 533 too, like Gallery and Video, are, y'know, "normal daily usage" far from needing maximum performance. If top speed's limited to those, the benchmark is useless.
Running browser, gallery, and camera at 533mhz? They really need to tweak their codes.
The big problem is that while the cpu might hit 533Mhz, the entire phone might not. For example, the CPU package might not have enough thermal draw to maintain 533Mhz, having to run at 400Mhz for periods of time to prevent overheating.
If the "run policy" is to switch between 533 and 400 based on some arbitrary temperature based (or time slice based) policy, no big deal. If the switch is "benchmark programs run fast, everything else is slow", then we know that they are looking for performance benchmarks which do not actually reflect the preformance of the device, which is fraud.