Slashdot Mirror
Consumer Reports Updates Its MacBook Pro Review (consumerreports.org)
Reader TheFakeTimCook writes: Last month, the new MacBook Pro failed to receive a purchase recommendation from Consumer Reports due to battery life issues that it encountered during testing. Apple subsequently said it was working with Consumer Reports to understand the results, which it said do not match its "extensive lab tests or field data." According to an article from Consumer Reports, Apple has since concluded its work, and says it learned that Consumer Reports was using a "hidden Safari setting" which triggered an "obscure and intermittent bug" that led to inconsistent battery life results. With "normal user settings" enabled, Apple said Consumer Reports "consistently" achieved expected battery life. Apple stated: "We learned that when testing battery life on Mac notebooks, Consumer Reports uses a hidden Safari setting for developing web sites which turns off the browser cache. This is not a setting used by customers and does not reflect real-world usage. Their use of this developer setting also triggered an obscure and intermittent bug reloading icons which created inconsistent results in their lab. After we asked Consumer Reports to run the same test using normal user settings, they told us their MacBook Pro systems consistently delivered the expected battery life." Apple said it has fixed the Safari bug in the latest macOS Sierra beta seeded to developers and public testers this week.
Why was this setting enabled in the first place? It sounds like the setting in question isn't on by default, nor should it be. Why would Consumer Reports have enabled it for their testing? Unfortunately, it will certainly be a bigger deal that Consumer Reports did not recommend the MacBook Pro than it will be that the recommendation has been amended. The damage may be done in this case, but should Consumer Reports be trusted going forward? It raises the question of what other unusual things may be done in the testing of other products, and if that might affect the quality of the recommendations. Perhaps it's an isolated case, but it's confusing as to why this occurred at all.
So that's what you need to do to get Apple to fix your bug. A couple years I found a bug in their version of sqlite3, it stopped accepting international characters (chinese, japanese, specifically). I tried to submit a bug via their bug reporting console, and I got this error message. So I sent an email to the address listed there, explaining the situation, and I got a response,
"Please report that through our bug console." The console was still broken.
"First they came for the slanderers and i said nothing."