Slashdot Mirror
Microsoft Dismisses Consumer Reports' Surface Complaints, But Doesn't Offer Much Evidence
Microsoft has publicly responded to Consumer Reports, saying that it disagrees with the publication's Surface reliability findings. But the company hasn't offered much in the way of evidence. In a blog post, Surface chief Panos Panay writes: In the Surface team we track quality constantly, using metrics that include failure and return rates -- both our predicted 1-2-year failure and actual return rates for Surface Pro 4 and Surface Book are significantly lower than 25%. Additionally, we track other indicators of quality such as incidents per unit (IPU), which have improved from generation to generation and are now at record lows of well below 1%. Surface also ranks highly in customer satisfaction. 98% of Surface Pro 4 users and Surface Book users say they are satisfied with their device, and our Surface Laptop and new Surface Pro continue to get rave reviews. Long-time watcher Paul Thurrott writes: Does changing the time frame from "by the end of the second year of ownership" to "1-2 year failure rate" skew the results because more failures happen later in a product's lifetime? Also, he introduces the notion of "return rates" here. By definition, the feedback that Consumer Reports receives is from product owners, not those who have returned products. If someone is almost two years into device ownership, they are not returning the product. They're just using it. And dealing with it. So consider the issue muddled, in just one carefully-constructed sentence. Which I believe was crafted to confuse the issue. But there is more. "Additionally, we track other indicators of quality such as incidents per unit (IPU), which have improved from generation to generation and are now at record lows of well below 1 percent," Panay offers. It's not possible to understand how an "incident" relates to a "failure." Mostly because he doesn't explain the term. Likely because doing so would betray that this is an apples to oranges comparison. [...] I will point the reader to Welcome to Surfacegate, my description of Microsoft's feeble attempts to ignore and then slowly fix endemic issues with those exact two Surface models. And anecdotally, I'll point to the fact that the three Surface Book models I've used have all had reliability problems. But the biggest issue I have with "customer satisfaction" is that it's kind of a bullshit measurement when it comes to premium products.
Productivity up sharply, profits up 17% since the Surface transition with very few problems. I expect a nice bonus
Does changing the time frame from "by the end of the second year of ownership" to "1-2 year failure rate" skew the results because more failures happen later in a product's lifetime?
Failure rates typically follow a bathtub curve, where most manufacturing/design issues are found early in the product life, then age related failures start increasing late in the life cycle. In between is where failures are typically the lowest.
Is why 3 generations of this device can't properly go to sleep mode. Every other wake is either: device dead or wifi dead.
But of course, you can blame it to testing methodology...
sweet tender yummy BALLS!
I guess with the Surface, now that they have full control of the hardware, they can't blame anyone besides themselves for problems the user faces. All the drivers, hardware must have been fully tested by Microsoft's most capable QA team (what's left of it) before the Surface leaves the factory. So what could possibly go wrong?
We started with about eight Surface Pro 3's and being that I like to experiment with cost savings we got two of the Surface Non-Pro's with Atom processors inside to test for intern/assistant level stuff.
We have two of those Pro's still functioning and I think we have one of the Atom's still in use but I'm not sure where it is, I may have to dig up inventory records.
That being said we have MacBook Pro's that are four to five times the age of these oldest Surface in use still being used as well as Lenovo T400's, some Dell's that are just about as old and a couple of other clunkers hanging on in lesser roles. You have to see a Surface as a disposable system.
I have a 64GB Surface Pro 1 at home that's still works like a champ and with the I5 processor it still functions as a champion coffee shop cruiser. That being said I care about my equipment. My daughter has a Surface Pro 4 that having received it at Christmas time has already outlasted a lot of the 3's in hours of use we've had around here many times over.
I'm to say that a Surface is a good unit for people who give a shit about their equipment, but not in a company environment where replacement is someone else's problem. It's like buying a used rental car - which I have done. You just sort of expect to have to replace the tires and brakes even if it only has 20,000 miles on it. Buy a new Surface for yourself they're pretty awesome and the software problems that plagued them on Windows 8 seem to have gone away with WIndows 10, just don't waste your money in a corporate environment. *
*I have not tested the Surface laptops (attached keyboard types) or the all-in-one desktops
The preceding post was not a Slashvertisement.
The Surface Pro 4 and Surface Book were introduced in October 2015. Surface Pro 3 in June 2014.
Consumer Reports does their survey towards the beginning of their year. So "by the second year of ownership" means mostly Surface Pro 1 and 2 units. "1-2 year failure rate" means mostly Surface Pro 3 and 4 units.
Customer satisfaction surveys (usually the J.D. Power survey) tend to be skewed by:
It's also worth noting that Consumer Reports cited "start-up and freezing problems" as reasons for pulling their recommendation. Unless there's a hardware flaw (unlikely since the components are all standardized and work fine in millions of other computers), that points to problems with Microsoft's drivers. And wasn't addressed in Microsoft's reply.
I have been looking at the Surface for awhile. It looked OK.
Then I got one at work to replace my aging Dell Latitude. (Surface Pro 4)
The keyboard is junk. It's flimsy and moves when you type on it; the trackpad, which is required if you're using it as a computer (which I have to) is very narrow in depth and is difficult to use when compounded by the fact everything flexes when you use it.
Using this device on anything but a flat table or desk is right out.
Also, it's covered in Microfiber. WTF. This stuff attracts lint, dust, and pet hair like nobody's business and I shudder to think how grimy it's going to be in a year or so.
In short, Microsoft turned what looked like an OK tablet into a horrible computer.
That I'm stuck with.
Maybe for years.
Sigh.
..don't panic
covered in chocolate!
they smell so bad even hookers wretch
Not entirely. I suspect the majority of the issues can be traced to the Skylake CPUs in the Surface Pro 4. The issues with that generation CPU in mobile are pretty well known and it took a while for Intel to release drivers that either resolved, or worked around, the issues. Complicated further by Microsoft insisting on using modified versions of the Intel drivers, adding further delay.
Microsoft isn't totally without blame, but I would say it's like an 80/20 split between Intel and Microsoft respectively.
If they ain't salty I ain't sucking em.
The NFL has a contract with Microsoft to use their Surface tablets on the sidelines. They tout it as a way for the team to get almost real time information on play calling, whether offense or defense. Unfortunately, and directly related to this article, there are those, particularly coaches and quarterbacks, who bemoan the unpredictability of the Surface and its many malfunctions.
Even worse, people were initially calling them iPads or iPad-like which certainly didn't sit well with the marketing crew at Microsoft.
Microsoft can claim all they want their Surface is doing well, but from real world experiences, where timely information is invaluable, or in the case of Consumer Reports where Surface owners report the numerous problems they have, there is only so much spin which can be done to try and spackle over these poorly performing devices.
We will bankrupt ourselves in the vain search for absolute security. -- Dwight D. Eisenhower
I have an iPad.
I've had multiple Surface products starting at the first Pro model. They are very versatile and "the ultimate Windows experience" but that experience doesn't speak to much in terms of quality.
The operating system aside (Windows indexer breaking, several driver problems, basic functionality missing and no ability to use the Feedback Hub without turning on full telemetry, etc), the hardware is/was riffled with problems. The Broadcom multi-communications chip (WiFi/Bluetooth) is a mess. It's been revision after revision, firmware patched a million times, etc. It STILL has problems connecting to networks, staying connected, "daydreaming," almost unusable with IPv6 until really recently, etc.
The power/dock connector is REALLY finicky on the newer models (bumping it disconnects the dock and bounces all the devices), the original Pro 1/2 connector is garbage, the display port pass through is more of a "when it works" and "when it works properly." IE, 4K is almost impossible to get working and requires a dance of turning off devices, unplugging/replugging, rebooting, etc. The DPI scaling stuff is a nightmare (yeah it's the OS but it's part of this experience).
Ontop of all this, the hardware is glued together and nearly impossible to work on, not without breaking it. Even knowing this "going in," it's frustrating when it's out of warranty and now they want half the price of a base model device to look at it. Oh, the hardware is extremely fragile. Dropping from 1 ft will shatter the screen. I've had this happen multiple times to me and several colleagues. If you baby the device, you might be fine, except I know someone who's device shattered in between the office, sitting on a car seat and taking it out at home. I don't know that you can "win" here.
The warranty is possibly the worst aspect of this device. Without the extended warranty, you will never get any service for anything on the device. They will claim it's impossible to have any defects, manufacturing problems, etc. With the warranty, they will fight you tooth and nail to replace it, asking you to re-install Windows even when the display is shattered as if that would make it any better. After they agree to replace it, they'll randomly lose your order (as in they don't even have a tracked incident for you), they'll lose your return even though the tracking information shows it received and signed for, the exchange will go great and you have your replacement and months later they charge you the full replacement cost (even though the incident has been closed for a month or so).
Oh, and one last thing, I had a device out of warranty by 1 day. Yes, I know it was out of warranty by a day, but the manager in store (Portland) refused to not only look up the device's extended warranty (they couldn't find it in their system and I had to look it up for them), they completely refused to work with me in any way. They basically treated me like I hadn't bought the device from them at all and forced me to either have to pay full price for a replacement (that they said they couldn't do there in store) or buy another device. Great customer service and support all around.
I fully believe their low return rates... knowing how they get them that low, that's something everyone should be aware of before going into a Surface device. You'll get a hard sell on accepting your broken device, even while in the warranty period.
they can't blame anyone besides themselves for problems the user faces.
I guess that all that is left, then is to deny deny deny, and if someone finds errors, then their testing methods must be broken.
... I suspect the majority of the issues can be traced to the Skylake CPUs in the Surface Pro 4. The issues with that generation CPU in mobile are pretty well known...
Microsoft was the one who put those CPUs into the Surface. I didn't, you didn't and Intel didn't. So you seem to be saying that Microsoft knowingly put a CPU into the Surface that had known reliability issues, without doing any QA testing of those CPUs before using them. Wow, that sounds even worse than I had thought.
If I was Microsoft I would be less concerned about the Consumer Reports survey, and more concerned how many of those people surveyed felt compelled to give the Surface negative marks and how many of those people were spreading the word about their bad experiences.
Well I don't think they put their most capable QA on it. The most capable QA team has been there a long time, and it would be foolish to pull them off their project that they're experts on and put them on a brand new hardware platform ("uh, boss, you know we're software QA and not hardware, right?").
And QA doesn't matter if the management doesn't follow through and fix the bugs and put a premium on quality. Quality at Microsoft has always been an afterthought. Improving quality gets in the way of constantly adding new and useless features. When in history has any customer said "wow, this Microsoft product is high quality!" That's not saying the products are outright bad, they may be useful but customers have learned to live with the hiccups and lack of polish.
both our predicted 1-2-year failure and actual return rates for Surface Pro 4 and Surface Book are significantly lower than 25%.
Back when I was working developing industrial control products, we considered anything above a few percent (like less than 5%) to be "something's wrong here", prompting an evaluation of design, components (and software, if applicable), manufacturing processes, QC processes, etc.
Something as alarming as 25% would have likely caused the entire product to be at least temporarily pulled-from-sale immediately, and more-than-likely, never sold again. In fact, even the most-failure-prone product I was aware of at that place had a failure rate of around 10%, and it was basically cancelled and never mentioned again...
Oh, and BTW, I remind everyone that these products were for use in an INDUSTRIAL environment, which is a LOT harsher than what your average consumer can dish-out. By far.
... I suspect the majority of the issues can be traced to the Skylake CPUs in the Surface Pro 4. The issues with that generation CPU in mobile are pretty well known...
Microsoft was the one who put those CPUs into the Surface. I didn't, you didn't and Intel didn't. So you seem to be saying that Microsoft knowingly put a CPU into the Surface that had known reliability issues, without doing any QA testing of those CPUs before using them. Wow, that sounds even worse than I had thought.
Hmmm. Then why don't the Macs that use those CPUs have the same issues?
we've got 85 surface 2s at work - from 2013 - and we've only replaced one (the one we dropped on concrete)
i own five of them and the only problems i've had have been my own fault
i just don't believe this for a flat second
That's disastrous, assuming it's AFR.
Excluding disk drives, 1.5% to 2% is a decent number.
thegodmovie.com - watch it
Our experience hasn't reflected the Consumer Reports numbers. It has been far worse.
My SP3:
Replaced SP3 due to failure in the fan making a loud noise.
Replaced keyboard due to complete failure to type.
Replaced keyboard again (SP4 keyboard) due to mouse failing to click.
Power brick cable subject to a recall.
Replaced pen (SP4 pen) due to failing to pair.
Currently living with infuriating wakeup bug that results from having an SP4 TypeCover on an SP3 which Microsoft know, acknowledge is a driver issue and have failed to fix in 18 months.
My partner's SP4:
Replaced SP4 due to bootloop that even a re-image couldn't fix.
Replaced pen on day 1 as it was DoA.
Replaced pen when it refused to pair.
Power brick cable is frayed and starting to become a fire hazard.
Arc Touch mouse is failing to click properly (about to be replaced).
So far we've had a 100% failure rate on Surface Pro devices, and a near 100% failure rate on Surface Pro accessories. To MS's credit they were very forthcoming with RMAs and in my opinion out of all the failed devices I've sent back they have been by far the fastest with turnaround on RMA'd equipment.
I do love the device though and hope the quality gets better, but I'm not optimistic.
While annoying SW bugs generally get fixed. HW design weaknesses kind of stuck. The Surface has nifty features but appears fragile which should be understood by prospective buyers. I have a 5 yr old Vaio SVZ 1311AJ no issues but can't open to clean without risk of breaking due to very tight connections. Can clean moderately using pressurized air but over time dust collects inside so heats up more now. Would like to get a new device since overdue for a failure but SP4 pro seems risky.
I'm to say that a Surface is a good unit for people who give a shit about their equipment,
Life is too short to waste on equipment deliberately designed to not be robust.
Accidents happen and to not design equipment for such situations is just planned obsolescence and it sucks.