Witness Ridicules 'Hands-On' Reviews of Surface

Freshly Exhumed writes "Danny Sullivan over at Marketing Land has been tipped over the edge by various colleagues: 'After seeing yet another "hands-on" review of the Microsoft Surface tablet, I thought it would be interesting to shed more light on what exactly the journalists who assembled in Hollywood this week for the Surface launch event actually got to do with the tablets. In short, not a lot. Come along as I explain the hands-off reality of what I saw.' In response to Sullivan's criticisms, TechRadar contributor Mary To Many rebuts that merely touching something that does not operate nor even truly exist equates to an actual hands-on review. So, what do Slashdotters expect a "hands-on" review to reveal and/or include?"

Comments

"Reviews"

[tsa]

Most "Reviews" I see on the 'net are just summaries of what you find in the product folder, nothing more. So what's new about this?

Re:"Reviews"

[fuzzyfuzzyfungus]

It's probably going to get even worse now that so much of the new hotness is in functionally embedded systems that ship tied to hardware...

With software, the vendor essentially faces the choice of keeping it under wraps, or of having pre-release builds leaking all over the damn place. Maybe if it's just a handful of partner companies with some seriously mean NDAs; but once it gets to the journalist level you can forget about it(analogous to the Oscar screeners that get leaked every damn time, despite being subject to just about everything short of armed guard...)

Hardware, though, with the exception of the occasional unit that...um...goes missing in a bar, is easier to keep a tight grip on. Plus, very-late-prototype hardware, in limited quantities, is inevitably available to the vendor a modest period before retail hardware is available. That sort of thing is perfect for rewarding, or punishing, media outlets. Play ball, and you'll be seeing a sample unit in the mail in time to have your benchmarks and review done before the other guys get back from Best Buy. Don't, and you can read your competitor's reviews while you wait in line to buy something to review...

This sort of thing has gone on for years in the gaming-enthusiast-wanking sector of CPU, GPU, and motherboard reviews; but it hasn't really been all that relevant. Enthusiasts care a lot; but there just aren't a lot of enthusiasts, so it was mostly just sideshow, and the truth usually managed to surface by the time whatever the product was trickled down to the '80% of the speed, 20% of the price' market where people actually buy things.

Now that the 'mainstream' devices are tightly embedded hardware units, though, there will likely be a lot more room for the same sort of shenanigans that rule enthusiast hardware wank to worm their way into mainstream tech coverage.

Re:"Reviews"

[ackthpt]

Tech reviewers step on each others toes to be "first". They all have the quality of "the first post" on slashdot.

While some First Posts have actually had some merit, a lot of First Reviews are missing some real world sit-down-and-see-what-this-can-do impressions. As far as I care a Real Hands On Review means the reviewer has it for the weekend, or such, and takes it around with them, tries various things in various settings. You get more insight from someone who has actually done something with an item rather than been part of the cattle herd at the official event.

Obviously a functional unit

[Karmashock]

MS's problems are really kind of bizarre. It's not for lack of talent or trying they just keep screwing up. It has to be management. You don't get such systematic across the board f' ups unless management is behind it.

Re:Obviously a functional unit

[Samantha Wright]

It's management alright. It's been management for years. Microsoft consistently hires the best people in the field (well, those that Google doesn't snag—prior to that, though, they were nearly unchallenged, and consequentially MS has had a huge number of very respectable older researchers and engineers, including a large contingent of ex-DEC people) and then squashes them with bad managers, who spend so much time politicking and infighting that they can't recognize genius like the Courier.

Unfortunately this is an increasing trend in the whole software industry; the very recent example of Diablo 3's utter failure to live up to hype, even though it's now the fastest-selling game in history, can largely be attributed to management changes in Activision. The underlying problem seems to be hiring management and leadership from non-computing sectors instead of promoting from within, although in MS's case it's more like a long-term family feud.

Re:Obviously a functional unit

[Karmashock]

I largely agree.

The MBAs are very useful for administration but that doesn't mean they need to be in charge. Subordinate them to secretarial functions and give actual department control to people that actually know what is going on.

Re:Obviously a functional unit

[Omestes]

Unfortunately this is an increasing trend in the whole software industry; the very recent example of Diablo 3's utter failure to live up to hype, even though it's now the fastest-selling game in history, can largely be attributed to management changes in Activision [teamliquid.net].

Er... you realize you have a contradiction in that statement, right? "Utter failure to live up to hype" and "fastest selling game in history" don't really work together. Also you ignore the fact that the Blizzard side of Activision-Blizzard, and the Activision side of it are separate, and wholly autonomous. Or at least I haven't seen a single credible, informed, source stating otherwise. Even your link amounts to little more than a conspiracy theory, linking disparate facts into a structure without any actual evidence towards the premise. Yes, there might be the same masters (which Blizzard has had before), but this doesn't really mean much since you don't actually know whats going on. For all we know they are hands-off of Blizzard since they are the most profitable game company in existence so they must be doing something right (as evident from Diablo 3, which is justifiably making money hand over fist).

That said, and back on topic: I think MS has a behemoth problem, they are so large, and their fingers are in so many pies, that they lack the ability to quickly adapt to anything. I'm going to get flamed here, but many of their "failures" have actually been pretty decent, but they failed to grasp the market, or marketing, needed to make it work. They try to win by sheer weight, and not by finesse. This works sometimes (Office, the IE domination of yore, Exchange), but generally fails in a non-business arena. The iPod and the Zune weren't really that different, with one being better in some areas, and the other in others, but the Zune completely died because you can't compete on pure specs, especially with Apple (the masters of image and sexy). I suppose this does boil down to managment, but also to the fact that they are MS... Its hard to say "Microsoft" (insert picture of Bill Gates/Balmer)" and "sexy" in the same breath, much less sell it to the masses.

Re:Obviously a functional unit

[Raenex]

The technician traced the problem to too many wifi users at once which isn't a big deal.

Only if you live inside the reality distortion field. What, you want to use your WiFi at a conference, or a busy place like an airport, or the university cafeteria? You can't, but no big deal. Oh, and you're holding it wrong.

Re:Obviously a functional unit

[UnknowingFool]

So you're saying that a clogged wifi network causing glitches during the middle of a presentation of product using wifi(which isn't a fault of the product but the network devices) is totally the same thing as a product freezing in the middle of a presentation. What world do you live in?

Re:Obviously a functional unit

[Omestes]

... where sometimes the hype is so great that they score gigantic initial sales and most of those people say it sucks.

I'm not really sure that "most" people actually would say it sucks. In real life (i.e. ignoring Metacritic, Diablo forums, and Amazon) I haven't met anyone who actually thinks it sucks, or is nearly as bad as the internet makes it seem. Same goes for most of the people I've met in random public games. Hell, actually playing it, I can't say it sucks (not as good as some things, much better than others).

Part of the problem is that their audience is much, much, bigger than when their "classic" games came out. And much of the internet is a very different, and much more idiotic and hostile place. A lot of that new audience are WoW players, and thats what they wanted, or expected. A lot of it is nostalgia trumping clarity; "it isn't as good as Diablo 2, therefore it sucks". This ignores that fact that Diablo 2 had a fair share of problems on launch, and also wasn't actually the shining pinacle of gaming that everyone remembers. Nothing can beat nostalgia. It also isn't Diablo 2.5, since they decided to actually try new systems for once. Part of it is that people have philosophical differences with where Blizzard is going (no LAN, real money auction house). Inferno (the end game) is too hard... Classes aren't balanced (its a single player/coop game, who cares?).

There are also a lot of silly conspiracy theories running around, and a lot of uncorroborated group think (the "hacks" scandal). I've actually never seen anything quite like the vocal brouhaha that followed the release of the game. Especially considering that many people HATE the game, but put in 180 hours. I don't think I've actually ever played a game that much, especially in a period of time less than a month, since I was 15. And meanwhile Blizzard is laughing their way to the bank.

There is too much ire to be simply people not liking it, or merely sucking. There are thousands of games that suck, that die in obscurity. But for some reason this one needs the internet to run around making mouth noises about it. Playing the game, and ignoring my shining memory of Diablo 2, I can honestly say that it is a pretty damn decent game.

Re:Obviously a functional unit

[Karmashock]

Perhaps a better phrase would be "hollowed out"... the insides sucked out leaving a shell.

Everything that was special about blizzard will soon be little more then a memory if they don't change course and quickly. D3 is a continuation of the WoW path. Blizzard might have made most of their money from WoW but WoW isn't why customers trusted them enough to sign on to WoW.

They built that on the back of their RTS and hack and slash franchise history. They pulled that community. It will only accept so many betrayals. And once it's gone well, it has to be reearned. And most companies never earn that sort of trust. Ever. So the ability to reearn that sort of thing once lost is minimal.

Re:Obviously a functional unit

[Omestes]

he available statistics [xfire.com] suggest that a lot of people bought the game, and then have failed to continue playing it. It sold very quickly at first, but now no one wants it.

You might be right, but I can't tell from that. I don't know how they are measuring this, and I have no clue what Xfire even is. Also, I don't know what to compare that too, how does a game launch of comparable size compare. Again, for all I know, you are 100% correct.

Many of the patch changes nevertheless appear to correspond to the agenda put forth by Activision's new management in the aforementioned conspiracy theory, even if Activision doesn't actually have direct influence in Blizzard's management: continued nerfing despite promises not to do so [battle.net], removing drops from destructible objects, and only slightly modifying the difficulty of Elite monsters all appear to imply that the company wants to force players to participate in the real money auction house in order to finish the game.

I think the community has a lot to do with the nerfs. Inferno was supposed to be impossible, and everyone complained about it NONSTOP, and very loudly. If anything they made the penultimate difficulty LESS difficult, which, one would think, would lessen the need for cash purchases of items. There has been a lot of changes that the community has spun into "Blizzard wants money!!", that can also be seen at face value. Nerfing some builds did open other options, as much of the Demon Hunter nerfs did, they increased the amount of perceived viable builds. I'm okay with this. I wouldn't be okay, though, with them nerfing things to benefit the RMAH (real money auction house). I haven't seen much evidence of this. Companies always patch games, especially Blizzard. They are known for being aggressive patchers, look at the patch notes for Diablo 2 and expansion.

I've heard that Blizzard's testers couldn't actually complete the game on the hardest difficulty, and that they knowingly shipped the game in that state.

When they said that, my mind screamed "hyperbole". But I still, if true, wouldn't take that as a negative. Further, you don't need to really ever use the auction house, the RMAH, or any other social feature of the game to have fun. It might be more work, but it still fun to try. I'm working my way through Act II, and haven't spend a cent of real money, and only a small amount in the in-game money AH. I probably die more than the people who want to spend money, but I'm fine with that.

I probably sound like a "fanboy" or some such here. I do have complaints about the game, but I figure its pointless to voice them since they've been said by others ad nauseum. I also really can't tell how it stacks against Diablo II, since nostalgia rears its ugly undependable head. I'm also much older than I was back then, I have a life now, I have hobbies that don't involve sitting in front of a computer in a dark room, I have experience with more things, and my tastes have changed. I want to say Diablo 2 was better, but I can't really. I was in the Torchlight 2 beta, and I didn't enjoy it as much as I felt I should have, or would have 10 years ago.

At this point, with Microsoft, I really, truly believe that people are so conditioned to hate their products that they can't rightly succeed any more with their current brands. This is the fate of all greedy computer companies, I think.

I don't even know if its hate... On places like Slashdot, sure, but out there in userland... I'm pretty sure it is mostly apathy. When you (Joe Sixpack) think of Microsoft you think of Windows, something that you are forced to use at work, or Office, something that you are forced to use at work. Or you think of that incomprehensible beige box sitting in your home office, that your kid has to fix for you. The problem with Microsoft, for most people, is probably that it leaves no impression on them.

Useless blog link for a non-story. Why did /. post

[Anonymous Coward]

All this relevent information was already covered by Engadget, TheVerge, etc, etc. None-story here. They did a press event. No prices, no date of availablility, and they didn't want folks to actually touch the demo models running beta versions of an OS on beta hardware. Shocker.

Re:Oh please, get a life.

[medcalf]

I don't think it's MS hate. If someone put their hands on a new model Jaguar, with no engine and which they weren't allowed to sit in, and then called it a road test, their credibility (the reviewer,'s not Jaguar's) would be dead with me from then on. MS announced something that might be vaporware, in the sense of never coming to market, or might in fact be the device that unseats the iPad. But that's not the issue: deceptive reviews are the issue. Is the keyboard as cool as it looks or an unusable monstrosity? The reviewers in question have no way of telling, but are acting as if they do. That's what annoys.

Re:Oh please, get a life.

[93 Escort Wagon]

Oh come on. This was not like an Apple event at all. The Surface demo made a huge deal about the keyboard, how much better it was than the iPad's soft keyboard - and then the journalists weren't allowed to try it out, even for a second?

Read the story next time before commenting please.

Re:That's easy.

[adamstew]

That makes about as much sense as someone saying that they got to sit in a parked sports car that didn't have a steering wheel and they weren't allowed to turn on the stereo or push any buttons.... but since the car was parked on the road it's a "road test".

Write up is a troll

[kamapuaa]

She doesn't say "merely touching an unworking product makes it hands-on review" at any point. She says that she can give a review that's "hands on" even with just a short time using the product, as long as she's clear it's just an impression and isn't an in-depth review. If you read the review, it's full of qualifiers like "At this stage Microsoft is being very cagey and no-one has had much time using Surface RT yet, but from our experience of trying it out."

Just another unfair article summary by some Slashdot basement dweller with an anti-Microsoft agenda.

Re:Write up is a troll

[rudy_wayne]

She says that she can give a review that's "hands on" even with just a short time using the product, as long as she's clear it's just an impression and isn't an in-depth review.

Holding a device, that isn't turned on, for a few seconds, is not a "hands-on review", except in the world of unethical scumbag journalists.

Re:Oh please, get a life.

[thetoadwarrior]

Sounds like the truth upset a fanboy.

Re:"Journalism"

[Media_Scumbag]

Journalism's been dying for years, but like a frog in a pan of water, it will not flex a muscle to save itself. Though it boils violently in it's own excrement, its' audience can scarcely be bothered, as the spectacle isn't awful enough.

Pardon my anecdote:

I was at E3 2000 when MS revealed another (pardon the pun) "game changer" in much the same way as this "iPad-killer": The X-Box.

There was no case, no controller (it was a Logitech PC controller) and myself and 20 or so journalists sat in a makeshift theatre watched a fly-through demo highlighting what we all knew was a basic PC Direct-X graphics engine. No one steered the flythrough, none of us were allowed to touch the controller or the clunky plexiglass and PC-guts that sat on a small, cloth-draped a/v rack. None of our questions could really be answered, either. To this day, I'm not at all sure why they didn't call individual reporters up to breakout rooms or hotel suites, because those of us who weren't in our early 20's were thoroughly unimpressed.

I'm sure someone gave them props. After all, E3, gaming and the Web (still) were booming, and fact-checked news and Comdex were showing their age.

Read the Web articles of the NYT, WashPo, WSJ, - any of the leading print publications from the past 30 years or more. How often do you see grammatical, spelling, or factual errors? I see them with exponentially increasing frequency. I think it's indicative of the "death of print," and more distressingly, the "dumbing-down of America." No one cares about quality reporting anymore. They want HuffPo, Brietbart, TMZ, and Gawker. They want blood.

Bradbury was right.

... it boils down to ...

[Taco Cowboy]

The problem being highlighted is that journalists were(understandably, given the pressure for ad impressions; but very arguably unethically) overstating the amount of information they were actually bringing to their readers. Regurgitating press releases makes you a flack; but it isn't inherently unethical. Re-labelling press releases as 'news' and then regurgitating them is another matter entirely...

Actually, the entire thing boils down to ... many of those who call themselves "journalists" have neither the journalistic integrity nor the will power to become a real journalist

Those who tell stories should be known as "Story Teller", not journalists .

To be a true journalist is actually not easy - it's always a tight-rope act when the situation demands an independent view

It is one of the reasons why the so-called "journalism" we have today are mostly crap