Slashdot Mirror
Are Complex Games Doomed To Have Buggy Releases?
An anonymous reader points out a recent article at Gamesradar discussing the frequency of major bugs and technical issues in freshly-released video games. While such issues are often fixed with updates, questions remain about the legality and ethics of rushing a game to launch. Quoting:
"As angry as you may be about getting a buggy title, would you want the law to get involved? Meglena Kuneva, EU Consumer Affairs Commissioner, is putting forward legislation that would legally oblige digital game distributors to give refunds for games, putting games in the same category in consumer law as household appliances. ... This call to arms has been praised by tech expert Andy Tanenbaum, author of books like Operating Systems: Design and Implementation. 'I think the idea that commercial software be judged by the same standards as other commercial products is not so crazy,' he says. 'Cars, TVs, and telephones are all expected to work, and they are full of software. Why not standalone software? I think such legislation would put software makers under pressure to first make sure their software works, then worry about more bells and whistles.'"
What exactly is the downside to forcing a company to give refunds for the broken merchandise that it sells?
"His name was James Damore."
A. Yes.
"Cars, TVs, and telephones are all expected to work, and they are full of software. Why not standalone software?"
What's the definition of work anyways? Most products sold nowadays suck. I just got rid of a laundry machine that was 40 years old. I'll be lucky if the new one lasts 1/4th as long. I bought a LCD monitor that worked until the 1 year warranty was up. My cellphone functions, but its software is crummy and buggy. It even freezes up sometimes. (No, it isn't a smartphone.)
Software is more like writing a book. Some books are written with superior quality "code." The "computer" reads that code, and depending on how well that code was written, the "computer" can read it more quickly and determine the proper function faster. (In this case, correctly interpreting the knowledge inside the book.) Some books are total shit. For example, just about every book that's used in education. Especially ones written by professors. They don't "work."
I do have one sentiment. I absolutely think that games should be returnable. But that's from an idealistic standpoint. I fully understand many, many people are going to play the game all the way through then return it to Wally World.
AST:
'I think the idea that commercial software be judged by the same standards as other commercial products is not so crazy,' he says. 'Cars, TVs, and telephones are all expected to work, and they are full of software. Why not standalone software? I think such legislation would put software makers under pressure to first make sure their software works, then worry about more bells and whistles.'"
Next he will be claiming that it is safer to use a properly modular operating system.
http://michaelsmith.id.au
Isn't there a bit of a difference there between the examples (TVs, DVD players, etc) and PC software? Everything else is a self-contained unit with no dependencies on anything else or (in the case of DVD players) accepts things that are within tight tollerances (and if your disk isn't, then it'll skip, but that's the disk's fault).
Software, on the other hand, has no control over the environment that it is put in (unless it is an OS X app, which is somewhat consistent), with huge permutations of other software, hardware components, and dodgy background processes, plus user fiddling. It isn't quite as easy to get things flawless in that situation (although some companies can improve on what they do now).
Also, how will this relate to OSS? Will I never release a final version of my app because I can't afford the liability and so it'll always be in beta because there could be bugs left?
It's not just games: most consumer electronics nowadays are a mix of software and hardware and often enough it's the software part that is released unfinished (read: buggy).
Software on non-life-critical applications has been given a free ride for two long - if it's not acceptable that a DVD player refuses to start at odd moments or randomly stops working, why would the same be acceptable in a computer game (which is just another for of entertainment) or an OS?
As somebody working in IT, who has worked in the industry in both IT Services and IT Products, I've seen again and again the main behaviours that lead to buggy software releases:
a) No real software development process resulting in unpredictability with regards to the real finish date.
b) Bad requirements definitions, stuffed with incomplete, inconsistent and unclear "desires", with way too much time wasted in "would be nice" requirements leading to last minute requirements changes as people discover the missing/bad bits.
c) Little or no real testing, mostly done by amateurs (or worse, developers).
d) Hard deadlines set by sales and marketing which, coupled with the points above, results in releases of unfinished products.
The reason why this happen is very simple: companies can get away with this, so management (from top to bottom) can get away with being disorganized, unstructured, "shoot-from-the-hip" cowboy-like, non-proactive and outright incompetent.
(yes, I AM sour about this)
Funny enough, buyers of software products and services are so used to be royally done by the industry that some of the worst offenders in this space are actually the larger IT companies, not the smaller ones: in a playing field were buyers expected and valued quality in software, the higher-quality companies would outcompete and outgrow the low-quality ones, and yet what we see is the opposite.
The prices are what the market will bare. They would not "ramp up prices" because that would be sub-optimal.
The reality here is that they are not paying for the ordinary quality control process because they can legally get away with it. It has nothing to do with retail pricing.
"His name was James Damore."
Comment removed based on user account deletion
So Andy Tanenbaum is now a mere "tech expert"? That's a big step down from "CS god."
For the uninformed, ast wrote a kinda good book on operating systems called "Operating Systems: Design and Implementation." I believe this one guy from Finland wrote an OS called Linux based on another OS called Minix discussed in that book (and even got into the flamefest of the century with the Finnish guy!). And then there's a bunch of other stuff you may or may not know about, such as the Amoeba distributed OS, a free anonymous p2p network called Turtle, and probably a few other knick-knacks along the way.
Seriously, give the Man due credit.
Nope, Nvidia.
"we've got trenchcoats and bad attitudes" - John Constantine, HellBlazer
I bought Mass Effect only to find out that the game simply does not run. My computer is as close to flawless as it could possibly get, it's been running for years and has successfully played many games with many different engines, I have done workarounds for crashes and bugs and all sorts of things, it's a tried and true PC.
Just an anecdote from the other side of the fence, and not saying this is necessarily your case. Certainly not defending Mass Effect, I've never tried the game personally.
We've had numerous users report serious defects in our products over many years, and faced all sorts of threats and insults, only for the fault to be eventually traced to the user's "tried and tested hardware." Each program that you may use exercises different components of your PC in different ways. Sometimes subtle differences can make a massive difference in results; the difference between working fine and not even starting up. Should the developer pay because you have some mildly faulty ram?
We've also seen vastly different behavior from hardware/drivers built to the same spec but sourced from different manufacturers, or from the same manufacturer but over different periods. Sometimes these deviations are within the spec but not covered by reasonable testing; often these deviations are outside the spec completely. Should the developer pay because one or more of your components do not follow the specs, or deviate significantly from what was standard practice at the time the software was developed?
As a user, I have to agree that it sucks when products don't work as advertised. I agree that there should be a mechanism for complaint against any vendor, whether their product be physical or virtual. But I'm not sure that I agree that there should be an absolute right of refund at the user's discretion. That's just open for abuse - whether deliberate or incidental.
I'm also not particularly fond of DRM and yet that would seem to be the only way that a vendor could offer true "returns" of a software-only product.
It's probably worth noting that I'm not claiming that all bugs are the user's fault; but it's certainly not the case that all bugs are the application developer's fault, either.
Have any of you played Guitar Hero 3 for the Wii?
It's a fairly simple game, right? You have an .mp3 file, and paired with it you have a file containing a list of tuples (time, subset of buttons {1,2,3,4,5}), then you "play back" those two files simultaneously and see if the users strums while holding down the correct subsets within some well-defined window of time.
You can put the game in a "broken" state (requiring you to back out to the main menu); I don't recall exactly how, but I think it's when you, from practice mode, change the practice speed, you get dumped back to a dysfunctional practice mode screen.
If you tell the game your monitor (TV) has a certain delay, when you practice at less than 1x regular speed, apparently the game thinks it should not just scale the time differences in the list-of-subsets file but also that your monitor takes longer time to show pictures. Morons.
And the menu structure is big, menu items are inconsistently named, and the structure itself is poorly aligned with what people want to do. Bad usability. Example: I want to give up on a song, so I choose quit; "Do you really want to quit; unsaved progress will be lost?" (wtf, there's no way to actually save progress...). Well, "Yes I want to quit". "Ok, where do you want to quit to? Main menu, song list, or retry this song?" What??? If I wanted to retry the song, I would have selected the "retry song" menu item. The only reason having a choice here is good is because it takes so unbearably long to navigate from the song list to the main menu.
And couldn't they have added an option to compensate for broken TVs which not only have picture lag, but have slightly desynchronized audio and picture? Would that really have been too hard? (Well, apparently...)
For such a brilliantly designed game play, the implementation (and the design of the things that go around the game play) is unbelievably crappy. I'm seriously doubting whether they tested it.
(And what was that thing about shipping discs with mono audio?)
Seriously, avoid GH3/Wii. If you must show off by completing (or FC'ing) TTFAF on expert, do it some other platform. It's for your own good.
if it's not acceptable that a DVD player refuses to start at odd moments or randomly stops working, why would the same be acceptable in [...] an OS?
How do you define "Fitness for its purpose" when the purpose is defined differently by each individual user? That's both the power and challenge of software: it can do anything. General-purpose OSes are meant to let you do anything.
They're also big enough (i.e. consisting of a large number of interacting components) that if you want to define exactly what users can and can't expect (and can/can't do), you'll end up with either an insanely long list, or overly broad items on that list.
Either "No warranty unless file C:\etc\blah matches this context-sensitive grammar" (repeat 1e6 times over for different files) or "No warranty if the user tinkers with C:\Windows".
Also, what if you get hit by malware which does something that would void the warranty if you did it yourself, and then the malware deletes itself?
Defining Acceptable User Behaviour and Acceptable Software Behaviour is going to be arduous. What would be gained?
Would people refund Windows and replace it with... a different-but-just-as-broken OS? A different-and-less-broken-but-still-broken OS? Or an OS that doesn't do anything? It's not like there are per-unit software manufacturing defects...
I think the vendors that constantly have buggy initial releases are the same consistently.
EA? I expect a buggy release or a release that doesn't run well or at all.
Blizzard? Mostly ships pretty functional games or expansions these days. Blizzard has enough money and enough of a following that they don't have to shove software out before it's ready. Their recent betas seem to have fewer bugs than other studios' releases.
Blizzard?!? They haven't shipped a game in nearly 5 years! ...and WoW was patched near constantly for the first year of its life.
Expansions, maybe - but games? It's been a long time!
The quality of the code is a function of its cost, too.
For example, the code written for NASA hardware (i.e.: space shuttles), have more documentation than the size of the hardware itself (so, we're looking at a large pile of documents next to the shuttle). It's tested for years, it only works on tested CPUs (i.e.: 20 years old proven 8086s), and the actual "waterfall" method (which is generally a disaster for any other project) is properly applied.
That total brings the cost of each source code line to average $1000. (Same for medical appliances, etc).
The cost of a commercial off the shelf software is much (much much) less than $100.
But, even under such strict control, we had to debug the Mars rovers due to unforeseen bugs during their initial flight.
Anybody here on Slashdot can do the math, and fill in the gaps to calculate the future price of games (for a reference they are $60/unit now).
Call it a programmer's intuition, or just calculating chances. If 100 games work on a system and 1 doesn't, there's a good chance it's something to do with the game.
This is the first assumption that you'd jump to, but there's no real evidence that it's true. Perhaps the "one game" simply uses more ram than the others, and thus hits a "faulty bit" in one of the ram chips. Perhaps the graphics drivers have an off-by-one condition that causes them to over-read a vertex buffer and into unallocated memory. Even on "identical" machines, this may not crash unless the application's allocations match an exact pattern that causes the bad read to touch an unmapped page. We've seen both of these in practice.
Sure, there are plenty of bugs - pretty much any software has some, and some software more than others, but it's fair to say that the "my machine is fine, it must be the software" mentality leads to some very poor conclusions from time to time, even though it's an understandable position for a user to take.
Now try and prove that the game is broken to a person for who it worked flawlessly for 3 whole run throughs. Good luck.
Yup, we certainly see some of this as well. Loyal users can sometimes be fast to jump on people reporting valid bugs.
What exactly is the downside to forcing a company to give refunds for the broken merchandise that it sells?
Well, the industry would say piracy. I might buy Call of Duty, then, said it was "broken", and returned it. Granted, this should be the norm, but the industry would see things differently. This is why the shareware model is nice. You can see if the game actually works before you pay for it.
This is my sig.
No company is perfect, but there is always one company that ends up the flag bearer for PC game releases. Last decade, it was Origin. These days, I'd say the company that has the best processes for getting games to the PC would be Blizzard.
I don't intend this to sound fanboish, but they have had some very smooth release cycles in recent memory. Last year's release of WoTLK for example.
You failed. Blizzard hasn't shipped a game in 5 years. Expansions for WoW, yes. Games, no.
What we're looking at here is offshoring a lot of highly specialized work to essentially shops in the third world that have not the same level of expertise, and we get crap at a cheaper price. We compete in the west not by improving the quality, but, by making things cheaper themselves. So, we have less testing, less documentation, just code and ship and little for iterative development. Plus, we have more corporate style methodologies that reward the schedule more than the product. Pretty much, if you build stuff stupid, you get stupid stuff.
Maybe there will be a shakeout where we realize that consumer IT is much more demanding than corporate IT is, and that methodologies that work fine for corporate clients, like Agile / SCRUM, or Waterfall, don't really apply so much to consumer products. A consumer product is done when it is done.
This is my sig.
Should the developer pay because you have some mildly faulty ram?
Other product vendors have to pay if the customer decides they don't like the color of their purchase. So yeah, it's the cost of doing business.
It's understood that computers are complex and not every program will work in every situation, but why do you think the vendor is entitled to the money of someone who for whatever reason can't use your product? It would be a lot to ask to required the vendor to figure out what screwy component is causing problems and to make their product run on everyone's computer, but it's not a lot to expect them to only make money from people who are actually getting the bare minimum that they paid for.
Pretty much everyone knows where to get pirated versions of whatever software they want to run, so I'm pretty sure the only people hurt by not allowing refunds are honest customers.
We hope your rules and wisdom choke you / Now we are one in everlasting peace
Mass effect worked flawlessly for me on multiple computers of varying specs (No ATI cards though) From a bottom end, to a mid range to a ultra high end computer.
It's something with your setup, not the game.
They were tested and verified; thousands and thousands of other people played Mass Effect with no problem. Nobody is using you as a Beta. It is not possible to test all combinations, and moreover the fix, when you find one, will be on your end, because if it were the code everyone would be experiencing the same difficulty. The inability of even technically competent gamers to grasp this basic point is the best argument against enforced refunds in my opinion. People are able to tell when a phone doesn't work; with a game all they know is that it isn't working. You can't tell where the problem lies, so you just blame the game reflexively. Natural, but very often wrong. If the company wants to give you a refund that's fine but you'd both be better served if you weren't so adamant that the game was the problem and actually worked to find the real culprit; you might get to play a great game and the company would retain your money and your custom.
You do realize that every single game that runs fine for you is running terribly for someone else. If a bug isn't very widespread it probably isn't the game's fault. Some games really are buggy messes but Mass Effect isn't one of them.
To be fair GGP said hardware or driver. ATI has bad drivers and Nvidia has bad hardware.