Slashdot Mirror
ESRB To Automate Game Rating
The Entertainment Software Rating Board, which has struggled to keep up with the flood of games produced for app stores and other online markets, is now taking steps to automate the rating process.
"Starting on Monday the ratings board plans to begin introducing computers to the job of deciding whether a game is appropriate for Everyone, for Teens or for Mature gamers (meaning older than 16). To do this the organization has written a program designed to replicate the ingrained cultural norms and predilections of the everyday American consumer, at least when it comes to what is appropriate for children and what isn’t. ... the main evaluation of hundreds of games each year will be based not on direct human judgment but instead on a detailed digital questionnaire meant to gauge every subtle nuance of violence, sexuality, profanity, drug use, gambling and bodily function that could possibly offend anyone. The questionnaire, to be filled out by a game’s makers (with penalties for nondisclosure), is like a psychological inquest into the depths of all the things our culture considers potentially unwholesome."
The first attempt to automate the posting of an appropriate rating for each new game. I think so.
six days in Falujah would get T(13+) for moderate-to-high amounts of violence, no sexual themes, limited or no use of profanity, no drug use, no gambling and no bodily functions.
45 5F E1 04 22 CA 29 C4 93 3F 95 05 2B 79 2A B2
I'm fed up with decisions being made by questionnaires and computers. I think we should stop tolerating analyses of health, fitness, credit, intelligence, etc based on simplistic tests and numbers. The expert system is one of the most horrible simplifications of human judgement ever to grace the confused world of AI, and is almost without exception implemented with some bias to fulfil a pre-determined aim and reinforce some prejudice.
This is shifting the work to the game developers, whose staff has to fill out the extremely long questionnaires. Which might make one wonder, what's the point of the rating board in the first place?
The board says that publishersâ(TM) answers to the digital questionnaire will determine a gameâ(TM)s rating and that a human wonâ(TM)t review it until after the game is out the door.
As stated in a draft of the boardâ(TM)s news release, âoeAll games rated via this new process will be tested by E.S.R.B. staff shortly after they are made publicly available to verify that disclosure was complete and accurate.â
Their computer spits out a rating based on a questionaire and nobody double checks until after the public launch?
The ESRB is turning itself into a rubber stamp organization.
[Fuck Beta]
o0t!
What happens if you manage to design something that just straight throws the system off and rates it totally wrong? Did they ever stop to think that cultural norms are not a static type of thing? Seems to me that this is just a recipe for disaster
Why dont we just put the answers to the questionnaire online and then any parent who cares enough to read them will know exactly what they are buying. That way no one will be judging at what age you can play a game and the ... unpleasantness of games is no longer reduced to a number. Parents who are sensitive to topics like drug abuse or gun control or sex can read the questionnaire and decide for themselves on a per topic basis.
But it will still cost 2,000 dollars to have a game rated.
It's understandale right now, since they have to play the whole... No wait, they just watch a video of the game. But s omeone has to make the video... oh, yeah... The publisher does that. Well, buying retail copies for records, and future checks of... No, they demand three retail copies after the rating is completed. Well, they are located on Madison Avenue.
There's no follow up to that one. They are a non-profit on Madison Avenue that charge you 2,000 dollars for you essentially rate your own game, the ngive them the right to fine you 30,000 dollars if you make a mistake.
Nice. Fuckng. Racket.
I'm really getting annoyed that we are importing American culture, their values and norms.
It's not that they are conflicting with my culture there is already a lot of overlap, but there is a difference in values and norms. And it annoys me that problems are being made of things that are no problem or less of a problem in my culture and vice versa.
Cultures are meant to be different let's respect that and keep it so.
Game developers/publishers already submit a long questionnaire and a video detailing every instance of everything that might affect the rating. They're already on their honor to do this honestly. All this move involves is removing the human element, which was intended to be objective anyways, and replace it with automated computer analysis. They honestly probably already have an algorithm to determine how many swears gives a Teen rating or Profanity label; counting the exact number can be done by voice recognition, if it's not already part of the questionnaire.
Corruption is convincing someone that the selfless ideal is the same as their selfish ideal.
If anyone is interested in how that conversation ended.
/Angry glance. /presses button on chair, spins around to reveal a giant screen. The image of a young programmer appears on the screen.
Activision exec: My lord Kotik, the 117th iteration of Medal of Duty is ready for release.
Bobby Kotik: Has there been any changes to the gameplay?
Activision exec: Uh, my lord Kotic
Bobby Kotik:
Activision exec: One of the junior developers thought of some improvements.
Bobby Kotik: I SAID NO CHANGES.
Bobby Kotik:
Junior Developer: ahhh, um, Lord Kotik, I, I, I wasn't expecting you.
Bobby Kotik: You have failed me for the last time Junior Developer.
Junior Developer: URK... gargle... klick.
Bobby Kotik: See that this does not happen again Executive.
Calling someone a "hater" only means you can not rationally rebut their argument.
Way to show you just don't care.. "hey, fill out this survey so a computer can determine how to rate your game. No, we aren't going to play it."
From the wikipedia article:
"To obtain a rating for a game, a publisher sends the ESRB videotaped footage of the most graphic and extreme content found in the game. The publisher also fills out a questionnaire describing the game's content and pays a fee based on the game's development cost:[5]
$800 fee for development costs under USD $250k
$4,000 fee for development costs over $250k"
So, the game developer is going to do all the work and pay you to certify their game and you aren't doing anything but running a website and pocketing money? You're trading on the name you've built as a "reliable standard" and you're going to be gone as soon as Sony/Microsoft/Apple/any other app store marketplace, realizes they can take your piece of the pie and do this same thing and take money for it.
I could understand if not enough games were being submitted and you were contemplating going out of business because nobody used you anymore, but you're claiming the exact opposite. Too many people are giving you money wanting you to rate games so you're stepping out of the game rating business?
I don't have any kids and have never cared what rating a game received, but I consider this move to be counterproductive to the people who are paying you. The first slip-up isn't going to be a publishers ass it's going to be the ESRB when people ask "who's minding the store?" and the answer is nobody.
. . . we will have stuff that pleases no one . . .
"Bland New World"
Schroedinger's Brexit: The UK is both in and out of the EU at the same time!
Do what a lot of shitty companies and research programs have been doing: outsource it to MTurk (Amazon's "artificial artificial intelligence") site. The task would be something like, put a checkmark next to a series of stills from a video excerpt of the game playthrough. If there's a checkmark there, it gets an AO rating. The worker is paid 2 cents for her effort.
This reminds me of companies playing Google's algorithm to get a higher page-rank.
Will we see people playing the ESRB engine in the future? - There has to be some sort of risk mitigation or analysis at any decent company on how to best produce desired outcome.
I mean no program is person, some games need human decision making, and a tool should have been developed to supplement human input rather than replace it.
Just my opinion. (Public terminal, AC sorry)
[Generic scenes of violence]: Yawn.
[Graphic dismemberment]: No problem.
[Guns and car chases]: I see worse on Nickelodeon/Disney these days.
[Well-endowed woman with skimpy skin colored top on]: WHOA WHOA WHOA. NOT FOR KIDS. RATED M. RATED M!!!!!!!!
My 3rd old commits more violence in the 1hr after he wakes up and the 1hr before bed to his own family than you'll ever find in a game like "Call of Duty." The idea that you're going to protect a child from something that they are literally doing from birth is asinine. How would the ESRB rate the average schoolyard playground?
Slashdotters have limited or no ineteraction with girlfriends, but heavy use of evil lairs.
My first Journal Entry ever, in 8 years! http://slashdot.org/journal/365947/aphelion-scifi-fantasy-horror-poetry-webzine
I am wondering if the criteria will be published. Could we possibly see a game developed specifically with the intention of hitting every button possible?
Is being rated by the ESRB mandatory? Can they rate your game without your consent??
Will it be like some auto essay gradeing systems where you can game it to pass? Also auto Software QA and Testing systems can pass a APP that fails but not to the point of tripping the QA system.
So will there be a category for promiscuity? Adultery? Bearing false witness (cruelly lying)? Inter-species love? Homosexual love? I bet you that only the last one will be asked about, but on any way of looking at the issue (weather biblical fundamentalist or humanistic or whatever) that sort of focus makes no sense. Should interest groups now call in and request certain lines which play to their issues? Can someone force them to ask about abortion on the questionnaire, for example? Or about cancer, which in many families could be considered an "adult" topic? About uncritical jingoism? About philosophical arguments for why God doesn't exist? About the uncritical promotion or demonization of some religion or culture?
I'm trying to think of some troubling categories that almost certainly won't be on the checklist. I'm sure that there will be a category for graphic violence, about whether blood and dismemberment is shown, etc. But whatever, we know it's a game. But I'd like to also know whether some character will be killed or "taken away" which the player actually started to care about. That's the sort of thing that traumatizes kids: stuff like Bambi. Of course that won't be rated.
We all know that we don't need anything graphic to be freaked out. Just think of Lovecraft or Poe. Some fiction just succeeds at scariness, but no checklist will catch it. So I think that this checklist idea is just an abdication of responsibility by the ESRB. But it doesn't matter. I can't think of anyone who thought that they were a valuable source of information about games.
How does a questionnaire and automated system deal with the tone and nuance of a game through a bunch of questions? Maybe it might work as a prescreening system but there still has to be some measure of human review. If necessary charge games developers to receive their rating and hire staff according to demand.
Even with human intervention, transparency is paramount.The rule base and the source code should be publicly disclosed. If they can't do that then it really shouldn't be trusted at all. After all, how were these rules judged to be representative of public opinion. What does public opinion mean anyway? I expect public opinion in most cities is vastly more liberal than it is in the middle of the bible belt.
moderate-to-high amounts of violence
I watch the news. Check.
FCC-regulated TV news does exercise discretion as to whether to show the gorier shots.
sexual themes
Fuck? YEAH!
use of profanity
Fuck, YEAH!
Now you've got that song from Team America: World Police running through my head.
gambling
I cross the street, drive a car... does that count?
Western culture generally doesn't consider pure risk to be forbidden gambling. As I understand it, forbidden gambling is any sort of risk that A. isn't a pure risk and B. isn't tied to a business concern's profit.
drug use
Do coffee and beer count?
Maybe and yes. Irresponsible drug use counts, as does use of age-restricted drugs.
bodily functions
Who doesn't?
Not flamboyantly in front of other people in the way that a video game about a monkey flinging her own poop might.
a more detailed explanation on the back of the box
Walmart keeps its console games behind locked glass. Is each parent supposed to ask an associate to take every single game off the shelf, one by one, just to read the descriptor on the back? No, a parent in a Walmart store will probably look at the one-letter summary if anything.
Bobby Kotik: Has there been any changes to the gameplay?
It's possible to tone down the violence by changing the setting while leaving the gameplay unchanged. One extreme is the tack taken by early first-person shooters, such as Battlezone and Faceball 2000, which didn't have any blood or even humanoid forms. A slightly less extreme example is the localization of Contra into Probotector for Germany and neighbors, where most characters were turned into android robots.
I wonder if this also covers unintentional content that gets released to the public through less than obvious means? (bugs, hacks/cracks)
I mean, when the hot coffee mod was dropped, independent of the developer or publishers control, didn't that send the ESRB into a tizzy?
Hi, I Boris. Hear fix bear, yes?
Hi all,
So where can I find a copy of this questionnaire? The article doesn't seem to link to it. It would be cool to have a list of things I could do to potentially offend people.
I think what this will end up doing is making developers revert to either their own rating system or not rating their games at all.
Dropping ESRB will have three consequences:
My 3rd old commits more violence in the 1hr after he wakes up and the 1hr before bed to his own family than you'll ever find in a game like "Call of Duty."
How? In my experience, the violence of a single-digit-year-old is nonlethal,* unlike the violence in Call of Duty games.
* Apart from reports of accidents involving unsafely stored firearms.
And the one letter summary is a great place to start (and stop in most cases). You're not going to find Sexual Content in anything below an M-rated game. You won't find blood in anything below T-rated, and even then they stipulate "Minimal Blood" in T-rated games. The content descriptors are there to help make fine selections after you've narrowed the field - If you're comfortable with your child hearing profanity but not seeing violence then they can help you make your choice.
Once you're looking at a particular game (or a choice among a few), it's pretty trivial to ask the Wal-Mart employee to open the case and let you flip over the box.
I worked in videogame stores or big box retailers in one way or another for 6 years through high school and college. Admittedly, it's been years since I worked in a retail environment selling games, and maybe parents have changed since I sold games. I ran into plenty of parents who were concerned about content, and even a good many who asked about the ratings. In all my time, though, I never encountered someone for whom the content descriptor was their first step in the purchasing decision. It was a final step, a check on whether the game(s) they'd selected were appropriate.
In my experience, a parent was either concerned about the content of a particular game ("Joey said to get BloodSlaughter 5 for his birthday - is it appropriate?"), is asking for a recommendation from the staff ("What's good for a 13 year-old boy? Uh-huh, and is there a lot of adult stuff?") or was familiar enough with the games him/herself and not in need of running down each game. Maybe there's someone out there who starts their purchasing process by making a pile of games without Sexual Themes or Animated Violence descriptors, but it seems so backwards because it would result in a massive pile of things to sort through rather than choosing games that look appealing and deciding if they're appropriate. A Dora the Explorer licensed game is unlikely to have any objectionable content descriptors, but that doesn't mean that most parents will be actively considering it for their 12 year-old son.
In other words, no one's reading the back of every game in the Wal-Mart case because, even for the most game-clueless parents, half or more of the games in the case probably aren't potential purchases to begin with.
Walmart keeps its console games behind locked glass.
For accuracy's sake, this is not true at all Walmarts. The one I work at doesn't keep games behind glass, and I don't think any of the other local ones do, either (that's another 4+ stores in a 1.5 hour driving radius).
They're supposed to be putting the glass back up at all stores, though. Nationwide, apparently the theft rate went up ~1000% after the glass was removed.
a parent in a Walmart store will probably look at the one-letter summary if anything.
I think it's more accurate to say that the parents just buy what their kids want. As a cashier, I have to go through the rating speech if there's a kid present and 90% of the time I get as far as "You're aware that this game has a rating of..." before I get cut off. Heck, I had one parent send her ~11 year old to buy his M-rated game by himself. Only once have I had an actual discussion with parents about game ratings, and they found that they much preferred the content in M-rated games to the content in T-rated games for their son. Even the kid agreed that he'd found more questionable content in T-rated games than M-rated ones.
For context, I live in a very rural part of New York, about 30 minutes south of Canada.
Once you're looking at a particular game (or a choice among a few), it's pretty trivial to ask the Wal-Mart employee to open the case and let you flip over the box.
Assuming you can even find an electronics associate who isn't busy with another customer. Or knows anything about their department.
In my experience, a parent was either concerned about the content of a particular game ("Joey said to get BloodSlaughter 5 for his birthday - is it appropriate?"), is asking for a recommendation from the staff ("What's good for a 13 year-old boy? Uh-huh, and is there a lot of adult stuff?") or was familiar enough with the games him/herself and not in need of running down each game.
In my experience (and I've been working as a cashier the last ~7 months at a Walmart), parents usually don't give a damn. I cite the mother who sent her ~11 year old to buy his M-rated game by himself (I made him go get his mom before I made the sale... she didn't seem to understand why I needed to talk to her before making the sale). Or maybe they've just made their decision by the time they get to the register. I've really only run into one set of parents that was really knowledgeable about the rating system despite not being gamers themselves.
Nethack would probably get NC17 (or whatever their equivalent is) because it has:
1) Drugs (Magic mushrooms and potions that make you hallucinate),
2) Incubi and Succubi that have (implied) sex with you,
3) Violence against police* (the "keystone cops" that show up if you directly steal from a shop),
4) Cannibalism,
5) Sex changes, and
6) Devil worship and human sacrifice--by the player!**
Yet the whole thing is done with ascii character "graphics", and is purely tongue-in-cheek, and is about as dangerous for the kiddies as a Warner Brothers cartoon.
* This could have actually gotten it banned entirely under a short-lived California law.
** Don't try this unless you're playing a chaotic character--but still...
In my experience (and I've been working as a cashier the last ~7 months at a Walmart), parents usually don't give a damn.
Sure, that's probably the biggest group. It's also why many stores require cashiers to ask for ID. If a parent doesn't care, well, there's not much you can do beyond follow your employer's policy and your own code of ethics.
The original post complained about how well a single letter rating could summarize the content. I pointed out that it isn't meant to, and that there's a fine-grained system in place to clarify where the rating came from. Someone else complains that Wal-Mart's anti-theft system prevents you from using the fine-grained content descriptors on a large scale, but those aren't really relevant until you've already selected a game and are trying to decide if it's appropriate. It's not really fair to fault what is a pretty robust rating system for apathy on the part of a parent or a lousy anti-theft system.
Assuming you can even find an electronics associate who isn't busy with another customer. Or knows anything about their department.
And that would be but one of the reasons why I won't even bother with Wal-Mart. Incidentally, they're the only major retailer (besides Toys R Us) that I'm aware of that still uses the glass case method. Most either use individual anti-theft cases, dummy boxes, or something like Target's new system (the game boxes are all tethered to a storage cabinet) so that customers can pick up, handle, and read the game box.
Anyone else think this sounds kind of cool from a gamer's perspective?
I'd like a game with high violence, low sexual themes, medium amounts of profanity, and some gambling on the side, please.
Assuming there's an accessible database.
So they're pushing their job back on the game developers. Great.
I'm sure there's an iOS or Android app that will help you develop the social skills you so plainly lack.
would it be allowed to play with itself or would it be rated mature only ?
Free speech was meant to be free for all... how can anyone grow up in a nanny state ?