Slashdot Mirror
Researchers Snag 60 TB of Everquest 2 Behavioral Data
A group of researchers who went from game developer to game developer looking to acquire data for studying online social interaction got more than they bargained for. Sony Online Entertainment keeps extensive server logs of everything that happens within Everquest 2. When the researchers asked if there was anything they could look at, SOE was happy to share the entire EQ2 database — upwards of 60 TB — for their perusal. In addition to basic gender and age queries — who interacted with whom, and when — the scientists are also trying to find ways to track more subjective characteristics, such as performance, trust, and expertise. "To get estimates of them, the team is experimenting with trying to track physical proximity and direct interactions, such as when characters share experience from an in-game victory. To give a concrete example of the data's utility, Srivastava described how he could explore the phenomenon of customer churn, something that's significant for any sort of subscription-based service, like cell phones or cable TV. With the full dataset, the team can now track how individual customers dropping out of the game influenced others who they typically played or interacted with. Using this data, the spreading rate and influence factor could then be calculated, providing hard measures to work with."
Update: 2/18 at 21:04 by SS: Sony contacted us to set the record straight about the shared information. All information that could identify players was removed from the data given to the researchers. Chat logs were not shared at all. Read on for SOE's full statement. "The information Sony Online Entertainment provided for the research project was scrubbed of all PII (Personally Identifiable Information) prior to being provided to the researchers. For example, no content of any player chat logs were shared with the researchers. The information shared consisted of data such as which in-game characters chatted with each other and the volume and frequency of the contacts. None of this information was connected to, or linked with, the real names or other PII, of any players. Basically, the researchers looked at the connections between players and how their online networks were built and used, not the content of any actual conversations that these players may have had with one another (that content was not made available to the researchers). Additionally, some EQ2 players voluntarily participated in a blind survey concerning their playing habits and demographic backgrounds. This information was anonymously gathered and shared between SOE and the researchers. Finally, corporations such as SOE and the researchers' universities have controls and safeguards in place to help to protect the privacy of individuals who voluntarily participate in surveys for research projects such as this one."
Update: 2/18 at 21:04 by SS: Sony contacted us to set the record straight about the shared information. All information that could identify players was removed from the data given to the researchers. Chat logs were not shared at all. Read on for SOE's full statement. "The information Sony Online Entertainment provided for the research project was scrubbed of all PII (Personally Identifiable Information) prior to being provided to the researchers. For example, no content of any player chat logs were shared with the researchers. The information shared consisted of data such as which in-game characters chatted with each other and the volume and frequency of the contacts. None of this information was connected to, or linked with, the real names or other PII, of any players. Basically, the researchers looked at the connections between players and how their online networks were built and used, not the content of any actual conversations that these players may have had with one another (that content was not made available to the researchers). Additionally, some EQ2 players voluntarily participated in a blind survey concerning their playing habits and demographic backgrounds. This information was anonymously gathered and shared between SOE and the researchers. Finally, corporations such as SOE and the researchers' universities have controls and safeguards in place to help to protect the privacy of individuals who voluntarily participate in surveys for research projects such as this one."
People didn't trawl through 60 terabigadits of data to find out things about being social. They went down to road to the local shindig and tried to talk to a girl.
Now, get off my porch.
Moved to http://soylentnews.org/. You are invited to join us too!
The Female elves aren't what they appear to be. LOL
Don't meet offline, trust me on this!
gold farmers?
People didn't just go down the road to the local shindig. They organised meetings online and went on booze binges with fellow socially inept MMO players. We knew we were awkward, at least we were awkward Together!
FTFA: Mostly, the gamers seemed healthy; their body mass index was better than the US average and, although they were slightly more depressed than average, they were also less anxious.
How on earth is Sony measuring their customer's body mass index? I know 60 TB is a lot of data, but I don't believe that even the most magical of algorithms can derive player body mass indexes or whether they're "slightly more depressed than average" from it. I call bullshit.
Sony keeps a giant database of what everyone is doing...yikes.
what a waste of space.
"social" data?
from a MMO?
my god man....I can find thousands of better reasons to use 60TBs of space and not have to use the best one of "for p0rn."
shoot...mapping the world...mapping the oceans, mountains, space, al gore's head, folding, seti, another wayback project, etc.
In particular, if the researchers have access to user/character names (or worse, real ones!), Sony could be in violation of the Children's Online Privacy Protection Act. I'm not a lawyer, but:
(F) any other identifier that the Commission determines permits the physical or online contacting of a specific individual;
Particularly if names can be correlated to age, it seems unwise. Without assurance that personally identifying information is not in the data, any parent of an EQ player should be concerned that they were sent to "a collaborative group of academic researchers at a number of institutions".
So in the future when they have this all nailed down, if I am freaking awesome and everyone counts on me for something and they determine me leaving the game would cause like a dozen other people to quit the game cuz it's just not the same without me, they might start paying me to keep playing instead of me paying a subscription fee? :-D that'd be awesome lol.
Google's Super Secret Search Algorithm: SELECT @search_results FROM internet WHERE @search_results = 'good'
Actually, it was probably a survey, but you do raise a valid issue there about privacy. Exactly how much data did Sony share?
We were outraged that AOL shared some improperly anonymized search strings, which in some cases could lead to the guy's real identity. Here, if they even knew who to survey, it means that Sony just handed over RL names on a silver platter.
What else? Chat logs? 60 TB is a hell of a lot of data. And probably a lot of things you said at one point, but didn't think them too carefully, and probably didn't mean them in the long run.
As an easy example, think of all the people who've bought gold, or played a flirty female when they're male, or flirted with one who turns out to be male. Lots of public embarrassment potential. Or what if you talked about sex to someone who, as it turns out, was 8 years old? I'm sure some people would love to jump to conclusions there.
Sent some tells about sometimes wanting to kill your classmates or co-workers? I'm sure those will be worth some hysteria when the next school shooting comes around. Talked about drugs or about homosexuality? I'm sure some future potential employers would make a fuss about that. Confessed playing or chatting from work 'cause you're the network admin and the logs don't apply to you? Well, now Sony's logs do. Etc.
And how much of the billing data is in there, anyway? Enough for someone to steal your identity? But even if it was just enough to contact those people IRL and survey them, I'm guessing at least the email address must have been in there. I'm sure some spammers and phishers would love to have it too.
Basically even if you trust that those researchers probably won't do that, the circle of people with access to someone's private data just grew. It only takes one irresponsible git or disgruntled admin, or even an insecure network which someone can break into and look around, for that circle to grow even further. How many steps until someone does do something unethical with it?
Sony already did their part in not giving a flying fuck about protecting their customers' privacy, after all. It only takes one or two more people with the exact same cavalier attitude, before it bites some people in the arse for just trusting Sony.
A polar bear is a cartesian bear after a coordinate transform.
I can see how Sony was happy, but I can't see how raping people's privacy for a buck is ethical in any form or shape. And in some parts of the world, it's probably even illegal.
Because make no mistake, it was for money. Simply put, "Let's see if these people can figure out how we can make more money in subscriptions." Apparently the prospect of a quick buck is all it takes for a corporation to sell your data.
From the biblical 30 silvers, to Sony doing it for the _hope_ of getting 15 dollars a month out of you, not much has changed. In fact, it seems to me like competition just drove the prices down ;) Each individual person that Sony sold, _maybe_ they can get another subscription out of 1% of them, and even then for a month or two. Effectively Sony sold everyone for the hope of 30 cents or so a piece. Makes me respect Judas more by comparison.
A polar bear is a cartesian bear after a coordinate transform.
You both make good points about the privacy issues. Games like this are wide open to be data mined. But then the whole Internet is effectively an extension of this datamining. The whole Internet is becoming like a giant storage system to the data miners.
Its like the old say, knowledge is power. Unfortunately some people fail to realize everything they do online giving someone else small fragments of power over them (and the small fragments add up) because that is the way the companies want it. They want to data mine everyone. No company or government is looking out for people's privacy. They don't want to protect privacy because they are the people who want to abuse everyones privacy. They all want as much information on everyone as possible. Knowledge is power. The only time they ever want to consider controls on privacy, is when its their own privacy at stake.
Its also the boiling frog principle as each small loss of privacy on its own doesn't seem important. It'll only be years later, when we all look back and they have our every thought and movement stored in their databases that we can see just how much total power it gives companies and governments. But by then it'll be too late.
The more information people give away, the more easy they are to control and manipulate. Which is exactly what companies and governments want. The irony is, psychologists have shown the people who think they're not easily manipulated have been shown they are the most susceptible to manipulation. (It makes sense because the most susceptible to manipulation have not learned to look out for manipulation).
Here's a paper on the subject...
"Creating critical consumers: Motivating receptivity by teaching resistance."
http://www.niu.edu/user/tj0bjs1/papers/sc04.pdf
Datamining is becoming a huge growth industry because knowledge is power and the minority of people in power want to keep (and grow) their money and power over people. Its been the same thoughout history but now they have access to more information than their predecessor's could have every dreamed possible.
Plus its going to get worse. Much worse. Even the UK Lords constitution committee has warned at how far surveillance of people is going. Where the UK is now, the US and other countries are going to rapidly follow...
http://news.bbc.co.uk/1/hi/uk_politics/7872425.stm
Would be the month WoW came out. I lost most of my EQ2 buddies to WoW.
There are places where the networks are not touching,and there are places where they are-Boeing's Lori Gunter
> Why did you think you have any reasonable claim to privacy in an MMO?
While I admit that not all speech is protected and some of your examples might have a bit of validity, your sense of proportions is way out of line.
> If you "sent some tells about sometimes wanting to kill your classmates or
> co-workers" not only would I not expect you to retain your privacy and anonymity,
> but I would expect Sony -- and anyone who read your tells -- to feel an obligation
> to notify the law.
Sony, maybe. From Sony's point of view, the tradeoff is relatively simple: what is the risk that this user will actually murder someone, and what kind of monetary damage could it cause Sony if that happened and Sony didn't report anything to the police beforehand, versus the cost of constantly policing the chats of that online game. But if you think your average police station is interested in investigating what might be tens of thousands of such reports (remember, you wanted every single user who heard this to report it), versus patrolling the streets and other more usual police duties, you're just being stupid. (I could understand your supporting users snitching to Sony, however, especially if the ToS encouraged or required it.)
> Talk "about sex to someone who, as it turns out, is 8 years old"? You should spend
> a night in jail just to cure you of your stupidity. Don't they teach you in school
> specifically *not* to make any assumptions about the age and gender of someone you
> "meet" online?
If this was in one of Disney's MMOs, well, OK, then jail is reasonable. If this was in a setting where the ToS explicitly forbids minors, then again, you're just over-the-top. It is unlikely in the extreme that any court would find you guilty of anything, unless you really have good cause to suspect that the person in question was a minor.
> Do not write anything in any online forum anyplace, even under a pseudonym, that
> you would not be comfortable having viewed by your teacher/boss/wife/husband/neighbor.
Again, fearmongering. You are correct that there is a risk of everything you post being connected with you, even if you use Tor or whatever, but, depending on circumstances, this risk could be very, very small. Like with everything else, you have to balance the risk of the post being connected with you versus the benefit of posting what you want to say. Just like when you decide to get in your car and drive to work.
> You do not have any right to privacy online (somehow this slipped past
> the Founding Fathers) and you are a fool if you expect any.
The legal system has a lot of inertia, and only just now starts to address this issue. So even if there is no recognized legal right to anonymity online right now, there very well could be one in another 20-50 years.
> Apparently the prospect of a quick buck is all it takes for a corporation to sell your data.
The irony is that Sony is legally required to do that if it seems that that would increase its profits. The only thing which would justifiably prevent it would be a reasonable fear of bad publicity causing a decrease of revenue, instead. Hmm, how many Slashdotters play Sony MMOs? And exactly how many fewer will play Sony's games because of this?
I understand your point, and it's a very valid one, but:
1. Supposedly, they also found such stuff as what the average gamer age is, that they're slightly depressive, or what their BMI is.
It doesn't exactly sound like that's strictly about the characters, sorry. (I'd think that all those dwarves and barbarians would have a higher BMI than these guys found;) As I was saying in another post, even if they then surveyed those people, the fact remains that they knew who and where to contact. I.e., at the very least that WillowCakes341 can be found for a survey under the name Random J Sixpack under the email address randy6p@example.com.
2. Plus, see the thing about AOL and their improperly anonymized data. AOL hadn't handed over the usernames too, but thanks to including the "vanity searches" for one's own name, it was pretty trivial to find out a lot of identities.
If that 60TB of data is truly raw and unfiltered, you can find lots of people telling their names and/or email addresses to each other, discussing their family issues, etc. A lot of people are simply entirely too social and naive for their own good when online. I wouldn't be too surprised if for a lot of those customers you can find everything from name to RL address to name of their spouse and children in some chat logs, just because they thought it's a private tell to a friend.
3. And, again, I would bet that a lot of that includes information not about just themselves, but about friends and family who never consented to having their lives made public. Even if I were to take the nasty attitude that, basically, "if X was stupid enough to put his life online, X deserves what's coming to him now", I fail to see why should people Y and Z suffer too just for having been acquaintances with X.
E.g., in the past mom drove me nearly neurotic with telling everyone every single freaking detail about me. I grew up with the idea that pretty much every single neighbour, teammate of mom's, acquaintance of an acquaintance, or even random strangers on the bus, would know even at which time I took a shit yesterday. I'm not exaggerating. In retrospect, probably most people didn't even actually listen, because few things are more boring than a mom talking non-stop about her kids. But if you do the same online, and that's a lot of information archived and immortalized for anyone to trawl through.
A polar bear is a cartesian bear after a coordinate transform.
1. I hope you realize that exactly that was what allowed the Soviet Union to keep its citizens in check.
Yes, everyone knows about Stalin's brutal repression, but that was toned down a lot after Stalin. They discovered that you can control people easier by making them think you have a dossier on them, and they can eventually get bitten in the arse by something they said in the past. And that even something which doesn't outright warrant a one-way trip to Siberia can bite them in the arse in some other way. Like maybe they won't ever be allowed to travel abroad again, or they'll never get a promotion now, or whatever.
It actually worked better than Stalin's executions and mass deportations, surprisingly.
People learned to do exactly what you seem to advocate: don't say anything you wouldn't be comfortable explaining to the nice commissar. Don't assume anything about someone else, other than that they might get you in trouble. (E.g., by being an agent provocateur and trying to get you to say something that'll remain on your record for ever.) Distance yourself from anyone discussing those forbidden things, you wouldn't want to go on record as associating with that kind of people.
It just made them unable to organize in any form or shape. That guy talking against the Party became not a guy to rally around, but someone who'll probably get you in trouble if you join him. You don't want that attached permanently to your record.
As an example of how well it worked, think Sakharov. He was a very loud in speaking against the regime... but nobody joined him anyway. The party was feeling secure enough about it to only slap him with an "exile"... to a relatively decent job in the fourth biggest city in Russia. Not in Siberia either. And while he did get a bunch of visits from the police, none seemed to be brutal or anything. Probably more to show everyone else that Sakharov _is_ being watched, and the party will know if you associate with him. It worked like a charm. Millions of people who secretly aggreed with him, didn't want to actually have it added permanently to their dossier.
And you have to bear in mind that we're talking about the dead-tree kind of dossier in the USSR's case. They had neither the manpower to actually supervise everyone and record everything, nor the search engine to actually find anything unless you actually gave someone a good reason to read yours. It worked anyway.
The recent trend of everything being recorded and indexed on the internet, could create a chilling effect of much more epic proportions.
2. Even if you take the approach of "if you were stupid enough to say X online, then you deserve whatever's coming to you", in reality it's trivially easy for someone to say stuff about you without your having consented. There'll be tells, blog posts, etc, discussing various things you've said or done or were mis-heard of saying.
So in effect if you want to have any kind of protection, you have to not say anything debatable to anyone, never assume about any person you meet IRL that they're not going to send an online tell about it, etc. Welcome to the wonderful world of talking only about the weather, and never trusting anyone. As I was saying, that's what turned the USSR into a mass of isolated and easily bullied individuals too.
A polar bear is a cartesian bear after a coordinate transform.
If chat logs were handed over to someone who is not SOE, then this transaction is a huge breach of privacy. Phone numbers, addresses, userids and passwords, admisison of illegal acts (smoking pot, drinking underage, sex underage, etc) all of these things are surely in the chat logs. I cancelled my EQ2 subscription about 3 weeks ago because of my own financial concerns. (The economy bites, if you hadn't noticed.) I told my guild about it and that I intended to return once it looks like I had some stability in my economic situation (no rent increases, no layoffs at work, etc). This announcement, though... it guarantees that I won't be returning to EQ2. I think I'm ready to swear off SOE offerings altogether. Is it time that we went back to pc gaming without network ties? That's what I've been doing with my playtime since cancelling my online subscription. I've rediscovered some old favorites. Maybe a smart game studio will see this change as a developing trend to take advantage of. I'm ready for more (good) pc games. When's the last time that you saw a dungeon delver where you got to fully customize your entire party for its adventure? The last I can remember is Wizardry 8 which released back in 2001. I guess it's time to pull it off my dusty shelf and install it for more fun game time without exposing my personal habits so somebody else can profit from selling my information.
Really? It seems to me that among other things you were already dismissing as "lacking the common sense God gave to a dog" were even discussing drugs or homosexuality.
Make up your mind. Either we're talking about Dark Elves cybering and other uninteresting stuff, or about topics which affect legislation and taxes.
E.g., it seems to me like debating homosexuality is how those guys got any rights in the first place. Otherwise we'd still be at the stage where it's a capital offense.
E.g., discussing drugs seems to me like an important issue, since my tax money essentially goes into sending some people to jail. I would think that destroying someone's life for having smoked pot, would merit more thought than that.
Among other things, because of this thing called "democracy". The people are supposed to decide if the majority wants those laws or not. But you can't have an informed and rational choice if one half of the debate is essentially bullied into shutting the fuck up, for fear that they'll make themselves unemployable if they take that side. That's the way towards groupthink and bad laws.
Plus, to quote Adlai E. Stevenson, "A free society is one where it is safe to be unpopular." Once you've essentially made it unsafe to say anything unpopular even in a supposedly private tell or email, that society has ceased to be free.
So if you see these kinds of issues as on par with "people disguised as Dark Elves "cheating" on their wives through chat channels in a video game"... well, to put it mildly, your sense of proportions is _way_ off.
A polar bear is a cartesian bear after a coordinate transform.
Want nao!
That's a lot of a/s/l tells...
> Sony is legally required to follow it's shareholders wishes
And barring a shareholders' meeting actually deciding something to the contrary, IMO most courts would assume that those wishes are to earn as much money as possible, without doing anything illegal or morally repugnant. Meaning, morally repugnant to Joe Sixpack, not to your average Slashdotter. Or do you have some kind of precedent you could post as evidence to the contrary?
... until you arrive at the cybersex chatlogs. Then it just gets really awkward.
For those of you who every day click accept to SOE's End User License Agreement and dont read what it says I will copy paste some Quotes from it. You can find all of the EULA and Terms of Services at http://www.station.sony.com/termsofservice.vm
"when you communicate within any game or any other communication feature within The Station (e.g. live chat, instant message services and the like), even "privately" to another person, you do so with the understanding that those communications go through our servers, can be monitored by us, you have no expectation of privacy in any of those communications and, accordingly, you expressly consent to monitoring of communications (including technical support and customer service communications) that you send and receive"
That is for every Online game Sony Provides. Also For those who want to say Sony is the only ones doing this, refer to your EULA for Blizzard or any other online service Including XBox 360 or even the Wii
So Sony cheerfully hands over all the logs, including chat. Had you known every single conversation you had, in public or private channels, in the presence or absence of any other living being, yea every letter you typed (including all the combat commands you flubbed) would be given to others to read, would you still pay them your money? I'm very pissed about this and I only played for two months. If I were still a subscriber I'd be about ready to drive to SOE headquarters and "thank" them for guarding my privacy so diligently.
Every child learns, but not Sony apparently, that just because something is possible doesn't mean that it's good. (I.e. Sony has the legal right to do this, but should they? Someone decided "yes".)