Slashdot Mirror
Microsoft Tracking Behavior of Newsgroup Posters
theodp writes "Ever get the feeling your Usenet newsgroup list is being watched? By Microsoft? If so, consider yourself right. An interesting but troubling CNET interview with Microsoft's in-house sociologist goes into how the software giant is keeping a close eye on newsgroups and other public e-mail lists, tracking and rating contributors' social habits and determining "people who the system has shown to have value." Those concerned that it's not a good idea for computers to track their belongings and whereabouts are advised that they may ultimately have to fragment their identities, keeping multiple IDs and e-mail addresses."
Hopefully the general contempt for proprietary, inferior solutions will drive them towards some better stuff.
Get thee glass eyes, and, like a scurvy politician, seem to see things thou dost not.--King Lear
" Those concerned that it's not a good idea for computers to track their belongings and whereabouts are advised that they may ultimately have to fragment their identities, keeping multiple IDs and e-mail addresses."
Who isn't already doing this?
With the advent of spam most people I know abandonned their first email address years ago. I have one for each service I use (including slashdot).
Hmm a system that tracks who's posts are of value and who's are not. I would suggest a scheme where they mark people's post as "Interesting", "Informative", and other such words. Maybe some way to mark them as "Funny" and even "Flame bait" or "Troll" if they are just obnoxious posters trying to get a fight going.
What do you think? Would it work?
Oh wait!
My god, you are so naive.
BOO! TERRO
The AURA just sounds like the CueCat Digital Convergence people who wanted to put a bar code on everything. Again, MS is not the company I'd like to see doing this.
*Rather Offtopic - but Digital Convergence used to advertise the CueCat with an 'Angel coming down to earth from heaven to barcode everything' and the well-known Digital Angel RFID people have also made a religious reference in the company's name. The hue and cry of Christian's 'the number of the beast' references beg the question:
Who the hell is doing marketing for these people? I remember getting an icky feeling when I saw the 'infomercial' for the CueCat, and similarly the Digital Angel website. And I'm not the 'churchy' type. I can only imagine what the fundies think...
* This idea is copyrighted. Use of this idea may not be used to more attractively market 'evil' technology, or put a chip in my head. Thanks.
Woof.
--
If code was hard to write, it should be hard to read
Don't tell me that you post on Usenet and expect those posts to be "private"! Give me a break. If ANYONE wants to read and study how people interact on this most public of forums, I fail to see how anyone can object.
And if Microsoft weren't doing this, wouldn't there be articles appearing with titles such as "Microsoft ignores valuable customer feedback available free on Internet"? I am no big fan of Redmond, but I think they are almost forced to do something like this to avoid being blindsided.
sPh
So it's like Karma on Slashdot, but on a more stealth level, like Google PageRank.
It's more like a Google PageRank implemented Newsgroup posters instead of Web Sites, and run by Microsoft instead of Google. Microsoft is just adding true statistics and tracking to the already existant "human credibility" of posters.
Newgroups posts are public.
I don't see this as a problem.
-Pete
Soccer Goal Plans
Anyone can do this.... But since it's Microsoft, it's doubleplus ungood.
Is a suitable state of mind when large and powerful groups decide they want to spy on you.
:)
I'm sure MS already spies on Slashdot and tracks every profile here. I have four, and switch between them carefully, unt sometimez I speek in forin lanjuajes just to confooze them.
On the other hand, this reminds me strangely of a scene from Dilbert.
Serf1: Boss, I need to monitor newsnet.
PHB: why?
Serf1: So we can track our competitors, manipulate public opinion, and run smear campaigns against political opponents.
PHB: sounds fine...
Serf1: It will take nine months, that's ok?
PHB: yes, get someone to help you if you need it.
later..
Serf2: So, did you get it?
Serf1: Yes, we're now official newsnet spies.
Serf2: porn on, dude!!! alt.binaries, here I come.
Serf1: I've asked for some new hard drives too...
Ceci n'est pas une signature
How is this "troubling"? They are researching a way to make USENET and such more effective. They aren't interested in the fact that cmdrtaco@slashdot.org posted to alt.sex.unicorns 10 times last month.
This is good valid research, the type that applied research CS programs should be doing. Thismay actually make a difference in a deployed product.
I think we should tone done the M$ and SCO crap for a while.
Okay, anybody who signs up for a message board with thier real information, or creates a mail account with thier information, or posts to newsgroups with real information is just asking for this sort of thing to happen. I'm pretty tired of going to websites and having to sign up. I NEVER put in any real information, and encourage EVERYBODY to put in fake information. Why do they ask for this information? So that they can do exactly what MS is doing.
Now don't get me wrong, I don't think that this is some sort of plot of evil. Well it sorta is, but the whole motivation behind any kind of information gathering is money. They want to spend less on advertising by targeting only the people who will show interest in thier products. The more they watch people like this the more money potential they have.
The best way to keep your privacy from becoming an issue and all of these information databases getting merged on you is to NEVER, EVER give out your real information to ANYBODY, especially on the internet, unless it's a secure SSL transaction, and you really trust the source.
Bide your time well, Linux zealots, for the mighty power of SCO's IP will reign down upon thee!
Darl McBride
Chief Executive Officer
Caldera International, Inc.
People READ my public POSTINGS?
I'm JUDGED by what I say in PUBLIC?
MY GOD!
The only thing that bothers me is that MSFT pisses away stockholder cash on this, unless they can somehow turn it into legitimate market research.
BTW, they read slashdot too. If the editors cared about this sort of "invasion of privacy", they'd remove the AC posting limit.
And why does a site so rabid over the issue of online anonymity have to refer to anyone who chooses to post as such as a coward?
I don't need no instructions to know how to rock!!!!
I figure we have nothing to worry about. If Microsoft wrote the tracking software, then it probably doesn't work anyway.
---
Lousy rotten karmic retribution.
I often check my logs to see where visitors are comming from and if it's a message board I stop by and read what people are saying to see what motivated them to go to my site.
Many companies (stars often check out what fans are saying around the net) are probably scoping out message boards/newsgroups to see what people are saying about their products. And plenty of people have opinions about various products but most people are less than stellar when it comes to intelligently expressing why they feel the way they do.
"It sucks" is not helpful to companies in their quest to improve their products. And people who bitch about everything or praise everything also aren't worth paying attention to.
It's called market research. This is a non story. "I want to have an opinion about X but X better not read it!" is just dense.
Ben
Work Safe Porn
- This monitoring goes on exclusively in the msnews.microsoft.com domain, plus a few others that are also run by the company. While NetScan is sometimes pointed to MS-oriented news servers (news.devx.com is an example), Microsoft is not "monitoring USENET".
- Marc Smith is a very sharp guy who has done a lot of interesting work with the social dynamics of online communities. Goggle him for more info. And if you have questions about what NetScan does, give it a whirl and form your own conclusions.
- At the moment, NetScan is used by the MVP program to follow members' posting history. The MVP program is not exclusive to NNTP, however.
- I can't see how this goes into the "YRO" section - if Microsoft is monitoring the news servers it operates and that bothers you - don't post there. This is hardly the land of the Microsoft advocate or even user for that matter. This is like reporting that I'm painting my bedroom bright red - WTF do the neighbors care about that?
Yet another hysterical ad revenue generating headline, brought to you by the Slashdot "editors".What's so alarming about this?
It's no different than any social study on the general public. It's done in academia all the time.
If someone thinks their Usenet posts are so damn sensitive or private they don't want people to look at or study them later, don't post to Usenet or use an anonymizing service.
In Soviet Russia, I ruled you
This is old news. With .Net, Windows Update and Lord Knows what else, it should be no suprise to anyone that Redmond is poring over any and all soft-content being created using any of their apps.
Not only is it a near limitless cache of information, there is near limitless ways to use it. They can market new crap, er, products to us; determine how to repackage and (attempt to) re-sell information to anyone who may buy.
You post info to misc.transport.road, for example, on the lastest news regarding the Maumee River crossing project (the massive I-280 bridges in Toledo, Ohio), you'll get spammed, er notified about Micro$oft Streets and Trips 2004.
Post a concert review on another newsgroup, and you might get something from Ticketmaster. And guess who gets a cut: some software company in Redmond.
Not to be paranoid or a conspiracy theorist, but it should be evident to anyone with even a couple of firing synapses that Microsoft is no longer a software business. Software is only a stepping stone to a larger avenue of revenue: human thought, human knowledge, human behavior, and the exploitation thereof in any way whatsoever - so long as it provides a revenue stream.
The problem with socialism is that they always run out of other people's money. - Margaret Thatcher
Tracking users who provide value?
You mean like keeping track of poster through karma ratings?
The next day he was showing Ben Schneiderman some of this stuff at the open house. A bunch of us looked on as they chatted, planned visits, golf outings and talked about how it all worked.
Depending on the queries he gave it, this one program would chew through data from usenet. and give back all kinds of stats and then draw relationships It even did graphical representaitons of users' actvity. Density of posts in a single thread versus starting new threads, frequency of posts, replies vs. new messages etc would be denoted by distance from the main timeline, darkness and width of the circel and so forth. You would look at a wide but faint circle and say (and I may be off in how the key worked, but ...) "This guy sticks to the topic over a long period of time" or you could denote the flame warrior or the vagrant by their graphical representation and so forth. The way the data was processed was really cool and how quickly you could start to decipher the keys was really interesting.
The Big brother implications ... well that's a whole 'nother thing there too isn't it?
Que Deus te de em dobro o que me desejas
[May God give you double that which you wish for me]
At least they're not big brother.
Rate me higher Microsoft. Hire me Microsoft. I want to have your children Microsoft. I know your watching this site Microsoft. I'm identity # 285-75-4210.
You say things that offend me and I can deal with it. Can you?
One of the first tasks of any individual joining a group is to determine the pecking order within which authority is distributed. This is a critical task that humans have been doing since before they were human.
What's being talked about here is reverse engineering trust heirarchies, algorithmically, simply from a discussion corpus extracted from Usenet.
This is very, very cool stuff. It is a hard application of a soft science, and if its results match empirical data, it represents a greater level of understanding about the human mind.
This is something to celebrate and take interest in, not malign simply because it's Microsoft that's behind it.
I do remind the security paranoid that reputation management remains one of the few characteristics obsessively protected in otherwise anonymous systems.
Yours Truly,
Dan Kaminsky
DoxPara Research
http://www.doxpara.com
Now that they've confirmed that they do this, there's only one thing to remember:
Before you fly up to Redmond for your interview, make sure you post a year's worth of insightful commentary on major relevant newsgroups, with your name and email attached
--
Use Vobbo for Video Blogs
An interesting article from the School of Common Sense shows that your public actions are being monitored by everybody who sees them!
"The privacy implications of this are staggering," says some guy who gets inflammed by things. "People could figure out all sorts of patterns about your life. Why, if they observe you going to the pet store, they could actually figure out that you likely own a pet! Next thing you know, you'll be getting subscription offers for pet magazines. Nobody needs that."
People who fear this massive intrusion of privacy have several options open to them. First, the use of full face masks, and body costumes, can confuse potential observers. Make sure to change masks and costumes frequently. Visiting stores and locations that you wouldn't normally visit can 'sour' their tracking data as well.
"If you have children, drop them off at a school that they don't attend," says Imflammatory Boy, "and tell them to walk to their real school."
Vintage computer games and RPG books available. Email me if you're interested.
While it's true that such simple messages and "Fr1st p0st" equivalents can't be mapped, they can easily be filtered out as noise, since there's no content anyway.
However, if you're posting reviews to Amazon or ePinions your text is likely to have analyzable content.
I know someone who has done this type of analysis and discovered people who reply to their own posts in dicussion boards under different IDs to make it look like they had some kind of consensus. When confronted with the analysis, they admitted the ruse.
This is not some conspiracy thing where Microsoft is trying to collect secret data on individuals.
The article is about this guy at MS and what he does there. The are several projects he is involved with.
One is the Netscan tool. This is available for use by the general public. You can run it yourself and seen what it can and can not do.
http://netscan.research.microsoft.com/
I beleive that it was orginally created in part to help identify helpful people in the user community so they could be rewarded (becoming and MVP for instance) They do not discriminate against you based on what platform you use as a desk top or what OS your website is hosted on. Just if you regularly post stuff and reply to posts.
I do not know much about the other tool except what is in the article.
The other tool is very much unrelated to newsgroups and like the cue cat on steriods execept I do not belive data goes to the parent company.
Say...anyone know if there is still a display case near the elevator in one of the basements of the EE building at Caltech that displays less-than-successful projects of the faculty and students? If so, does it still include Carver Mead's 4K write-only memory card (from back in the days when 4K was a lot of memory)? (No, it wasn't INTENDED to be write-only).
What we've done is highlight the 40 threads that got the most number of messages in this period--day, week, month, year. And we'll say, Here are 40 really big threads.
Well, at least he's found a meal ticket. I mean almost anybody's who's spent ANY time on USENET knows that the size of a thread is a poor predictor of useful or interesting content. While there is a chance that the thread is interesting, there is also a VERY good chance that it's a mishmash of flames and massivily offtopic digressions. This is clearly demonstrated by the netscan application referenced in the article.
Roving Web-Teleoperated Robot
Well, given the very pro Microsoft stances that many folks have here in response to anything critical of Microsoft, I have wondered if they are paying attention to Slashdot as well. Especially considering that many of the rabidly pro-MS posts are posted as AC.
:-)
Modded as Offtopic and flamebait? Oh, no. It's worse than I feared. Not only are they paying attention to Slashdot, they have infiltrated the ranks of moderators!
Visit Jonesblog and say hello.
You just pulled that out of your ass, and you know it. There are so many gigantic misunderstandings underlying that statement that I can't even begin to attack it, so suffice it to say, a simple Bayesian analysis more than likely cannot identify people based solely on what they write.
Ok, I'll give you a hint. Suppose we apply this method to Slashdot. There are about 650000 Slashdot readers. You are talking about calculating the class-conditional probability for every user on Slashdot. The differences in class-conditional probability (per user) are going to unbelievably small -- so small that any results you achieve are going to be statistically meaningless.
Bayesian techniques work okay for classifying when you've only got two or three buckets. But when you try to apply it to say, thirty buckets (much less 650000!!) it breaks down really quickly.
Also, remember that the true name for the technique is "Naive Bayesian inference." In this case (heh, in most cases) the term "naive" doesn't mean "clever and infallible."
Yes, I do research on text analysis algorithms with applications to anti-spam filters, so I do have some clue what I'm talking about.
We were having a lot of trouble tuning our psychoanalysis routine. There was this one user on slashdot that kept crashing the system. We finally decided that the user is one of the worst recorded cases of multiple personality disorder. Some of the personalities were found to be incredibly psychotic and anti-social, others brilliant. Basically all over the map. Finally we just had to filter out all messages from Mr. Coward.
I am Barlo Mung. Barlo Mung is me. It's my email address. It's my counter strike nic.
I'm not going to pretend to be anyone else.
Want to dredge up all the postings I've made anywhere on the internet? Go ahead. WTF do I care. If I didn't want people to read it and know it came from me, Barlo, I would not have posted it.
At least on their own newsgroups (the microsoft.* hierarchy) they've been doing this for years. Back over 6 years ago I was a Windows programmer (don't worry, a long time ago I saw the light and now am a linux programer!) Anyway... Because of the work I was doing at the time with Windows and was also answering a lot of questions in the microsoft.* newsgroups I attracted MS's attention. They made me an MVP (Most Valuable Professional) back around '95.
One of the things MVP's were told was that MS tracked our posting habits in their newsgroups. They used our e-mail addresses for this. The tracking was purportedly to help determine if our MVP status would be retained from year to year. (it's an annual award) Since they acknowledged way back when that they were tracking users on their own newsgroups it really doesn't surprise me all that much that they'd expand it to cover more groups.
Actually, given that Google has an archive of many of the newsgroups it really wouldn't be all that difficult for pretty much anybody do track individual posting habits, etc. Just run some searches for the e-mail address of the user in question.
For a few months I did this for MS Games. I searched newsgroups and fan message boards to see what players were talking about. If everyone was pissed off and wanted a new patch, that's what I reported. If people were excited about a certain feature, I reported that, too. If you ever flamed MS for something you didn't like, I might have sent it to them.
One choice quote from memory... "WE NEED A PATCH. GOD IF YOU SHOVED SOME COAL UP THERE ASSES YOUD GET A DIAMOND!!!LOL"
It paid $10/hr, and I needed the money.
are seeing this for what it is: "No big deal"
/crashes for some reason, it reboots the box anyhow, i.e. a problem with GDI reboots the box either way.
This is NOT big brother. This is about building valuable meta information on top of usenet. Why ? Because one of the things MS heard long ago is that people liked linux because they could go to a newsgroup and get help with it, often from the people that wrote the component in question ? What did MS do ? They responded - MS employees now monitor the microsoft.public news groups. We respond to posts, try and solve problems for people, answer questions, debug code, etc etc. I myself can be found occasionally posting in the Visual Basic newsgruops (where we have lots and lots of non-full-time or beginning programmers that really need just a little bit of help to get them going).
The people that _write_ the VB compiler are now monitoring VB newsgroups to try and help connect with real customers and to really understand how people use and dislike MS products.
Managing and making sense out of the whole mess that is usenet is a nightmare, and MS Research is doing some good work in this area. MS has some internal software that treats usenet posts as "issues" and determines if they've been resolved or not, if they need followup, etc etc. One interesting thing we've found is taht there are many issues resolved by "the community", i.e. non-MS employees that are subject matter experts. I don't know the details on this but I think we make an effort to track who is and isn't a great contributor and maybe they get some sort of compensation or recognition or something.. like i said i don't know the details of that at all..
In any case, the point of this usenet data mining is to try and analyze the incredibly huge sea of usenet. We want to figure out what kinds of problems people have, what people are causing noise, what people are really helping other, etc etc. There is no nefarious invasion of privacy here, the only thing that is analyzable is what people explicitly post to a public forum...
Look at my userid - i was a slashdot reader long before i work where i currently do. Back then, the MS bashing and second guessing definitely took place, and i even participated. I'm still a slashdot reader but I do get awfully tired of the sheer volume and irrationality of negative-MS stuff that happens here.
When I started at MS, I found out awfully fast that many of my arguments against MS were speculative, but mostly it was me being factually wrong and talking out of my ass. I remember in my original interviews i was trying to lecture an NT developer about how putting GDI in kernel for NT4 was stupid because it would lead to crashes. How pompous of me! It was something I read on some stupid website or industry rag. Later I found out (from reading Inside W2k -- excellent book) that it was irrelevant because if the session manager sees that the GDI user-land process exits
So after 8+ years of hating MS and talking out of my ass, followed by 3+ years of working at MS and realizing how much i was talking out of my ass, I'm doing two things:
1) talking out of my ass less
2) telling others that are clearly talking out of their ass that they are doing so, so that they can
2a) stop spreading misinformation
2b) have their eyes opened that nobody is impressed by their incorrect speculations and their emotional campaigns of disinformation
I know im not preaching to a sympathetic audience here, but honestly, the speculation, questions, etc people have about MS could be answered truthfully and honestly if some of you would bother to ask, or do some research. But unfortuneately i know all to well (because i used to do it) that its easier, and certainly more fun, to beleive everything you _want_ to beleive about MS that bolsters your own predetermined mindset. If, for example, you find yourself referring to an article that The Register wrote, please stop and ask yourself what the hell the regis
My opinions are my own, and do not necessarily represent those of my employer.
They've been doing it for years. If this one email will just get around to 25,000 people, Bill Gates will send everybody a $1000 check. It shouldn't be surprising that they're monitoring Usenet, too, probably just to send checks to people there.
Do you have ESP?