Data Mining Amazon.com Wish Lists

Dr. Webster writes "In his article "Data Mining 101: Finding Subversives with Amazon Wishlists," Tom Owad of Applefritter outlines a way in which one could build detailed personal profiles of hundreds of thousands of U.S. citizens in a matter of hours. Reading habits, personal tastes and even political party affiliation could be inferred from the results, and through the use of Yahoo! People and Google Maps, one could even map out geographically where people with certain interests or affiliations live, down to their address. Most surprisingly, the process of doing this is completely legal, and doesn't even violate Amazon's Conditions of Use."

Comments

Mining voluntary information on a public website?

[Saven Marek]

Mining voluntary information on a public website? Come back and tell us when you can mine the info as easily from say real amazon sales records of what I actually did buy not what I might want the public to think I am buying.

This is not a story. This is not news that matters

[Doomedsnowball]

Breaking news! People conducting surveys report other people freely giving away personal information! That could be an article from http://www.theonion.com/. Shocking. Call it a "wishlist" not a "voluntary survey about what you like" and it's an amazing invasion of privacy.

Re:Mining voluntary information on a public websit

[insertwackynamehere]

i was gonna say the same thing, I'm really for people's rights online (i'm like insane about freedom of speech and stuff if you ask my friends), but I honestly dont see what people expect. If you put information online for the public, this is what happens, no conspiracy, no illegal or suspicious activity. This information is voluntarily released for up for grabs, if people want to use it for that I dont see a problem.

Re:This is not a story. This is not news that matt

[Reverend Darkness]

Actually, if you had read the article, you would have seen that Mr. Owad does not reference "invasion of privacy" at all. What he does do is help people understand how information they share online can be used to create a general profile, and even to link them to others. The point of the article is to educate people. But, like others, you were probably just going on the snippet... I mean, why read the whole story when you can see the headlines via RSS, right?

Point of the article

[terradyn]

Most of the comments seem to be along the lines of: "What use is it to mine wishlists?" You're missing the point of the article. His main idea is from this section of the article:

This is what's possible with publicly available information, but imagine if one had access to Amazon's entire database - which still contains every sale dating back to 1999 by the way. Under Section 251 of the Patriot Act, the FBI can require Amazon to turn over its records, without probable cause, for an "authorized investigation . . . to protect against international terrorism or clandestine intelligence activities." Amazon is forbidden to disclose that they have turned over any records, so that you would never know that the government is keeping records of your book purchases. And obviously it is quite simple to crossreference this info with data available in other databases. On a final note, the FBI is now hiring computer scientists to implement a project that sounds very similar to what I just did.

Re:Point of the article

[zCyl]

I still don't get it.

The FBI can find out where I live without Amazon or Google.

Yeah, but the FBI isn't supposed to know that you bought a book called "Why Bush is a Tyrant." The point being, it becomes quite dangerous if the government is allowed to keep tabs on what you read, because the political freedom which comes from freedom of speech requires that ideas can be exchanged and learned without fear of consequences.

Re:Point of the article

[crazyphilman]

I believe that what most of us find irritating about the Patriot Act is that many of the powers Bush asked for after 9/11 were not used to pursue terrorists, but rather political groups that the president disliked. For example, the airport no-fly list was mostly not used to prevent terrorists from flying (they all had false ID anyway) but rather, to prevent hippies and other malcontents from attending protests.

Now, I believe you and I both completely agree that asshole fundamentalists with bombs should be repeatedly forced to endure body cavity searches performed by a very large man hopped up on speed, with his choice of implements (no lube).

HOWEVER, I also believe that Bush should be prevented from abusing the powers he has received. I believe the FBI should be absolutely barred from using these powers against American citizens unless they can document, to a judge's satisfaction, that the person is in fact up to no good and has actually committed a crime (or is about to).

Now, what's wrong with that? Most of the funamentalists with bombs aren't in fact Americans. So my little restriction of government power won't affect your desire to not be blown up in the slightest.

I await your reply...

Most subversive anarchists...

[CupBeEmpty]

...that I know freely subscribe to Amazon.com wish lists. They are like "lets overthrow the government that wants to jail us" but they are also all over "Lets let everyone know how we feel about corparations and the government by making wish lists that not only incriminate us but play into the hands of the very corporate droogs we hate... makes sense right." Anyone thinking they will get useful information about truly dangerous groups from Google Maps or Amazon Wish Lists needs to take a breather and sit down for a minute.

What?

[Perseid]

From the article:
On a final note, the FBI is now hiring computer scientists to implement a project that sounds very similar to what I just did:

"Currently, the FBI is strengthening systems engineering in order to tie new systems together architecturally and ensure that standards for custom and packaged applications are enforced, and it needs engineers to accomplish this goal, the agency said.

(etc...)

Where does he read data mining into this? I read that the FBI wants to update their computers to make their databases better. Their databases.

This article strikes me as scare mongering, and until I hear that the government plans on breaking the knuckles of people who read Aldous Huxley, I don't care about what's merely possible.

Just to point out

[TubeSteak]

It is one thing to 'mine' information from Amazon, it is another thing entirely, to mine useful information.

Even his crude filtering techniques can yield worthwhile leads for police/FBI. He says that the first result for bible is "The Cannabis Grow Bible: The Definitive Guide to Growing Marijuana for Recreational and Medical Use".

Is it so hard to imagine that a certain fraction of people with that book on their wishlist may either be growing weed, or have it in their possesion? Or that a percentage of people 'wishing' for the Improvised Munitions Handbook (printed by our favorite Uncle Sam @ the DoD) aren't chemists or demolitionists?

/doesn't have an Amazon wishlist and never will

WTF is wrong with you?

[Lord Kano]

Most surprisingly, the process of doing this is completely legal, and doesn't even violate Amazon's Conditions of Use.

It shouldn't be surprising, it's common sense. Why in the fuck should it be illegal or against Amazon's conditions of use to read information in someone's wish list? The whole point of a wish list is so that other people will know what books you want.

LK

Re:WTF is wrong with you?

[typical]

Very simple principle. Lots of data is individually acceptable, but when compiled or processed, is unacceptable.

For example, say you maintain a Slashdot identity that you don't link to your real name. While no one post of yours may be sufficient to tie your identity to your name, the sum total may be sufficient.

Or security cameras. Most people don't worry about *one* security camera, but a lot of people get concerned when they are constantly being monitored by cameras which are tied together by computer to monitor where they go each day.

It may be more relevant that it may appear.

[igrigorik]

It may not be 'real news' but I don't think it should be dismissed as completely irrelevant. (Like 95% of current commentators have done).

First, on relevance of wishlists:
Granted that wishlists are not the most accurate estimates of your preferences, what is? My list contains over 50 books, and for the most part they are all related to each other. In fact, I would say that by looking at my list you would have a pretty accurate gauge to measure my interests. Am I an anomaly? Possibly. (Though I doubt it)

But it still makes you wonder how then does Amazon produce dozens and dozens of relevant suggestions to each of your books. For example, I often add a book to my shopping cart just to see the "what other people have bought when they bought this book x". Click, Wishlist, click, Wishlist. I think it's naive to dismiss wishlists completely. In fact, I'm sure that you will be able to successfully data mine data obtained from the wishlists and extract interesting and useful information.

Now, the actual experiment:
An interesting observation that I've recently read about developments in AI: "It stops being AI once it hits the mainstream". It's true, and it's happening here. The idea does not capture anything new, but the application is interesting. You can find out what people are reading and where. (And that's a powerful tool!) It open a big can of worms: advertising, targeting social groups, other 'moral' and 'immoral' uses. To those who know how to utilize it, this might prove to be a goldmine.

Re:Mining voluntary information on a public websit

[AndreiK]

Is it considered bad if I recognize and have read about half of those books?

Well 99% of the people here don't get it

[SmallFurryCreature]

Even comments to your posts don't get it. All this guy did was prove just how easy it is to use a seemingly harmless database to prove your a commie. Oh wait, get my mind out of the 50's, a terrorist. Or did the boogyman change name again? Pedo's are an eternal favorite and you can't really defend the rights of pedo's unless you wanna be lynched.

No the FBI or anyone else would never bother with amazon wish list. They would simply get the sales records. This guy does not have access to those so he uses what he can to prove his point.

Yes it is scary. Especially for those of us who have family (or more to the point do not have family) killed for expressing the wrong ideas.

I however don't think we should blame the FBI or similar agencies, they are the instruments of us the people. It is we who have voted the current goverments into power. Corruption you say? Well then it is you and me that have allowed that to happen. I do not believe in the mythical innocent citizen. Others have died for freedom. No reason we should be allowed to sit on our backsides and complain our freedoms are taken away. FIGHT

Not that I will of course. I know deepdown that what is happening is wrong and also know that I am one of the cattle. Perhaps it will make it easier when I am put in a cattle wagon to be gassed.

The problem with fighting for your freedom is that one persons freedom fighter is another persons terrorist.

I ain't got an answer or a solution except to suggest "PAY CASH". Even if your part of the herd there is no reason to make it any easier for them to send you off to the slaughterhouse.

Will it happen? It has happened countless times before. Check the McCarty trials. The treatment of Japanse americans vs German americans. The gunning down of american citizens by police during peace protests. The way england handled the RIA and labor strikes. All of them pretty recent.

Something scary might happen in our lifetimes. Or not. This is one tiny example to prove that it won't be hard on the technical side. Now all we need to is to elect leaders crazy enough to do it. /me looks at the current leaders of the "free" west. Too late.