Slashdot Mirror
When Purchase Recommendations Go Bad
nixman99 writes "An article on MSNBC describes what happens when 'View Similar Products' recommendations go bad. From the article: 'The company said it was alerted to the problem early yesterday afternoon after word began spreading among bloggers. When visitors to Walmart.com requested Planet of the Apes: The Complete TV Series on DVD, four other movies were recommended under the heading Similar Items. Those films included Martin Luther King: I Have A Dream/Assassination of MLK and Unforgivable Blackness: The Rise and Fall of Jack Johnson.'"
Connection:
Planet Of The Apes - Social Commentary.
Martin Luther King - Import changer of society.
Were you to be a glass is half full kind of person, that sounds like a connection. I could entirely accept that enough customers to trigger a connection algorithm are interested in social commentary to the degree that both titles appeal.
Were you to be a glass is half empty kind of person, clearly the system is racist.
Fortunately, we have a media that's only interested in postive and uplifting stories so they'd never focus purely on the negative, for shock value, without considering other possible alternatives.
And, for added amusement, type "Civ 4" in to Amazon and see what recommendations come up further down the list. It may too be racist. It may be a deeply humorous commentary on lonely guys playing Civ 4. Or it may be some other connection that we haven't figured out yet.
But then that's the whole point of data mining... Finding connections that humans tend to be entirely too preoccupied by their assumptions to be able to see beyond.
Check out the "also try" recommendations. :)
d s+with+guns&sm=Yahoo!+Search&toggle=1&ei=UTF-8&fr= FP-tab-web-t
http://images.search.yahoo.com/search/images?p=ki
I submitted this story a few days ago, and linked to the original Wal-Mart story. Literally, 10 minutes later Wal-Mart had changed their recommendations to Friends.
Is it a simple un-supervised algorithm that creates relationships based on customer's choices? Then shouldn't the whole American public be to blame? In other words did the people who buy "Planet Of The Apes" also buy the book about MLK, implying an association between black people and apes? The fact of the matter remains that most people in U.S. are racist - period. Even the ones who preach PC are racist even if just at the subconscious level. There have been studies done that shows this.
This makes me think of an interesting point: in one of the previous articles on Slashdot someone said how it is possible to extract so much data out of people's wish lists. But how about also gaining an insight into the American global subconscious by looking at the items people choose when they shop at the stores like WalMart, Amazon and others? I see someone in Sociology being interested in this...
I had the same problem working out what people's beef was with King Kong - apparently enslaving a huge gorilla when it's the only such example of a huge gorilla is somehow related to enslaving African people. Maybe I'm naïve but I didn't see that connection.
There was an even more explicit example of same - Farnham's Freehold by Robert A. Heinlein. In that a white family + black slave from 1950s America get transported into the future, where a black supremacist version of Islam (yeah, I don't see the connection either) has forced all Caucasians into slave roles and the black slave is treated like a prince. The head of the household is shocked to find that while he's given everything he wants he's property of someone else.
Rather than asking why thin-skinned reactionaries aren't rational enough to understand that the theme of 'Planet of the Apes' examines how society deals with racism, I would rather discuss the technical problem that is likely to come up again and again. That problem would be 'Exception Filtering'.
Examples:
a)Filtering out Metallica named files off of Napster.
b)Filtering out Chinese bloggers off of MSN.
c)Filtering out Planet of the Apes from similar themed Walmart DVDs.
Questions:
1)Is it even possible to filter successfully, against a majority that wants access?
2)Should we pretend that 'Exception Filtering' is possible, and place blame on programmers, so as to avoid dealing with the true societal problems.
3)If we do filter, who will decide for us? The government? Which government(s)? Big companies? Every easly offended minority?
I wonder how many geeks there are, do we count as a minority? Maybe then we could muster some political clout, and get something accomplished, rather than complaining about how technology ignorant polititions are.
Isn't this a corruption of Hanlon's Razor which states that:
"Never attribute to malice that which can be adequately explained by stupidity"
In this case, it could be construed that either the system, or the people making the malicious links are the stupid element - both could come to the racial conclusion by misinterpreting the data. Alternatively, the system might be too smart, working in a logical way such that elements in subject matter for both Planet of the Apes and Martin Luther King both deal with social commentary, alienation and segregation.
Either way, the comments by the spokesperson that the system was malfunctioning and not working as it was supposed to are probably incorrect; it work exactly as it was programmed, but it was either too stupid or too smart for us to comprehend adequately.
you are forgetting that this does not provide a complete cross section of all american or otherwise consumers.
This would only cross section those consumers who shopped online at those various stores. Even assuming one third of americans purchased ten percent of all household purchases on websites, you would have an indicative three percent of all purchases to make up for one hundred percent of all american characteristics? Does it really make sense that people anywhere, US, worldwide or in any particular town or "net-hood" only eat pizza and drink coke or pepsi? are you telling me that places like hard rock cafe don't actually sell food, they only talk about it?
The point i'm making is not that many people order their groceries online, and with the exception of pre-ordering and pre-paying for your food while making online reservations - which is a system i have not heard of, although someone is bound to do it soon - so you're assumption based on the above comment is that all purchases online are indicative of all people in a group somewhere, means that nobody on the planet or whichever region ever eats. So why are we all still here?
Just because an idea sounds good on paper for doing research, this is not a valid idea for judging all consumers. Now i'm going to leave out how the Gartner Group or some other group of a similar rep could do some polling of this nature for another poster to have a chance to refute my own claims, I just want to point out that I see both sides, I just think the parent post was not to well considered. Thanks, my $.02
/ begin side rant
I personally thought that Planet of the Apes was a good sci-fi movie of what if, not a social commentation nor an analogy of slavery. I have never sat down and wondered if it was a possible commentary on post-war (WWII) Europe, or an example of Communism gone bad, or what it would be like if my belly-button lint froze the sun or anything else.
All of those PC people out there that are so hung up on OOOOOHHH, WHAT DID HE SAY? can get off their soapbox and come back to work now. Unless they're too good for work. Like those people who had to get BUSSED from one natural disaster site, only to be in the middle of the next natural disaster site in the US southeast because they DIDN'T WORK SO THEY DIDN'T OWN THEIR OWN CARS. I know that I personally volunteered to drive my whole family from the SE US to somewhere safer, because I didn't want to have so many of our cars helping cause congestion on the highway, knowing what we were getting into, but there's a social commentary waiting to happen, the people whose government assisted living was washed away in New Orleans, LA
This has been a rant provided to you by one pissed off but levelheaded southerner - not a RACIST, just someone who has to work and expects all other able bodied citizens of the planet to as well.
/ End rant, thanks for pardoning me
2^3 * 31 * 647
Imagine their president ordering this to take place. Wouldn't he just make statements directly to the press, like Henry Ford did?
Isn't it more likely that 1) An employee either mistakenly or on purpose set this up, or 2) The system had a failure (ie: a pointer to a link list got messed up?
Since Wal-Mart has not told how this foul up occured, we may never know. But what I do see that gives Wal-Mart some credence that this is not an slur by the company is that they immediatly addressed the problem when they found out about it.
It allows individuals to perpetuate a culture of people who have low self esteem and use PC social taboos to manipulate others in a way that makes them feel better, but only because they have power over others and not because they are overcoming their own problems.
Twinstiq, game news
This is so true! It drives me nuts when people refer to others as "African-American" simply because they are black. It makes Americans look so incredibly dense and uneducated.
My uncle is African-American, as are some of my good friends. They were born in Etheopia, Tchad, Algeria, South Africa, Egypt, Nigeria, Tanzania, Kenya, and a host of other places IN AFRICA. I have other friends from Haiti, the Dominican Republic, and Jamaica. And do you know what? Not a single one of them refers to him or herself as "African-American". They have a national identity, not a continental one!
The Berber is as "white" as an Scandanavian. Some of my Ethiopian friends are incredibly Arab in appearance. One of my South American friends looks Scottish, and the other is a gorgeous raven-haired (white) Jew. nSungu is from Tanzania, and is one of the most strikingly beautiful women I have ever met, with skin that I can only describe as somewhere between cappucino and caramel, while another good friend, from Tchad, has skin the color of charcoal. They're as different from each other as they are from a Korean or an Hmong, yet our society feels comfortable lumping them into a single category.
Let's face it, "African-American" is a horrible descriptor for most people. Using it only serves to differentiate the "them" from the "us". If we stopped focusing so much on what makes us different instead of what brings us together, the term wouldn't even be necessary. If you were born in the Americas, you are "American". If you were born in Africa but emigrated to the Americas, you are "African-American". That's where it ends. Not all black people are African-American, and not all African-Americans are black. Let's stop with the stupid racial stereotypes.
There's money to be had in charging racism. Now that real racism (towards blacks) is hard to find, the racism industry has to look at the most miniscule and trivial things to keep the media attention on them and the bucks flowing in.
Real racism is killing people because of their race, like what happens if a white guy walks through Memphis or south Chicago. It's separating people by race like what happened in the U.S south before the 60's and in South Africa during apartheid. It's dragging some guy to death behind a pickup because he's black.
Racism is not listing Planet of the Apes and M.L. King on the same web page.
If these jackasses causing the hullabaloo see themselves as apes, and want to take offense every time they see a reference to apes, fine. Let them. There's all kinds of freaks in this world and we usually take the attitude that "as long as they're not bothering anybody let them be".
Let's do this here. Let them be. No news coverage. No wailing and gnashing of teeth over a "racist incident" where no racism exists.
Lincoln started the War between States to prevent secession, which up to that point was considered my many states to be within the rights of the States as set forth by the 9th and 10th Amendment.
Lincoln HATED blacks. The Emancipation Proclamation even kept states within the Union as slave states! The Wiki for the EP covers much of this.
Lincoln forced the South the secede as he planned on taxing and setting tariffs on the South in order to pay off his cronies for "improvements" in the North. The South threatened to leave the Union if Lincoln was elected on his platform -- Henry Clay's American System. Lincoln loved Clay and Hamilton, they wanted a mercantilist society in America. They wanted a central bank (so they could legally counterfeit money) they wanted corporate welfare for their cronies, they wanted a huge military to expand the empire. The Whigs fell apart and became the Republicans -- who still love all 3 items (central banking, empirism and corporate welfare).
If you want a great read, pick up DiLorenzos' "The Real Lincoln." You can get it at most Borders book stores. I offer a nice deal, too. If you buy the book and DON'T like it, tell me what you paid and I'll buy it back from you plus shipping (within reason, no books over $30). I'll give it to someone who is interested.
DiLorenzo posts regularly on LRC: www.lewrockwell.com Do a search for his name and he often has information on Lincoln. His book has hundreds of footnotes including quoting Lincoln himself.
The tragedy of the War between States is the lies and myths that the Lincoln-lovers in education and history writing tend to spread. Lincoln was the worst tyrant, and completely destroyed the Union over corporate welfare. The book is amazing and a complete eye opener.
To claim that people who see racism in anything and everything, like King Kong, is due to their own racism is not wrong. If you look hard enough, you can find racism in anything if you are determined to see it.
You don't make the poor richer by making the rich poorer. - Winston Churchill