Slashdot Mirror
When Purchase Recommendations Go Bad
nixman99 writes "An article on MSNBC describes what happens when 'View Similar Products' recommendations go bad. From the article: 'The company said it was alerted to the problem early yesterday afternoon after word began spreading among bloggers. When visitors to Walmart.com requested Planet of the Apes: The Complete TV Series on DVD, four other movies were recommended under the heading Similar Items. Those films included Martin Luther King: I Have A Dream/Assassination of MLK and Unforgivable Blackness: The Rise and Fall of Jack Johnson.'"
Amazon recommended some adult entertainment to go with the Madagascar (rated U) when I ordered the other day. My other interest was nature books, so how it put two and two together no one knows.
Connection:
Planet Of The Apes - Social Commentary.
Martin Luther King - Import changer of society.
Were you to be a glass is half full kind of person, that sounds like a connection. I could entirely accept that enough customers to trigger a connection algorithm are interested in social commentary to the degree that both titles appeal.
Were you to be a glass is half empty kind of person, clearly the system is racist.
Fortunately, we have a media that's only interested in postive and uplifting stories so they'd never focus purely on the negative, for shock value, without considering other possible alternatives.
And, for added amusement, type "Civ 4" in to Amazon and see what recommendations come up further down the list. It may too be racist. It may be a deeply humorous commentary on lonely guys playing Civ 4. Or it may be some other connection that we haven't figured out yet.
But then that's the whole point of data mining... Finding connections that humans tend to be entirely too preoccupied by their assumptions to be able to see beyond.
Why are people apologising for this recommendation? IMHO, this is actually a fairly good recommendation!
POTA is a movie about civil rights, in this case across species, not races. One species (the monkeys/gorillas) effectively enslaves another species (humans) and the base message of the movie is about the struggle for emancipation by this enslaved species.
So exactly how is a movie about enslavement and emancipation not related to real life civil rights issues?
I'm not American so I'm not really exposed to this over-the-top sensitive PC stuff, but this seems just silly to me. Franky, I find the people who did the complaining about this issue offensive and ignorant.
Sparks:Gadget:Beer Maker
When Planet of the Apes first came out it was revolutionary. It took the Lords of Creation - White men - and put them in a situation where they were the oppressed, the minority. Someone else was in charge and no worse, perhaps better, than the astronauts. The movie asked questions and had a discussion of race in America that would have been unthinkable without the fig-leaf of science fiction.
So yes, it was appropriate. Those who are offended never looked deeper than the skin. Which is sort of the problem.
The man who never alters his opinion is like the stagnant water and breeds Reptiles of the Mind -- William Blake
pull out your rose tinted glasses and try to see it this way:
The Planet of the Apes is a social commentary in the form of a sci-fi film, MLK was a historic figure who made great efforts to make society more equal.
Trying to view a glass that's half full I'll try to see that as a connection that some software somewhere made. Of course the victocrats(glass half empty types) will see nothing beyond the titles of the connected products. To them I say get over it and try to look beyond the superficial.
There are lies, damned lies, and statistics.
When I looked up the 1979 film "The Cracker Factory," which about a woman who drifts in and out of asylums, I got the following recommendations:
8 Mile
Over the Top
Bean
So we can blame the unthinking machines and the corporations that use them for our own cultural and racial bigotry. Nice.
it's a blue bright blue Saturday hey hey
I don't get it - what exactly is so offensive about those recommendations? Could it be that's only offensive in a climate that is so obsessed with political correctness that you cannot make perfectly innocent recommendation without some people reading whatever malicious intention into it? Honestly, I don't understand this, but I think it makes me a little bit happier that I'm living in Europe...
planet of the apes was loosely considered an allegory on race relations, or at least recognized to have spurred discussion on race relations, (although I don't exactly see how anyone thought it would be a good idea to have it be suggested by using apes).
anyway if you categorized these things in terms of hierarchies or in terms of degrees of separation, and they wanted to boost the relevance of MLK stuff, they'd boost the levels of search depth to find connections, even tenuous connections, to make things that had even a remote connection to one of MLK's supercategories recommend the MLK media.
technology can make people look pretty damn stupid, but as a progressive, I'm pretty embarrassed by the progressives that were so sure they saw overt evidence of deliberate and corporate-sponsored racism in this. I'm not saying there wasn't a racist in wal-mart that thought it would be funny to manually link POTA to MLK, but it's not even close to the only possible explanation. All people have to do is remember the old grapevine game to realize how easily an intent or an idea can corrupt itself by just being passed three or four links down a chain.
skkkoooonnnggggkkk ptui
Check out the "also try" recommendations. :)
d s+with+guns&sm=Yahoo!+Search&toggle=1&ei=UTF-8&fr= FP-tab-web-t
http://images.search.yahoo.com/search/images?p=ki
... or funny?
g .gif
E.g. http://www.speakeasy.org/~curby/swg/text/jellypon
I vote for funny.
--
"Extra Anus Kills Four-Legged Chick" -- Headline
I submitted this story a few days ago, and linked to the original Wal-Mart story. Literally, 10 minutes later Wal-Mart had changed their recommendations to Friends.
Is it a simple un-supervised algorithm that creates relationships based on customer's choices? Then shouldn't the whole American public be to blame? In other words did the people who buy "Planet Of The Apes" also buy the book about MLK, implying an association between black people and apes? The fact of the matter remains that most people in U.S. are racist - period. Even the ones who preach PC are racist even if just at the subconscious level. There have been studies done that shows this.
This makes me think of an interesting point: in one of the previous articles on Slashdot someone said how it is possible to extract so much data out of people's wish lists. But how about also gaining an insight into the American global subconscious by looking at the items people choose when they shop at the stores like WalMart, Amazon and others? I see someone in Sociology being interested in this...
Star Trek (the episodes that are not pure action or particle of the week thrillers) does this a couple of times. I am reminded of the color difference episode where we meet two races locked in a fight to the death, the one being black/white and the other being white/black.
TNG had an episode to show how stupid judging people on their sexual preferences is but showing a race that is purely homosexual (a 1 gender species that still used two people to procreate is off course the ultimate same sex race) with the sexual weirdos being those who tended to have heterosexual feelings.
This is indeed the eye of the beholder, it took me a while to figure it out even what the problem was. Apparently blacks are apes.
MMO Quests are like orgasms:
You may solo them, I prefer them in a group.
Surely the racism is in the eyes of the people complaining about this, rather than in the programmed system that is probably matching keywords?
You always get a slightly strange recommendation when shopping on sites with this feature. It is to be expected, categorisation can only go so far...
On Amazon.com, when browsing for Essential .NET Amazon.com was nice enough to tell me that fellow purchasers also wore "Clean Underwear". I was a bit disturbed that Eddie Bauer felt it was needed to specify that the underwear I was buying was in fact clean.
Rather than asking why thin-skinned reactionaries aren't rational enough to understand that the theme of 'Planet of the Apes' examines how society deals with racism, I would rather discuss the technical problem that is likely to come up again and again. That problem would be 'Exception Filtering'.
Examples:
a)Filtering out Metallica named files off of Napster.
b)Filtering out Chinese bloggers off of MSN.
c)Filtering out Planet of the Apes from similar themed Walmart DVDs.
Questions:
1)Is it even possible to filter successfully, against a majority that wants access?
2)Should we pretend that 'Exception Filtering' is possible, and place blame on programmers, so as to avoid dealing with the true societal problems.
3)If we do filter, who will decide for us? The government? Which government(s)? Big companies? Every easly offended minority?
I wonder how many geeks there are, do we count as a minority? Maybe then we could muster some political clout, and get something accomplished, rather than complaining about how technology ignorant polititions are.
so he can educate everyone on how Walmart doesn't care about black people.
I love 'The West Wing'. In fact, I like it so much that I've got every single dvd box set (1-6). All purchased from Amazon.
So what did they recommend to me?
This. Yeah - great thanks.
Mojo-jojo... and I'm ashamed to even admit knowing that.
Isn't this a corruption of Hanlon's Razor which states that:
"Never attribute to malice that which can be adequately explained by stupidity"
In this case, it could be construed that either the system, or the people making the malicious links are the stupid element - both could come to the racial conclusion by misinterpreting the data. Alternatively, the system might be too smart, working in a logical way such that elements in subject matter for both Planet of the Apes and Martin Luther King both deal with social commentary, alienation and segregation.
Either way, the comments by the spokesperson that the system was malfunctioning and not working as it was supposed to are probably incorrect; it work exactly as it was programmed, but it was either too stupid or too smart for us to comprehend adequately.
So far I've seen 5 posts modded up pretty high for saying that this isn't as offensive as it sounds, and is even in some ways appropriate. Give me a break. Is racial insensitivity so DEAD in your country that you can't see how putting four influential black icons onto the same page as a B-movie about monkeys is offensive?
I'm not saying it was maliciously done. Without seeing the algorithm, nobody can know for sure, but I know enough about data mining to know that random stuff crops up. But for the Love of CHRIST show a little empathy.
I bet people would be singing a different tune if it were four documentaries about 9/11 mixed with Mahmoud Darwish's The Shahid?
you are forgetting that this does not provide a complete cross section of all american or otherwise consumers.
This would only cross section those consumers who shopped online at those various stores. Even assuming one third of americans purchased ten percent of all household purchases on websites, you would have an indicative three percent of all purchases to make up for one hundred percent of all american characteristics? Does it really make sense that people anywhere, US, worldwide or in any particular town or "net-hood" only eat pizza and drink coke or pepsi? are you telling me that places like hard rock cafe don't actually sell food, they only talk about it?
The point i'm making is not that many people order their groceries online, and with the exception of pre-ordering and pre-paying for your food while making online reservations - which is a system i have not heard of, although someone is bound to do it soon - so you're assumption based on the above comment is that all purchases online are indicative of all people in a group somewhere, means that nobody on the planet or whichever region ever eats. So why are we all still here?
Just because an idea sounds good on paper for doing research, this is not a valid idea for judging all consumers. Now i'm going to leave out how the Gartner Group or some other group of a similar rep could do some polling of this nature for another poster to have a chance to refute my own claims, I just want to point out that I see both sides, I just think the parent post was not to well considered. Thanks, my $.02
/ begin side rant
I personally thought that Planet of the Apes was a good sci-fi movie of what if, not a social commentation nor an analogy of slavery. I have never sat down and wondered if it was a possible commentary on post-war (WWII) Europe, or an example of Communism gone bad, or what it would be like if my belly-button lint froze the sun or anything else.
All of those PC people out there that are so hung up on OOOOOHHH, WHAT DID HE SAY? can get off their soapbox and come back to work now. Unless they're too good for work. Like those people who had to get BUSSED from one natural disaster site, only to be in the middle of the next natural disaster site in the US southeast because they DIDN'T WORK SO THEY DIDN'T OWN THEIR OWN CARS. I know that I personally volunteered to drive my whole family from the SE US to somewhere safer, because I didn't want to have so many of our cars helping cause congestion on the highway, knowing what we were getting into, but there's a social commentary waiting to happen, the people whose government assisted living was washed away in New Orleans, LA
This has been a rant provided to you by one pissed off but levelheaded southerner - not a RACIST, just someone who has to work and expects all other able bodied citizens of the planet to as well.
/ End rant, thanks for pardoning me
2^3 * 31 * 647
The only reason this story is getting attention is Walmart is the current American icon of corporate evil and greed. America is a nation of victims who have nothing better to do than blame their personal failures on everyone else. This story should be humorous.
usually when people use racial stereotypes they're just trying to be funny, even if their humor is fucked up by other people's standards. but the ggp post presumeably wasn't(unless he was going for very subtle irony). therefore: racist.
the phrase "afican-american" implies that all black people in america are immigrants from africa, and while most of their ancestors were from africa that doesn't mean they need to be labeled "african-americans" anymore than white people need to be labeled "european-americans". the phrase also implies that all africans are black, which is a racist generalization. but whatever
Instead of making the unlikely assumption that Walmart has a racist policy based on the recommendation of 3 films buy a computer, did anyone stop to ask why the system did this? I mean perhaps the films do have something in common, does anyone star in more than one of them? Do they have the same release date/year? or DVD release date? Do they share composers, directors or crew? Are they all catogorised under "American History"? Maybe the most fucking obvious reason is that several people who bought Planet of the Apes also bought these other films!!
The press is always ready for a scandel and never ready to actually follow it up with some investigative journalism. I guess its cheaper to just re-broadcast a video feed and pay the royalties or print something direct from AP.
This comment does not represent the views or opinions of the user.
It allows individuals to perpetuate a culture of people who have low self esteem and use PC social taboos to manipulate others in a way that makes them feel better, but only because they have power over others and not because they are overcoming their own problems.
Twinstiq, game news
Walmart doesn't care about black people.
The theory of relativity doesn't work right in Arkansas.
This is so true! It drives me nuts when people refer to others as "African-American" simply because they are black. It makes Americans look so incredibly dense and uneducated.
My uncle is African-American, as are some of my good friends. They were born in Etheopia, Tchad, Algeria, South Africa, Egypt, Nigeria, Tanzania, Kenya, and a host of other places IN AFRICA. I have other friends from Haiti, the Dominican Republic, and Jamaica. And do you know what? Not a single one of them refers to him or herself as "African-American". They have a national identity, not a continental one!
The Berber is as "white" as an Scandanavian. Some of my Ethiopian friends are incredibly Arab in appearance. One of my South American friends looks Scottish, and the other is a gorgeous raven-haired (white) Jew. nSungu is from Tanzania, and is one of the most strikingly beautiful women I have ever met, with skin that I can only describe as somewhere between cappucino and caramel, while another good friend, from Tchad, has skin the color of charcoal. They're as different from each other as they are from a Korean or an Hmong, yet our society feels comfortable lumping them into a single category.
Let's face it, "African-American" is a horrible descriptor for most people. Using it only serves to differentiate the "them" from the "us". If we stopped focusing so much on what makes us different instead of what brings us together, the term wouldn't even be necessary. If you were born in the Americas, you are "American". If you were born in Africa but emigrated to the Americas, you are "African-American". That's where it ends. Not all black people are African-American, and not all African-Americans are black. Let's stop with the stupid racial stereotypes.
From Wikipedia
The term "African American" has been in common usage in the United States since the late 1980s, when greater numbers of African Americans began to adopt the term self-referentially. Malcolm X favored the term "African American" over "Negro" and used the term at an OAAU (Organization of Afro American Unity) meeting in the early 1960s, saying, "Twenty-two million African-Americans - that's what we are - Africans who are in America." Former NBA player/coach Lenny Wilkens is another who used the term as a teenager when filling a job application. Many Blacks began to abandon the term "Afro-American", which had become popular in the 1960s and '70s, for "African-American," because they desired an unabbreviated expression of their African heritage that could not be mistaken or derided as an allusion to the afro hairstyle. The term became increasingly popular, and by the 1980s, Jesse Jackson and others pressed for its adoption and acceptance.
Forgive me for lecturing, but I'll stereotype a bit here and suggest that the majority of /. readers don't fall into the group of people who can see offense in this situation either out of ignorance, or unfamiliarity with minorities and their history. I know there's a large contingent out there that believes the white male is an "oppressed" group in America due to affirmative action, Title IX, or other assorted anti-harassment and anti-discrimination laws or rules. I'm sure the strain must be unbearable...
I love this site and my fellow slashdotters and I come here every day -- but sometimes things are just wrong.
I Am Not A Conspiracy Theorist (IANACT?) but there could be something more sinister at work here than some computer algorithm linking the social commentary of "The Planet of the Apes" with Martin Luther King's role in the civil rights struggle. Discrimination and offensive racial stereotyping are not dead issues -- they often lie just beneath the surface because there are many who still believe that some people are inferior to others simply because of their ethnicity, skin color or gender. And speaking as an African American (and I don't get up on this soapbox often, folks), this was offensive and I am not amused.
We all know the posters on this site wouldn't let Microsoft off the hook so easily or rush to defend them so quickly if the folks in Redmond were behind this.
Now, let the bashing begin! Who needs positive karma?
A few years ago, Amazon offered me this bizarre, yet strangely appropriate, recommendation. I believe the screenshot I took speaks for itself (yes, it's real and undoctored):
http://img241.imageshack.us/img241/8897/sa6iz.png
Based on what's been said in a couple of other comments, it sounds like the Wal-mart recommendations system operates via a set of assigned keywords or metadata. Having worked on a similar type of system in the past--they probably have a defined set of keywords that can be assigned for each DVD. When the DVD is entered into the database, it is the data entry person's responsibility to chose a proper set of keywords for the DVD.
This type of system develops strange biases in several ways, most noteably through human interpretation. Say you have a keyword "black/white relations." One data entry person might only assign that keyword to nonfictional documentaries, while another might assign that keyword to based-on-real-life movies as well. And another person who's particularly sensitive to the underlying messages of movies might assign that keyword to Planet of the Apes (as well as possibly to box collections of ST:TOS).
Somebody selects one of those movies, and gets a bizarre selection of "related" movies which simply reflects the fact that three different people viewed the use of a defined keyword and thus assigned it in three different ways. It's hard to even design business rules to prevent this from happening because it overly limits what the system was designed to do. If a business rule says that only nonfiction documentaries and based-on-real-life movies can receive the "black/white relations" tag, you might end up missing a movie like Crash. If the business rule says instead that you can't assign a tag based on the "underlying" message of a movie, how do you define underlying message? Racism or "black/white relations" (my bet is that the Wal-mart keyword was closer to "black/white relations" rather than "racism" because all the movies that apparently popped up as suggestions were about that particular subset of racism) is the in-your-face message of Planet of the Apes. It's so thinly masked by the story that I'm not sure I'd define it was the "underlying" message. I'm the type of person that probably *would* assign Planet to the "black/white relations" tag, because its consideration of that theme is about the only redeeming factor of the movie.
Of course, I grew in an area where--due to integration--racism was a pretty major issue and I thought I'd learned most of the various "bad" terms that members of one race (hell, one European background) called members of another race (or other European backgrounds) when I was young. "Monkey" had definitely fallen out of use in my area by the 1980s; first time I was ever introduced to it as a racist term was online about four years ago.
From the actual perspective of a racist, it makes obvious sense.
They're black or brown. They've got that flat nose thing going. They wear little or no clothing. They come from Africa, living side by side with each other. They're clearly related to us, but also clearly more primitive.
On a continent with many different primate species, it would make sense if they were all related, and the fact that one of them can interbreed with us doesn't mean they're the same, any more than it means that horses and donkeys are the same. Who knows, since they are an intermediate race, they can also interbreed with some monkey species.
Hopefully unneeded disclaimer: Those are not my opinions, but those I believe are/were typical racist thoughts on the matter
I think the problem with everyone's view of this is that it is being seen in one way only by most. This is an innocent match of titles that offended some, and those who say people are overreacting by making this a national issue are correct. In addition it is something that many minorities, particularly African Americans will find offensive. Those with a healthy attitude would laugh at the poor pairing and call it a day, but they should still be a little offended. Would we be less likely to care if we had "real problems", of course we would. In our society many of us have less "real problems" than some other parts of the world or even some of those in our own country. This is a reflection of the wealth of the society as a whole, and that the majority are not facing the "real problems" some people face. As one looks at various countries around the world the number of "real problems" they face is a function of the wealth of the nation. Are those of us who find this offensive oversensitive, I don't think so. As was said earlier, when people have been likened to monkeys in the past, and still face some significant inequalities in society are likely to have a very different perspective on the subject. One of the posts mentioned the different types of racism and separated perceived racism from actual racism. Unfortunately the two are not mutually exclusive, one is born from the other. The xenophobic nature of man is very prevalent throughout the world and its hsitory. The Catholics and the Protestants in England and Ireland, and the Afrikaans, English, and Africans in South Africa are good examples. These are extreme so lets look at the U.S.A.; The Irish immigrants when they first began arriving in numbers were disliked and shut out, the Chinese, the Italians, etc.. The African-Americans are a unique case having been forcefully brought here and treated sub human. This first began in the 1640's and grew exponentially til about 1790 and continued til 1865, only to be replaced by various forms of indentured servitude and of course segregation. All of that being peppered with violence towards African-Americans and the damage done by being relegated to a second class status, the effects we are still dealing with today. About 300 years of this has led to the second type of racism. The perception can be incorrect and at times oversensitive, but let us not think that it is born from a vacuum, it is created by the history. Things have become better, but it has been a very short number of years, not even one lifetime, since the civil rights movement and we still do have work to do. In time I do think it will fade, but you can't erase 300 years with less than a sixth of that time. When my friends who are in the majority ask me to help them to understand what racism today is like, and how it can be disturbing, frightening, apalling, and sometimes humourous, I usually fail. I think until you have been refused service or entry somewhere, had trouble finding work, or been physically/verbally threatened or harassed by not only other citizens but the authorities as well it is hard to really understand. In summary, it makes sense to be offended, but not to make it national news.