Slashdot Mirror
When Profiling Goes Wrong
huskymo writes "This morning's Wall Street Journal is carrying a funny story on TiVo and Amazon's automatic customer profiling. As most Slashdot readers probably know, TiVo keeps track of which programs you record and--if you haven't told it not to--records other programs it thinks you'd like. The article describes users that TiVo's mistaken for Korean, for gay, even for "a pregnant gay man.""
Funny as hell.
Free Mac Mini
To read the story without having to register for the (pay) site, use this link:
3 6872356908,00.html
http://online.wsj.com/article_email/0,,SB10382619
That's not the article! Here's the real text from wsj.com:
If TiVo Thinks You Are Gay, Here's How to Set It Straight
What You Buy Affects Recommendations
On Amazon.com, Too; Why the Cartoons?
By JEFFREY ZASLOW
Staff Reporter of THE WALL STREET JOURNAL
Basil Iwanyk is not a neo-Nazi. Lukas Karlsson isn't a shadowy stalker. David S. Cohen is not Korean.
But all of them live with a machine that seems intent on giving them such labels. It's their TiVo, the digital videorecorder that records some programs it just assumes its owner will like, based on shows the viewer has chosen to record. A phone call the machine makes to TiVo, Inc., in San Jose, Calif., once a day provides key information. As these men learned, when TiVo thinks it has you pegged, there's just one way to change its "mind": outfox it.
Mr. Iwanyk, 32 years old, first suspected that his TiVo thought he was gay, since it inexplicably kept recording programs with gay themes. A film studio executive in Los Angeles and the self-described "straightest guy on earth," he tried to tame TiVo's gay fixation by recording war movies and other "guy stuff."
"The problem was, I overcompensated," he says. "It started giving me documentaries on Joseph Goebbels and Adolf Eichmann. It stopped thinking I was gay and decided I was a crazy guy reminiscing about the Third Reich."
He mentioned his TiVo tussle to a friend, who told an executive at CBS's "The King of Queens," who then wrote an episode with a My-TiVo-thinks-I'm-gay subplot.
A lot of gadgets and Web sites now feature "personalization technologies" that profile consumers by tracking what they watch, listen to or buy. The software, embedded in sites such as Amazon.com and CDNOW.com, then recommends other books, videos and music based on a customer's tastes.
The Willies
Many consumers appreciate having computers delve into their hearts and heads. But some say it gives them the willies, because the machines either know them too well or make cocksure assumptions about them that are way off base. That's why even TiVo lovers are tempted to hoodwink it -- a phenomenon that was also spoofed this year on another TV show, HBO's "The Mind of the Married Man."
Mike Binder, creator and star of that show, had set his home TiVo to record his 1999 movie, "The Sex Monster," about a man whose wife becomes bisexual. After that, Mr. Binder's TiVo assumed he would enjoy a steady stream of gay programming. Unnerved, he counteracted the onslaught by recording the Playboy Channel and MTV's spring break bikini coverage. It worked, he says. "My TiVo doesn't look at me funny anymore."
His wife, however, was taken aback when she saw all the half-naked women he was ordering through TiVo. He told her those women meant nothing to him: "I'm just counterprogramming because TiVo thinks I'm gay." She was unamused. The incident inspired an episode of his show.
Though some users contend TiVo has sex on the brain, TiVo's general manager, Brodie Keast, explains that the box is merely "reacting to feedback you give it." Still, the machine employs algorithms -- searching several thousand key details (favorite actors, movie and TV genres) -- that leave some people wondering whether it is judging their predilections.
Mr. Karlsson, 26, says he "pre-emptively" found all the religious shows in his TV listings and used the "thumbs down" button on his remote control to tell TiVo he has no interest in them. (Giving three thumbs down is the best way to block a program.) After that, his TiVo recorded movies about creepy homicides. "They all have titles like 'Murder on Skeleton Isle,' " says the computer system administrator in Cambridge, Mass.
He uses the "thumbs" button to tell TiVo he hates such films. He also orders cooking shows, which softens TiVo's view of him. "I don't want it thinking I'm an ax murderer," he says.
Mr. Cohen, 30, has a TiVo that mysteriously assumed he wanted Korean news programs. The Philadelphia lawyer gave thumbs down to anything Korean, and his TiVo got the message. Sort of. "The next day, it recorded the Chinese news," he says.
TiVo's 500,000 subscribers use the box primarily to record programs they specifically request, and many laud its ability to pause live broadcasts and record a show's entire season. Still, in TiVo-focused online chat-rooms and in secretive admissions to one another, some say they resent being pigeonholed by TiVo's suggestions.
'A Pregnant Gay Man'
Like TiVo, other techno-profilers run hard with limited information. Ray Everett-Church of Fremont, Calif., who is gay, ordered "Queer as Folk" videos from Amazon.com. Understandably, the site began suggesting gay-related calendars and books. Then he bought a baby book for a pregnant friend. So for weeks, the site also recommended parenting books. He says it was as if Amazon.com decided he was "a pregnant gay man."
He fought back, he says, "by inundating it with additional data. I searched for other stuff -- on politics, computers -- so it would stop throwing baby books at me. Now it thinks I've abandoned the baby and I'm preparing for a career in politics."
Mr. Everett-Church, a privacy consultant for businesses, predicts that as techno-profiling increases, more people will purposely muck up their profiles. They'll fear ordering books on mental illnesses or sexual preferences because they'll wonder if they'll somehow be publicly identified.
All techno-profiling companies contacted for this article said that information gleaned is for the customer's personal use only. Still, even Amazon.com founder Jeff Bezos knows the potential mortification factor.
For a live demonstration before an audience of 500 people, Mr. Bezos once logged onto Amazon.com (amazon.com) to show how it caters to his interests. The top recommendation it gave him? The DVD for "Slave Girls From Beyond Infinity." That popped up because he had previously ordered "Barbarella," starring Jane Fonda, a spokesman explains.
Dawn Freeman, 23, a tax analyst in Lexington, Ky., has bought lowbrow videos, such as "American Pie," from Amazon.com. But she was aghast when the site suggested Tom Green's gross-out performance in "Road Trip."
"I thought, 'I know I don't like high cinema, but have I really reached the point where I'd like to watch Tom Green lick a mouse?" To even out her Amazon profile, she went through the site finding "witty independent films."
Her TiVo also thinks she's a sophomoric-humor-loving 12-year-old, she says. It keeps giving her cartoons. "I know it's dumb to take it personally, but it's in your face. These are supposedly objective computers saying, 'This is what we think of you.' "
Dissing Ice Cube
A.J. Meyer, a 35-year-old Web site developer in Minneapolis, ordered the DVD for "Scarface," the Al Pacino gangster movie, from Netflix.com (netflix.com). After that, the site kept recommending movies about gangster rappers. He stopped the assault by giving negative ratings to all movies starring Ice Cube. (Netflix allows members to rate any of its 12,000-plus titles with one to five stars -- whether they have rented a film or not. That helps the site calculate future recommendations.)
After Mr. Meyer ordered a documentary about New York from Amazon.com, it pitched him countless documentaries -- even one on the history of the thimble. He stopped the Ken Burnsification of his profile by searching the site for plasma TVs. "That way, I identified myself as a high-tech guy," he says. "The thimble is more low tech."
Virginia Heffernan, TV columnist for Slate.com, doesn't understand why some people are resistant to techno-profiling, or find it creepy. She didn't look for any deep meaning when her TiVo kept giving her TV shows in Polish. And after buying self-help books on Amazon.com, she accepted that every time she logged on, the site pitched products to make her a more self-fulfilled human being.
"I like the idea that someone cares," she says. "Even a machine."
TiVo users can program the machine to skip certain channels entirely. But many users don't bother to figure out how to do it, or are too intrigued by TiVo's recommendation process, says a spokesman. TiVo is paid to promote programs and products it calls "advertainment" on a special screen. But the company says none of these are given to users as suggestions.
Some people have given up trying to manipulate personalization technologies. Dino Leon, a hair-salon owner in Birmingham, Mich., says his TiVo quickly figured out that he and his partner were gay. They were OK with that, but just for fun, they tried to confuse the software by punching in "redneck" programs, like Jerry Springer's talk show.
TiVo wasn't fooled, and kept recording gay shows. Mr. Leon believes the box was giving them a message: "You're definitely gay. And you're watching too much TV."
By JEFFREY ZASLOW
Staff Reporter of THE WALL STREET JOURNAL
Basil Iwanyk is not a neo-Nazi. Lukas Karlsson isn't a shadowy stalker. David S. Cohen is not Korean.
But all of them live with a machine that seems intent on giving them such labels. It's their TiVo, the digital videorecorder that records some programs it just assumes its owner will like, based on shows the viewer has chosen to record. A phone call the machine makes to TiVo, Inc., in San Jose, Calif., once a day provides key information. As these men learned, when TiVo thinks it has you pegged, there's just one way to change its "mind": outfox it.
Mr. Iwanyk, 32 years old, first suspected that his TiVo thought he was gay, since it inexplicably kept recording programs with gay themes. A film studio executive in Los Angeles and the self-described "straightest guy on earth," he tried to tame TiVo's gay fixation by recording war movies and other "guy stuff."
"The problem was, I overcompensated," he says. "It started giving me documentaries on Joseph Goebbels and Adolf Eichmann. It stopped thinking I was gay and decided I was a crazy guy reminiscing about the Third Reich."
He mentioned his TiVo tussle to a friend, who told an executive at CBS's "The King of Queens," who then wrote an episode with a My-TiVo-thinks-I'm-gay subplot.
A lot of gadgets and Web sites now feature "personalization technologies" that profile consumers by tracking what they watch, listen to or buy. The software, embedded in sites such as Amazon.com and CDNOW.com, then recommends other books, videos and music based on a customer's tastes.
The Willies
Many consumers appreciate having computers delve into their hearts and heads. But some say it gives them the willies, because the machines either know them too well or make cocksure assumptions about them that are way off base. That's why even TiVo lovers are tempted to hoodwink it -- a phenomenon that was also spoofed this year on another TV show, HBO's "The Mind of the Married Man." [TiVo Remote] Remote Control: Viewers help TiVo understand their tastes by giving TV shows thumbs up or down.
Mike Binder, creator and star of that show, had set his home TiVo to record his 1999 movie, "The Sex Monster," about a man whose wife becomes bisexual. After that, Mr. Binder's TiVo assumed he would enjoy a steady stream of gay programming. Unnerved, he counteracted the onslaught by recording the Playboy Channel and MTV's spring break bikini coverage. It worked, he says. "My TiVo doesn't look at me funny anymore."
His wife, however, was taken aback when she saw all the half-naked women he was ordering through TiVo. He told her those women meant nothing to him: "I'm just counterprogramming because TiVo thinks I'm gay." She was unamused. The incident inspired an episode of his show.
Though some users contend TiVo has sex on the brain, TiVo's general manager, Brodie Keast, explains that the box is merely "reacting to feedback you give it." Still, the machine employs algorithms -- searching several thousand key details (favorite actors, movie and TV genres) -- that leave some people wondering whether it is judging their predilections.
Mr. Karlsson, 26, says he "pre-emptively" found all the religious shows in his TV listings and used the "thumbs down" button on his remote control to tell TiVo he has no interest in them. (Giving three thumbs down is the best way to block a program.) After that, his TiVo recorded movies about creepy homicides. "They all have titles like 'Murder on Skeleton Isle,' " says the computer system administrator in Cambridge, Mass.
He uses the "thumbs" button to tell TiVo he hates such films. He also orders cooking shows, which softens TiVo's view of him. "I don't want it thinking I'm an ax murderer," he says.
Mr. Cohen, 30, has a TiVo that mysteriously assumed he wanted Korean news programs. The Philadelphia lawyer gave thumbs down to anything Korean, and his TiVo got the message. Sort of. "The next day, it recorded the Chinese news," he says.
TiVo's 500,000 subscribers use the box primarily to record programs they specifically request, and many laud its ability to pause live broadcasts and record a show's entire season. Still, in TiVo-focused online chat-rooms and in secretive admissions to one another, some say they resent being pigeonholed by TiVo's suggestions.
'A Pregnant Gay Man'
Like TiVo, other techno-profilers run hard with limited information. Ray Everett-Church of Fremont, Calif., who is gay, ordered "Queer as Folk" videos from Amazon.com. Understandably, the site began suggesting gay-related calendars and books. Then he bought a baby book for a pregnant friend. So for weeks, the site also recommended parenting books. He says it was as if Amazon.com decided he was "a pregnant gay man."
He fought back, he says, "by inundating it with additional data. I searched for other stuff -- on politics, computers -- so it would stop throwing baby books at me. Now it thinks I've abandoned the baby and I'm preparing for a career in politics."
Mr. Everett-Church, a privacy consultant for businesses, predicts that as techno-profiling increases, more people will purposely muck up their profiles. They'll fear ordering books on mental illnesses or sexual preferences because they'll wonder if they'll somehow be publicly identified.
All techno-profiling companies contacted for this article said that information gleaned is for the customer's personal use only. Still, even Amazon.com founder Jeff Bezos knows the potential mortification factor.
For a live demonstration before an audience of 500 people, Mr. Bezos once logged onto Amazon.com (amazon.com) to show how it caters to his interests. The top recommendation it gave him? The DVD for "Slave Girls From Beyond Infinity." That popped up because he had previously ordered "Barbarella," starring Jane Fonda, a spokesman explains.
Dawn Freeman, 23, a tax analyst in Lexington, Ky., has bought lowbrow videos, such as "American Pie," from Amazon.com. But she was aghast when the site suggested Tom Green's gross-out performance in "Road Trip."
"I thought, 'I know I don't like high cinema, but have I really reached the point where I'd like to watch Tom Green lick a mouse?" To even out her Amazon profile, she went through the site finding "witty independent films."
Her TiVo also thinks she's a sophomoric-humor-loving 12-year-old, she says. It keeps giving her cartoons. "I know it's dumb to take it personally, but it's in your face. These are supposedly objective computers saying, 'This is what we think of you.' "
Dissing Ice Cube
A.J. Meyer, a 35-year-old Web site developer in Minneapolis, ordered the DVD for "Scarface," the Al Pacino gangster movie, from Netflix.com (netflix.com). After that, the site kept recommending movies about gangster rappers. He stopped the assault by giving negative ratings to all movies starring Ice Cube. (Netflix allows members to rate any of its 12,000-plus titles with one to five stars -- whether they have rented a film or not. That helps the site calculate future recommendations.)
After Mr. Meyer ordered a documentary about New York from Amazon.com, it pitched him countless documentaries -- even one on the history of the thimble. He stopped the Ken Burnsification of his profile by searching the site for plasma TVs. "That way, I identified myself as a high-tech guy," he says. "The thimble is more low tech."
Virginia Heffernan, TV columnist for Slate.com, doesn't understand why some people are resistant to techno-profiling, or find it creepy. She didn't look for any deep meaning when her TiVo kept giving her TV shows in Polish. And after buying self-help books on Amazon.com, she accepted that every time she logged on, the site pitched products to make her a more self-fulfilled human being.
"I like the idea that someone cares," she says. "Even a machine."
TiVo users can program the machine to skip certain channels entirely. But many users don't bother to figure out how to do it, or are too intrigued by TiVo's recommendation process, says a spokesman. TiVo is paid to promote programs and products it calls "advertainment" on a special screen. But the company says none of these are given to users as suggestions.
Some people have given up trying to manipulate personalization technologies. Dino Leon, a hair-salon owner in Birmingham, Mich., says his TiVo quickly figured out that he and his partner were gay. They were OK with that, but just for fun, they tried to confuse the software by punching in "redneck" programs, like Jerry Springer's talk show.
TiVo wasn't fooled, and kept recording gay shows. Mr. Leon believes the box was giving them a message: "You're definitely gay. And you're watching too much TV."
Write to Jeffrey Zaslow at
jeffrey.zaslow@wsj.com
For non-programmers:
A bitmap is a data structure where a collection of bits is stored. This allows for more compact storing of information. For example a 32 bit word can be used to store 32 true-false values. This is more efficient than storing an array of 32 bytes with TRUE or FALSE in them. Bitmaps are not limited to storing true-false data. A 32 bit word can store 8 four bit values as well.
In the pregnant gay man example, the bitmap likley had bits for male/female, gay/straight and pregnant. Set them all to 1's and you get a pregnant homosexual male.
Uninitialized variables are caused when a function accesses a variable before explicitly setting it. This is a common problem in C/C++ and can result in some odd behavior. An uninitialized variable could result in the bits being set even though the program never explicitly set them.
Remember, You are unique...just like everyone else.
It is most certainly a copyright violation, and not covered by fair use, since it is the entire article, with no commentary.
:)
That being said, thanks for posting it
Some people have a way with words, and some people, um, thingy.
As the article points out, the suggestion algoritm isn't perfect, but if it gets off target, it's fairly easy to correct... even though the users in the article obviously hadn't figured out the most efficient way to do so. The suggestion system works by allowing the viewer to press the "Thumbs Up" or "Thumbs Down" button. Strangely enough you can give a show up to three thumbs up or down (most people I know only have two thumbs;). The algorithm uses these ratings to find shows that have been catagorized the same way as shows that the user has rated highly. One thing that most people don't realize is that any show selected for recording automatically gets one thumb up. Naturally, for this system to work, show catagorizations have to be accurate, which isn't always the case.
The users in the article who recorded lots of shows to counter the ratings were doing things the hard way. A much easier way is simply to go to the suggestions screen where TiVo supplies a list of recommended shows that it thinks the viewer might want to see. From there, it's easy to just give three thumbs down to each of the shows that the viewer doesn't like. On a lucky day the show that caused the problem in the first place will appear as a re-run, so the problem can be fixed quickly. This can be repeated until the suggestions screen only shows stuff that the viewer likes.
Signatures are a waste of bandwi (buffering...)
Who gave this a score +5 Interesting? If you clicked on the link and actually read the story, you would know that www.wsj.com is not free, but online.wsj.com is free (atleast for this story).
When you first get a TiVo, it doesn't have a lot to go on, so it seems to use some generic suggestions to see what you think of them. After a few weeks of regular use, especially if you have set up multiple Season Passes and/or asked to record shows, it will fine-tune the suggestions. Also, if you delete a recorded suggestion without watching it (or more than 5 minutes), the TiVo notes that and adjusts future suggestions (though not as much as a ThumbsDown).
It is not a good idea to give three thumbs-down on a lot of shows, this will tend to deterioriate the suggestions algorithm. One thumb is usually sufficient, but keep in mind that TiVo doesn't know, for an individual show rating, WHY you thumbed it, so it adds or subtracts weighting from entries for genre, actors, directors, etc.
As an example - my wife has season passes for various home improvement shows, such as Changing Rooms. So we get lots of suggestions for other home and garden shows, including various Martha Stewart shows. My wife hates Martha, so we give her shows one thumb down, but the TiVo doesn't know it's because of Martha. It's the collective weight of other ratings that tune the suggestions.
For a while, there was a hidden feature called TeachTiVo, that allowed you to rate individual actors, directors, titles and genres. The UI wasn't complete (and was buggy), and you had to "enable backdoors" to get at it at all. The whole feature was removed in recent versions, unfortunately. I'd like to see something like it return in the future.