Netflix Replacing Star Ratings With Thumbs Up and Thumbs Down (variety.com)
An anonymous reader quotes a report from Variety: Get ready to say goodbye to star ratings on Netflix: The company is getting ready to replace stars with Pandora-like thumbs ups and thumbs downs in the coming weeks. Previously-given star rating will still be used to personalize the profiles of Netflix users, but the stars are disappearing from the interface altogether. Netflix VP of Product Todd Yellin told journalists on Thursday during a press briefing at the company's headquarters in Los Gatos, Calif., that the company had tested the new thumbs up and down ratings with hundred of thousands of members in 2016. "We are addicted to the methodology of A/B testing," Yellin said. The result was that thumbs got 200% more ratings than the traditional star-rating feature. Netflix is also introducing a new percent-match feature that shows how good of a match any given show or movie is for an individual subscriber. For example, a show that should close to perfectly fit a user's taste may get a 98% match. Shows that have less than a 50% match won't display a match-rating, however.
My experience with Netflix's star rankings is they matched my inclinations pretty well... except when it came to Netflix-produced content. With that stuff, Netflix invariably told me their "best guess" was between 4.7 and 5.0 stars, every time - but, after watching it, I don't think I gave any of it even 4 stars.
So perhaps they're trying to hide the way they're gaming the system to favor their own products.
#DeleteChrome
Great shades of Tivo! Man, I miss that awesome interface...
A/B Testing is an amazing tool. How many A/B tests have to be stacked with a 99/1 split to result in a 10% assurance that someone is going to be confused and disgusted with the change.
This one was a 66/33 split. Recalculate. How many ...
The closer we get to 50/50 the faster we make the user experience worse by listening to A/B tests without reason.
How many A/B Tests have been run by our favorite applications to hate.
God: "I don't leave footprints!"
Netflix will control the narrative, with minimal input from pesky users. How long before there's only a heart symbol? How long before they take away comments like IMDB because you don't want them.
What Netflix thinks I want to see and what I actually want to see lines up less than half the time. I want to be challenged and I want the interface to give me an opportunity to step outside of my comfort zone.
I don't see how reducing this to thumbs up/down is going to help that in any way.
All the ratings from every single viewer watching every single show and the best algorithm to process that data in the world won't help if you don't have a wide enough variety of content to recommend to your users.
At one point in Netflix's history, the number of 'Action movies with a strong female lead' on netflix was so low, but they knew that's what I liked, they started recommending to me rom-coms such as 'Bridget Jones Diary'
So basically, they figured that 3/5 starts is a "thumbs up" and there will be more positive ratings (and making their offerings look better) while abstracting (hiding) the "meh" factor. Gotta love marketing.. I remember an old phrase from my math tutors: Statistics can lie. :D
"Imagination is more important than knowledge" - Einstein
I'm sure there are many exceptions amongst the Slashdot user base, but hardcore critical thinkers aside, people don't use anything but "5 Stars" and "1 Star" anyway. They love it or they hate it. This change will make the experience more honest...now we KNOW it's just a bunch of crap. I'll be much less tempted to believe the recommendations now. That said...the recommendations Netflix used to give me, using the algorithm they used back in 2005...those were uncannily accurate.
Cloudiot: A person who does not see offsite storage as a way to lose control over access to his or her own data.
A web-site changes its interface... Stuff that matters? Seriously?
In Soviet Washington the swamp drains you.
Great. Now the PC world has invaded online movie reviews. Sorry, haven't seen Lena Dunham or Amy Shumer, mostly because I dislike their material. Maybe it's a generational thing. Netflix is now invalidating their recommendations because some whiny Hollywood types got their feelings hurt. Has anyone ever seen Heaven's Gate? Seriously, if Netflix can't stand the heat of their ratings system, why even have a ratings system? Is Reed Hastings (Co-founder and CEO) so needy for West Coast adulation that he's gutting their system? Sad.
The current rating system works for me. I generally only consider shows with 4 stars or above. It seems to correlate with my expectations. I have tried to watch stuff with 3 stars only and found it lacking. If the new system blurs 3 stars versus 4 then I will end up watching more crud that I don't want to. However, I guess we will have to see how it plays out.
They gave thumbs up/down ratings for movies, instead of ratings out of 4 or 5 stars (or A-F) like everyone else
One of the reasons why I think thumbs up/down works better than 5 stars is that everyone has a different idea of what 2, 3, and 4 stars should mean, and tend to skew towards 4 or even 4.5 stars as a midpoint ("average" product) instead of 3 as you'd expect from the scale's range. e.g. Amazon's 5 star system sounds like a really effective tool at first glance, until you learn that the average rating for all products is 4.47 stars. (4.36 for the 70% which are non-incentivized + 4.74 for the 30% which are incentivized = 4.47)
Most rating systems have run across this same problem. Some attempt to correct for it by normalizing (IMDB does this) to try to stretch the votes which are clumped up at the high end of the scale. But this creates other problems as some people submitting new ratings base it on the normalized scale, while others base it on their personal scale (skewed high).
With no clear consensus on what exactly the middle grades/ratings actually mean, it makes sense to just simplify the scale and make it binary.
I don't see how reducing this to thumbs up/down is going to help that in any way.
More engagement -- "thumbs got 200% more ratings than the traditional star-rating feature." Anecdotally, I often finish a show and don't rate it, as I find myself wondering if I "just" liked (disliked) it, or if I REALLY liked (disliked) it -- but a binary choice is often pretty obvious for me.
That said, I think it's often best to use some third-party tools/blog posts/friends/etc. for determining what to watch.
In physics, many a times S/N ratio is extremely poor for an individual bit of data. In such cases, it is better to 1 bit quantization rather than multi-bit for optimal performance. In star rating, you are giving more weight to people rating it at extremes. With thumbs up and thumbs down, the weight is same for everyone.
Theoretically, the optimum performance is to use "e" levels of quantization which I am not sure how to achieve. For integers, the 3 levels (like, dislike, no-opinion) is the most optimum when each sample has very low S/N.
Unqualified ratings systems are painfully limited whether they are stars or thumbs. Context is needed to know *why* people liked or disliked something. I can read reviews to get a small sample of this (assuming people have taken the time to write good reviews), but letting users tag content allows us find stuff we like so much more effectively. Tags for sub-genre, themes, memes, good acting, bad SFX or anything else people might be looking for. I wish Amazon would do the same for books too.
I think it was 2 years ago, it's so long I don't remember it anymore, but I really hated when that happened.
I liked to see what stuff had one star rating, because it usually told me it wasn't worth watching, now I have a HUGE list of "continue watching..." that won't go away, simply because there's so much trash that Netflix "guesses" that I want to watch.
Bet the thumb system was introduced to stop people from complaining about too few movies and series, and watch "whatever" is available instead of being selective about what we watch.
What this world is coming to - is for you and me to decide.
Yes, I know there is probably a team responsible for the interface.
The problem is that when you have a well thought out interface and that team still has a job, they have do work this means they have to make it worse (Hi Windows!). "We have created the perfect interface, now the team has some proposed changes.."
So they will go about endlessly fixing what ain't broke and then not going back to what was working because that would admit fault.
So now on PS3 the movies begin playing before you can read all of what they are about. They should have tried this on their CEO and let him read 3 seconds of each of his emails before holding a loud phone playing a movie over the top of what he is reading.
It used to not do this at all.
Then it started doing it when you selected the movie to read about it.
Now it does it on the main menu when you try to read descriptions... Someone really needs to die in a fire.
Next they will require a shock collar on your scrotum to use netflix and it will shock you after 5 seconds of not picking something.
After that the shock collar will just be shocking all the time.
So yes, removing meaningful ratings because your average human just 1 or 5 stars everything hurts everyone who is smarter than a bowl of oatmeal.
At this point I'm just watching Amazon Prime and they started out with the scrotal shock collar!
This is obviously what Reed Hastings meant as innovation that traditional theaters couldn't deliver.
Use an extension like this one that gives IMDb ratings on the Netflix page.
Slashdot, fix the reply notifications... You won't get away with it...
What's next -- removing the thumbs down???
What the fuck is the point of having ratings if you are just going to make them homogeneous???
Just because _you,_ Netflix, don't let me rate a movie 0/5 doesn't mean it deserves a 1/5. IF I hate a movie it should get 0/5.
The WHOLE point of a 5 star rating is to provide fine-grained-ratings not some bullshit dual artificial rating. There is a HUGE difference between me liking a movie 50% and 100%. Some movies are 3/5 (60%), 4/5 (80%), and very few are 5/5 (100%). Lumping them these ALL together is fucking retarded. HOW is that helping the system find stuff I _really_ like vs stuff I _kind of_ like???
Thumb up/Down only is freaking useless unless it also allowed for 2 Thumbs up, 2 Thumbs down, and Meh (1/2 thumb).
Just like Google Music, there is no distinction between "heard before and it was meh". Like or Dislike is just not anywhere near distinct enough to account for taste.
I too am sometimes unsure whether to give something a 3, 4, or 5 star, but the real problem for me was the lack of a neutral option. There are many shows that I didn't particularly like, but didn't dislike either. Yes a 3 is the middle of a 5-star system and so should fit that, but they specifically label 3 stars as "like", and 2 stars is specifically labeled as "dislike". Without an actual neutral option, I either don't bother, or I just pick 3 anyway since it seems less incorrect to say I liked it than to say I didn't like it.
Right, because we only ever "like" or "dislike" things. As humans, we never have any ambiguity in how we feel about things.
This is stupid, and a ham-handed attempt to "dumb it down" in order to boost the perceived rating. Ugh.
Just cruising through this digital world at 33 1/3 rpm...
It should be a two tier system. Click thumbs-up / thumbs-down, then pick a star rating. So you can mark it as "remember this, also remember a general bad/good" then say how much you actually liked it.
What makes me mad about Pandora is I can't mark "remember this" without changing the recommended songs given to me. I don't want to change my channel, I just want to be able to listen to it later.
No, that link you posted to a web comic we've all seen a hundred times is not "obligatory."
That's exactly the issue. My interpretation of 5 stars goes like:
1 Star - Shit
2 Stars - Bad, but with some minor redeeming features
3 Stars - Mediocre, nothing special, OK if in a certain mood
4 Stars - Good, solid, not groundbreaking, but good
5 Stars - Amazing, great, top of the heap
You see that our interpretations vary. We don't assign the same value to the ratings, which is a problem is a shared rating system. One persons "eh, it's OK" is another person's "would never watch again".
With a binary system, it becomes a lot easier to compare and combine ratings. Did you like it or did you dislike it?
Eat the rich.
And we continue the dumbing down of america.
How is it any different?
1 - Sucks a lot
2 - Sucks Less
3 - Good but has suckitude
4 - No Suckitude
5 - Great
4,5 is Thumbs in the general Up direction.
1,2 is Thumbs in the general Down direction.
3 is MEH
So you lose any distinction between just how much its liked/disliked or whether it was merely average.
Why is it important to you to classify a difference between how much you like/dislike it? Isn't it enough to see your rating and then remember for yourself why you gave it that rating?
I tried rating my music library, first with a 5 star scale, then a 4 star and a 3 star scale. In all cases, I spent more time fussing about ratings, than I did about the actual music.
So I actually ended up going with the simplest solution of just marking my favorites and leaving the rest well enough alone. Now I'm on Spotify, where I mark my favorites for inclusion in "my music" and just ignore the rest.
Eat the rich.
https://www.xkcd.com/937/