Slashdot Mirror

← Back to Stories (view on slashdot.org)

How YouTube's Algorithm Really Works (theatlantic.com)

Posted by msmash on from the how-things-work dept.

YouTube wants to recommend things people will like, and the clearest signal of that is whether other people liked them. From a report: Pew found that 64 percent of recommendations went to videos with more than a million views. The 50 videos that YouTube recommended most often had been viewed an average of 456 million times each. Popularity begets popularity, at least in the case of users (or bots, as here) that YouTube doesn't know much about. On the other hand, YouTube has said in previous work describing its algorithm that users like fresher content, all else being equal. But it takes time for a post to build huge numbers of views and signal to the algorithm that it's worth promoting. So, the challenge becomes how to recommend "new videos that users want to watch" when those videos are new to the system and low in views. (Finding fresh, potentially hot videos is important, YouTube researchers have written, for "propagating viral content.")

Pew's research reflects this: About 5 percent of the recommendations went to videos with fewer than 50,000 views. The system learns from a video's early performance, and if it does well, views can grow rapidly. In one case, a highly recommended kids' video went from 34,000 views when Pew first encountered it in July to 30 million in August. The behavior of the system was explicable in a few other ways, too, especially as it adapted to making more clicks inside YouTube's system. First, as Pew's software made choices, the system selected longer videos. It's as if the software recognizes that the user is going to be around for a while, and starts to serve up longer fare. Second, it also began to recommend more popular videos regardless of how popular the starting video was.

62 comments

  1. No not really by Anonymous Coward · · Score: 3, Insightful

    I look up one video on how to fix something and all i get now are howto videos. I just needed one video to fix it. Its fixed stop sending me fixit videos and let me get back to my cat videos. Right meow

    1. Re:No not really by Z00L00K · · Score: 3

      That's not the worst with the YT algorithm - the worst is that it suggests videos I already watched like a year ago. Only a rare few of them are worth watching again.

      --
      If builders built buildings the way programmers wrote programs, then the first woodpecker would destroy civilization.
    2. Re:No not really by Anonymous Coward · · Score: 0

      I don't login and for some types of videos Youtube specifically tries to divert me from them. Trying to stay on a line of current news has become impossible. Other videos seem to link almost exclusively to videos that have already been watched at this particular ip address.

    3. Re:No not really by Anonymous Coward · · Score: 0

      Exactly. Except not limited to a year ago. "Uh, what is the best way to decide what he likes?" "Um, what did he watch before?" "Oh, ok, that must be what he likes; we'll suggest that."

    4. Re:No not really by Hasaf · · Score: 1

      An example of a lot worse than that is mentioned in the article. The following is from it and illustrates the problem extremely well:

      My favorite example of how informationally toxic YouTube's algorithm is this:

      Imagine you're high school freshman and got a school assignment about the Federal Reserve.

    5. Re:No not really by AmiMoJo · · Score: 1

      It really helps to open the little menu and tell YouTube you are not interested.

      However, the response is delayed. It seems like YouTube has a cron task that regularly sets up new recommendations for you, and it only runs once or twice a day so you have to wait for it.

      --
      const int one = 65536; (Silvermoon, Texture.cs)
      SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
    6. Re:No not really by Anne+Thwacks · · Score: 1
      the worst is that it suggests videos I already watched like a year ago. No, even worse than that is that I click the "not interested" button, and it says OK, Ill take than into account and then shows me the same damn video choices again Not just stupid, but doubling down on annoyingly stupid!

      And If I have clicked on "No, I don't want that" four or five times, it still sends me the same, 5 year old dross.

      Also, why does the search/home not offer "only today's stuff please"? 10 day old stories about Trump are not even useful to Trump supporters or bots.

      --
      Sent from my ASR33 using ASCII
    7. Re: No not really by ememisya · · Score: 1

      I mean how did we get by when either promoted or recommended videos weren't all you were shown? People just saw new videos and/or sort by views? lol That's the only way YouTube is tolerable for me still. I filter for today, and search different letters of the alphabet to find new videos. Sometimes "of" or "the" works pretty well too. Otherwise you can only access videos YouTube locks you down to. It's free anyways, what are we complaining about?

    8. Re:No not really by Askmum · · Score: 1

      So true. You look at one cat video and the next 10,000 suggestions are cat videos.
      But apart from that, you might think that when you show someone 10 suggestions and he chooses none of them, that person is not really interested in those suggestions so when he reloads you show him something else.
      Noooo. Next 10 suggestions are exactly the same. Sometimes even marking the video as "not interested" doesn't do it, it comes right up again.
      Youtube's algorithm is flawed on a basic level.

  2. How YouTube's Algorithm Really Works by taustin · · Score: 4, Insightful

    Very, very poorly.

    Their criteria is what they think they'll make the most money on, and to hell with user preferences. I know this because I can tell it I'm not interested in a particular video, or entire category, then refresh the page, and that video or category will be back. Again and again and again, for months, even though I tell it I'm not interested every time.

    1. Re:How YouTube's Algorithm Really Works by Anonymous Coward · · Score: 0

      More likely user error in your case.

    2. Re:How YouTube's Algorithm Really Works by Anonymous Coward · · Score: 0

      At least Steam lets you ignore things properly. Even to the point of marking it as such in case you forgot.

      Don't keep pushing what your users don't want.

    3. Re:How YouTube's Algorithm Really Works by TechyImmigrant · · Score: 1

      At least Steam lets you ignore things properly. Even to the point of marking it as such in case you forgot.

      Don't keep pushing what your users don't want.

      I still haven't worked out how to stop it bringing up early access crap.

      --
      I should use this sig to advertise my book ISBN-13 : 978-1501515132.
    4. Re:How YouTube's Algorithm Really Works by rogoshen1 · · Score: 1

      yep, would be nice as well if you could have it hide indie stuff too.

      No, I have zero interest in pixel art games, and I'm slightly confused as to why there needs to be 70 gorillian of them.

    5. Re:How YouTube's Algorithm Really Works by Anonymous Coward · · Score: 1

      Yea, very true. I'd like to ban Bright Side from appearing on Youtube for me and possible go as far as blocking any video sponsored by Bright Side. Clearly there's no meaningful way to do this. Youtube doesn't care if people post Clickbait content nor do they put any effort into people spamming obviously false content. Then there's just the stuff I'm not interested in. 99.9999% of Youtube is a clusterfuck.

    6. Re:How YouTube's Algorithm Really Works by taustin · · Score: 1

      Then YouTube lies to me when it tells me they have noted the input and will reflect it in future recommendations.

      Moron.

    7. Re:How YouTube's Algorithm Really Works by taustin · · Score: 1

      If that's what they're peddling right now, you don't.

    8. Re:How YouTube's Algorithm Really Works by Anne+Thwacks · · Score: 1
      More likely user error in your case.

      you mean when he said he was not interested, it was "fake news"?

      Please give me a capcha where I can lick "I am a bot"!

      --
      Sent from my ASR33 using ASCII
  3. not interested broken by slothman32 · · Score: 4, Insightful

    This is as good a place as any.
    For the last month or so when I clicked the "not interested" box the row collapsed but when I closed then re-opened YT it was there again.
    I don't like seeing the "live gaming" and other things but they keep coming back.

    --
    Why don't you guys have friends or journals?
    1. Re:not interested broken by TechyImmigrant · · Score: 3, Interesting

      >I don't like seeing the "live gaming" and other things but they keep coming back.

      I don't like anime, but when you do like a few Japanese bands, the algorithm has only one mapping in its hash table. Japanese --> anime.

      --
      I should use this sig to advertise my book ISBN-13 : 978-1501515132.
    2. Re:not interested broken by Anonymous Coward · · Score: 0

      Same thing has been happening to me recently, and it's a complete PITA.

      Even stupider than recommending stuff I've repeatedly clicked 'not interested' is the way it's now recommending channels I'm already subscribed to.

      Oh, and has anyone else been getting notifications of new videos from channels they're not subscribed to ?

    3. Re:not interested broken by sexconker · · Score: 1

      No, the mapping is TechyImmigrant --> weeaboo.

    4. Re:not interested broken by jonwil · · Score: 1

      Even worse is when YouTube intentionally makes certain channels harder to find (e.g. not recommending videos from those channels to people, unsubscribing people from those channels at random, not notifying those people of new videos from those channels).

      Evidence I have seen is that this happens for channels that dont opt for monetization (i.e. dont earn YouTube ad revenue), that are in any way controversial or on the nose or might be unpopular with advertisers (e.g. gun related channels) or that for whatever reason YouTube doesn't like.

  4. It's a shit show right now by SmaryJerry · · Score: 3, Insightful

    YouTube took a wrong turn somewhere with it's algorithm. Recommendations have less variety than basic on air tv channels. The algorithm always assumes you want something similar to what you've previously watched which is a major mistake. Yes I watched a clip of a late night talk show. But I don't need late night talk shows recommended to me constantly. Google has the same issue with advertising. Yes I bought a product but I don't need to buy multiple of them so stop advertising it to me constantly for a month afterwards.

    1. Re: It's a shit show right now by Anonymous Coward · · Score: 0

      In Soviet YouTube, algorithm recommends you!

    2. Re:It's a shit show right now by del_diablo · · Score: 1

      Similarity is by itself not a mistake. But generally the way the Youtube algorythm works is that IF you watch something, that thing will taint your viewer log and permanently mark it. So instead of watching something, you click on a bad video, watch a few minutes, exit, and then that users entire catalog is suddenly "The new recommended feature" for quite some time. And this is where the similarity problem comes in: You then get more bad videos from other users because it has similar tags, without any real quality.

      The problem with "similar videos" is that the tag system has no way to perceive quality or depth. I might like high quality videos about the chemistry of steel milling, but once i have watched a few i wouldn't be able to stand basic introductions. And because Youtube is youtube, it only length/views/tags to gauge if a video is similar. So it can't tell a introduction or a low level video from a advanced tutorial or video.
      If you enter a liberal media such as film craft, the problem escalates rapidly, because a lot of people will do basic analyses of the craft, but very few has the vocabulary or critical perception needed to do it properly. You will have analyze of things that survive purely on production values vs pacing, but the youtube algorithm can't understand this, so you get a lot of people doing "babies first analyze of this series" where production values or pacing is never discussed.

      Youtube might have replaced things like hobby magazines for specific niches, but its ability to dive into the topics or expand is horrid lacking. You can basically find a video, and then find that the algorithm is incapable of giving you more to watch.

  5. I forget who by rsilvergun · · Score: 3, Insightful

    but somebody once observed that some of the brightest minds of our generation are figuring out how to get me to watch cat videos (and the 15 second ad spot before them) instead of curing cancer. To say nothing of all the math wiz's working on quicker High Frequency Transactions for Wall Street.

    --
    Hi! I make Firefox Plug-ins. Check 'em out @ https://addons.mozilla.org/en-US/firefox/addon/youtube-mp3-podcaster/
    1. Re:I forget who by Anonymous Coward · · Score: 1

      but somebody once observed that some of the brightest minds of our generation are figuring out how to get me to watch cat videos (and the 15 second ad spot before them) instead of curing cancer. To say nothing of all the math wiz's working on quicker High Frequency Transactions for Wall Street.

      Because we're all individuals who need the same thing - money in the capitalistic world that we live in - because that's where the glory is. Corporations make money - they give money - they are entities for themselves. Humanity in general doesn't offer money - businesses do. Spend your time - expend your life - rinse repeat. Could you have made the world a better place? Sure! Would you have sacrificed your life doing so? Absolutely.

  6. Re:I don't need them to tell me by Anonymous Coward · · Score: 0

    Take your FOREVER BEEF with creimer to Wikipedia. We don't need this shit on Slashdot.

  7. The content is more important than the algorithm. by Anonymous Coward · · Score: 0

    The content is more important than the algorithm indexing it.

  8. I like the variety by Sir_Eptishous · · Score: 1, Funny

    So when I search for "Live Slayer Reign 1986" I get:

    - An instructional video on making a dove tail joint
    - A small theater production of The King and I
    - 4 different videos of stick thin pubescents doing covers of Taylor Swift on out of tune acoustic guitars
    - The last ten minutes of Heavy Metal Parking Lot.
    - A video on how the Federal Reserve is bringing back vampires from DNA

    --
    We play the game with the bravery of being out of range
    1. Re:I like the variety by Anonymous Coward · · Score: 0

      Wait? Federal Reserve is bring back vampires from DNA? Reposting this on Facebook and Twitter now! The world needs to know.

      #ObamaIsSecretVampire
      #TrumpAncientThousandYearVampire

    2. Re: I like the variety by Anonymous Coward · · Score: 0

      That's nothing; if you search "stick thin pubescents doing covers of Taylor Swift on out of tune acoustic guitars" you get:

        - a visit from the party van

  9. Children... by Crash+Dummy+Redux · · Score: 1

    The first video mentioned in the TFA is: "Bath Song | +More Nursery Rhymes & Kids Songs - Cocomelon (ABCkidTV)". Of course, children. YouTube's largest unofficial demographic that will watch anything put in front of them. One popular children channel has 6 billion views.

  10. Re: The content is more important than the algorit by Anonymous Coward · · Score: 0

    Not if you can't find it

  11. Wrong by Anonymous Coward · · Score: 0

    How YouTube's algorithm really works:

    Watch one video of ships being put into the sea because it looked interesting....get recommended ship videos for the rest of your fucking life.

  12. woohoo by Anonymous Coward · · Score: 0

    My cat video has 67 views after eight years.

  13. Youtube algo not very good by Anonymous Coward · · Score: 0

    It is great for recommending videos I don't want to watch.

    Also. What is it with tech companies trying to predict everything? They will never succeed.

    1. Re:Youtube algo not very good by Anonymous Coward · · Score: 0

      Some of this is pretty clever, and some not so clever.
      Recommending a video that millions of people have watched is not clever.
      Watching how long a user is on a page and serving up longer videos is very clever.
      Somewhere in between you have lots of scenarios, such as:
      User doesn't know what they want to watch - how could you possibly predict this by analyzing what other people watch.
      User knows what kind of video they want to watch but not what quality they need or style.
      User knows exactly what they want to watch but don't even know if it exists and they wouldn't consider a similar video to be truly similar (a classic assumption is that people will watch the best available video).

      Getting users to search for videos under these scenarios and watching the patterns they follow would be interesting.

      Here's an experiment (too expensive to perform, probably):
      Take a really popular movie that hasn't been advertised yet (like a new star wars or fast/furious or whatever) and put it on youtube without telling anyone and see what the algorithm does.

  14. Help me to help you by CustomSolvers2 · · Score: 1

    Easy-to-implement, user-friendly and even likeable feature helping you to really have a worthy recommendation system: checkboxes allowing users to enable/disable channels/videos. Please, just let me train your system for you! You keep showing me the recommendations as so far (when being logged-in or not, triggered by just one time I watched something which I didn't like or by hundreds of videos really relevant to me), but add 1-click-to-check-uncheck boxes. If I uncheck something, you don't show me that stuff again. Deal?

    --
    Custom Solvers 2.0 = Alvaro Carballo Garcia = varocarbas.
    1. Re:Help me to help you by CustomSolvers2 · · Score: 1

      After reading other comments, I realised that you have a (pretty crappy) version of what I was proposing but only for logged users. I am almost never logged in when watching videos and didn't know about it. So, improved version of my suggestion: could you please enable that feature also for not-logged-in users? (You are caring about what I watch anyway). It would also be excellent if you could add that X (honestly, I prefer the checkboxes) to individual videos, additionally to full channels.

      --
      Custom Solvers 2.0 = Alvaro Carballo Garcia = varocarbas.
    2. Re:Help me to help you by Anonymous Coward · · Score: 0

      Hi I'm Adam from YouTube Comment Experience and I wanted to thank you for sharing these ideas. Do you prefer seeing X or checkmark? Can you describe what your ideal userflow experience would be like Thanks again for helping us shape YouTube.

    3. Re:Help me to help you by CustomSolvers2 · · Score: 1

      Hi I'm Adam from YouTube Comment Experience and I wanted to thank you for sharing these ideas. Do you prefer seeing X or checkmark? Can you describe what your ideal userflow experience would be like Thanks again for helping us shape YouTube.

      Yes, Adam, this is exactly what any company should always do for its clients: making sure that their (user) experience is as good as possible. They should do that spontaneously without users' help like what is happening now. If you are so lucky to have first-hand-knowledge about what you should do to improve and you don't maximise that information, you would be doing a terribly bad work. And your recommendation system does certainly need quite a few tweaks. Just take a look at many other comments in this thread. BTW, you are welcome.

      --
      Custom Solvers 2.0 = Alvaro Carballo Garcia = varocarbas.
  15. Conform to the herd by Anonymous Coward · · Score: 0

    or be adsorbed.

  16. Youtube recommends what they want us to watch by thadtheman · · Score: 1

    Not what we want to watch. Even when we want to watch it, what they recommend is a like a double layer 35" thick crust sauasge, beef, pepperoni pizza. We want it, but we avoid it because it isnot good for us. But they keep shoving it under our noses.

    1. Re:Youtube recommends what they want us to watch by eaglesrule · · Score: 1

      Rather like how "youtube creators for change" propaganda videos keep mysteriously popping up into my subscription feed.

      Or how "authoritative news sources" get recommended over channels that have organically grown their subscriber count and views by virtue of practicing actual journalism, rather than what passes for it on cable news.

      Or how "trending" is almost always vapid pointless nonsense often with few views or was originally posted long before new and more relevant content, as if someone has a job of making sure the wider audience never gets exposed to any ideas that might challenge them to think past the usual drivel.

      I really long for a platform where ideas can propagate for better or for worse without someone putting their thumb on the scale. Maybe if Bitchute doesn't get the Gab treatment and get strangled in the crib there is a chance that it could be that platform. Just maybe.

    2. Re:Youtube recommends what they want us to watch by Anonymous Coward · · Score: 0

      No, Youtube keeps recommending me hot garbage no matter how many times I tell it I'm not interested in it, yet every time I load the homepage or scroll the sidebar recommendations that shit is back in the list, I've even gone through my entire viewing history and pulled out anything that I might have viewed that came from outside of my usual subscriptions and interests, yet the useless recommendations remain.

  17. Youtube thinks I'm a conspiracy theorist by Anonymous Coward · · Score: 0

    Most recommendations are vapid d-list online-celebrity self promotions and videos "explaining" the world from conspiracy nuts. I regularly feel insulted when I see the "recommended for you" tag. I guess that's what I get for blocking ads and deleting cookies. Fuck you too, Google.

    1. Re:Youtube thinks I'm a conspiracy theorist by Anonymous Coward · · Score: 0

      I do the same and have actually had youtube give me a survey question to answer about whether I believe youtube could help me find what im looking for. At the same time that the question was asked youtube seemed to be actively thwarting my attempt to find a song I didn't know the title to.

  18. Re:I don't need them to tell me by Anonymous Coward · · Score: 0

    Negative, I came here just to laugh at the creimer post

  19. Re: It work by posting wikipedia articles below by Anonymous Coward · · Score: 0

    Give it some time; sooner or later it will get better, and then it will suggest links to purchase antipsychotic medication.

  20. starts with catagories by houghi · · Score: 1

    There are only 20 catagories,if that. This will drown out most thibgs. Want to see some huling or australian rules? Well, tough.

    Adding a lot of subgroups and sub sub groups would sallready make it easier to include some and exlude others.

    I woukd like some music, but not others. And searching with google.com gives better results than with youtube using the same search..
    It is also obvious it is tracking you. On a clean install, a search wil show the same prefered videos on the side as being logged in for a month. And i follow some non-connected suff.
    Makeup 4wheel driving, computer related, ,house building, toy trains, gaming. So how come theygive me the same makeup channel I am subscribed, as well as several other videos of other channels I am subscribed to to on a PC with a blank image after watching a soccer video? Somethung I never watcj, normally. Obviously not logged in anywhere.

    --
    Don't fight for your country, if your country does not fight for you.
  21. No surprise by Anonymous Coward · · Score: 0

    Back in the heyday of AltaVista and some others, I could find what I really wanted. Yes, it took more work on my part, but finding something I really want, if it isn't very popular, with Google is very unlikely. YouTube, unsurprisingly, is becoming increasingly the same. I don't want what's the most popular. Ever. I want what I'm looking for. If I want YouTube to suggest something, I'll ask for it ("when I want your opinion, I'll beat it out of you"). Lately, when looking for how-to videos, I put in precise keywords, but I still have to run through pages of popular somewhat-related junk to find what best matches my request. Booo. Google and YouTube are the most successful in returning, near the top, what I request when what I request is related to a subject that is never going to become very popular. How about a switch on the screen, "Popular" or "Precisely"?

  22. Iteration by dhaen · · Score: 1

    Sounds like an iterative function. Let's hope it never explodes.

    1. Re:Iteration by williesmith1 · · Score: 1

      youtube also uses browser fingerprinting and ip address for recommendations, it doesn't seem they included this in their experiment, so each walk would be influenced by previous walks. i find the autoplay on youtube combined with the deep learning for recommendations to be an odd combination...the ai is basically reinforcing itself if the user doesn't do anything. you get a situation where the user can walk away from the computer and the AI assumes the user liked all these videos it played. soundcloud downloader

  23. Youtube is on its way to failure. by sycodon · · Score: 1

    As more and more videos are uploaded, the ration of good to Garbage gets worse.

    Eventually you won't be able to find anything good for all the garbage.

    --
    When Fascism comes to America, it will call itself Anti-Fascism, and tell you to give up your guns.
    1. Re: Youtube is on its way to failure. by Zero__Kelvin · · Score: 1

      That makes no sense. There is literally no logic backing your ckaim.

      --
      Guns don't kill people; Physics kills people! - John Lithgow as Dick Solomon on Third Rock From The Sun
  24. gigo by Anonymous Coward · · Score: 0

    Youtube's recomendations give me a nose bleed. I try to avoid them at all cost.

  25. Autism really makes the algorithm go nuts by Anonymous Coward · · Score: 0

    I'm autistic, so I love repetition of some of my most favorite videos to a point that normies tune out like a clerk at a store during Christmas season. That makes YouTube show me really old videos that I've probably already seen, which is okay to some extent. But I also like discovering new videos from new people, and some of my favorite videos ever came from mostly random suggestions. I find these random suggestions to be far too sparse. There is no way to tell YouTube that I like the surprise and wonder of YouTube circa 2008 instead of the "here's more of the same old shit, oh and Jimmy fucking Kimmel because we're basically shitty TV now" YouTube of 2018. Here, I'll go reload the page right now. Recommended contains: four videos I've seen before, a fuckton of gaming stuff I don't really want to watch (YouTube thinks that because I (re-)watch old AVGN, [Retro]Ahoy, Slope's Game Room's Kickscammers series, and Larry Bundy Jr. videos, I want to watch a LOT of gaming videos, but most gaming videos are fucking garbage that I don't ever want to watch), a couple of old videos by channels I like, and a few subscriptions I have that put out like four fucking videos a day.

    Long gone are the days when I could just magically happen upon FreddeGredde's song medley on classical guitar which led me to discover AVGN because it was one of the songs. Now it's all "the AI overlord at YouTube thinks you'll like the same old shit, a bunch of seemingly related shit that you keep punching thumbs down on and closing, and here's some more goddamn fucking Jimmy Kimmel because DON'T FORGET TO PAY FOR YOUTUBE RED AND TV AND ALL THAT HURR DURR IMMA CORPORATE WHORE"