Slashdot Mirror

← Back to Stories (view on slashdot.org)

Google Scraps Controversial Policy That Gave Free Access To Paywalled Articles Through Search (theverge.com)

Posted by msmash on from the up-next dept.

For years, Google has provided a nifty trick to get around subscriptions for newspapers and magazines. But the company is now doing away with it. From a report: Google is ending its controversial First Click Free (FCF) policy that publishers loathed because it required them to allow Google search results access to news articles hidden behind a paywall. The company is replacing the decade-old FCF with Flexible Sampling, which allows publishers instead to decide how many (if any) articles they want to allow potential subscribers to access. Google says it's also working on a suite of new tools to help publishers reach new audiences and grow revenue. Via FCF, users could access an article for free but would be prompted to log-in or subscribe if they clicked anywhere else on the page. Publishers were required to allow three free articles per day which Google indexed so that they appeared in searches for a particular topic or keyword. Opting out of the FCF feature was detrimental because it demoted a publisher's ranking on Google Search and Google News.

72 of 97 comments (clear)

  1. One can only hope. by msauve · · Score: 4, Insightful

    Hopefully, Google will also recognize paywalled sites and refuse to index them, or at least put them at the bottom of the results.

    --
    "National Security is the chief cause of national insecurity." - Celine's First Law
    1. Re:One can only hope. by Grishnakh · · Score: 2, Insightful

      Also hopefully Trump will trip and fall into a fucking cuisinart,

      This is a horribly thing to wish for.

      If it happened, we'd have Pence as our president, which would be even worse.

    2. Re:One can only hope. by NicknameUnavailable · · Score: 1

      Hopefully, Google will also recognize paywalled sites and refuse to index them, or at least put them at the bottom of the results.

      TFA literally says they are giving them an alternate means to have their sites indexed. It's the exact opposite of good news because it just blocks an avenue by which to read things for free which would otherwise be behind a paywall.

    3. Re:One can only hope. by Anonymous Coward · · Score: 1

      Atleast google has that down arrow you can click to view related articles, at least one of them is likely to not be pay walled. But they do need to come of with some way to notate pay walled articles, and while they're at it flag the sites that pester you to turn off your ad block to view an article. Nothing like getting a paragraph or two into an article just to have that dumb ass shit pop up.

    4. Re:One can only hope. by grahamsz · · Score: 1

      It'd be great if you could let google know which paywalled sites you subscribe to so that those still appear but others choose not to.

    5. Re:One can only hope. by JoshuaZ · · Score: 1

      Pence would be horrible in many respects, but he wouldn't be nearly as incompetent or blustering. The sheer risk of nuclear war by itself is so severe with Trump, that I'd be completely willing to put up with Pence's theocratic tendencies if it meant we wouldn't have that risk.

    6. Re:One can only hope. by alexo · · Score: 1

      If the choice is between an incompetent evil and a competent one, I'd rather endure the former.

      (A better choice would be a competent good, but that is not currently an option).

    7. Re:One can only hope. by Merk42 · · Score: 2

      This.
      People shouldn't be paid for their work. I'm entitled to free content!

    8. Re:One can only hope. by AvitarX · · Score: 1

      I'd love a search tool that let me filter out the pay sites I won't be able to read.

      --
      Wow, sent an e-mail as suggested when clicking on "use classic" banner, and got a fast response that addressed my msg
    9. Re:One can only hope. by JohnFen · · Score: 2

      Oh, please.

      If the means by which people are choosing to get paid involves subjecting their audience to the online advertising industry and the tracking that goes with it, that's their own fault, not mine. Such advertising is far from the only way to get paid, it's just the easiest for the websites.

      I don't object to reasonable advertising. I will not tolerate the tracking, though, and until it stop then I am keeping my adblocker in place and not disabling it for anybody. If that means I'm locked out of some sites, then so be it.

    10. Re:One can only hope. by tepples · · Score: 1

      I don't object to reasonable advertising. I will not tolerate the tracking, though, and until it stop then I am keeping my adblocker in place and not disabling it for anybody. If that means I'm locked out of some sites, then so be it.

      That's why I use the tracking protection built into Firefox. It's enabled by default in Private Browsing windows and can be enabled through about:config for use outside Private Browsing. It and the similar Disconnect extension should cover ad networks and ad exchanges that track users across sites. This gives the user plausible deniability against accusations of freeloading, as it a publisher (operator of a website that carries ads) can still serve self-hosted ads to tracking blocker users.

    11. Re:One can only hope. by Merk42 · · Score: 1

      TFA is about paywall sites. Sites that get their revenue from subscriptions instead of advertising.

    12. Re:One can only hope. by swillden · · Score: 2, Insightful

      Also hopefully Trump will trip and fall into a fucking cuisinart,

      This is a horribly thing to wish for.

      If it happened, we'd have Pence as our president, which would be even worse.

      No, it wouldn't. Pence is a more manageable sort of crazy.

      --
      Note to ACs: I usually delete AC replies without reading them. If you want to talk to me, log in.
    13. Re:One can only hope. by swillden · · Score: 1

      If the choice is between an incompetent evil and a competent one, I'd rather endure the former.

      The choice is between an unpredictable evil and a predictable one. The latter is much easier to manage.

      --
      Note to ACs: I usually delete AC replies without reading them. If you want to talk to me, log in.
    14. Re:One can only hope. by tepples · · Score: 1

      Some website operators and Internet VOD providers insist on both a subscription to see anything and an additional subscription to view documents without ads.

    15. Re:One can only hope. by JohnFen · · Score: 1

      I use Firefox's tracking protection as well, but I consider it incomplete -- so I use NoScript in addition to it. I don't use an extension specifically designed to adblock (NoScript covers that as a side-effect), but I do use a rather huge hosts file to redirect ad company domain names to localhost (I do this on my phone as well).

    16. Re:One can only hope. by tepples · · Score: 1

      I do use a rather huge hosts file to redirect ad company domain names to localhost

      Ad companies appear to have already started to defeat hosts by using pseudorandom subdomains. APK's solution can't block these, but Firefox tracking protection can:

      For each HTTP load, Firefox looks up multiple URL fragments based on Safe Browsing regex lookup. This allows us to blocklist all subdomains of a tracking domain without enumerating each one.

      (I do this on my phone as well).

      What method do you use for this? VPN or root?

    17. Re:One can only hope. by JohnFen · · Score: 1

      Ad companies appear to have already started to defeat hosts by using pseudorandom subdomains. APK's solution can't block these

      This is true (although I bristle at calling this "APK's solution", since this has been around longer than APK has). So far, this has been pretty easy to compensate for with wildcard entries, but I see the day coming when that won't be a decent solution anymore. I've been working on other ideas to handle it in the longer term.

      What method do you use for this? VPN or root?

      Both. When I'm away from my own WiFi, my phone sets up a VPN connection to my home network. When that's in play, then the ad networks are blocked by my network's firewall. The phone also has a hosts file installed to cover instances where I can't connect to the VPN for one reason or another.

    18. Re:One can only hope. by Merk42 · · Score: 1

      Sure, some do, but the original comment and article is about paywalled sites as a whole, making no distinction between sites that are paywall only and paywall and advertising.

    19. Re:One can only hope. by AmiMoJo · · Score: 1

      They can get paid, they just can't waste my time baiting me into visiting their web site only to be hit with a demand for payment.

      Aside from anything it's false advertising and SEO scamming. Google doesn't appreciate it when you present different content to their indexing bot and to browsers.

      --
      const int one = 65536; (Silvermoon, Texture.cs)
      SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
    20. Re:One can only hope. by Merk42 · · Score: 1

      I got this circular in the mail advertising products they had in stock, but then when I went to the store I had to pay for them?! Such false advertising!

    21. Re:One can only hope. by AmiMoJo · · Score: 1

      Junk mail is well understood to be advertising things that are for sale. Search results with snippets of articles are understood to take you to those articles to read.

      It would be fine if Google put a little icon to indicate non-free content next to such results, and allowed you to filter them out.

      --
      const int one = 65536; (Silvermoon, Texture.cs)
      SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
    22. Re:One can only hope. by AmiMoJo · · Score: 1

      Most of it is just trying to charge you for free stuff anyway. How often does a Slashdot story include "story may be paywalled, click here to read the same thing for free"?

      --
      const int one = 65536; (Silvermoon, Texture.cs)
      SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
    23. Re:One can only hope. by Merk42 · · Score: 1

      So no effect on its ranking, but just an icon next to it? I'd honestly be fine with that.

    24. Re:One can only hope. by AmiMoJo · · Score: 1

      A paywall will get the site down-ranked naturally anyway. People will link to it a lot less often.

      --
      const int one = 65536; (Silvermoon, Texture.cs)
      SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
  2. Best Solution by Luthair · · Score: 4, Insightful

    Give me as a user the optional to hide sites with paywalls.

    1. Re:Best Solution by Anonymous Coward · · Score: 1

      Indeed. Or at least put the result at the bottom of the page with a big red P next to it.

    2. Re:Best Solution by alexo · · Score: 1, Redundant

      Give me as a user the optional to hide sites with paywalls.

      You are the product, not the customer.

      You have as much say about Google's practices as the cows have about the MacDonald's menu.

    3. Re:Best Solution by JohnFen · · Score: 1

      I think that's why the commenter said "optional". Personally, a whitelist wouldn't be useful to me unless it didn't require me to have an account with the search engine and didn't require me to tell the search engine what sites I'm paying.

      I'd much rather have an blanket on/off switch that excluded all paysites.

  3. What a coincedence by Bill+Hayden · · Score: 4, Insightful

    I too have a decades-old policy: I don't use pay-walled sites.

    --
    Protect your browser with the Force Safe Search add-on
    1. Re:What a coincedence by JohnFen · · Score: 1

      Since the choice we're given is often between ads that spy on you and paying money, I have taken to paying money for sites that are important to me.

      But only if paying money allows me to keep my browser defenses up.

    2. Re:What a coincedence by tepples · · Score: 1

      With the 30 cent swipe fee that credit card networks charge, how can a user afford to read one article from each of 20 sites?

    3. Re:What a coincedence by JohnFen · · Score: 1

      They can't , at least until someone comes up with a reasonable micropayments system for websites. I'm still baffled by why nobody seems to be able to make this work, given that laundromats have been doing it for years now.

      But I'm not going to pay to read a single article. I pay sites that I read regularly, and there aren't 20 of those.

    4. Re:What a coincedence by JohnFen · · Score: 1

      Yes, this is common but not universal. I won't pay unless doing so eliminates the ad-related tracking.

  4. Self destructing link aggregation. by Zurkeyon3733 · · Score: 1

    Sexxy! ;-P

  5. NoScript by Bing+Tsher+E · · Score: 5, Informative

    I have found that if you enforce javascript blocking using NoScript, some sites that only want you to be able to view a certain 'count' of articles for free just can't keep track and don't block you.

    1. Re:NoScript by Anonymous Coward · · Score: 1

      I have found that if you enforce javascript blocking using NoScript, some sites that only want you to be able to view a certain 'count' of articles for free just can't keep track and don't block you.

      Block the cookies they use to keep track of you, block the javascript they use to do shit you don't need, block pretty much all third party shit because it's pretty much parasites, block Flash because only a fucking idiot runs Flash on the general internet.

      If it ends up being paywalled, block the whole site and use your back button.

      Tools like http switchboard allow you to block a tremendous amount of shit on the internet which simply adds no value. I simply refuse to participate in the marketing bullshit on the internet, if that means there are sites I can't access, so be it.

      Set your cookie policy to be whitelist only, because most of the cookies you'll be getting are purely tracking and analytics. That count of articles is quite often done at the cookie level.

    2. Re:NoScript by JohnFen · · Score: 1

      The internet is completely dependent on Javascript now...sad.

      I think you mean the web, not the internet.

      That aside, I still think that assertion is overblown. Yes, there are some sites that break completely when you don't allow Javascript, but most sites that I encounter handle it without breaking. You just don't get all those fancy animations and such (which, in my opinion, is an improvement).

      It is true that the first couple of months of use NoScript can be a pain in the ass, but once you get it dialed in, it rarely gets in the way.

  6. Re:Publishers Were Not Following the Rule Anyway by Archangel+Michael · · Score: 1

    incognito mode helps get around Paywalls especially when they have "free" versions for new visitors.

    --
    Agent K: A *person* is smart. People are dumb, stupid, panicky animals, and you know it.
  7. Sigh, Google by mysidia · · Score: 4, Insightful

    I DONT WANT content in search results that I can't actually view.

    Fine, get rid of FCF if you want, but then either blacklist subscription sites from search indexes, OR require indexed content match what I can see and
    give me a checkbox to omit them from search results (preferably checked by default).

    1. Re:Sigh, Google by bigdavex · · Score: 1

      I don't want that either. But I don't pay google's bills, so I might not get to pick.

      --
      -Dave
  8. Re:Adblockers? by Grishnakh · · Score: 1

    I'd much rather we get something closer to the BBC, more than the circus-for-money we currently have in American media.

    The problem with that is that it'd be government-owned, and most Americans wouldn't trust it at all.

    Remember, at least half of Americans believe that 9/11 was staged by the government.

  9. Mostly there is no paywall by nospam007 · · Score: 1

    Just delete your cookies and you're good to go.

  10. Re:Publishers Were Not Following the Rule Anyway by bored_lurker · · Score: 1

    Not sure what you consider far right or Christian. Looking at the top 10 results in google news (signed out) shows NPR, Bloomberg (x2), BBC, CNN, NY Times (x2), Al-Bawaba, and Washington Post (x2). None of these strike me as either of those and some quintessentially the opposite.

    --
    --- Tolerance is the axiomatic "virtue" of those without convictions ---
  11. Hope they integrate with with Personal Blocklist by Paul+Burney · · Score: 1

    As with many others here, I don't want results from sites that I can't visit. I understand that Google wants the data, but there's no reason that I need to see that mixed in. Google currently has a feature for Chrome users that not many people seem to be aware of called the Personal Blocklist. You can get the extension from Google here:

    https://chrome.google.com/webs...

    After you've installed it, when you're on a google search results page, you'll see a small link to "block example.com" under each result. No more articles from Forbes or pictures from pinterest that you can't browse through. Hope that helps someone!

    --
    <?php while ($self != "asleep") { $sheep_count++; } ?>
  12. Re:Publishers Were Not Following the Rule Anyway by Grishnakh · · Score: 1

    I constantly see (signed out; I never sign into Google News or other services at work), including in today's feed along side the usuals (BBC, NYT, WaPo, CNN, Reuters, etc.):

    Fox News
    ChristianityToday.com
    The Narrative Times
    RT

    I frequently also see Breitbart and Washington Times, though not today.

  13. Look for "Cached" by tepples · · Score: 1

    I have good news and bad news.

    But they do need to come of with some way to notate pay walled articles

    Good news: Google Search on desktop browsers fairly reliably notates paywalled articles through the lack of "Cached" in the down arrow menu.

    Bad news: The down arrow menu doesn't appear in mobile browsers.

    and while they're at it flag the sites that pester you to turn off your ad block to view an article.

    Bad news: Now that Google has established its pay-per-article system known as Contributor, Google has actually joined the anti-adblock brigade. It even recommends that users of Firefox Private Browsing click a button labeled "Disable protection" to allow access to a site. (This is because most anti-adblock can't tell the difference between tracking blockers, which block only ad networks and exchanges that track users across sites, and actual ad blockers, which additionally block self-hosted ads.) Now that TV Tropes is using anti-adblock, I'm even more glad that I switched to All The Tropes years ago.

  14. CPB + viewers like you by tepples · · Score: 1

    PBS and NPR exist. They're not government agencies, though they are funded by a mix of contributions from government programs and viewers like you (thank you).

    1. Re:CPB + viewers like you by Grishnakh · · Score: 1

      Watch for them to be defunded soon.

  15. Noarchive is flagged by tepples · · Score: 1

    I will only be truly disappointed if Google does not clearly flag them in search results

    Paywalled documents use <meta name="robots" content="noarchive">. Google Search flags results with noarchive on desktop (look for lack of "Cached") but not on mobile.

    1. Re:Noarchive is flagged by JohnFen · · Score: 1

      Those tags are commonly used on sites that aren't paywalled, too.

    2. Re:Noarchive is flagged by mysidia · · Score: 1

      That's a good indicator.... now if Google would please modify search so that NOARCHIVE documents are listed in search results with No snippet, AND the Search result will only be returned based on keywords in the Title, NOT a document search against content made visible only to Google.

  16. Something rank here by Impy+the+Impiuos+Imp · · Score: 1

    Publishers were required to allow three free articles per day which Google indexed so that they appeared in searches for a particular topic or keyword. Opting out of the FCF feature was detrimental because it demoted a publisher's ranking on Google Search and Google News.

    Wait wait hold on a sec. I thought Google played innocent on rankings by claiming it was all algorithms.

    --
    (-1: Post disagrees with my already-settled worldview) is not a valid mod option.
    1. Re:Something rank here by tepples · · Score: 1

      I guess the algorithm used to be that if the document presented to Googlebot is much longer than the document presented to a Chrome user who opts into telemetry, cloaking is happening. Now it's still an algorithm: Google Search will ignore CSS classes marked as paywalled through JSON-LD when making that determination.

  17. Re:Publishers Were Not Following the Rule Anyway by bored_lurker · · Score: 1

    I do sometimes see Fox and Washington Times but at no higher rate, more often less, than NY Times, BBC, etc. I also see articles from Al Jazzera and the Hindu Times.

    Personally I use Google News specifically because it gives me a view from diverse perspectives. If it only showed sources from the left or the right I would stop using it. I believe the truth is usually somewhere in the middle and I can't find that if I only get news from only one perspective.

    --
    --- Tolerance is the axiomatic "virtue" of those without convictions ---
  18. Which sites use noarchive w/o conditional access? by tepples · · Score: 1

    Among sites that appear in Google Search results displayed to me, I have perceived the noarchive value as noticeably correlated with conditional access methods, such as paywall or anti-adblock. If there were no desire for conditional access, a rational site operator would allow archiving even if only to shift the hosting burden for old documents to archive operators. For an example of such shifting, see here:

    The old OurPla.net is archived here (and the most active part, my wiki-weblog here). (Thank you, archive.org.)

    Your experience appears to differ from mine. Which sites using noarchive that lack conditional access do you commonly see in results from Google Search?

  19. Who is the customer? by DidgetMaster · · Score: 1

    As usual, many of the comments reflect the idea that us peons are Google's customers. We want Google to cater to OUR needs, instead of Google's real customers...those who pay Google real money to show up in search results. It is funny to hear people who never want to pay a cent for anything expect their needs to be considered by (and be a top priority for) various businesses who are in it for the money.

  20. Re:Publishers Were Not Following the Rule Anyway by Grishnakh · · Score: 1

    It's not necessarily "crazy" (not like infowars.com garbage), but it absolutely is religious in nature. Something about "God's goal" is not "news", it's religious claptrap, even if in the end they are agreeing with my personal opinion on homosexuality. It's nice to see the Christians (or some of them) turning against prosperity gospel, but again, that's religious stuff, not general "news".

    Personally, I think it'd be perfectly fine to put that kind of thing into a news aggregator as something that users can choose to see in their feeds. But don't make it part of the default.

  21. Re:Which sites use noarchive w/o conditional acces by JohnFen · · Score: 1

    I'm not able to come up with a list for you at the moment, but I do see it often. I do it on my own sites as well: I use "noarchive" to prevent the caching of pages that change frequently (typically, this is the front page), and allow caching on pages that don't. This is the pattern I usually see with non-paywalled sites.

  22. There are no subscriptions I'm willing to pay for by irrational_design · · Score: 1

    There are no subscriptions I'm willing to pay for, other than water, gas, electricity, garbage and Internet. If I'm not willing to subscribe to newspapers, cable TV, streaming TV/movies, magazines, etc. I'm definitely not going to consider subscribing to your online site. I really hope google makes it possible for you to be hidden from me.

  23. Re:Publishers Were Not Following the Rule Anyway by JohnFen · · Score: 1

    Christianity Today is absolutely coming from a specific perspective (please name the new organization that is not), but I think they're a decent news outlet nonetheless.

    They aren't sneaky about what their slant is, and despite their slant, they actually engage in solid, honest, and relatively unbiased journalism.

    I'm not Christian, but I still consider them one of the more respectable outlets these days.

  24. Re:Publishers Were Not Following the Rule Anyway by JohnFen · · Score: 1

    Damn, my brain farted hard in that comment. I confused Christianity Today with a different site. Please ignore everything I said there.

  25. Re:Publishers Were Not Following the Rule Anyway by Archangel+Michael · · Score: 1

    Yeah, paywalls keep the stupid out. Nothing more.

    --
    Agent K: A *person* is smart. People are dumb, stupid, panicky animals, and you know it.
  26. Re:Publishers Were Not Following the Rule Anyway by Grishnakh · · Score: 1

    You must be thinking of the Christian Science Monitor. That's something else altogether. CSM has been around for decades, long before the WWW.

    I'd never even heard of ChristianityToday.com until I noticed Google News pushing it in the last few months. AFAICT, it's just "news" for Christians. That's OK, I guess, but it's not "general interest", any more than "IslamToday.com" or "ScientologyToday.com" would be, and a general-interest news aggregator has no business putting their stories in the default feed. Allowing users to select it is another matter.

  27. Re:Publishers Were Not Following the Rule Anyway by JohnFen · · Score: 1

    Yes, I was.

  28. paywalls by tina+juarez · · Score: 1

    Oh goody.. Now I will NOT be tempted to follow more bad news.... This will greatly aid my withdrawal from the internet as it has become.

  29. Won't paywall articles get downranked? by GuB-42 · · Score: 1

    Google says that paywalled news sites won't get downranked. It may be true in the sense that there won't be an explicit penalty.
    However, a common reaction after hitting a paywall is to go back to the search page and find something else. Normally, in that situation, googles downranks the offending site, considering that it didn't match the user's needs.

  30. I don't believe all content should be free... by el_smurfo · · Score: 1

    ...but when I hit a paywall, I always close the window. There are so many other ways to get information today, it's not worth me trying to figure out if the content is worth buying if I can't even get to hit. That goes double for that stupid Forbes loading screen as well.

  31. Metered paywall on profile views by tepples · · Score: 1

    Given the recent article "LinkedIn Says It's Illegal To Scrape Its Website Without Permission", LinkedIn probably has some sort of quota on profile views by the general public, like a metered paywall.

  32. Re:Publishers Were Not Following the Rule Anyway by Grishnakh · · Score: 1

    Exactly. That's why I like to see other news sources besides American mainstream ones, so BBC, Al Jazeera, and Hindu Times are interesting to read for this reason. But Breitbart and Infowars are just a complete waste of time, as is Fox News. Fox is so obviously slanted, and the other two are just tabloid trash making up bullshit (like Infowars continuing with the 9/11 "truther" conspiracy-theory nonsense after all these years).

  33. Re:Which sites use noarchive w/o conditional acces by mysidia · · Score: 1

    I use "noarchive" to prevent the caching of pages that change frequently

    That is absolutely not what noarchive is for -- there are are other directives to control caching.
    Noarchive is for asserting that projects and tools such as archive.org shall not save and make available historic versions of
    a web page allowing users who explicitly want to see old versions to see them.

    The page creator has that right legally to say nobody should redistribute archived versions of their page, which
    is what that tag is for --- but as far as I'm concerned anyone setting Noarchive is being Evil / anti-internet by
    making their site part of the disease that is information that can be lost -- in most cases trying to squeeze their idea of maximum Profit, which is
    not what the world wide web is for, and not the kind of content I want my Google searches to bring me to, unless there's no other option.

  34. Not google's fault by YoungManKlaus · · Score: 1

    How is google supposed to know that a page is browser-UA-sniffing and blocking anything that isn't google bot? Personally I'd love to see if google abandoned their custom UA in favor of faking a regular browser so this kind of BS stops.