Is the Web Heading Toward Redirect Hell?

Ant snips from Royal Pingdom this excerpt: "Google is doing it. Facebook is doing it. Yahoo is doing it. Microsoft is doing it. And soon Twitter will be doing it. We're talking about the apparent need of every web service out there to add intermediate steps to sample what we click on before they send us on to our real destination. This has been going on for a long time and is slowly starting to build into something of a redirect hell on the Web. And it has a price."

Comments

How do you get offenders to stop?

[alain94040]

Funny just this morning I noticed that it took at least 5 redirects or more for Google to let me login to Analytics. It felt like my browser had a life of its own!

The real problem though are the link shorteners. I'd like to vote with my feet and never click on them, but for many, they are like drugs, because they let you track your influence (how many people clicked) in real-time. It's especially bad on slower connections such as smartphones. Not everyone has 1MB/s.

Any ideas on how to convince people to stop?

--
Don't work on your startup project without a safety net

Re:How do you get offenders to stop?

[duguk]

Not everyone has 1MB/s.

Any ideas on how to convince people to stop?

Surely it's the latency, not the bandwidth that is the problem with 301s?
They can't be much more than a few hundred bytes!

Re:How do you get offenders to stop?

[TooMuchToDo]

Any ideas on how to convince people to stop?

Create a web service where you can provide a shortened URL and it will respond back with the full URL. Make sure this web service caches the redirect for at least 24 hours. You instantly kill any reason for the redirect to be there (their counts will no longer be accurate).

If someone wants to use this sort of service, I'd be happy to throw it together and provide it for free.

Re:How do you get offenders to stop?

[sarx]

I agree; but to be fair, I think it is easy for people with a little less knowledge to heuristically lump bandwidth and latency together, especially if they aren't dealing with (say) satellite links, because links with very low latency are in practice somewhat more likely to have high bandwidth. So if it is wrong, it is at least understandably wrong.

Re:How do you get offenders to stop?

[tom17]

Like this? http://unshorten.com/

I just 'thought of' that only to find it was done aages ago...

Re:How do you get offenders to stop?

[religious freak]

There is an RFC out there (I forget the number off the top of my head) which limits redirects to five. IE6 went above spec and allowed ... 20... I think. IE8 has shortened to allow 10 redirects. FF and Chrome allow the same or less. There is a limit on redirects by RFC, but many websites don't follow the rule and many browsers are forced to compensate because of this.

Ironically, I was just recently accessing a gmail based email system with an Android phone and suddenly I get the message "too many redirects". So now there's no way my google phone can access my google mail. -1 for that one Google.

Re:How do you get offenders to stop?

[dyingtolive]

Nice. If that got turned into a firefox plugin to realtime decrypt the links...

Re:How do you get offenders to stop?

[skids]

Not to mention, when a shared medium or statistically multiplexed PtP link of low bandwidth has congestion, latency is higher than on a higher bandwidth link, which has a much shallower queue built up and/or takes less time to wait for the 1500 byte packet that just started being transmitted to get out of the way. The distinction is only really relevant when you're discussing technicals of TCP window scaling and bandwidth delay product. Certainly not to the end user: "slow" is "slow".

Re:How do you get offenders to stop?

[CannonballHead]

But it's easy to explain the difference, so it's not entirely understandable.

It's like understanding the difference between top speed and acceleration. Not a terribly hard concept.

The real problem is the "internet" is a magic black box. Most people don't understand it's really just a big network, and works like a network... actually, somewhat similar to a much-quicker-delivery postal system, in simplistic terms. Except that there's a "request" thing, not just a "send" thing.

Re:How do you get offenders to stop?

[spazdor]

What an uncharacteristically even-handed Slashdot response!

You must be a noob.

Re:How do you get offenders to stop?

[CraftyJack]

Funny just this morning I noticed that it took at least 5 redirects or more for Google to let me login to Analytics. It felt like my browser had a life of its own!

Sure, but you're already saving 2-5 seconds per search with Google instant, so you still come out ahead.

Re:How do you get offenders to stop?

[Anonymous Coward]

There's also the added DNS lookups to consider.

Re:How do you get offenders to stop?

[shog9]

Close enough: https://addons.mozilla.org/en-US/firefox/addon/9549/

Re:How do you get offenders to stop?

[kurokame]

You know those exploding consoles on Star Trek? Did you ever wonder why someone would invent exploding keyboards? Now you know.

Re:How do you get offenders to stop?

[Anonymous Coward]

So you're saying the internet really is a series of tubes!

Re:How do you get offenders to stop?

[eth1]

Yeah. Load the page as normal, then, in the background, replace the redirects with direct links.

As a bonus, if enough people started using it, it would so bork up the tracking stats (and the load on the redirect servers) that using redirects like that would be less appealing in the first place.

Re:How do you get offenders to stop?

[omnichad]

Oh, so THAT'S why Slashdot doesn't support a wider Unicode character set.

Re:How do you get offenders to stop?

[Chninkel]

disable javascript on google.com: right-click and save url works like a charm :-) (although this also remove search autocompletion and instant ...)

Re:How do you get offenders to stop?

[zmooc]

Sounds just as brilliant as the classical idea to preload all links from a page... Addons like this kill the Internet. They're worse than the problem they're trying to solve.

Re:How do you get offenders to stop?

[martin-boundary]

Any ideas on how to convince people to stop?

This would require a browser plugin to create a dictionary, by converting the short URLs into their long forms, and share that dictionary with others. Ideally, only one person would actually click through the shortened URL to learn what the long URL is, while everyone else would take advantage of that knowledge.

Basically, this amounts to creating a community driven middle man for the URL shortening middle men. The required technology isn't more sophisticated than spam blocklists, which have been done before.

If 80% of the most popular shortened links are community cached, then 80% of the most valuable browsing statistics will be unavailable to the shortened URL provider, and they'll have trouble monetizing their "added value".

Re:How do you get offenders to stop?

[dgatwood]

Simple. Same reason latency for packets varies, but in reverse. There's a huge volume of mail that moves from San Francisco to NYC, so they have air mail routes that optimize this. Because the distance from San Francisco to, for example, San Diego is relatively short and relatively low volume, they take it by truck, so it takes longer to get there, and probably stops in LA on the way (which is probably a latency disaster due to the amount of mail they process---going through LA can be like a corrupt BGP packet causing backbone traffic to be routed through your little home DSL router). With network traffic, it's the reverse; longer hauls are more likely to go to satellite, which spikes the latency way up. Shorter distances are more likely to be by cable, which has a lower latency. The point is that the haul mechanism determines the latency, and different links have different haul mechanisms depending on distance and expected load.

Re:How do you get offenders to stop?

[Surt]

If he thought it was actually a physical series of tubes, he was largely correct, as that's in fact what it is. Lots of plastic tubes filled with glass or copper.

Re:How do you get offenders to stop?

[nametaken]

I don't love them either, but this won't happen. For nearly everyone browsing the web, shortened urls aren't a problem. It's a process that has added utility for both parties and "just works".

Or put another way, when people stop liking Twitter AND knowing how many clicks they've gotten through urls... THEN it'll stop.

Re:How do you get offenders to stop?

[Steeltoe]

Fuck off!

Creds. NOW!!!

Re:How do you get offenders to stop?

[gullevek]

Write in japanese and suddenly you can write a whole novel in 140 characters.

It's a shame too...

[bziman]

I refuse to click on any "shortened" link, because I want to know PRECISELY where I'm going to end up. Thank you Slashdot and goatse.cx. If it's important enough to go visit, it's important enough to spell out properly. And thank you, but I don't live my life via SMS, so the few extra characters is worth my piece of mind.

Re:It's a shame too...

[The MAZZTer]

TinyURL has a cool feature to help with this. For TinyURLs, of course.

Re:It's a shame too...

[piquadratCH]

With bit.ly, you can simply append a "+" to the URL and get an information page showing, between other information, which URL hides behind it, e.g. http://bit.ly/cTGasX+. I don't know if they have a setting to always display this info page, but I'm sure there are usersrcipts and bookmarklets out there that automatically append a "+" to every bit.ly link.

Re:It's a shame too...

[mpicker0]

I prefer to use "lengthened" URLs: http://hugeurl.com/.

Re:It's a shame too...

[zerocool^]

Myself, I prefer shady urls.

http://www.shadyurl.com/

http://slashdot.org/
becomes
http://5z8.info/aryanbr0ther00d_l9e5b_malicious-cookie

Techie price greater than luser price

[apoc.famine]

For those of us who use things like NoScript, the price can be that we don't get there. Ever.

I know that when I go to a site that can't work unless I allow a half dozen or more other sites to run scripts, I sometimes decide that it's not worth my time. When I click a link that then has to contact several domains, (sometimes ones I have specifically blocked) I might stop right there and close the tab.

The web isn't just headed towards redirect hell - it's turning into a damn sketchy web of tentacles working their way into every page. When I find ones that I'm not comfortable having around, I don't go back.

I'm not sure I like what the web has become. Thanks to NoScript, I at least know what it's become.

Re:Techie price greater than luser price

[Spazntwich]

I've noticed this as well, and just consider it the price I have to pay to avoid losing my nerd credentials along with my tiny bank balance.

But it is becoming more prevalent, and I'm not sure what the solution is. Part of me worries this is one of the setup steps in someone's grand scheme to make the internet "dangerous" enough that the "only solution" is to grant absolute internet authority to agency x. You know. To protect the children from all the sexual predators hiding kiddie porn in bit.ly links.

Re:Techie price greater than luser price

[interkin3tic]

My university seems to have come up with a plan to advertise themselves to staff and students who already work/study here: provide no direct link to the university e-mail. They want you to go to the front page, to see the latest news you're not interested in and ways to make donations to the university (hint hint), then login, and you'll be taken to more irrelevant news, links for course tools, and another link for e-mail, which will redirect you one or two times before getting to a google mail system.

I have it automatically forward everything to a normal gmail account that is easier to access anyway, so it doesn't matter except for those few times when I need to access it directly. I usually make a mental note to spend 5 minutes finding out a more direct way and memorizing it, but then never get around to it.

Re:Techie price greater than luser price

[omnichad]

Sorry - I meant to say:

https://mail.google.com/a/schoolmaildomain.edu/

www.linuxtoday.com is the champion

[bogaboga]

Folks at linuxtoday.com have been doing this for a long time. It's one reason I fled the site. Instead of taking me to where I wanna go directly, they make me click twice on the same site. This I believe, enables them to collect 'vital information' to present to their advertisers.

The bad thing is that they lost me and many others in the process.

By the way, it's intentional for me not to link to them from Slashdot directly.

To be honest the web was getting a bit fast...

[youn]

... so they had to find yet another way to slow things down... so the web could live up to its reputation of "world wide wait" ;)

More ads, again...

[Drakkenmensch]

[Wait 30 seconds or click here to skip to comment]

optimize google

[emkyooess]

The Optimize Google add-in for Firefox gets rid of some of their hellish redirects. Sadly, it doesn't update frequently and seems prone to breaking.

My Idea

[wbav]

I want to create a redirect loop. Just imagine, google to tinyurl to bit.ly to dilv.it back to google.

Or you could always just make a really long way to get to someone who'll never give you up, never let you down.

Re:My Idea

[NevarMore]

http://breakingcode.wordpress.com/2010/01/11/having-fun-with-url-shorteners/

Re:My Idea

[clone53421]

Done.

http://tinyurl.com/25lsp67
http://3.ly/2e5g64f

bit.ly adds its own little blab page if it detects multiple redirects, which is entertaining in its own way I guess... never-ending loop:

http://bit.ly/9bV4Re

The preview feature is fun too...

http://3.ly/RwuW
http://tinyurl.com/k2w9uiz

Re:My Idea

[clone53421]

Infinite recursion that defeats browser infinite recursion detection:

http://3.ly/3x5qdno
http://tinyurl.com/36n5j5y

(the tinyurl long link is <html><body><script>var t=setTimeout('window.location="http://3.ly/3x5qdno";',50);</script></body></html> encoded in a data: URI)

It's money, not knowledge

[Morth]

If someone is paying me for the clicks I send to their site, I need to count it so that I know how much I should charge, and they need to count it as well to know I'm not lying. They could make the count on the destination page, but usually it's far more easy to make a special service for it.

A redirect page is usually just a couple of hundred bytes large. Cookies might add some clutter, but probably still less than 1k in each direction, still fits in a single packet. I don't see the problem here.

Sampling can be good

[amplusquem]

Google and Facebook both use these "intermediate steps" to weed out malware infested sites and warn the user. Sampling can also be useful in judging if something is NSFW, or more importantly, rickrolling prevention.

Not all that new

[shoptroll]

Jeff Atwood hit on this issue in a blog post last year: http://www.codinghorror.com/blog/2009/06/url-shorteners-destroying-the-web-since-2002.html

Why it has to be so technically impractical?

[Anonymous Coward]

Less fragile and less of an unnecessary intermediary on this Web 2.0 (or whatever) age would be to catch the click of a link with onclick, set a cookie, and open the original, intended link. When user would again come back to the site, this cookie would be dumped to the site that so much wanted the information it was clicked. Even if the user would have some sort of embedded resource from this site open somewhere else, it could harvest the information and send it back.

Instead, we seem to be ending up with endless chains of redirectors and opaque identifiers that are bound to organizations that don't necessarily exist in a year. What a joy to use technology which is driven by needs of utter morons and greed of those interested to press most information out of the morons...

advertising funds nothing serious

[FuckingNickName]

and there is no useful (i.e. non-light-entertainment) content created primarily through advertising revenue. Slashdot developers who have made their money over the last decade producing tat by not overestimating the intelligence of the general public cannot bear to admit this, but you simply cannot produce high-calibre content when your primary aim is to suck in as many as possible of the kind of people who take notice of adverts.

Murdoch, often maligned for his lack of business sense but mysteriously still richer than all of us, seems to have tried and failed at pushing the subscription model. Obviously there are other viable models for producing information on the web such as government sponsorship (BBC, academia) and well organised groups of hobbyists (e.g. ham radio), but how will the sites who do not already have a dedicated subscription base through off-line heritage sustain themselves? Or maybe the answer is that they will not, the moment they take their eye off the advertiser as customer and start worrying directly about satisfying the desire for the reader to intellectually advance himself.

Facebook

[Xacid]

To play the devil's advocate - facebook's redirects started as a way to filter out all the spam links.

Said the Rich Executive:

[Even on Slashdot FOE]

Oh dear God yes! We can abuse JavaScript more to get more clicks!

wasn't there a time....

[bickerdyke]

when it was considered a security hole if you DIDN'T use a redirect on your page? IIRC there used to be an attack vector where malicous sites used links from freemail pages to steel session IDs from the referer-headers.

Re:wasn't there a time....

[JesseMcDonald]

To be fair, that is nothing more than a workaround for several other major security issues:

1. The referrer header itself. This header serves no useful purpose, and leaks information that the destination website has no need to know. There is no way to use the referrer information securely, since it can be trivially forged, but it does serve as an invaluable tool for malicious attacks and unwanted tracking.

2. Session IDs should be validated to prevent hijacking. At the very least the session ID should be ignored if it comes from a different IP address than the one which created the session. It's not a perfect solution, given dynamic IPs, NAT, and proxies, but it would block most attacks without inconveniencing normal users.

3. No private information, including session IDs, should ever go in the URL. HTTP POST requests or cookies are a better solution here. (Naturally, cookies should be valid only until the end of the session unless the user explicitly indicates otherwise.)

Infinite loop

[ron-l-j]

I think we could just write a Java program that will loop through a range of your top 10 urls every minute. Then you would never have to click. Call it autoBrowser. Because no one should actually click on things.

Re:The AJAX Solution

[farble1670]

your browser's SOP (same origin policy) prevents you from doing this. scripts aren't allowed to make net connections to sites outside the domain of the current page. this is to reduce XSS (cross-site scripting) attacks.

i understand there are standards in the making to allow such things, securely.

Shady URL's

[sirrunsalot]

Personally, I find the trend of redirecting to innocent sites via shady URL's much more alarming: http://5z8.info/foodporn_e0g0l_taliban-meetup

(I promise I'll get modded "troll" by someone who glanced at the link and assumed the worst. Hard to blame them, but I do love using those links whenever possible...)

It's called onmousedown!

[spage]

Yes you are really missing something! Just by viewing source you should notice on the a tag

  onmousedown="return rwt(this,'','','','3','AFQjCNElSuk8pqYMVk5pKG9sycYfDSh7zg','UsteGasJKDRPW0uis7I9Ig','0CCsQFjAC')"
  class="l"
  href="http://example.com/the/original/URL"

So on mouseover you see the original URL, but on click, function rwt ("rewrite"?) sends you to Google first with all that tracking crap, which then redirects you on your way.

If I right-click and Copy Link Location, I get a Google URL in Firefox with this tracking crap. If I feed that to curl, I don't get a status 301 redirect, I get a small piece of HTML back containing both a script that changes the window.location and a meta http-equiv refresh tag.

Disable JavaScript to disable all this.

Re:It's called onmousedown!

[spage]

Hmm, it varies. In Firefox if I'm logged in to Google a search result has an href with a plain URL but the onmousedown rewrites as I described above. If I log out the href is a Google URL and there's still an onmousedown rewrite. But in Konqueror where Google knows nothing about me, I get a plain href and no onmousedown handler.

So maybe another way to avoid Google tracking is use an obscure browser?

And yet, there's no need!

[mcrbids]

It would be trivial to do something with javascript - put an onclick handler that does an xmlHttpRequest to save the "needed" information without even needing to worry about header redirects and the like. The link can be something like

<a href="http://www.thesite.com/path/to/page.html" onclick="return notifyBigBrother(this);">

where notifyBigBrother() is a function that sends the click info to the search engine site. Why isn't this done?

that's why I like the MVP HOSTS file

[PJ6]

They get most of the major offenders in the list. Sure it breaks some links, but it's worth it.

The ones I hate most...

[Reziac]

...are those that come in perfectly legitimate email, stuff that I actively subscribed to. They already know where I came from, their own damned email. Why does it need to go through a redirecting clicktracker?

Furthermore, it lets even legit emails send me somewhere not only unanticipated but also a pain in the ass, like links that unexpectedly open a whopping great PDF.

Many thanks to folks who posted links to two URL de-obfuscator services, which are now permanently on my toolbar.

http://unshorten.com/index.php
http://www.longurlplease.com/