Slashdot Mirror
Broken Links No More?
johndoejersey writes "Students in England have developed a tool which could bring the end to broken links. Peridot, developed by UK intern students at IBM scans company weblinks and replaces outdated information with other relevant documents and links. IBM have already filed 2 patents for the project. The students said Peridot could protect companies by spotting links to sites that have been removed, or which point to wholly unsuitable content. 'Peridot could lead to a world where there are no more broken links,' James Bell, computer science student at the University of Warwick, told BBC News Online. Here is another story on it." See also the BBC story.
There are two parts to this tool, one of which is bad quite and one of which is quite good.
First, replacing links. This is a rather quite bad idea. Here's why, with an example.
In general, we can all agree that the technology behind Google is pretty impressive. It has its own "More Pages Like This" feature, which we can assume is at least somewhat similar to this one. Complex content analysis amoung billions of pages, to determine which are similar and which are different.
So, suppose we had a link to Major League Baseball, www.mlb.com on our page. And suppose, for whatever reason, that their site went away (perhaps a few more players' strikes?).
Well, what does Google suggest as a replacement? Check it out here.
First the National Football League (NFL), then the National Basketball Association (NBA), and then the National Hockey League (NHL). Followed by the ESPN sports network, and NASCAR racing.
Obviously if wanted to link to a site about baseball, all of those (other than ESPN) are really entirely irrelevant.
But if we wanted to link to a site about professional sports organizations, all of those (other than ESPN) are QUITE relevant.
Can this software know our intent?
Hardly.
You really have to question the ability of machines to select relevant links.
The situation is this: If someone goes to the trouble to manually create links in the first place, those should not be automatically changed to other sites that some computer program thinks may be related. Links shouldn't be inserted automatically; if someone needs more information on something you haven't linked to, they can use a search engine. And then your company isn't liable to look idiotic by linking to irrelevant sites.
Now, the other aspect of this product.
Removing dead or changed links is quite another matter. Automated removal of links is a great idea and quite useful. For example, consider when someone's domain name expires and it is taken over by a porn site. It'd be great to have a program that automatically removes links to it from your site. Like this tool, this could be based on a percentage of changed content--if the content changes significantly, remove the link quickly and automatically. If the content changes some intermittent amount, flag the link as needing review by the webmaster.
But in those both case, the software should present the webmaster with a list of such questionable links, those it has removed from the site temporarily, and then allow the webmaster to select replacement links.
Manually. With relevance.
Hang on. On similar lines, I've a great idea. Suppose I type a nonexistent hostname into my browser. Wouldn't it be good if the DNS server just gave me its best guess instead of an error message. Or some kind of Site Finding search engine. That'd be even better than
Athletic Scholarships to universities make as much sense as academic scholarships to sports teams.
A team in the Netherlands built an application that listens to contact centre conversations, picks out relevant keywords and automatically prompts the call centre agent with possible answers.
Does this app take the form of a paper clip? Because that would be a great idea!
Wouldn't this idea work a lot better with semantic web markup attached to links and also to intranet pages?
Agile Artisans
I think the link is broken... :)
My biggest problem is when I follow a link to a website that's no longer there. Yeah, moved pages happen, but I don't think they happen as often as deleted pages, expired domains, deleted websites, etc.
Do not meddle in the affairs of sysadmins, for they are subtle, and quick to anger.
This sounds a little like SiteFinder from Verisign. Click a broken link and isntead of a helpful error message you get whatever content IBM thinks is appropriate. Certainly this could be useful, but it could also end up as just another vehicle for advertising.
Suppose you have broken link http://somesite.com/foo/bar.html, some sites return a list of search results from within 'somesite.com' matching 'foo' or 'bar'. Quite clever, and much more useful than a plain old 'page not found' error.
This just takes that one step further by doing the searching at the referring end instad.
Like tinyurl, but one letter less! http://qurl.co.uk/
I decided it'd be too hard for software to decide whether a change was significant. I wonder how this software does it - presumably, you can change the threshold?
"Peridot could lead to a world where there are no more broken links". Yes, it could. Peridot could also lead to a world where broken links are not manually and intelligently spotted and repaired, but automatically repaired. Automatic resolution of what a link "ought" to point to is never going to be accurate (look at search engines), and could make a company website a minefield of confusion and frustration for the user.
Only time will tell, I suppose.
Some algorithm cruising through my website, rearranging files as it sees fit?
Sounds like a recipe for utter disaster in the worst case, and a source of mildly embarassing incidents at best.
How about this algorithm just report dead links to a human instead of trying too hard to be clever?
This sounds like someone had to come up with a final project, and settled on this one.
Maybe I'm being overly naive, but checking for broken links doesn't seem all that spectacular to me. It wouldn't take long to write a script to find all the broken links on a page.
The only parts that seemed worth while are replacing the links automatically, and testing if links are relevant.
I'm not so sure I'd trust a computer to do those things though. I'd much rather have the links flagged and checked by a human.
Slashdot Syndrome: the sudden, extreme urge to correct someone in order to validate one's self.
Any good Content Management System should already take care of any internal broken links automatically, or notify the webmaster so he'll be able to take care of it manually (in the case of page deletion, etc).
The only kind of people who'd go out of their way to use this software, probably have already use some sort of CMS.
A link points to document X.
If document X moves, and the link is invalid, a search for the link might actually find document X, and therefore, you have your benefit, and you would have saved a 404.
However - if a document becomes deprecated and deleted, then how can you assume the link is valid?
Or indeed, if the document has no relevant substitute.
A genealogy providing a link to another Willian Wallace wouldn't be good news if the original page went missing.
A better system is automated 404 alerting to the webservers administrator.
A bad link gets hit, bam, what document, from where. You can work things out intelligently, not automatically.
I think this is silly, perhaps grasping at straws, I see no reason why we would replace all our links to google 'I feel lucky' searches, so why do something like this?
This is the essence of what they have, and all they have done is coulded the search IP field (which is important) with 2 more patents, again increasing costs and endangering open source innovation, the true innovative playing field.
Of course, I could be wrong.
#hostfile 0.0.0.0 primidi.com 0.0.0.0 www.primidi.com 0.0.0.0 radio.weblogs.com
this isn't about replacing links on the internet as a whole... it's about replacing links on your company website, or at least reviewing those links.
not everything that happens in the world is an attempt by big brother to steer internet traffic to verisign or microsoft.
Spyware/Adware and IE already give you search results and links. The only difference is that this automatically places you at a different link without a choice.
Hmmm... Pie...
Damn you slashdotters!!! I work at IBM and the intranet server is down! I can't believe you've managed to cause the automatic load-balancer to kill the intranet in favour of a slashdot article.
Damn you!!
And purple hatstands
ErrorDocument 404 script.pl
Where script.pl parse the wanted URL and ask an indexing engin to find the most relevant page associated with the query...
Trolling using another account since 2005.
I'd prefer a more helpful 404 page, maybe with some links to the homepage or main sections of the site on it.
Sort of a "cannot find hello.jpg, click here to go back to the main page".
My point being, if the document I'm looking for is not there, I want to know it's not there. I don't want to read something else, thinking it's what I meant to read.
Usually when I'm googling around and clicking stuff I'm looking for the answer to some coding or computer related problem. I don't want to click on a link for "configuring Samba 3.0 with AD support", and wind up on a "Configuring Samba 2.2 with LDAP" and waste my time following bad advice.
I don't need no instructions to know how to rock!!!!
http://www-ai.cs.uni-dortmund.de/DOKUMENTE/malzahn _2003a.pdf
Basically, the thesis evaluates different methods to build a kind of "finger-print" of a page. The finger print is used to find the page with google if it is gone, or has changed significantly.
The internet wayback machine was used to learn distinguishing disappeared pages from pages changing slightly over the time.
Just use the W3C's link-checker.
For those running a real browser, just make this a link, preferably in your personal tool bar.
) {v oid(Qr=prompt('Url...',''))};if(Qr)location.href=' http://web.archive.org/web/*/'+escape(Qr)
javascript:Qr=document.URL;if(Qr=='about:blank'
Now when I click on a link that isn't there, I select my Archive search button and it shows me the Wayback Machine's history of that link. Of course it works only if the url hasn't been modified by the server. If it has it's another couple steps (copy link, ^T, archive search, paste url in pop-up dialog)
tcboo
You could create your links using googles im feeling lucky feature, assuming it was just a generic link site looking for interesting sites rather then specific articles.
s +For+Nerds&btnI=Google+Search
e.g:
http://www.google.com/search?hl=en&ie=UTF-8&q=New
And voila, you'll site will take you to the most popular related site to news for nerds, automagically, if slashdot died one day, another site would take it's place in the google rankings. FF.
On a slightly related note, a Firefox extension that searched links ahead and removed the link rendering for those that return a 404 might be handy (albeit fairly evil).
On a less related note, I've long been disappointed that some 300 series status codes in HTTP are so under-exploited, both by clients (e.g. automated bookmark management) and people running web sites.
Soon the target network would be back up, but all your links would be lost and randomly changed to something less useful. Good Invention!