Broken Links No More?

johndoejersey writes "Students in England have developed a tool which could bring the end to broken links. Peridot, developed by UK intern students at IBM scans company weblinks and replaces outdated information with other relevant documents and links. IBM have already filed 2 patents for the project. The students said Peridot could protect companies by spotting links to sites that have been removed, or which point to wholly unsuitable content. 'Peridot could lead to a world where there are no more broken links,' James Bell, computer science student at the University of Warwick, told BBC News Online. Here is another story on it." See also the BBC story.

Comments

Can someone say "Bad Idea Jeans"?

[Anonymous Coward]

There are two parts to this tool, one of which is bad quite and one of which is quite good.

First, replacing links. This is a rather quite bad idea. Here's why, with an example.

In general, we can all agree that the technology behind Google is pretty impressive. It has its own "More Pages Like This" feature, which we can assume is at least somewhat similar to this one. Complex content analysis amoung billions of pages, to determine which are similar and which are different.

So, suppose we had a link to Major League Baseball, www.mlb.com on our page. And suppose, for whatever reason, that their site went away (perhaps a few more players' strikes?).

Well, what does Google suggest as a replacement? Check it out here.

First the National Football League (NFL), then the National Basketball Association (NBA), and then the National Hockey League (NHL). Followed by the ESPN sports network, and NASCAR racing.

Obviously if wanted to link to a site about baseball, all of those (other than ESPN) are really entirely irrelevant.

But if we wanted to link to a site about professional sports organizations, all of those (other than ESPN) are QUITE relevant.

Can this software know our intent?

Hardly.

You really have to question the ability of machines to select relevant links.

The situation is this: If someone goes to the trouble to manually create links in the first place, those should not be automatically changed to other sites that some computer program thinks may be related. Links shouldn't be inserted automatically; if someone needs more information on something you haven't linked to, they can use a search engine. And then your company isn't liable to look idiotic by linking to irrelevant sites.

Now, the other aspect of this product.

Removing dead or changed links is quite another matter. Automated removal of links is a great idea and quite useful. For example, consider when someone's domain name expires and it is taken over by a porn site. It'd be great to have a program that automatically removes links to it from your site. Like this tool, this could be based on a percentage of changed content--if the content changes significantly, remove the link quickly and automatically. If the content changes some intermittent amount, flag the link as needing review by the webmaster.

But in those both case, the software should present the webmaster with a list of such questionable links, those it has removed from the site temporarily, and then allow the webmaster to select replacement links.

Manually. With relevance.

Re:Can someone say "Bad Idea Jeans"?

[GigsVT]

The "related" search isn't what you should be looking at.

Try this.

Re:Can someone say "Bad Idea Jeans"?

[Anonymous Coward]

You probably bookmark your A/C posts so you can slip back and check them. You're not fooling anyone, you know.

Great

[gowen]

Peridot could lead to a world where there are no more broken links,

... just links that don't got where the author intended. Gee, thats ... just great.

Hang on. On similar lines, I've a great idea. Suppose I type a nonexistent hostname into my browser. Wouldn't it be good if the DNS server just gave me its best guess instead of an error message. Or some kind of Site Finding search engine. That'd be even better than ... :)

I liked this one:

[underpar]

A team in the Netherlands built an application that listens to contact centre conversations, picks out relevant keywords and automatically prompts the call centre agent with possible answers.

Does this app take the form of a paper clip? Because that would be a great idea!

Semantic Web?

[jarich]

Wouldn't this idea work a lot better with semantic web markup attached to links and also to intranet pages?

well

[Anonymous Coward]

I think the link is broken... :)

Take this with a grain of salt

[blankman]

This sounds a little like SiteFinder from Verisign. Click a broken link and isntead of a helpful error message you get whatever content IBM thinks is appropriate. Certainly this could be useful, but it could also end up as just another vehicle for advertising.

Well that sounds perfectly dreadful

[Illserve]

Some algorithm cruising through my website, rearranging files as it sees fit?

Sounds like a recipe for utter disaster in the worst case, and a source of mildly embarassing incidents at best.

How about this algorithm just report dead links to a human instead of trying too hard to be clever?

This sounds like someone had to come up with a final project, and settled on this one.

CMS

[Anonymous Coward]

Any good Content Management System should already take care of any internal broken links automatically, or notify the webmaster so he'll be able to take care of it manually (in the case of page deletion, etc).

The only kind of people who'd go out of their way to use this software, probably have already use some sort of CMS.

It will work, but that isn't good, here is why

[tod_miller]

A link points to document X.

If document X moves, and the link is invalid, a search for the link might actually find document X, and therefore, you have your benefit, and you would have saved a 404.

However - if a document becomes deprecated and deleted, then how can you assume the link is valid?

Or indeed, if the document has no relevant substitute.

A genealogy providing a link to another Willian Wallace wouldn't be good news if the original page went missing.

A better system is automated 404 alerting to the webservers administrator.

A bad link gets hit, bam, what document, from where. You can work things out intelligently, not automatically.

I think this is silly, perhaps grasping at straws, I see no reason why we would replace all our links to google 'I feel lucky' searches, so why do something like this?

This is the essence of what they have, and all they have done is coulded the search IP field (which is important) with 2 more patents, again increasing costs and endangering open source innovation, the true innovative playing field.

Of course, I could be wrong.

German readers...

[dukoids]

may want to take a look at the master's thesis of Nils Malzahn (from 2003, in German) to see (in detail) how this actually can work:

http://www-ai.cs.uni-dortmund.de/DOKUMENTE/malzahn _2003a.pdf

Basically, the thesis evaluates different methods to build a kind of "finger-print" of a page. The finger print is used to find the page with google if it is gone, or has changed significantly.

The internet wayback machine was used to learn distinguishing disappeared pages from pages changing slightly over the time.